Skip to content

Migrate Valence Security Sentinel connector to CCF Push (Codeless Con…#14528

Open
shlomimatichin wants to merge 1 commit into
Azure:masterfrom
valencesec:master
Open

Migrate Valence Security Sentinel connector to CCF Push (Codeless Con…#14528
shlomimatichin wants to merge 1 commit into
Azure:masterfrom
valencesec:master

Conversation

@shlomimatichin

Copy link
Copy Markdown
Contributor

…nector Framework)

Replaces the deprecated Log Analytics HTTP Data Collector API (Workspace ID + Primary Key) connector with a CCF Push connector using DCE/DCR-based auth via the Azure Monitor Logs Ingestion API. This follows Microsoft's deprecation of the sharedKeys-based connector pattern.

Changes:

  • DELETE: ValenceSecurity.json (legacy Data Collector API connector)

  • ADD: ValenceSecurity_PUSH_CCP/ folder with 5 CCF files:

    • ValenceSecurity_connectorDefinition.json (kind: Customizable)
    • ValenceSecurity_PollerConfig.json (kind: Push)
    • ValenceSecurity_DCR.json (two streams: ValenceAlerts + ValenceAuditLogs)
    • ValenceSecurity_table_ValenceAlerts.json
    • ValenceSecurity_table_ValenceAuditLogs.json
  • UPDATE: Analytic rule to reference new table (ValenceAlerts_CL) and CCF connector ID (ValenceSecurityCcfPush); field names updated to CCF schema

  • UPDATE: Workbook to use ValenceAlerts_CL and new severity field name

  • UPDATE: Solution data file to point to CCF connector definition; v3.0.3

  • REGENERATE: Package (mainTemplate.json, createUiDefinition.json, testParameters.json, 3.0.3.zip) using the V3 packaging tool

    Required items, please complete

    Change(s):

    • See guidance below

    Reason for Change(s):

    • See guidance below

    Version Updated:

    • Required only for Detections/Analytic Rule templates
    • See guidance below

    Testing Completed:

    • See guidance below

    Checked that the validations are passing and have addressed any issues that are present:

    • See guidance below

Guidance <- remove section before submitting


Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:

Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR. Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process. Providing associated Issues that this resolves also easily connects the reason.

Change(s):

  • Updated syntax for XYZ.yaml

Reason for Change(s):

Version updated:

  • Yes
  • Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly. If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

  • Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

  • Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:


…nector Framework)

Replaces the deprecated Log Analytics HTTP Data Collector API (Workspace ID +
Primary Key) connector with a CCF Push connector using DCE/DCR-based auth via
the Azure Monitor Logs Ingestion API. This follows Microsoft's deprecation of
the sharedKeys-based connector pattern.

Changes:
- DELETE: ValenceSecurity.json (legacy Data Collector API connector)
- ADD: ValenceSecurity_PUSH_CCP/ folder with 5 CCF files:
  - ValenceSecurity_connectorDefinition.json (kind: Customizable)
  - ValenceSecurity_PollerConfig.json (kind: Push)
  - ValenceSecurity_DCR.json (two streams: ValenceAlerts + ValenceAuditLogs)
  - ValenceSecurity_table_ValenceAlerts.json
  - ValenceSecurity_table_ValenceAuditLogs.json
- UPDATE: Analytic rule to reference new table (ValenceAlerts_CL) and CCF
  connector ID (ValenceSecurityCcfPush); field names updated to CCF schema
- UPDATE: Workbook to use ValenceAlerts_CL and new severity field name
- UPDATE: Solution data file to point to CCF connector definition; v3.0.3
- REGENERATE: Package (mainTemplate.json, createUiDefinition.json,
  testParameters.json, 3.0.3.zip) using the V3 packaging tool

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Migrates the Valence Security Sentinel solution from the deprecated Log Analytics HTTP Data Collector (shared keys) connector to a CCF Push connector using DCE/DCR + Azure Monitor Logs Ingestion API, and updates solution assets (analytic rule, workbook, packaging) to align with the new tables and schema.

Changes:

  • Replaces legacy connector with CCF Push connector artifacts (connector definition, poller config, DCR, custom tables).
  • Updates analytic rule + workbook to query ValenceAlerts_CL and new field names (e.g., severity).
  • Regenerates solution package assets and bumps solution/package version to 3.0.3.

Reviewed changes

Copilot reviewed 12 out of 14 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
Solutions/Valence Security/Workbooks/ValenceAlertsWorkbook.json Updates workbook queries to new table/field names.
Solutions/Valence Security/Analytic Rules/ValenceAlerts.yaml Updates rule connectorId/table + field mappings for new schema.
Solutions/Valence Security/Data/Solution_ValenceSecurity.json Points solution to the new connector definition and bumps version.
Solutions/Valence Security/Data Connectors/ValenceSecurity.json Deletes legacy shared-keys connector definition.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_connectorDefinition.json Adds CCF Push connector definition (UI, queries, permissions, instructions).
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_PollerConfig.json Adds Push connector configuration (stream + auth placeholders).
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_DCR.json Adds DCR defining streams/transforms for Alerts + AuditLogs.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_table_ValenceAlerts.json Adds ValenceAlerts_CL custom table definition.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_table_ValenceAuditLogs.json Adds ValenceAuditLogs_CL custom table definition.
Solutions/Valence Security/Package/mainTemplate.json Updates packaged ARM to deploy new connector resources and bump versions.
Solutions/Valence Security/Package/createUiDefinition.json Updates solution install UI description and text blocks.
Solutions/Valence Security/Package/testParameters.json Adds packaging test parameters (location/workspace/etc.).

Comment on lines +933 to +936
"guidValue": {
"defaultValue": "[[newGuid()]",
"type": "securestring"
},
Comment on lines +941 to +947
"auth": {
"type": "object",
"defaultValue": {
"appId": "[[parameters('auth').appId]]",
"servicePrincipalId": "[[parameters('auth').servicePrincipalId]]"
}
},
}
},
{
"name": "[[concat(parameters('innerWorkspace'),'/Microsoft.SecurityInsights/', 'ValenceSecurityPushConnector', parameters('guidValue'))]",
Comment on lines +1004 to +1012
"streamName": "Custom-ValenceAlerts",
"dataCollectionEndpoint": "[[parameters('dcrConfig').dataCollectionEndpoint]",
"dataCollectionRuleImmutableId": "[[parameters('dcrConfig').dataCollectionRuleImmutableId]"
},
"auth": {
"type": "Push",
"appId": "[[parameters('auth').appId]",
"servicePrincipalId": "[[parameters('auth').servicePrincipalId]"
},
"dataConnectorTemplateNameConnectorDefinition1": "[concat(parameters('workspace'),'-dc-',uniquestring(variables('_dataConnectorContentIdConnectorDefinition1')))]",
"_dataConnectorContentIdConnections1": "ValenceSecurityCcfPushConnections",
"dataConnectorTemplateNameConnections1": "[concat(parameters('workspace'),'-dc-',uniquestring(variables('_dataConnectorContentIdConnections1')))]",
"dataCollectionEndpointId1": "[concat('/subscriptions/',parameters('subscription'),'/resourceGroups/',parameters('resourceGroupName'),'/providers/Microsoft.Insights/dataCollectionEndpoints/',parameters('workspace'))]",
Comment on lines +510 to +516
"name": "Valence-DCR",
"apiVersion": "2022-06-01",
"type": "Microsoft.Insights/dataCollectionRules",
"location": "[parameters('workspace-location')]",
"kind": "[variables('blanks')]",
"properties": {
"dataCollectionEndpointId": "[variables('dataCollectionEndpointId1')]",
Comment on lines +8 to +13
"connectorDefinitionName": "ValenceSecurityCcfPush",
"dcrConfig": {
"streamName": "Custom-ValenceAlerts",
"dataCollectionEndpoint": "{{dataCollectionEndpoint}}",
"dataCollectionRuleImmutableId": "{{dataCollectionRuleImmutableId}}"
},
"type": "Microsoft.OperationalInsights/workspaces/providers/contentTemplates",
"apiVersion": "2023-04-01-preview",
"name": "[variables('dataConnectorTemplateSpecName1')]",
"name": "[concat(parameters('workspace'),'/Microsoft.SecurityInsights/', variables('dataConnectorTemplateNameConnectorDefinition1'), variables('dataConnectorCCPVersion'))]",
"isWizard": false,
"basics": {
"description": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ValenceSecurityLogo.svg\"width=\"75px\"height=\"75px\">\n\n **Note:** Please refer to the following before installing the solution: \r \n • Review the solution [Release Notes](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Valence%20Security/ReleaseNotes.md)\r \n • There may be [known issues](https://aka.ms/sentinelsolutionsknownissues) pertaining to this Solution, please refer to them before installing.\n\n[Valence Security](https://valencesecurity.com/) SaaS security platform: Valence is the first SaaS security company to combine SSPM and advanced remediation with business user collaboration to find and fix SaaS security risks. SaaS applications are becoming decentrally managed and more complex, which is introducing misconfiguration, identity, data, and SaaS-to-SaaS integration risks. The Valence SaaS Security Platform provides visibility and remediation capabilities for business-critical SaaS applications such as Microsoft 365, Google Workspace, Salesforce, GitHub and Slack. With Valence, security teams can empower their business to securely adopt SaaS.\n\n**Data Connectors:** 1, **Workbooks:** 1, **Analytic Rules:** 1\n\n[Learn more about Microsoft Sentinel](https://aka.ms/azuresentinel) | [Learn more about Solutions](https://aka.ms/azuresentinelsolutionsdoc)",
"description": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ValenceSecurityLogo.svg\"width=\"75px\"height=\"75px\">\n\n**Note:** Please refer to the following before installing the solution: \n\n• Review the solution [Release Notes](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Valence%20Security/ReleaseNotes.md)\n\n • There may be [known issues](https://aka.ms/sentinelsolutionsknownissues) pertaining to this Solution, please refer to them before installing.\n\n[Valence Security](https://valencesecurity.com/) SaaS security platform: Valence is the first SaaS security company to combine SSPM and advanced remediation with business user collaboration to find and fix SaaS security risks. SaaS applications are becoming decentrally managed and more complex, which is introducing misconfiguration, identity, data, and SaaS-to-SaaS integration risks. The Valence SaaS Security Platform provides visibility and remediation capabilities for business-critical SaaS applications such as Microsoft 365, Google Workspace, Salesforce, GitHub and Slack. With Valence, security teams can empower their business to securely adopt SaaS.\n\n**Data Connectors:** 1, **Workbooks:** 1, **Analytic Rules:** 1\n\n[Learn more about Microsoft Sentinel](https://aka.ms/azuresentinel) | [Learn more about Solutions](https://aka.ms/azuresentinelsolutionsdoc)",
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants