[CyberSixgill Actionable Alerts] - Creating a new CCF Data Connector#14524
Open
dhanunjaya1054 wants to merge 2 commits into
Open
[CyberSixgill Actionable Alerts] - Creating a new CCF Data Connector#14524dhanunjaya1054 wants to merge 2 commits into
dhanunjaya1054 wants to merge 2 commits into
Conversation
v-atulyadav
added
Codeless Connector Framework (CCF) Connector
Solution
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR migrates the Cybersixgill Actionable Alerts solution from the legacy Azure Function connector to a Codeless Connector Framework (CCF) connector, introducing a unified parser and updating content (workbooks/hunting queries) to use that parser.
Changes:
- Added a new CCF connector (connector definition, poller config, DCR, and custom table) for Cybersixgill actionable alerts.
- Added parser functions to unify legacy Azure Function data (
CyberSixgill_Alerts_CL) with the new CCF table (CyberSixgillAlertsV2_CL) underCyberSixgill_Alerts. - Updated workbooks and hunting query to query the unified parser; bumped solution version to 3.1.0 and added release notes.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| Solutions/Cybersixgill-Actionable-Alerts/Workbooks/ActionableAlertsList.json | Updated workbook queries/columns to use CyberSixgill_Alerts parser fields. |
| Solutions/Cybersixgill-Actionable-Alerts/Workbooks/ActionableAlertsDashboard.json | Updated dashboard visuals to query CyberSixgill_Alerts and new normalized columns. |
| Solutions/Cybersixgill-Actionable-Alerts/ReleaseNotes.md | Added v3.1.0 release note entry describing CCF connector + unified parser. |
| Solutions/Cybersixgill-Actionable-Alerts/Parsers/parser_CyberSixgillAlertsFunction.json | New unified CyberSixgill_Alerts parser combining legacy + CCF parents/children. |
| Solutions/Cybersixgill-Actionable-Alerts/Parsers/parser_CyberSixgillAlertsExpandedFunction.json | New expanded parser to fan out sub-alerts from the CCF table. |
| Solutions/Cybersixgill-Actionable-Alerts/Hunting Queries/ActionableAlerts.yaml | Updated hunting query to use CyberSixgill_Alerts and added required CCF connector/table. |
| Solutions/Cybersixgill-Actionable-Alerts/Data/Solution_Cybersixgill_Actionable_Alerts.json | Added parser artifacts, added CCF connector definition, bumped solution version. |
| Solutions/Cybersixgill-Actionable-Alerts/Data Connectors/CybersixgillAlerts_CCF/table_CyberSixgillAlertsV2.json | New custom table schema for CCF-ingested alerts. |
| Solutions/Cybersixgill-Actionable-Alerts/Data Connectors/CybersixgillAlerts_CCF/Cybersixgill_PollerConfig.json | New REST poller config to fetch alerts + nested detail calls. |
| Solutions/Cybersixgill-Actionable-Alerts/Data Connectors/CybersixgillAlerts_CCF/Cybersixgill_DCR.json | New DCR declaration + transformKql mapping API payload → table schema. |
| Solutions/Cybersixgill-Actionable-Alerts/Data Connectors/CybersixgillAlerts_CCF/Cybersixgill_ConnectorDefinition.json | New CCF connector UI definition including sample/graph queries. |
| { | ||
| "type": "Microsoft.SecurityInsights/dataConnectors", | ||
| "apiVersion": "2022-10-01-preview", | ||
| "name": "[[concat('parameters('workspace')', '/Microsoft.SecurityInsights/','CyberSixgillAlertsV2_CL' , uniqueString(parameters('organizationId')) )]", |
Comment on lines
+29
to
+33
| "queryParameters": { | ||
| "sort_order": "asc", | ||
| "limit": "100", | ||
| "organization_id": "[[parameters('organizationId')]" | ||
| }, |
Comment on lines
+43
to
+47
| "paging": { | ||
| "pagingType": "Offset", | ||
| "offsetParaName": "offset", | ||
| "pageSize": 100, | ||
| "pageSizeParameterName": "fetch_size", |
Comment on lines
+24
to
+28
| "headers": { | ||
| "Content-Type": "application/json", | ||
| "Accept": "application/json", | ||
| "x-channel-id": "cea9a52effad4bc5e905a5a653f5cf9b" | ||
| }, |
Comment on lines
+69
to
+73
| "headers": { | ||
| "Content-Type": "application/json", | ||
| "Accept": "application/json", | ||
| "x-channel-id": "cea9a52effad4bc5e905a5a653f5cf9b" | ||
| }, |
| }, | ||
| { | ||
| "description": "Alerts by Threat Level", | ||
| "query": "CyberSixgillAlertsV2\n| summarize count() by ThreatLevel" |
Comment on lines
+43
to
+44
| "name": "CyberSixgillAlertsV2", | ||
| "lastDataReceivedQuery": "CyberSixgillAlertsV2\n| where TimeGenerated > ago(6h)\n| summarize Time = max(TimeGenerated)\n| where isnotempty(Time)" |
| "quote": "'", | ||
| "delimiter": ",", | ||
| "query": "CyberSixgill_Alerts_CL\r\n| where TimeGenerated > ago(720d)\r\n| project MyJson = parse_json(threats_s)\r\n| mvexpand MyJson\r\n| summarize by tostring(MyJson)\r\n| sort by MyJson asc", | ||
| "query": "CyberSixgill_Alerts\r\n| where TimeGenerated > ago(720d)\r\n| project MyJson = parse_json(Threats)\r\n| mvexpand MyJson\r\n| summarize by tostring(MyJson)\r\n| sort by MyJson asc", |
| "displayName": "CyberSixgillAlertsExpanded", | ||
| "category": "Microsoft Sentinel Parser", | ||
| "functionAlias": "CyberSixgillAlertsExpanded", | ||
| "query": "let _src = CyberSixgillAlertsV2_CL | extend ParentId = '', UniqueId = Id, SubAggregateAlertId = long(null), SubAlertDetails = dynamic({});let _subs = CyberSixgillAlertsV2_CL | where SubAlertsCount > 0 and isnotnull(SubAlerts) | mv-expand _sub = SubAlerts to typeof(dynamic) | extend ParentId = Id, UniqueId = strcat(Id, '__', tostring(_sub.aggregate_alert_id)), SubAggregateAlertId = tolong(_sub.aggregate_alert_id), SubAlertDetails = _sub | project-away _sub;union _src, _subs", |
| "clv2ws1" | ||
| ], | ||
| "outputStream": "Custom-CyberSixgillAlertsV2_CL", | ||
| "transformKql": "source | extend TimeGenerated = iff(isnull(todatetime(['date'])), now(), todatetime(['date'])) , Id = ['id'] , AlertDate = ['date'] , AlertTitle = ['title'] , AlertName = ['alert_name'] , AlertTypeId = ['alert_type_id'] , Content = case(isnotempty(tostring(['additional_info']['cve_id'])), tostring(['additional_info']['cve_description']), isempty(tostring(['content'])), tostring(['es_item']['content']), tostring(['content'])) , Severity = ['severity'] , ThreatLevel = ['threat_level'] , Threats = tostring(['threats']) , EsId = ['es_id'] , UserId = ['user_id'] , AlertCreationDate = now() , Time = tostring(['date']) , Read = ['read'] , Site = ['site'] , Assets = tostring(array_concat(iff(isnull(['matched_assets']['bins']), parse_json('[]'), todynamic(tostring(['matched_assets']['bins']))), iff(isnull(['matched_assets']['domain_names']), parse_json('[]'), todynamic(tostring(['matched_assets']['domain_names']))))) , MatchedAssetsBins = tostring(['matched_assets']['bins']) , MatchedAssetsDomainNames = tostring(['matched_assets']['domain_names']) , SubAlertsCount = iff(isnull(['sub_alerts']), 0.0, toreal(array_length(['sub_alerts']))) , SubAlerts = ['sub_alerts'] , StatusName = iff(isempty(tostring(['status']['name'])), 'Treatment Required', tostring(['status']['name'])) , ThreatActor = tostring(['es_item']['creator']) , ThreatSource = tostring(['es_item']['site']) , PortalUrl = strcat('https://portal.cybersixgill.com/#/?actionable_alert=', tostring(['id'])) , Cve = iff(isempty(tostring(['additional_info']['cve_id'])), '', tostring(['additional_info']['cve_id'])) , CveUrl = iff(isempty(tostring(['additional_info']['cve_id'])), '', strcat('https://portal.cybersixgill.com/#/cve/', tostring(['additional_info']['cve_id']))) , CyberSixgillCvss31 = iff(isempty(tostring(['additional_info']['cve_id'])), real(null), iff(isnull(['additional_info']['nvd']['v3']['current']), -1.0, toreal(['additional_info']['nvd']['v3']['current']))) , CyberSixgillCvss20 = iff(isempty(tostring(['additional_info']['cve_id'])), real(null), iff(isnull(['additional_info']['nvd']['v2']['current']), -1.0, toreal(['additional_info']['nvd']['v2']['current']))) , CyberSixgillDveScore = iff(isempty(tostring(['additional_info']['cve_id'])), real(null), toreal(['additional_info']['score']['current'])) , OrganizationName = tostring(['additional_info']['organization_name']) , ActorUrlWithContext = iff(isempty(tostring(['es_item']['creator'])), '', strcat('https://portal.cybersixgill.com/#/actor/', tostring(['es_item']['creator']), '/', tostring(['es_item']['site']))) , ActorUrlWithoutContext = iff(isempty(tostring(['es_item']['creator'])), '', strcat('https://portal.cybersixgill.com/#/actor/', tostring(['es_item']['creator']), '/')) , AdditionalInfo = ['additional_info'] , EsItem = ['es_item'] , MatchedAssets = ['matched_assets'] , Status = ['status'] , SourceSystem = 'RestAPI' , MG = '00000000-0000-0000-0000-000000000000' , ManagementGroupName = '' , Computer = '' | project TimeGenerated , Id , AlertDate , AlertTitle , AlertName , AlertTypeId , Content , Severity , ThreatLevel , Threats , EsId , UserId , AlertCreationDate , Time , Read , Site , Assets , MatchedAssetsBins , MatchedAssetsDomainNames , SubAlertsCount , SubAlerts , StatusName , ThreatActor , ThreatSource , PortalUrl , Cve , CveUrl , CyberSixgillCvss31 , CyberSixgillCvss20 , CyberSixgillDveScore , OrganizationName , ActorUrlWithContext , ActorUrlWithoutContext , AdditionalInfo , EsItem , MatchedAssets , Status , SourceSystem , MG , ManagementGroupName , Computer" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Required items, please complete
Change(s):
Updating huntig queries and workbooks accordingly.
Reason for Change(s):
Version Updated:
Testing Completed: