Difference when run with config file and without

[michasacuer]

So i have a bug where i provide a config file i have a lot of errors in my pipelines.

My Example config file for checkov:

{
    "tools": [
      {
        "tool": {
          "name": "Checkov",
          "version": "Latest"
        },
        "arguments": {
          "DownloadExternalModules": "false",
          "TargetDirectory": "$(Checkov.DefaultTargetDirectory)"
        }
      }
    ]
  }

and everything is fine but i am getting this error, even with false setting:

D:\a\_msdo\packages\nuget\Microsoft.Guardian.CheckovRedist_windows_amd64.3.2.144\tools\dist\checkov.exe --download-external-modules false --directory .\ --output-file-path D:\a\1\s\.gdn\.r\checkov\001\checkov.sarif
##[error]2024-07-08 11:42:17,665 [ThreadPoolEx] [WARNI]  Failed to download module git::https://ADOORG@dev.azure.com/ADOORG/PROJ/_git/kvmodule//src?ref=v0.3:None (for external modules, the --download-external-modules flag is required)

Task still yells that DownloadExternalModules is required even when it is provided and set to false. My pipeline is the simplest template as you can imagine:

parameters:
  TemplatesRepoName: ''

stages:
- stage: Microsoft_Defender
  displayName: Microsoft Defender for Cloud DevOps security
  condition: always()
  pool:
    vmImage: windows-latest
  jobs:
  - job: Microsoft_Defender_Scan
    displayName: Scan
    steps:
    - checkout: ${{ parameters.TemplatesRepoName }}
    - checkout: self
          
    - task: MicrosoftSecurityDevOps@1
      displayName: Microsoft Security DevOps
      inputs:
        config: configs/checkov.gdnconfig

From template above i got a lot of errors:

    Process:
      Convert:
        Converting any raw tool logs to Sarif format ...
        Completed converting raw tool logs to Sarif format.
      Import:
        No tool logs to process. 
    Break:
      Guardian is searching for results that meet the given criteria to break the build.
      Results Query Summary:
        Baselines: default
        Suppression Sets: default
        Policy: azuredevops
      Saved file D:\a\1\a\.gdn\msdo.sarif
      Found no breaking results.
      Active results: 0
      Skipped results: 0
        Baselined results: 0
        Suppressed results: 0
        Results excluded by tool filters: 0
        Results below minimum severity: 0
        Results classified as Pass: 0
        Results in flight: 0
##[error]Error running tool 1 of 1: checkov
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
##[error]BreakException: Guardian detected one or more breaking results.

When i do not provide config file, no issues, only error that DownloadExternalModules is required.

Why this task behaves differently when config file for checkov is provided?

[ptmetcalf]

I'm also seeing this same behavior. If I try to pass a config file even in the simplest form it causes it to fail and not upload any results.

{
  "tools": [
    {
      "tool": {
        "name": "Checkov",
        "version": "Latest"
      },
      "arguments": {
      }
    }
  ]
}

If it helps anyone else, I was able to use a checkov specific config file and it seems the scan picks up and uses that correctly. I just put a .checkov.yml file in my scan directory of my repo. This allowed me to set options as well as choose tests to skip.

directory: ./
output: sarif
soft-fail: true
framework: terraform
skip-check:
  - CKV_AZURE_190
  - CKV2_AZURE_1

You can see the details of the checkov specific config file on there site. Configuration using a config file