Add HINT validation result for dandiset.yaml in both validate and upload commands #1741

Copilot · 2025-11-12T20:53:17Z

When BIDS validator encounters dandiset.yaml (not part of BIDS spec), users now receive both the original ERROR from BIDS validator and a HINT suggesting to add the file to .bidsignore. The HINT is generated at the BIDS validator level, ensuring it appears in both dandi validate and dandi upload commands.

Changes

dandi/bids_validator_deno/_validator.py

Added HINT generation in _harmonize() function after processing each dandiset.yaml error
HINT ID: DANDI.BIDSIGNORE_DANDISET_YAML, severity: Severity.HINT, scope: DATASET
Provides actionable message with exact line to add to .bidsignore
Generates HINT at BIDS validator level, affecting all commands that use BIDS validation

dandi/tests/test_validate.py

Enhanced test_validate_bids() to verify both ERROR and HINT are generated
Updated to not expect modified error message (preserves original BIDS error)
Validates HINT properties and message content

Example Output

# ERROR (original from BIDS validator, unchanged)
ValidationResult(
    id="BIDS.NOT_INCLUDED",
    severity=Severity.ERROR,
    message="Files with such naming scheme are not part of BIDS specification..."
)

# HINT (new)
ValidationResult(
    id="DANDI.BIDSIGNORE_DANDISET_YAML",
    severity=Severity.HINT,
    message="Consider creating or updating a `.bidsignore` file in the root of your BIDS dataset to ignore `dandiset.yaml`. Add the following line to `.bidsignore`:\ndandiset.yaml"
)

Impact

The HINT now appears in both dandi validate and dandi upload commands, providing consistent guidance to users regardless of which command they use.

Original prompt

This section details on the original issue you should resolve

<issue_title>Hint user to create .bidsignore listing dandiset.yaml</issue_title>
<issue_description>Inspired by

Conflicting logging levels for BIDS validation errors #1734

When processing validation errors from bids-validator, seeing the error about dandiset.yaml being not part of BIDS specification, follow it with a HINT level validation result which would suggest to add dandiset.yaml into .bidsignore file. Solution should be accompanied with the unit test so we maintain consistent operation happen generation of validation messages changes for some reason</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes Hint user to create .bidsignore listing dandiset.yaml #1740

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

codecov · 2025-11-15T03:06:18Z

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 75.09%. Comparing base (3173812) to head (b2bf5a3).

Files with missing lines	Patch %	Lines
dandi/bids_validator_deno/_validator.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1741   +/-   ##
=======================================
  Coverage   75.08%   75.09%           
=======================================
  Files          84       84           
  Lines       11910    11920   +10     
=======================================
+ Hits         8943     8951    +8     
- Misses       2967     2969    +2

Flag	Coverage Δ
unittests	`75.09% <92.85%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

When BIDS validator encounters dandiset.yaml (not part of BIDS spec), now: - Keeps the existing ERROR about the file not matching BIDS patterns - Adds a new HINT level validation result suggesting to create/update .bidsignore Includes test to verify both ERROR and HINT are generated Co-authored-by: yarikoptic <39889+yarikoptic@users.noreply.github.com>

candleindark · 2025-12-11T21:21:47Z

@yarikoptic I don't get the point of this or #1740. If we don't want the problem with dandiset.yaml to show up as an error to the user, you can just modify the error originated from deno BIDS validator into a hint (we have already modified its message). Why add this additionally hint, which is a duplication and makes the structure of the code a bit awkward? If the reason for this duplication is that you want to preserve some information about the origin of the validation result, this is still unnecessary because we have the origin attribute in a ValidationResult object.

yarikoptic · 2025-12-11T22:00:52Z

We don't want to "cheat" bids validator, so if people run it directly they get the same result. AI we want to instruct for to mitigate that error consistently

asmacdo · 2025-12-12T19:39:13Z

Tested against dandiset 001442 (chosen because it is a smallish BIDS dataset, though lots of validation noise).

The PR works as described - the [DANDI.BIDSIGNORE_DANDISET_YAML] HINT now appears after the [BIDS.NOT_INCLUDED] error:

[BIDS.NOT_INCLUDED] .../dandiset.yaml — The dandiset metadata file...
[DANDI.BIDSIGNORE_DANDISET_YAML] ... — Consider creating or updating a `.bidsignore`...

IMO this is fine, the HINT probably will be easier to understand for unfamiliar users, even though the duplication and extra noise are somewhat undesirable.

trimmed raw output: master branch

2025-12-12 12:57:53,797 [ WARNING] No BIDS reference root provided.
2025-12-12 12:57:53,797 [    INFO] No schema path specified, defaulting to the bundled schema, `/home/austin/devel/dandi-cli/venvs/dev3/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2025-12-12 12:57:53,889 [    INFO] SUCCESS: All files are BIDS valid and no BIDS-required files are missing.
2025-12-12 12:57:53,889 [    INFO] No schema path specified, defaulting to the bundled schema, `/home/austin/devel/dandi-cli/venvs/dev3/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
[BIDS.NOT_INCLUDED] /home/austin/devel/dandi-cli/001442/dandiset.yaml — The dandiset metadata file, `dandiset.yaml`, is not a part of BIDS specification. Please include a `.bidsignore` file with specification to ignore the metadata file in your dataset. For more details, see https://github.com/bids-standard/bids-specification/issues/131#issuecomment-461060166.
[BIDS.JSON_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/dataset_description.json — A JSON file is missing a key listed as recommended.
subCode: GeneratedBy
issueMessage: Field description: Used to specify provenance of the dataset.

[BIDS.JSON_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/dataset_description.json — A JSON file is missing a key listed as recommended.
subCode: SourceDatasets
issueMessage: Field description: Used to specify the locations and relevant attributes of all source datasets.
Valid keys in each object include `"URL"`, `"DOI"` (see
[URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)), and
`"Version"` with
[string](https://www.w3schools.com/js/js_json_datatypes.asp)
values.

[BIDS.SIDECAR_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/sub-rat1/micr/sub-rat1_sample-data1_SEM.png — A data file's JSON sidecar is missing a key listed as recommended.
subCode: Manufacturer
issueMessage: Field description: Manufacturer of the equipment that produced the measurements.

[BIDS.SIDECAR_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/sub-rat1/micr/sub-rat1_sample-data1_SEM.png — A data file's JSON sidecar is missing a key listed as recommended.
subCode: ManufacturersModelName
issueMessage: Field description: Manufacturer's model name of the equipment that produced the measurements.

[... 600+ more BIDS.SIDECAR_KEY_RECOMMENDED warnings ...]

2025-12-12 12:57:54,386 [    INFO] Logs saved in /home/austin/.local/state/dandi-cli/log/2025.12.12-18.57.53Z-217494.log

trimmed output: PR branch

2025-12-12 12:58:27,284 [ WARNING] No BIDS reference root provided.
2025-12-12 12:58:27,284 [    INFO] No schema path specified, defaulting to the bundled schema, `/home/austin/devel/dandi-cli/venvs/dev3/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
2025-12-12 12:58:27,375 [    INFO] SUCCESS: All files are BIDS valid and no BIDS-required files are missing.
2025-12-12 12:58:27,375 [    INFO] No schema path specified, defaulting to the bundled schema, `/home/austin/devel/dandi-cli/venvs/dev3/lib/python3.13/site-packages/bidsschematools/data/schema.json`.
[BIDS.NOT_INCLUDED] /home/austin/devel/dandi-cli/001442/dandiset.yaml — The dandiset metadata file, `dandiset.yaml`, is not a part of BIDS specification. Please include a `.bidsignore` file with specification to ignore the metadata file in your dataset. For more details, see https://github.com/bids-standard/bids-specification/issues/131#issuecomment-461060166.
[DANDI.BIDSIGNORE_DANDISET_YAML] /home/austin/devel/dandi-cli/001442 — Consider creating or updating a `.bidsignore` file in the root of your BIDS dataset to ignore `dandiset.yaml`. Add the following line to `.bidsignore`:
dandiset.yaml
[BIDS.JSON_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/dataset_description.json — A JSON file is missing a key listed as recommended.
subCode: GeneratedBy
issueMessage: Field description: Used to specify provenance of the dataset.

[BIDS.JSON_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/dataset_description.json — A JSON file is missing a key listed as recommended.
subCode: SourceDatasets
issueMessage: Field description: Used to specify the locations and relevant attributes of all source datasets.
Valid keys in each object include `"URL"`, `"DOI"` (see
[URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)), and
`"Version"` with
[string](https://www.w3schools.com/js/js_json_datatypes.asp)
values.

[BIDS.SIDECAR_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/sub-rat1/micr/sub-rat1_sample-data1_SEM.png — A data file's JSON sidecar is missing a key listed as recommended.
subCode: Manufacturer
issueMessage: Field description: Manufacturer of the equipment that produced the measurements.

[BIDS.SIDECAR_KEY_RECOMMENDED] /home/austin/devel/dandi-cli/001442/sub-rat1/micr/sub-rat1_sample-data1_SEM.png — A data file's JSON sidecar is missing a key listed as recommended.
subCode: ManufacturersModelName
issueMessage: Field description: Manufacturer's model name of the equipment that produced the measurements.

[... 600+ more BIDS.SIDECAR_KEY_RECOMMENDED warnings ...]

2025-12-12 12:58:27,857 [    INFO] Logs saved in /home/austin/.local/state/dandi-cli/log/2025.12.12-18.58.26Z-217682.log

Diff between master and branch

5a6,7
> [DANDI.BIDSIGNORE_DANDISET_YAML] /home/austin/devel/dandi-cli/001442 — Consider creating or updating a `.bidsignore` file in the root of your BIDS dataset to ignore `dandiset.yaml`. Add the following line to `.bidsignore`:
> dandiset.yaml

candleindark

Proposed some changes.

Additionally, if we want this custom validation hint to show up in the dandi upload command as well, we need to move this logic to an earlier point the in the calling process. The current setup only affect dandi validate not dandi upload where the issue was originally reported at #1734 (comment).

dandi/validate.py

…per Isaac review

yarikoptic · 2026-01-07T17:21:27Z

per discussion with @candleindark the issuing of HINT should be done deeper inside where we validate BIDS datatasets so we effect also upload command, not just CLI validate. @claude Move that analysis and issuing of HINT validation into code where BIDS dataset is validated and result records are yielded.

claude · 2026-01-07T17:25:22Z

Claude encountered an error —— View job

I'll analyze this and get back to you.

yarikoptic · 2026-01-08T00:36:21Z

@copilot also please address now failing tests

- Moved dandiset.yaml HINT logic from validate() to _harmonize() in bids_validator_deno - This ensures HINT appears for both 'dandi validate' and 'dandi upload' commands - Removed modification of original BIDS error message (preserving original error) - Updated test to not expect modified error message - All 101 validation tests passing Co-authored-by: yarikoptic <39889+yarikoptic@users.noreply.github.com>

Copilot · 2026-01-08T00:43:17Z

@copilot also please address now failing tests

Fixed failing tests (commit bbe7870). The test was expecting the modified error message, but per review feedback we now preserve the original BIDS error. Updated test to not check for the modified message. All 101 validation tests now passing.

per discussion with @candleindark the issuing of HINT should be done deeper inside where we validate BIDS datatasets so we effect also upload command, not just CLI validate. @claude Move that anal...

Moved HINT generation logic to _harmonize() in dandi/bids_validator_deno/_validator.py (commit bbe7870). The HINT now appears for both dandi validate and dandi upload commands since it's generated at the BIDS validator level rather than in the CLI validate function.

Copilot AI assigned Copilot and yarikoptic Nov 12, 2025

Copilot started work on behalf of yarikoptic November 12, 2025 20:53 View session

Copilot AI changed the title ~~[WIP] Add hint for user to create .bidsignore for dandiset.yaml~~ Add HINT validation result suggesting .bidsignore for dandiset.yaml Nov 12, 2025

Copilot AI requested a review from yarikoptic November 12, 2025 21:08

Copilot finished work on behalf of yarikoptic November 12, 2025 21:08

yarikoptic added the patch Increment the patch version when merged label Dec 4, 2025

yarikoptic requested review from asmacdo and candleindark December 11, 2025 12:10

yarikoptic marked this pull request as ready for review December 11, 2025 12:10

yarikoptic force-pushed the copilot/add-hint-for-dandiset-yaml branch from 5d5dd81 to 957da04 Compare December 11, 2025 12:10

asmacdo approved these changes Dec 12, 2025

View reviewed changes

candleindark requested changes Dec 12, 2025

View reviewed changes

dandi/validate.py Outdated Show resolved Hide resolved

dandi/validate.py Outdated Show resolved Hide resolved

RF: do not adjust existing message, add a HINT one and adjust record …

6faecd2

…per Isaac review

Copilot started work on behalf of yarikoptic January 8, 2026 00:36 View session

Copilot AI changed the title ~~Add HINT validation result suggesting .bidsignore for dandiset.yaml~~ Add HINT validation result for dandiset.yaml in both validate and upload commands Jan 8, 2026

Copilot finished work on behalf of yarikoptic January 8, 2026 00:44

Merge branch 'master' into copilot/add-hint-for-dandiset-yaml

b2bf5a3

yarikoptic requested review from asmacdo and candleindark January 8, 2026 14:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add HINT validation result for dandiset.yaml in both validate and upload commands #1741

Add HINT validation result for dandiset.yaml in both validate and upload commands #1741

Copilot AI commented Nov 12, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 15, 2025 •

edited

Loading

Uh oh!

candleindark commented Dec 11, 2025

Uh oh!

yarikoptic commented Dec 11, 2025

Uh oh!

asmacdo commented Dec 12, 2025

Uh oh!

candleindark left a comment

Uh oh!

Uh oh!

Uh oh!

yarikoptic commented Jan 7, 2026

Uh oh!

claude bot commented Jan 7, 2026 •

edited

Loading

Uh oh!

yarikoptic commented Jan 8, 2026

Uh oh!

Copilot AI commented Jan 8, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add HINT validation result for dandiset.yaml in both validate and upload commands #1741

Are you sure you want to change the base?

Add HINT validation result for dandiset.yaml in both validate and upload commands #1741

Conversation

Copilot AI commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example Output

Impact

Comments on the Issue (you are @copilot in this section)

Uh oh!

codecov bot commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

candleindark commented Dec 11, 2025

Uh oh!

yarikoptic commented Dec 11, 2025

Uh oh!

asmacdo commented Dec 12, 2025

Uh oh!

candleindark left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

yarikoptic commented Jan 7, 2026

Uh oh!

claude bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yarikoptic commented Jan 8, 2026

Uh oh!

Copilot AI commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Nov 12, 2025 •

edited

Loading

codecov bot commented Nov 15, 2025 •

edited

Loading

claude bot commented Jan 7, 2026 •

edited

Loading

Copilot AI commented Jan 8, 2026 •

edited

Loading