Skip to content

YAML: Extract comments#21935

Merged
mbg merged 16 commits into
github:mainfrom
MathiasVP:add-yaml-comments
Jun 17, 2026
Merged

YAML: Extract comments#21935
mbg merged 16 commits into
github:mainfrom
MathiasVP:add-yaml-comments

Conversation

@MathiasVP

@MathiasVP MathiasVP commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

This PR adds support for extraction of YAML comments to all the languages which makes use of the YAML extractor.

To keep this PR simple I've decided to not include the new YamlComment class into the existing Comment classes for each language. I suggest we do that as a per-language follow-up if necessary.

(All the force-pushes is my fumbling with up- and downgrade scripts. The DCA runs should still be representative of the changes)

@MathiasVP MathiasVP added depends on internal PR This PR should only be merged in sync with an internal Semmle PR javascript Pull requests that update Javascript code Actions Analysis of GitHub Actions labels Jun 3, 2026
@github-actions github-actions Bot added the JS label Jun 3, 2026
@MathiasVP MathiasVP force-pushed the add-yaml-comments branch 2 times, most recently from 08b6b0f to 8049dec Compare June 4, 2026 11:54
@github-actions github-actions Bot added Python Ruby QL-for-QL Rust Pull requests that update Rust code labels Jun 4, 2026
@MathiasVP MathiasVP changed the title JS: Extract YAML comments (and use them in Actions) YAML: Extract comments Jun 4, 2026
@MathiasVP MathiasVP force-pushed the add-yaml-comments branch 2 times, most recently from c89f174 to 3a88dd3 Compare June 4, 2026 13:46
@MathiasVP MathiasVP force-pushed the add-yaml-comments branch 2 times, most recently from 0ec56f6 to b44f0dc Compare June 4, 2026 15:53
@MathiasVP MathiasVP marked this pull request as ready for review June 4, 2026 16:46
Copilot AI review requested due to automatic review settings June 4, 2026 16:46
@MathiasVP MathiasVP requested review from a team as code owners June 4, 2026 16:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the shared YAML extraction model to include YAML comments, adding a new yaml_comments relation/type to the relevant dbschemes and exposing a YamlComment class through the shared YAML QL library and per-language YAML adapters.

Changes:

  • Add yaml_comments (and @yaml_comment) to YAML dbscheme fragments and include comments in @yaml_locatable.
  • Extend the shared YAML library (codeql.yaml.Yaml) with a comment base type and a YamlComment class.
  • Add a JavaScript library test exercising YamlComment.getText().
Show a summary per file
File Description
unified/ql/lib/unified.dbscheme Adds yaml_comments and @yaml_comment to the unified YAML schema fragment.
shared/yaml/codeql/yaml/Yaml.qll Introduces comment support in the shared YAML QL API (CommentBase, YamlComment).
rust/ql/lib/upgrades/77e9a70be4b0cf5ecb1d4c1d841b2d970715a912/upgrade.properties Declares a Rust db upgrade for YAML comment extraction.
rust/ql/lib/rust.dbscheme Adds yaml_comments and extends @yaml_locatable for Rust DBs.
rust/downgrades/e1bce498ef78280ebe0a32b1d9d6f26c96eaf41f/upgrade.properties Declares Rust downgrade behavior for yaml_comments.
ruby/ql/lib/upgrades/29b7b6fc1982422368cb0a4644fd0c81f993c618/upgrade.properties Declares a Ruby db upgrade for YAML comment extraction.
ruby/ql/lib/ruby.dbscheme Adds yaml_comments and extends @yaml_locatable for Ruby DBs.
ruby/downgrades/d6f4c73dc33d28aebcffd53ba080eeecc99470f5/upgrade.properties Declares Ruby downgrade behavior for yaml_comments.
ql/ql/src/ql.dbscheme Adds yaml_comments and extends @yaml_locatable for QL extractor DBs.
ql/ql/src/codeql_ql/ast/Yaml.qll Wires YAML comment entities into the QL AST YAML adapter.
ql/ql/src/codeql_ql/ast/Ast.qll Wires YAML comment entities into the broader QL AST YAML module.
python/ql/lib/upgrades/eb5fc917c79bb23ce2de4a022f3e566d57a91be9/upgrade.properties Declares a Python db upgrade for YAML comment extraction.
python/ql/lib/semmlecode.python.dbscheme.stats Updates Python dbscheme stats to include @yaml_comment/yaml_comments.
python/ql/lib/semmlecode.python.dbscheme Adds yaml_comments and extends @yaml_locatable for Python DBs.
python/ql/lib/semmle/python/Yaml.qll Wires YAML comment entities into the Python YAML adapter.
python/downgrades/b7745eb2df865c97e50b7803956a82988716e29a/upgrade.properties Declares Python downgrade behavior for yaml_comments.
python/downgrades/b7745eb2df865c97e50b7803956a82988716e29a/semmlecode.python.dbscheme Provides the downgraded Python dbscheme variant without yaml_comments.
javascript/ql/test/library-tests/Comments/YamlComments.ql Adds a JS library test query selecting YamlComment text.
javascript/ql/test/library-tests/Comments/YamlComments.expected Expected results for the new JS YAML comment test.
javascript/ql/test/library-tests/Comments/comments.yml YAML fixture used by the JS comment test.
javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/upgrade.properties Declares a JavaScript db upgrade for YAML comment extraction.
javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/semmlecode.javascript.dbscheme Upgrade dbscheme snapshot including yaml_comments for JavaScript.
javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/old.dbscheme Prior JavaScript dbscheme snapshot for the upgrade.
javascript/ql/lib/semmlecode.javascript.dbscheme.stats Updates JS dbscheme stats to include @yaml_comment/yaml_comments.
javascript/ql/lib/semmlecode.javascript.dbscheme Adds yaml_comments and extends @yaml_locatable for JS DBs.
javascript/ql/lib/semmle/javascript/YAML.qll Wires YAML comment entities into the JS YAML adapter.
javascript/extractor/tests/yaml/output/trap/orig.yml.trap Updates extractor trap output to include extracted YAML comments.
javascript/extractor/tests/yaml/output/trap/emoji_buffer_boundary.yml.trap Updates extractor trap output to include extracted YAML comments (boundary case).
javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/upgrade.properties Declares JavaScript downgrade behavior for yaml_comments.
javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/semmlecode.javascript.dbscheme Provides the downgraded JS dbscheme variant.
javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/old.dbscheme Current JS dbscheme snapshot used as the downgrade source.
actions/ql/lib/codeql/actions/ast/internal/Yaml.qll Wires YAML comment entities into the Actions YAML adapter.
actions/ql/lib/codeql/actions/ast/internal/Ast.qll Exposes YAML comments as Comment in the Actions AST layer.

Copilot's findings

  • Files reviewed: 40/44 changed files
  • Comments generated: 1

Comment thread javascript/ql/test/library-tests/Comments/comments.yml
@MathiasVP MathiasVP force-pushed the add-yaml-comments branch from b44f0dc to 44c8a97 Compare June 4, 2026 16:56
@asgerf asgerf added the no-change-note-required This PR does not need a change note label Jun 8, 2026

@hvitved hvitved left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruby / Rust 👍

@tausbn tausbn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue, which can be addressed in a follow-up PR. Otherwise this looks good to me.

Comment on lines +283 to +286
yaml_comments (unique int id: @yaml_comment,
string text: string ref,
string tostring: string ref);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is autogenerated, so you need to update dbscheme.template as well.

However, given the circumstances, I think this could just be done in a follow-up PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. Thanks a lot! I will do that in a follow-up PR

@mbg mbg merged commit 1cb5be5 into github:main Jun 17, 2026
133 of 145 checks passed
MathiasVP added a commit that referenced this pull request Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Actions Analysis of GitHub Actions depends on internal PR This PR should only be merged in sync with an internal Semmle PR javascript Pull requests that update Javascript code JS no-change-note-required This PR does not need a change note Python QL-for-QL Ruby Rust Pull requests that update Rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants