Skip to content

Skip INTL0003 PascalCase validation for test methods#374

Merged
BenjaminMichaelis merged 10 commits intomainfrom
copilot/fix-intl0003-test-method
Mar 10, 2026
Merged

Skip INTL0003 PascalCase validation for test methods#374
BenjaminMichaelis merged 10 commits intomainfrom
copilot/fix-intl0003-test-method

Conversation

Copy link
Contributor

Copilot AI commented Feb 3, 2026

INTL0003 was flagging test methods with underscores (e.g., Method_Scenario_ExpectedResult) as PascalCase violations. Test methods commonly use underscores for readability and should be exempt.

Changes:

  • Added IsTestMethod() check in NamingMethodPascal analyzer to skip methods decorated with test attributes
  • Uses namespace-based detection to automatically recognize test attributes from xUnit, NUnit, MSTest, and TUnit frameworks
  • Any attribute from the Xunit, NUnit.Framework, Microsoft.VisualStudio.TestTools.UnitTesting, or TUnit.Core namespaces is automatically recognized
  • Includes fallback to common test attribute name checking for test environments where namespace metadata may be incomplete
  • Added test coverage for MSTest, xUnit, and NUnit frameworks
  • Updated docs to document the exception

Example:

[TestClass]
public class FooTests
{
    [TestMethod]
    public void FooThing_IsFooThing_HasFooThing() // No longer triggers INTL0003
    {
        Assert.IsTrue(true);
    }
    
    public void myNonTestMethod() // Still triggers INTL0003
    {
    }
}

This namespace-based approach provides better maintainability and automatically supports new test attributes added to these frameworks without requiring code changes to the analyzer.

Original prompt

This section details on the original issue you should resolve

<issue_title>Test Method triggers INTL0003</issue_title>
<issue_description>A Test method like "FooThing_IsFooThing_HasFooThing" Triggers INTL0003 in a test project when I don't think it should</issue_description>

<agent_instructions>FIRST write a test that fails to prove the bug. THEN write a fix, to fix the bug and make sure the test passes.</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Fix test method triggering INTL0003 error Skip INTL0003 PascalCase validation for test methods Feb 3, 2026
@BenjaminMichaelis
Copy link
Member

@copilot maybe we should loosen it up a bit more. Maybe we ignore underscores in any method that is decorated by an attribute in the XUnit, NUnit, MSTest, or TUnit namespaces.

Copy link
Contributor Author

Copilot AI commented Feb 3, 2026

@copilot maybe we should loosen it up a bit more. Maybe we ignore underscores in any method that is decorated by an attribute in the XUnit, NUnit, MSTest, or TUnit namespaces.

Updated the implementation to check attribute namespaces instead of specific attribute names. The analyzer now automatically recognizes any test attributes from Xunit, NUnit.Framework, Microsoft.VisualStudio.TestTools.UnitTesting, and TUnit.Core namespaces. This provides better coverage and maintainability.

Implemented in commit 99ba578 with a fallback to attribute name checking for test environments where namespace metadata may be incomplete.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • pe3vsblobprodcus354.vsblob.vsassets.io
    • Triggering command: /usr/share/dotnet/dotnet dotnet restore --no-dependencies /home/REDACTED/work/CodingGuidelines/CodingGuidelines/Tools/XMLtoMD/GuidelineXmlToMD.sln --packages /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/packages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal /p:TargetFrameworkRootPath=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:NetCoreTargetingPackRoot=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:AllowMissingPrunePackageData=true (dns block)
    • Triggering command: /usr/share/dotnet/dotnet dotnet restore --no-dependencies /home/REDACTED/work/CodingGuidelines/CodingGuidelines/Tools/XMLtoMD/GuidelineXmlToMD/GuidelineXmlToMD.csproj --packages /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/packages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal /p:TargetFrameworkRootPath=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:NetCoreTargetingPackRoot=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:AllowMissingPrunePackageData=true (dns block)
    • Triggering command: /usr/share/dotnet/dotnet dotnet restore --no-dependencies /home/REDACTED/work/CodingGuidelines/CodingGuidelines/Tools/XMLtoMD/GuidelineXmlToMD.Test/GuidelineXmlToMD.Test.csproj --packages /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/packages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal /p:TargetFrameworkRootPath=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:NetCoreTargetingPackRoot=/home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/emptyFakeDotnetRoot /p:AllowMissingPrunePackageData=true (dns block)
  • s8mvsblobprodcus38.vsblob.vsassets.io
    • Triggering command: /usr/share/dotnet/dotnet dotnet restore --no-dependencies /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/152969D1C94CACFB2DD8DDFC3D3B91A8/missingpackages_workingdir --packages /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/missingpackages /p:DisableImplicitNuGetFallbackFolder=true --verbosity normal --configfile /home/REDACTED/work/CodingGuidelines/.codeql-scratch/dbs/csharp/working/nugetconfig/nuget.config --force (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@BenjaminMichaelis
Copy link
Member

@copilot I don't see the case where we would need Fallback attribute names for test environments where namespace metadata may be incomplete

Copy link
Contributor Author

Copilot AI commented Feb 3, 2026

@copilot I don't see the case where we would need Fallback attribute names for test environments where namespace metadata may be incomplete

You're right that in production code the namespace metadata should always be available. However, the fallback is needed for our test infrastructure - the DiagnosticVerifier test helper doesn't add references to test framework assemblies (MSTest, xUnit, NUnit), so when our tests use [TestMethod] or [Fact] attributes, the ContainingNamespace is null.

We have two options:

  1. Keep the fallback (simple, works now, doesn't hurt production usage)
  2. Update DiagnosticVerifier.Helper.cs to add MetadataReferences for test frameworks (more complex, might have assembly loading issues)

Which approach would you prefer?

@BenjaminMichaelis BenjaminMichaelis force-pushed the copilot/fix-intl0003-test-method branch from 35c324e to 732c8d0 Compare March 10, 2026 20:25
@BenjaminMichaelis BenjaminMichaelis marked this pull request as ready for review March 10, 2026 20:30
Copilot AI review requested due to automatic review settings March 10, 2026 20:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the INTL0003 “Methods PascalCase” analyzer to exempt test methods (which often use underscores for readability) and adds supporting test infrastructure + documentation.

Changes:

  • Skip INTL0003 reporting for methods identified as test methods via test-framework attributes (xUnit/NUnit/MSTest/TUnit).
  • Add/extend analyzer tests to cover underscore-named test methods across multiple frameworks.
  • Add test framework package references and a helper to inject framework assemblies into Roslyn test compilations; document the exemption.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
docs/analyzers/00XX.Naming.md Documents the new exemption for underscore-named test methods.
IntelliTect.Analyzer/IntelliTect.Analyzer/Analyzers/NamingMethodPascal.cs Adds IsTestMethod() detection to skip INTL0003 for test methods.
IntelliTect.Analyzer/IntelliTect.Analyzer.Test/NamingMethodPascalTests.cs Adds parameterized tests validating no diagnostics for test methods with underscores and diagnostics for non-test cases.
IntelliTect.Analyzer/IntelliTect.Analyzer.Test/IntelliTect.Analyzer.Tests.csproj Adds reference-only framework packages to support metadata references in analyzer tests.
IntelliTect.Analyzer/IntelliTect.Analyzer.Test/Helpers/TestFrameworkReferences.cs Introduces a registry of test framework metadata references + test data.
IntelliTect.Analyzer/IntelliTect.Analyzer.Test/Helpers/DiagnosticVerifier.Helper.cs Injects the framework metadata references into Roslyn test projects.
Directory.Packages.props Centralizes versions for added test framework packages.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

BenjaminMichaelis and others added 9 commits March 10, 2026 14:33
…nderscores

Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
…upport

Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
…ption

- Add xunit.v3.core, NUnit, TUnit.Core as compile+runtime references in test project
  (PrivateAssets=all so no runner conflicts; IncludeAssets=compile;runtime for DLL availability)
- Create TestFrameworkReferences registry: single source of truth for all supported
  test frameworks (name, attribute syntax, using directive, MetadataReference).
  Adding a new framework only requires one entry here + the namespace in NamingMethodPascal.
- Inject all framework references into every DiagnosticVerifier.Helper CreateProject
  compilation, so namespace-based detection is exercised by real tests (not just fallback)
- Collapse 4 redundant framework-specific test methods into one [DynamicData] test
  driven by TestFrameworkReferences.All; new frameworks auto-covered at zero cost
- Add negative test: non-test method with underscores in a test class still warns
- Add user-defined [Test] attribute test: custom namespace attribute still warns
- Fix IsTestMethod fallback: gate name-based matching on TypeKind.Error only,
  preventing false negatives for user-defined attributes named Test/Fact/Theory/etc.
- Make testFrameworkNamespaces and commonTestAttributeNames static readonly fields
  to avoid per-invocation heap allocation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@BenjaminMichaelis BenjaminMichaelis force-pushed the copilot/fix-intl0003-test-method branch from 732c8d0 to 267e87f Compare March 10, 2026 21:34
…thod

- Fix StartsWith bug: 'Xunit'.StartsWith('Xunit') would also match 'Xunit2Extensions'
  etc. Now uses exact match or dotted prefix check to prevent false positive suppression.
- Remove s_commonTestAttributeNames fallback: now that DiagnosticVerifier adds real
  framework assembly references to every test compilation, TypeKind.Error can no longer
  occur in tests. The fallback was also risky in production — any user-defined [Test]
  or [Fact] attribute in an unresolved assembly would incorrectly suppress INTL0003.
- Rename s_testFrameworkNamespaces -> _TestFrameworkNamespaces to comply with INTL0001.
@BenjaminMichaelis BenjaminMichaelis merged commit d4e7320 into main Mar 10, 2026
3 checks passed
@BenjaminMichaelis BenjaminMichaelis deleted the copilot/fix-intl0003-test-method branch March 10, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test Method triggers INTL0003

3 participants