Remove ExtraCoverage category — run all functional tests in CI#1997
Draft
tyrielv wants to merge 6 commits into
Draft
Remove ExtraCoverage category — run all functional tests in CI#1997tyrielv wants to merge 6 commits into
tyrielv wants to merge 6 commits into
Conversation
The ExtraCoverage category excluded ~110 functional test methods (19 test classes) from the default test run and CI pipeline. These tests cover critical functionality — mount/unmount edge cases, dehydrate, repair, shared cache, disk layout upgrades, junctions, and more — but were never run in CI validation, only via the --extra-only flag. Remove the ExtraCoverage filtering so all functional tests run in the default pass, which is what CI already uses (sliced across 10 parallel jobs with 60-minute timeouts). This eliminates the risk of these tests silently atrophying. Changes: - Remove [Category(Categories.ExtraCoverage)] from all 18 test classes and 1 test method - Remove ExtraCoverage constant from Categories.cs - Remove --extra-only flag handling from Program.cs - Delete SharedCacheUpgradeTests.cs (zero test methods — dead code) - Update AuthoringTests.md to reflect the new default behavior Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Address the ~33 test failures surfaced by removing ExtraCoverage: FastFetch tests (15): Download the FastFetch artifact into the functional test directory in CI so FastFetch.exe is available. ConfigVerbTests (2): Extend NUnitRunner's slice grouping regex to include MultiEnlistmentTests alongside EnlistmentPerFixture, so Order-dependent tests within a class stay in the same slice. UpgradeReminderTests (3): Delete — tests the old NuGet-based upgrade reminder system which has been removed from the service. Atrophied tests (28): Mark with NeedsReactionInCI — these tests have stale expectations due to behavioral changes in mount, dehydrate, repair, disk layout upgrades, and shared cache. They need updated expectations in follow-up PRs: - MountTests (11 methods): mount error handling changed - DehydrateTests (8 methods): folder dehydrate behavior changed - RepairTests (3 methods): GVFS now tolerates corrupt git index - WindowsDiskLayoutUpgradeTests (2 methods): expected paths drifted - SharedCacheTests (1 method): mount after repair behavior changed Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Introduce SkipInCIAttribute — a custom NUnit CategoryAttribute that accepts a reason string, making it clear why each test is skipped and what needs fixing. Replace all NeedsReactionInCI usages (both new atrophied tests and pre-existing flaky tests) with descriptive SkipInCI annotations. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
RepairTests: GVFS now tolerates corrupt git index on mount (rebuilds from projection), so the mount-fail precondition is stale. Update the three corrupt-index tests to verify that repair restores the index file without asserting mount failure beforehand. SharedCacheTests.RepairFixesCorruptBlobSizesDatabase: Same pattern — GVFS now tolerates corrupt blob sizes DB. Remove mount-fail assertion, keep repair verification. MountTests: Remove two obsolete tests: - ProjFS_CMDHangNoneActiveInstance: ProjFS regression test no longer relevant to GVFS mount behavior - MountingARepositoryThatRequiresPlaceholderUpdatesWorks: placeholder update flow moved out of mount startup Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Git changed checkout output casing ('Branch' -> 'branch') and message
format. Use case-insensitive substring match instead of exact string
comparison.
Assisted-by: Claude Opus 4.6
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
tyrielv
force-pushed
the
tyrielv/remove-extra-coverage
branch
from
e281d05 to
b7277c9
Compare
MountTests: Two fixes to prevent test cascade failures in CI: 1. MountShouldFail now captures stderr in addition to stdout. Mount error messages moved to stderr, so the expected-message assertion was failing, which prevented cleanup code from running and left the enlistment metadata corrupt for all subsequent tests. 2. Wrap metadata-corruption tests (MountFailsWhenNoOnDiskVersion, MountFailsWhenNoLocalCacheRootInRepoMetadata, MountFailsWhenNoGitObjectsRootInRepoMetadata) in try/finally to guarantee metadata restoration even if assertions fail. SharedCacheTests.MountReusesLocalCacheKeyWhenGitObjectsRootDeleted: Mark SkipInCI — mount hangs when git objects root is deleted, causing slice 9 to time out. Needs deeper investigation. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
ExtraCoveragecategory excluded ~110 functional test methods (19 test classes) from the default test run and CI pipeline. These tests cover critical functionality — mount/unmount edge cases, dehydrate, repair, shared cache, disk layout upgrades, junctions, and more — but were never run in CI validation, only via the--extra-onlyflag.This PR removes the
ExtraCoveragefiltering so all functional tests run in the default pass. CI already slices tests across 10 parallel jobs with 60-minute timeouts, so there should be headroom for the additional tests.Changes
[Category(Categories.ExtraCoverage)]from all 18 test classes and 1 test methodExtraCoverageconstant fromCategories.cs--extra-onlyflag handling fromProgram.csSharedCacheUpgradeTests.cs(zero test methods — dead code)AuthoringTests.mdto reflect the new default behaviorWhat to watch for