Add abdiff utility#999
Merged
Merged
Conversation
testsuite/abdiff.py runs the same benign transfer with two rsync binaries (A = build under test, B = a baseline) and compares the OUTCOME -- exit code, stderr, --stats "Literal data", the destination tree (content + full metadata), the --itemize list, and (with --cost) peak process-group RSS. For benign input the two must be indistinguishable; any divergence is a regression candidate. It is a developer tool, NOT a runtests.py test (does not end in _test.py). Capabilities: - Scenario sweeps over options / path shapes / file types / sizes / modes / selection / placement / wire / transports, plus domain-knowledge pairwise + combo sweeps and a stochastic fuzzer/role matrix. - Transport lanes: local, ssh split (lsh.sh), stdio-pipe daemon, a REAL TCP daemon (bound port + greeting/handshake/auth challenge-response), and the restricted rrsync wrapper (support/rrsh.sh; each binary paired with its own version's rrsync via --rrsync-a/--rrsync-b, since rrsync ships in the script). - Stability gate: each binary is run N times and escalated on a candidate diff; nondeterministic scenarios are quarantined FLAKY, never reported as regressions. - Parallel (-j, default 20) with a per-run findings log; --loop runs until --timelimit (or Ctrl-C), feeding the pool a half-random / half-systematic stream of new combinations. As root an "all" run also folds in the root-only sweeps (priv, daemonchroot). - General coverage levers: a cost oracle (--cost, peak RSS over the whole process group), transport lifted as an orthogonal axis, a resume/redo sweep, and type-transition / nanosecond-mtime / scale (--scale N) fixtures. Documented in testsuite/README.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the utility I am using to automatically find regressions in new releases. It does a huge search over possible rsync options and find differences in behavior between two releases. It has already found several regressions in the upcoming 3.5.0 security release (all of which are now fixed)