Make slow pytest tests opt-in by default#1345
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request configures pytest to skip tests marked as 'slow' by default, introducing a new --run-slow command-line option to include them. It updates the documentation, helper scripts, and configuration files accordingly, and adds unit tests to verify the behavior. One issue was identified in conftest.py where a potential TypeError could occur if the -m option returns None, and a code suggestion was provided to handle this safely.
| markexpr = config.getoption("-m", default="") | ||
| if config.getoption("--run-slow") or _SLOW_MARKER_RE.search(markexpr): |
There was a problem hiding this comment.
If config.getoption("-m") returns None (which can happen in certain environments or when pytest is run programmatically), calling _SLOW_MARKER_RE.search(markexpr) will raise a TypeError: expected string or bytes-like object. We should ensure markexpr is a truthy string before performing the regex search.
| markexpr = config.getoption("-m", default="") | |
| if config.getoption("--run-slow") or _SLOW_MARKER_RE.search(markexpr): | |
| markexpr = config.getoption("-m", default="") | |
| if config.getoption("--run-slow") or (markexpr and _SLOW_MARKER_RE.search(markexpr)): |
ba85175 to
bb99609
Compare
Skip tests marked slow by default. Add --run-slow and keep explicit slow marker expressions working, while nightly and coverage checks still include slow tests.
bb99609 to
f085dac
Compare
Fixes #1327.
This makes tests marked
slowopt-in by default for normal pytest runs, while preserving explicit ways to include them:--run-slowincludes slow tests.-m slowand other explicit marker expressions mentioningsloware left to pytest.-m "not slow"; nightly pytest and incremental coverage still pass--run-slow.The new option lives in the repository-level
conftest.py, so pytest knows about it before collection and before wrapper scripts pass--run-slow.Local validation:
python -m pytest src/openfermion/testing/pytest_config_test.py -q-> 4 passedpython -m pytest src/openfermion/ops/representations/doci_hamiltonian_test.py -q-> 12 passed, 1 deselectedpython -m pytest src/openfermion/ops/representations/doci_hamiltonian_test.py -q --run-slow-> 13 passedpython -m black --check conftest.py src/openfermion/testing/pytest_config_test.pypython -m pylint conftest.py src/openfermion/testing/pytest_config_test.py.github/workflows/ci.yamland.github/workflows/nightly-pytest.yamlwith PyYAMLgit diff --check origin/main..HEAD