Our system tests currently run sequentially, in one GitHub Actions workflow and from one execution of the systemtests.py:
- Trigger GitHub Actions workflow
- Trigger
systemtests.py with a test suite
- Start Docker Compose services for each specific test combination
We could parallelize at any of these levels:
- Use a matrix in the GitHub Actions workflow and trigger different test suites in parallel.
- Advantage: potentially using more hosted runners and getting more refined output on GitHub
- Disadvantage: we can only parallelize on GitHub (which is generally what we need).
- Parallelize the
systemtests.py so that it calls Docker Compose in parallel.
- Advantage: Runs also locally
- Disadvantage: Will be messy, not portable to other runners.
@PranjalManhgaye what do you think?
Our system tests currently run sequentially, in one GitHub Actions workflow and from one execution of the
systemtests.py:systemtests.pywith a test suiteWe could parallelize at any of these levels:
systemtests.pyso that it calls Docker Compose in parallel.@PranjalManhgaye what do you think?