* Split CI tests to run them in parallel
The idea of this PR is to split tests into multiple sets that can be run
in parallel by GH. For this, all tests in test_models.py that would run
on GH get a pytest marker. The GH workflow matrix is factorized to run
only a single marker. That way, only a subset of tests should run per
worker, leading to quicker results.
There is also a worker that runs all the tests that are not inside
test_models.py.
* [skip ci] empty commit to abort ci
* Fix typo in marker name
* Split fx into forward and backward
* Comment out test coverage for now
Checking if it's responsible for regression in CI runtime.
* Remove pytest cov completely from requirements
* Remove cov call in pyproject.toml
Missed that one.
When tests finish, a report should be printed that shows the code
coverage of timm. This should give us a better idea where we should work
on test coverage.
I have tested this locally (on a subset of tests) and it worked.
Since the number of test dependencies was getting quite high, I created
a requirements-dev.txt and moved them there. GH action and
CONTRIBUTING.md are adjusted accordingly.
Furthermore, instead of extending the pytest invocation, I created a
pyproject.toml and added the coverage options there. For completeness, I
also added the black settings that come closest to the style of timm.
LMK if this is not desired.
For now, the coverage is only reported but not enforced. I.e. when a PR
is created that adds uncovered lines, CI will still succeed. We could
think about adding codecov or something like that, but it can be
annoying sometimes and the service was flaky for me in the past.
Ross Wightman
Benjamin Bossan
Benjamin Bossan