Try to use pytest-xdist for test suite

[pat-schmitt]

While trying to reduce the runtime of the test suite on GitHub Actions, I came across pytest-xdist. The idea is to distribute the tests across all available CPUs. On GitHub Actions you get two CPUs, meaning there is potential to halve the test run times.

However, for this to work, we need to use locking correctly for all files involved in the tests, which is not currently the case. This PR is meant to investigate whether we are able to make use of this.

@TimoRoth do you know anything about this, or do you have an idea how we can make this work?

• Tests added/passed
• Fully documented
• Entry in whats-new.rst

[TimoRoth]

The runners got 4 CPUs even.
I tried to use pytest-xdist in the past, but it never worked reliably.
Tests fail in random, non-deterministic ways.
A major overhaul of all tests would need to happen first, making them all fully independent from one another.
I believe right now the precise order of tests is necessary, since artifacts prior ones create are consumed by later ones.

[fmaussion]

I believe right now the precise order of tests is necessary, since artifacts prior ones create are consumed by later ones.

I don't think this is true (or rather if it is, we need to fix it), but many tests rely on a small sample of data created either in pytest fixtures or test classes init. It's too bad though, this would be a massive help...

[gampnico]

What about using filelock?

Here is also an example for writing to disk.

[TimoRoth]

Things created during init should not be a problem for xdist, since it really should be running init before running any tests.
So the source for the failures has got to be something else.