@pytest.mark.slow and --runslow option

[honno]

• Skips any test with the pytest.mark.slow marker by default like in NumPy
• Introduces --runslow option to run these slow tests
• Uses --runslow on our test CI

[honno]

n.b. currently no test in our suite uses pytest.mark.slow (save a commented-out one in test_scalarmath.py), but I played with this manually and it worked fine.

[ev-br]

Thanks! This is a very welcome change!

Locally with a recent main branch, I see several uses of @slow, will double-check a bit later.

torch_np/tests/numpy_tests/core/test_multiarray.py:1015
torch_np/tests/numpy_tests/core/test_multiarray.py:1015
torch_np/tests/numpy_tests/core/test_multiarray.py:1015
torch_np/tests/numpy_tests/core/test_multiarray.py:1015
torch_np/tests/numpy_tests/core/test_multiarray.py:1015
torch_np/tests/numpy_tests/core/test_multiarray.py:1015
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:1015: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

torch_np/tests/numpy_tests/core/test_multiarray.py:1296
torch_np/tests/numpy_tests/core/test_multiarray.py:1296
torch_np/tests/numpy_tests/core/test_multiarray.py:1296
torch_np/tests/numpy_tests/core/test_multiarray.py:1296
torch_np/tests/numpy_tests/core/test_multiarray.py:1296
torch_np/tests/numpy_tests/core/test_multiarray.py:1296
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:1296: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

torch_np/tests/numpy_tests/core/test_multiarray.py:4649
torch_np/tests/numpy_tests/core/test_multiarray.py:4649
torch_np/tests/numpy_tests/core/test_multiarray.py:4649
torch_np/tests/numpy_tests/core/test_multiarray.py:4649
torch_np/tests/numpy_tests/core/test_multiarray.py:4649
torch_np/tests/numpy_tests/core/test_multiarray.py:4649
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:4649: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow  # takes > 1 minute on mechanical hard drive

torch_np/tests/numpy_tests/core/test_multiarray.py:5705
torch_np/tests/numpy_tests/core/test_multiarray.py:5705
torch_np/tests/numpy_tests/core/test_multiarray.py:5705
torch_np/tests/numpy_tests/core/test_multiarray.py:5705
torch_np/tests/numpy_tests/core/test_multiarray.py:5705
torch_np/tests/numpy_tests/core/test_multiarray.py:5705
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:5705: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

torch_np/tests/numpy_tests/core/test_multiarray.py:6919
torch_np/tests/numpy_tests/core/test_multiarray.py:6919
torch_np/tests/numpy_tests/core/test_multiarray.py:6919
torch_np/tests/numpy_tests/core/test_multiarray.py:6919
torch_np/tests/numpy_tests/core/test_multiarray.py:6919
torch_np/tests/numpy_tests/core/test_multiarray.py:6919
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:6919: PytestUnknownMarkWarning: Unknown pytest.mark.valgrind_error - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.valgrind_error(reason="leaks buffer info cache temporarily.")

torch_np/tests/numpy_tests/core/test_multiarray.py:6944
torch_np/tests/numpy_tests/core/test_multiarray.py:6944
torch_np/tests/numpy_tests/core/test_multiarray.py:6944
torch_np/tests/numpy_tests/core/test_multiarray.py:6944
torch_np/tests/numpy_tests/core/test_multiarray.py:6944
torch_np/tests/numpy_tests/core/test_multiarray.py:6944
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:6944: PytestUnknownMarkWarning: Unknown pytest.mark.valgrind_error - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.valgrind_error(reason="leaks buffer info cache temporarily.")

torch_np/tests/numpy_tests/core/test_multiarray.py:6978
torch_np/tests/numpy_tests/core/test_multiarray.py:6978
torch_np/tests/numpy_tests/core/test_multiarray.py:6978
torch_np/tests/numpy_tests/core/test_multiarray.py:6978
torch_np/tests/numpy_tests/core/test_multiarray.py:6978
torch_np/tests/numpy_tests/core/test_multiarray.py:6978
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:6978: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

torch_np/tests/numpy_tests/core/test_multiarray.py:7857
torch_np/tests/numpy_tests/core/test_multiarray.py:7857
torch_np/tests/numpy_tests/core/test_multiarray.py:7857
torch_np/tests/numpy_tests/core/test_multiarray.py:7857
torch_np/tests/numpy_tests/core/test_multiarray.py:7857
torch_np/tests/numpy_tests/core/test_multiarray.py:7857
  /home/br/sweethome/proj/scipy/torch_np_compat/torch_np/tests/numpy_tests/core/test_multiarray.py:7857: PytestUnknownMarkWarning: Unknown pytest.mark.leaks_references - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.leaks_references(

[honno]

Locally with a recent main branch, I see several uses of @slow, will double-check a bit later.

Ah I see, screwed up the grep when I was searching is all 😅

[ev-br]

'later' turned out to be 'now'. Checked it locally, and it does cut the noise! Test suite is still dominated by error processing from xfailed tests, but this is a great change regardless. Thank @honno