The docs describe exactly your problem: https://docs.pytest.org/en/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option. Copying from there:
Here is a conftest.py file adding a --runslow command line option to control skipping of pytest.mark.slow marked tests:
# content of conftest.py import pytest def pytest_addoption(parser): parser.addoption( "--runslow", action="store_true", default=False, help="run slow tests" ) def pytest_collection_modifyitems(config, items): if config.getoption("--runslow"): # --runslow given in cli: do not skip slow tests return skip_slow = pytest.mark.skip(reason="need --runslow option to run") for item in items: if "slow" in item.keywords: item.add_marker(skip_slow)
We can now write a test module like this:
# content of test_module.py import pytest def test_func_fast(): pass @pytest.mark.slow def test_func_slow(): pass
Matt Messers.. 9
There's a couple ways to handle this, but I'll go over two common approaches I've seen in Python baselines.
1) Separate your tests by putting the "optional" tests in another directory.
Not sure what your project layout looks like, but you can do something like this (only the test directory is important, the rest is just a toy example layout):
README.md setup.py requirements.txt test/ unit/ test_something.py test_something_else.py integration/ test_optional.py application/ __init__.py some_module.py
Then, when you invoke pytest, you invoke it by doing pytest test/unit
if you want to run just the unit tests (i.e. only test_something*.py
files), or pytest test/integration
if you want to run just the integration tests (i.e. only test_optional.py
), or pytest test
if you want to run all the tests. So, by default, you can just run pytest test/unit
.
I recommend wrapping these calls in some sort of script. I prefer make
since it is powerful for this type of wrapping. Then you can say make test
and it just runs your default (fast) test suite, or make test_all
, and it'll run all the tests (which may or may not be slow).
Example Makefile you could wrap with:
.PHONY: all clean install test test_int test_all uninstall all: install clean: rm -rf build rm -rf dist rm -rf *.egg-info install: python setup.py install test: install pytest -v -s test/unit test_int: install pytest -v -s test/integration test_all: install pytest -v -s test uninstall: pip uninstall app_name
2) Mark your tests judiciously with the @pytest.mark.skipif
decorator, but use an environment variable as the trigger
I don't like this solution as much, it feels a bit haphazard to me (it's hard to tell which set of tests are being run on any give pytest
run). However, what you can do is define an environment variable and then rope that environment variable into the module to detect if you want to run all your tests. Environment variables are shell dependent, but I'll pretend you have a bash environment since that's a popular shell.
You could do export TEST_LEVEL="unit"
for just fast unit tests (so this would be your default), or export TEST_LEVEL="all"
for all your tests. Then in your test files, you can do what you were originally trying to do like this:
import os ... @pytest.mark.skipif(os.environ["TEST_LEVEL"] == "unit") def test_scrape_website(): ...
注意:将测试级别命名为“单位”和“集成”无关。您可以随意命名。您还可以有许多级别(例如每晚测试或性能测试)。
另外,我认为选项1是最好的方法,因为它不仅清楚地允许测试的分离,而且还可以为测试的含义和表示增加语义和清晰度。但是软件中没有“一刀切”的解决方案,您必须根据自己的具体情况来决定采用哪种方法。
HTH!
The docs describe exactly your problem: https://docs.pytest.org/en/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option. Copying from there:
Here is a conftest.py file adding a --runslow command line option to control skipping of pytest.mark.slow marked tests:
# content of conftest.py import pytest def pytest_addoption(parser): parser.addoption( "--runslow", action="store_true", default=False, help="run slow tests" ) def pytest_collection_modifyitems(config, items): if config.getoption("--runslow"): # --runslow given in cli: do not skip slow tests return skip_slow = pytest.mark.skip(reason="need --runslow option to run") for item in items: if "slow" in item.keywords: item.add_marker(skip_slow)
We can now write a test module like this:
# content of test_module.py import pytest def test_func_fast(): pass @pytest.mark.slow def test_func_slow(): pass
There's a couple ways to handle this, but I'll go over two common approaches I've seen in Python baselines.
1) Separate your tests by putting the "optional" tests in another directory.
Not sure what your project layout looks like, but you can do something like this (only the test directory is important, the rest is just a toy example layout):
README.md setup.py requirements.txt test/ unit/ test_something.py test_something_else.py integration/ test_optional.py application/ __init__.py some_module.py
Then, when you invoke pytest, you invoke it by doing pytest test/unit
if you want to run just the unit tests (i.e. only test_something*.py
files), or pytest test/integration
if you want to run just the integration tests (i.e. only test_optional.py
), or pytest test
if you want to run all the tests. So, by default, you can just run pytest test/unit
.
I recommend wrapping these calls in some sort of script. I prefer make
since it is powerful for this type of wrapping. Then you can say make test
and it just runs your default (fast) test suite, or make test_all
, and it'll run all the tests (which may or may not be slow).
Example Makefile you could wrap with:
.PHONY: all clean install test test_int test_all uninstall all: install clean: rm -rf build rm -rf dist rm -rf *.egg-info install: python setup.py install test: install pytest -v -s test/unit test_int: install pytest -v -s test/integration test_all: install pytest -v -s test uninstall: pip uninstall app_name
2) Mark your tests judiciously with the @pytest.mark.skipif
decorator, but use an environment variable as the trigger
I don't like this solution as much, it feels a bit haphazard to me (it's hard to tell which set of tests are being run on any give pytest
run). However, what you can do is define an environment variable and then rope that environment variable into the module to detect if you want to run all your tests. Environment variables are shell dependent, but I'll pretend you have a bash environment since that's a popular shell.
You could do export TEST_LEVEL="unit"
for just fast unit tests (so this would be your default), or export TEST_LEVEL="all"
for all your tests. Then in your test files, you can do what you were originally trying to do like this:
import os ... @pytest.mark.skipif(os.environ["TEST_LEVEL"] == "unit") def test_scrape_website(): ...
注意:将测试级别命名为“单位”和“集成”无关。您可以随意命名。您还可以有许多级别(例如每晚测试或性能测试)。
另外,我认为选项1是最好的方法,因为它不仅清楚地允许测试的分离,而且还可以为测试的含义和表示增加语义和清晰度。但是软件中没有“一刀切”的解决方案,您必须根据自己的具体情况来决定采用哪种方法。
HTH!
一个非常简单的解决方案是使用-k
参数。您可以使用该-k
参数取消选择某些测试。-k
尝试将其参数与测试名称或标记的任何部分进行匹配not
您可以使用来反转匹配(您也可以使用布尔运算符and
和or
)。因此,-k 'not slow'
跳过名称中具有“ slow”,名称中具有“ slow”的标记或类/模块名称包含“ slow”的测试。
例如,给定此文件:
import pytest def test_true(): assert True @pytest.mark.slow def test_long(): assert False def test_slow(): assert False
运行时:
pytest -k 'not slow'
它输出类似:(请注意,两个失败的测试都与过滤器匹配时被跳过)
============================= test session starts ============================= platform win32 -- Python 3.5.1, pytest-3.4.0, py-1.5.2, pluggy-0.6.0 rootdir: c:\Users\User\Documents\python, inifile: collected 3 items test_thing.py . [100%] ============================= 2 tests deselected ============================== =================== 1 passed, 2 deselected in 0.02 secOnds====================
由于热切的匹配,您可能需要执行一些操作,例如将所有单元测试放在一个名为目录的目录中unittest
,然后将慢速单元标记为slow_unittest
(以便偶然匹配名称恰好慢的测试)。然后,您可以使用它-k 'unittest and not slow_unittest'
来匹配所有快速的单元测试。
pytest示例标记的更多用法