Focusing on pytest

Published on April 30, 2020

Ruby’s Rspec gem has a nice feature called filter-run-when-matching:

RSpec.configure do |c|
  c.filter_run_when_matching :focus
end

If you add this to your .rspec configuration file, it will run tests tagged focus. If none exist, it’ll run the entire suite. This is great when working in a larger codebase. If you know a refactor is going to temporarily break the world, but you have a single test that you can iterate on to get it done, then the rest of the failures are just time-consuming noise. Tag that test with focus, and Rspec will suddenly switch to only running it. Combine this with a continual test runner, and it’s pure joy.

I missed this when I switched over to Python and pytest. pytest doesn’t have this built-in, but pytest is also much more configurable than Rspec. It turned out to be pretty easy to add this to my project:

# tests/conftest.py

def pytest_configure(config):
    """
    Register the `focus` marker, so we don't get warnings.
    """

    config.addinivalue_line("markers", "focus: Only run this test.")


def pytest_collection_modifyitems(items, config):
    """
    Focus on tests marked focus, if any.  Run all otherwise.
    """

    selected_items = []
    deselected_items = []

    focused = False
    for item in items:
        if item.get_closest_marker("focus"):
            focused = True
            selected_items.append(item)
        else:
            deselected_items.append(item)

    if focused:
        print("\nOnly running @pytest.mark.focus tests")
        config.hook.pytest_deselected(items=deselected_items)
        items[:] = selected_items

pytest allows you to create custom plugins for this sort of functionality. But it also allows you to register those same plugin hooks in your conftest.py files. The above code in your tests/conftest.py file will do the trick. pytest_configure() registers a new marker (focus). pytest_collection_modifyitems() allows you to change the list of tests after they’ve been “collected”. In our implementation, we filter to the focused tests if we detect that any of them have the focus marker.

If you combine this with a continual test runner (such as pytest-watch), then you can quickly switch between running the full test and just the test you care about by adding or removing a single @puytest.mark.focus annotation. Just one more bit of friction out of the way.

You can see this code in a Cloud Foundry service broker we’re developing for one of our clients, cloud.gov.