Skip pytest test in Azure DevOps pipelines: a quick guide

Introduction

Running all tests in a Continuous Integration (CI) pipeline is crucial, but there are scenarios where it’s not ideal to execute every test. This might be due to lengthy test durations or dependencies on resources such as data, APIs, or systems that aren’t available within the CI pipeline. In such cases, pytest provides the skipif annotation to selectively exclude tests based on specific conditions.

@pytest.mark.skipif(..., reason="...")
def test_that_should_not_run_in_ci():

Method

To skip a test when running in an Azure DevOps pipeline, you can create an expression that evaluates to true within that environment. Utilizing environment variables is an effective approach. To identify available variables, add a step in your pipeline YAML file that prints all environment variables:

  - script: |
      printenv
    displayName: 'Print environment variables'

This step will output information similar to:

##[section]Starting: Print environment variables
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.229.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
printenv
========================== Starting Command Output ===========================
SHELL=/bin/bash
AGENT_NAME=Ubuntu0202
AGENT_VERSION=3.225.0
AGENT_HOMEDIRECTORY=/var/build/Ubuntu0202
[...]
##[section]Finishing: Print environment variables

Example

Use the environment variable information to construct an expression and skip the test accordingly. Additionally, it’s crucial to provide a clear reason for skipping the test. Here’s an example:

import os

@pytest.mark.skipif(os.getenv("AGENT_NAME"), reason="Do not run in Azure DevOps Pipeline, because the needed secure DB is not accessible")
def test_that_should_not_run_in_ci():

Conclusion

By employing the skipif annotation with an environment variable condition, you can control which tests run in your Azure DevOps pipeline. Remember to include a meaningful reason for better understanding and maintainability. This ensures a streamlined testing process in your CI/CD workflow.