Writing and Running STS Tests

This guide explains how to write test cases using the STS framework and run them with tmt and Testing Farm.

Writing a Test Case

STS test cases follow pytest conventions with additional storage-specific fixtures and utilities. Here's a complete example (see also tests/iscsi/parameters/ for a real test):

1. Create the Test File

Create a Python test file (e.g., example-test.py) in your test directory, using sts-libs fixtures and utilities:

# Copyright: Contributors to the sts project
# SPDX-License-Identifier: GPL-3.0-or-later

from __future__ import annotations

import logging
from typing import TYPE_CHECKING

import pytest

from sts.fio.fio import FIO
from sts.utils.cmdline import run

if TYPE_CHECKING:
    from sts.loop import LoopDevice

logger = logging.getLogger(__name__)


@pytest.mark.parametrize('loop_devices', [1], indirect=True)
def test_example(loop_devices: list[LoopDevice]) -> None:
    """Test example using sts-libs features.

    This test:
    1. Uses loop_devices fixture to get a test device
    2. Verifies device exists using run()
    3. Runs I/O test using FIO
    """
    device = str(loop_devices[0].path)
    logger.info(f'Starting example test with device {device}')

    # Verify device exists
    result = run(f'lsblk {device}')
    assert result.succeeded, 'lsblk command failed'
    assert device.rsplit('/', maxsplit=1)[-1] in result.stdout

    # Run I/O test
    fio = FIO(filename=device)
    assert fio.run()

Here is another example — an LVM test that creates a logical volume on loop-backed storage and verifies basic operations:

from __future__ import annotations

from typing import TYPE_CHECKING

import pytest

from sts.lvm.logical_volume import LogicalVolume
from sts.utils.files import mkfs

if TYPE_CHECKING:
    from sts.lvm.volume_group import VolumeGroup


@pytest.mark.parametrize('loop_devices', [1], indirect=True)
class TestLvmBasic:
    """Basic LVM operations on loop-backed storage."""

    def test_create_and_format_lv(self, setup_loopdev_vg: VolumeGroup) -> None:
        """Create an LV, format it, and verify the report."""
        lv = LogicalVolume(name='testlv', vg=setup_loopdev_vg.name)
        lv.create(size='100M').assert_ok()
        lv.refresh_report()

        assert lv.report is not None
        assert lv.report.lv_name == 'testlv'

        assert mkfs(lv.path, 'ext4')

        lv.remove().assert_ok()

2. Create the FMF Metadata File

Create a main.fmf file alongside the test to let tmt discover it:

summary: My storage test example
description: |
  Detailed description of what the test validates.
  Include test setup, test steps, expected result and cleanup.
framework: shell
test: pytest ./example-test.py
component:
  - kernel \ Storage \ Device Mapper
tag+:
  - local
tier: 1
duration: 10m
check:
  - how: avc
    result: xfail
  - how: dmesg
  - how: coredump

FMF tag conventions:

• Use component with Bugzilla-style names (e.g. kernel \ Storage \ Device Mapper).
• Common tags are inherited via parent main.fmf files — use tag+: to add tags instead of tag: which replaces.
• Tests included in CI are selected by the plans/sts-ci/ci.fmf plan filters (tier and tag exclusions), not by a special tag.

3. Create a tmt Test Plan

Create a plan file (e.g., example-plan.fmf) under plans/ that selects which tests to run and how to provision the test environment:

summary: My storage test plan
discover:
  - how: fmf
    filter:
      - tag:local
provision:
  how: virtual
  image: fedora
execute:
  how: tmt

Alternative Plan Configurations

For container testing:

provision:
  how: container
  image: centos:stream10

For specific hardware requirements (Beaker):

provision:
  how: beaker
  image: RHEL-9%
  hardware:
    disk:
      - size: '>= 10 GB'
    memory: '>= 4 GB'

Skipping and Gating Tests

Use sts.marks helpers to skip or xfail tests based on distribution, version, or installed packages. These keep skip conditions consistent across the test suite.

In fixtures (imperative)

from sts.marks import requires_rhel


def my_fixture():
    requires_rhel(9)  # skips if on RHEL < 9
    ...

As decorators or pytestmark (declarative)

from sts.marks import is_rhel_below, requires_package
from sts.utils.version import VersionInfo

# Skip an entire module on old RHEL
pytestmark = pytest.mark.skipif(is_rhel_below(9), reason='requires RHEL 9+')


# Skip if a package is too old
@requires_package('stratisd', VersionInfo(3, 7, 0))
def test_stratis_feature(): ...

See the sts.marks API reference for all available helpers.

Selecting Tests with FMF Markers

The sts.utils.fmf_markers pytest plugin reads main.fmf metadata and applies tags and tiers as pytest markers. This lets you select tests with -m the same way tmt plan filters do:

pytest tests/ -m ci              # same tests as the sts-ci plan
pytest tests/ -m tier1           # all tier-1 tests
pytest tests/ -m "dm and tier1"  # tier-1 device-mapper tests
pytest tests/ -m "not hw_fc"    # exclude FC-hardware tests

The plugin requires the fmf library and silently does nothing if it's not installed.

Running Tests Locally

Using tmt (Recommended)

# Run a specific test plan
TMT_SHOW_TRACEBACK=full tmt run plan --name plans/iscsi/tier0 -dddvvv

# Login on failure for debugging
tmt run login --when fail --when error plans --name /plans/iscsi/tier0

Full documentation: tmt.readthedocs.io | Matrix: #tmt:fedora.im

Using Pytest Directly

SSH into the SUT, then:

dnf install -y python3-pip
pip install sts-libs

pytest example-test.py -v
pytest -o log_cli_level='debug' -vrA example-test.py

Running Tests with Testing Farm

# Basic submission
testing-farm request \
  --context distro=rhel-9 \
  --compose RHEL-9.7.0-Nightly \
  --git-url https://gitlab.com/rh-kernel-stqe/sts \
  --plan /plans/iscsi/tier0 \
  --arch x86_64,aarch64

# From your branch
testing-farm request \
  --compose Fedora-Rawhide \
  --git-url https://gitlab.com/your-username/sts \
  --git-ref your-feature-branch \
  --plan /plans/your-test-plan

Common Fixture Patterns

from __future__ import annotations

from typing import TYPE_CHECKING

import pytest

if TYPE_CHECKING:
    from sts.loop import LoopDevice
    from sts.lvm.volume_group import VolumeGroup


# Loop devices — parametrize count via indirect
@pytest.mark.parametrize('loop_devices', [2], indirect=True)
def test_with_loop_devices(loop_devices: list[LoopDevice]) -> None:
    device1, device2 = loop_devices
    # device1.path, device1.name, etc.


# Loop devices with custom size
@pytest.mark.parametrize('loop_devices', [{'count': 1, 'size_mb': 4096}], indirect=True)
def test_with_large_loop(loop_devices: list[LoopDevice]) -> None: ...


# LVM — setup_loopdev_vg creates a VG from loop devices
@pytest.mark.parametrize('loop_devices', [1], indirect=True)
def test_with_lvm(setup_loopdev_vg: VolumeGroup) -> None:
    vg = setup_loopdev_vg
    # vg.name, vg.create_lv(), etc.

Good Practices

• Descriptive test names: test functions should clearly indicate what they test
• Use fixtures: leverage sts-libs fixtures for storage setup/teardown
• Set realistic durations in main.fmf — tests that time out are hard to debug
• Start local: test with tmt run before submitting to Testing-Farm
• Use verbose mode: TMT_SHOW_TRACEBACK=full and -dddvvv for debugging

Troubleshooting

# Lint FMF metadata
tmt lint

# Verbose run with traceback
TMT_SHOW_TRACEBACK=full tmt run ... -dddvvv

# Dry run — show what would be executed
tmt run --dry

# Show discovered tests / plans
tmt tests ls
tmt plans ls

# Clean workdirs, guests and images
tmt clean