Skip to content

About

GitHub Action that selects Python tests affected by source changes using static dependency analysis.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Use this GitHub action with your project
Add this Action to an existing workflow or create a new one
View on Marketplace

Repository files navigation

Smart Test Trigger

Smart Test Trigger is an intelligent GitHub Action that statically analyses your Python codebase to determine the precise set of tests that need to be re-run based on the changes in a pull request. By avoiding full test suite runs on every minor change, it dramatically reduces CI execution time and compute costs.

The Problem (Business Value)

In modern Python monorepos, running the entire test suite on every commit is slow and expensive. When a developer changes a single isolated module, 90% of the test suite execution might be unnecessary.

Smart Test Trigger solves this by:

  1. Extracting the git diff of your current branch against the base branch.
  2. Parsing your Python code's Abstract Syntax Tree (AST) to build a fast, localized dependency graph without executing any code.
  3. Traversing the graph (Breadth-First Search) to find only the test files that depend directly or transitively on the modified files.
  4. Automatically falling back to a full test run if critical infrastructure files (like pyproject.toml, requirements.txt, or Dockerfile) are touched.

Quick Start

You can integrate this Action directly into your existing GitHub workflow. The example below shows the full integration including baseline tracking and a sticky PR report comment:

name: CI

on:
  pull_request:
    branches: [ main ]
  push:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0 # IMPORTANT: Deep fetch is required for git diff to work

      - name: Restore baseline cache
        uses: actions/cache@v4
        with:
          path: .smart-trigger-cache
          key: smart-trigger-baseline-${{ runner.os }}
          restore-keys: smart-trigger-baseline-

      - name: Smart Test Trigger
        id: smart_trigger
        uses: Zaywoo66/Smart-test-trigger@v1.1.0
        with:
          base-branch: ${{ github.base_ref }}
          repo-path: .

      - name: Run Pytest (Selective)
        if: steps.smart_trigger.outputs.full_run == 'false' && steps.smart_trigger.outputs.test_files != ''
        run: pytest ${{ steps.smart_trigger.outputs.test_files }}

      - name: Run Pytest (Full Suite — timed)
        if: steps.smart_trigger.outputs.full_run == 'true'
        id: full_run_timer
        run: |
          START=$(date +%s)
          pytest tests/
          END=$(date +%s)
          echo "duration=$((END-START))" >> "$GITHUB_OUTPUT"

      - name: Update Baseline
        if: steps.smart_trigger.outputs.full_run == 'true'
        run: python -m smart_trigger.cli_update_baseline --duration ${{ steps.full_run_timer.outputs.duration }}

      - name: Post Test Trigger Report
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const marker = '<!-- smart-test-trigger-report -->';
            const timeSaved = '${{ steps.smart_trigger.outputs.time_saved_estimate }}';
            const timeSavedLine = timeSaved === 'n/a'
              ? 'Недостаточно данных (нужен первый полный прогон)'
              : `${timeSaved} сек`;

            const body = `${marker}
            ## 🎯 Smart Test Trigger Report
            | Метрика | Значение |
            |---|---|
            | Всего тест-файлов | ${{ steps.smart_trigger.outputs.total_test_count }} |
            | Выбрано для прогона | ${{ steps.smart_trigger.outputs.selected_test_count }} |
            | Пропущено | ${{ steps.smart_trigger.outputs.tests_skipped_pct }}% |
            | Оценка сэкономленного времени | ${timeSavedLine} |
            | Основано на исторических прогонах | ${{ steps.smart_trigger.outputs.baseline_sample_count }} |
            `;

            const { data: comments } = await github.rest.issues.listComments({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
            });
            const existing = comments.find(c => c.body.includes(marker));

            if (existing) {
              await github.rest.issues.updateComment({
                owner: context.repo.owner, repo: context.repo.repo,
                comment_id: existing.id, body,
              });
            } else {
              await github.rest.issues.createComment({
                owner: context.repo.owner, repo: context.repo.repo,
                issue_number: context.issue.number, body,
              });
            }

Inputs

Input Description Default Required
base-branch Base branch (or ref) to compare the current HEAD against. main No
head-ref Current branch tip to diff from. HEAD No
output-format Format of the emitted test file list (space, newline, json, github-output). github-output No
repo-path Filesystem path to the repository root inside the container. /github/workspace No

Outputs

Output Description
test_files Space-separated list of test file paths that must be executed. (Populated when output-format is github-output).
full_run 'true' when an infra file change was detected or tests cannot be isolated; 'false' in selective mode.
trigger_reason Human-readable explanation for why a full run was triggered (e.g., requirements.txt changed).
total_test_count Total number of test files discovered in the repository.
selected_test_count Number of test files selected for execution in this PR.
tests_skipped_pct Percentage of test files skipped relative to the total.
time_saved_estimate Estimated seconds saved compared to a full-run baseline, or 'n/a' if no baseline has been accumulated yet.
baseline_sample_count Number of historical full-run samples used to compute the time-saved estimate.

Baseline Duration Tracking

Smart Test Trigger can estimate how much time each selective run saves by maintaining a rolling average of full test suite durations.

How it works:

  1. On full runs, the workflow times the pytest execution and records the duration via python -m smart_trigger.cli_update_baseline --duration <seconds>.
  2. The baseline data is stored in .smart-trigger-cache/baseline.json and persisted between runs using actions/cache.
  3. On selective runs, the time_saved_estimate output shows the estimated seconds saved based on the proportion of tests skipped.

First run: Before any full run has been recorded, time_saved_estimate will be 'n/a'. The estimate becomes available after the first full test suite execution.

Runtime Requirements

Because Smart Test Trigger is implemented as a Docker Container Action, please note the following:

  • It must run on a Linux runner (e.g., ubuntu-latest). Docker container actions are not supported on Windows or macOS runners.
  • The action automatically bundles its own Python environment and dependencies, meaning it will not conflict with the Python environment of your project.
  • It requires git inside the container (already included) and a deep checkout (fetch-depth: 0) in your workflow to perform the diff.

Local Debugging via CLI

You can run the utility locally during development.

  1. Ensure you have the dependencies (if any) installed.
  2. Run main.py directly from the smart_trigger module.
# Run in the root of your repo
python -m smart_trigger.main --base-branch main --output-format space

You can view the resulting test files printed to your terminal. To run your tests locally, you can pipe the output:

pytest $(python -m smart_trigger.main --output-format space)

About

GitHub Action that selects Python tests affected by source changes using static dependency analysis.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages