ActionBoxBlog

QADevOpsBackendPlatform

Flaky test retry approval in CI: quarantine failures safely

A copy-paste flaky test retry approval workflow for CI that limits retries, records the decision, and fails closed when quarantine is rejected.

CI turned green after three retries, and nobody knows the test failed once. Next week it fails four times, then five — and the suite is now a slot machine. The retry button is the symptom; the silent retry is the disease. A flaky test retry approval makes each retry a human decision on record: the failure, the log, the rerun — and why we're allowed to ship anyway.

The pattern: retries are requests, not reflexes

SituationWhat happensDecision needed
Test fails once, unknown flakeAsk a human: retry or investigateYes — with the log attached
Retry passes, flake confirmedRecord it, count it against a budgetNo — but the record is permanent
Retry fails againBlock the merge — real failureNo
Infra failure (timeout, network)Auto-retry onceNo — it's not a test failure
Flaky budget exceededBlock until a human investigatesYes

Step 1: fail loud, then ask

The first failure must not auto-retry. It stops the pipeline and creates a decision with the evidence:

yaml
name: test-gate
on:
  pull_request:

permissions:
  contents: read
  checks: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests once
        id: test
        run: npm test -- --reporter=json --outputFile=results.json
        continue-on-error: true

      - name: Parse failures
        id: parse
        if: steps.test.outcome == 'failure'
        run: |
          node -e "
            const r = require('./results.json');
            const failed = r.testResults
              .flatMap(s => s.assertionResults.filter(a => a.status === 'failed'));
            console.log(failed.map(f => f.fullName).join('\n'));
           require('fs').writeFileSync('failures.txt', failed.map(f => f.fullName).join('\n'));
           "
           echo "failed=$(wc -l < failures.txt)" >> "$GITHUB_OUTPUT"

       - name: Install Actionbox CLI
         if: steps.parse.outputs.failed > 0
         run: curl -fsSL https://actionbox.cloud/install.sh | sh

       - name: Ask before retry
         if: steps.parse.outputs.failed > 0
         id: approval
         env:
           ACTIONBOX_TOKEN: ${{ secrets.ACTIONBOX_TOKEN }}
         run: |
           result=$(actionbox ask "Retry ${{ steps.parse.outputs.failed }} failed test(s) as flaky?" \
             --option approve="Retry" \
             --option reject="Fail the job" \
             --context-json "[{\"type\":\"key_value\",\"items\":{\"failed_tests\":\"${{ steps.parse.outputs.failed }}\",\"suite\":\"unit\",\"attempt\":\"first failure\",\"log\":\"${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\"}}]" \
             --wait --timeout 15m --json)
           echo "decision=$(jq -r '.decision // empty' <<< "$result")" >> "$GITHUB_OUTPUT"

      - name: Rerun on approval
        if: steps.approval.outputs.decision == 'approve'
        run: npm test

The approver sees the count and the exact log link — and their "retry" is on the record, so a green build is never a guess about what passed.

Step 2: count flakes against a budget

A retry here and there is fine. A retry on every PR is a broken test hiding in the suite. Track them mechanically:

Keep the retry count in your CI or test-results store and compare it with the budget before creating the next Action. The current Actionbox CLI does not provide an actionbox list command; use the documented Actions API or History view when you need to reconcile the audit record.

The budget makes flakiness visible as a trend instead of a vibe — and gives the test owner a number to be responsible for.

Step 3: skips are decisions too

"Skip this test, we'll fix it later" is a policy decision, and it deserves a signature — not a test.skip in a drive-by commit:

yaml
      - name: Skip requires an on-record approval
        if: contains(github.event.pull_request.labels.*.name, 'flake-skip-approved')
        run: |
          echo "Skip approved with the linked decision on the PR."
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

A skipped test with an attached decision is a tracked debt. A skipped test with no decision is a hole in your coverage that nobody owns.

Full example: quarantine flow with a reopen date

bash
#!/usr/bin/env bash
# quarantine.sh — move a flaky test to quarantine with a decision + expiry
set -euo pipefail

TEST_NAME="${1:?test name}"

actionbox ask \
  "Quarantine ${TEST_NAME} and reopen in 14 days?" \
  --option approve="Quarantine (auto-reopens 14d)" \
  --option deny="Keep it in the suite" \
  --context-json "[{\"type\":\"key_value\",\"items\":{\"test\":\"${TEST_NAME}\",\"failure_rate\":\"4/10 runs\",\"owner\":\"team-ci\",\"reopen\":\"2026-08-30\"}}]" \
  --callback-url https://ops.acme.com/actionbox/quarantine-callback

Quarantine with an expiry is honest debt: the test comes back with a date, and the decision that moved it is findable forever.

Why this beats "add a retry plugin"

  • Green builds mean something — every retry that produced the green is named
  • Flakiness is measurable — a budget turns "the suite feels flaky" into a number
  • Skips have owners — no test disappears without a recorded decision
  • Real failures still block — a second failure is a merge stop, no questions asked

Try it

  1. Install the CLI on your runner
  2. Create a Source → ACTIONBOX_TOKEN
  3. Replace your retry plugin with the ask pattern

Create a free Source · GitHub Actions reference · Deploy approval gates in GitHub Actions · Dependency PR approvals without rubber-stamping

Try this workflow in minutes

Create a free Source, then run the exact commands from this post against the live API — no approval infrastructure to build.

S
Suson Sapkota

Founder, Actionbox