ActionBoxBlog

ML engineerData sciencePlatform

ML model promotion approval workflow: staging to production safely

A copy-paste ML model promotion approval workflow that gates staging-to-production releases with evaluation metrics and safe rollback behavior.

Suson Sapkota 3 min read Release management

ML model promotion is the rare deploy where "the pipeline is green" means almost nothing. Your eval suite passed on staging data; the question is whether the model is good for real traffic. That judgment call is exactly what a human gate is for — automated gates should check the metrics, and a human should own the promotion.

ML model promotion approval workflow

A promotion pipeline that:

  1. Trains and registers model v2 in the registry (automated)
  2. Runs eval against staging, collects offline + canary metrics (automated)
  3. Creates a promotion decision with the metrics attached (the gate)
  4. Promotes to production only on explicit human approval
  5. Auto-rolls back if the canary breaches guardrails

Step 1: the promotion gate

python
import os
from datetime import datetime, timedelta, timezone
from actionbox import Actionbox

box = Actionbox(api_key=os.environ["ACTIONBOX_TOKEN"])
decision = box.ask(
    title="Promote model v2 to production?",
    description="Evaluation passed; canary stable for 72h. Sign-off required before full rollout.",
    context=[{
        "type": "key_value",
        "items": {
            "model": "checkout_reranker",
            "version": "v2 (2026-08-09)",
            "offline_ndcg": "+4.2% vs v1",
            "canary_latency_p95": "38ms (target <50ms)",
            "canary_error_rate": "0.02%",
            "cost_delta": "+$31/day",
            "rolled_back_v1": "untouched",
        },
    }}],
    interaction={"type": "single_choice", "label": "Promotion decision", "options": [
        {"id": "promote", "label": "Promote to production", "style": "primary"},
        {"id": "extend_canary", "label": "Extend canary 24h"},
        {"id": "reject", "label": "Reject — keep v1", "style": "destructive"},
    ]},
    expires_at=(datetime.now(timezone.utc) + timedelta(hours=1)).isoformat(),
    on_expire={"type": "resolve", "response": {"type": "single_choice", "value": "extend_canary"}, "reason": "no response, extended canary"},
)

The approver sees the metrics next to the buttons — no dashboards to cross-reference.

Step 2: branch the pipeline on the decision

python
if decision == "promote":
    registry.promote(model_version="v2", env="production")
elif decision == "extend_canary":
    canary.extend(hours=24)
else:
    notify("#ml-team", "v2 rejected; staying on v1")
    raise SystemExit(1)

A rejection is a decision with a reason, recorded for audit — it's not a silent skip.

Step 3: the canary keeps its own guardrails

The human gate covers promotion; the machine covers execution. After promotion, let the canary auto-rollback on guardrail breach — the same approval API can run the rollback decision automatically via its callback:

bash
# Guardrail breach -> auto-rollback (pre-approved by runbook policy)
EXPIRES_AT=$(date -u -d '+15 minutes' '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || date -u -v+15M '+%Y-%m-%dT%H:%M:%SZ')
actionbox send "Canary error rate 3.1% — auto-rollback to v1" \
  --option rollback="Roll back" \
  --expires "$EXPIRES_AT" \
  --on-expire-json '{"type":"resolve","response":{"type":"single_choice","value":"rollback"},"reason":"auto-rollback policy"}' \
  --callback-url https://ml.acme.com/actionbox/rollback

Fail-closed: if the callback can't be reached, the rollback doesn't happen and the incident page fires.

Where the gate fits your stack

StackGate pointNotes
MLflow / Sagemaker registrystage → prod transitionregister + ask, then promote
Argo / KServe rolloutcanary → fullapprove before ramping to 100%
LLM prompt/registrynew prompt versionattach eval deltas (see below)
Batch re-training cronnightly retraingate on data-quality metrics

LLM-specific: prompt changes need gates too

Prompts regress like code, but silently. Gate a prompt version the same way:

python
decision = box.ask(
    title="Ship prompt v14 to production?",
    context=[{"type": "key_value", "items": {
        "task": "support-triage",
        "offline_win_rate": "0.62 (v13: 0.58)",
        "refusal_rate": "1.1% (target <2%)",
        "hallucination_hallmark": "none in eval",
    }}],
    interaction={"type": "boolean", "label": "Ship prompt v14?"},
)

Try it

  1. pip install actionbox in your promotion job
  2. Create a Source → ACTIONBOX_TOKEN
  3. Replace your "manual promotion ticket" with the ask gate above

Create a free Source · Python SDK reference · Approval gates for data pipelines

#mlops#model-promotion#approval-gates#llm

Try this workflow in minutes

Create a free Source, then run the exact commands from this post against the live API — no approval infrastructure to build.

Create a free SourceRead the docs
S
Suson Sapkota

Founder, Actionbox

Related workflows