Skip to navigation
Runs and results

Prove a fix

Rerun once and compare.

After a failed run you change the app. The proof is one rerun of the same scenario, and a comparison with the first run.

The rule

The verify skill allows one rerun after a fix. If that run still fails, your agent stops, reports the cause, the run ids and one next action, and waits for you. It does not loop.

Rerun and compare

  1. Run the scenario again with a new request_id.
  2. Call compare_runs with the earlier run and the new one:
compare_runs(project_id, baseline_run_id=<earlier run>, candidate_run_id=<new run>)

It shows what flipped: scenarios that failed and now pass, passed and now fail, or are unchanged. A scenario that now passes and has no other failure is fixed.

  1. Report both run ids with the verdicts. Do not say fixed on a run that is incomplete.

From CI, talotrace check gives the same answer as an exit code. See exit codes. See MCP tools for the arguments of compare_runs.