Skip to navigation

Runs and verdicts

A run passed only when it ended, executed to the end and its analysis found no failure.

A run executes scenarios against your app, in a cloud browser for a web app or on a cloud phone for an Android app. iOS is not available. Starting a run does not wait for it: start_run answers at once with a run id, a cursor and a link to watch it.

States

A run is admitted, then executes, ends and is analysed. watch_run follows it by cursor and reports each step, card and state change.

Being admitted is not passing. The run command and start_run only say that a run was accepted.

Verdicts

VerdictWhen
passedThe run ended, it executed to the end, its analysis is done and the analysis found a pass.
failedThe run ended and the analysis found a failure in your app.
incompleteThe run has no usable result: it was cancelled, it never ran, or the analysis did not finish. It is unverified, never a pass.

When a failure comes from TaloTrace itself, the verdict says so, and the run is not counted against your app.

Anything but passed means the change is not verified. A coding agent reports “unverified” and names the problem code and stops, instead of claiming success.

Gate on a verdict

talotrace check starts one run, waits for the run and its analysis under one deadline and exits with the verdict as its code: 0 only for a pass. See exit codes.

Next

Read a run, look at its evidence, or prove a fix.