AIR CLI: implement air run --watch (stream logs until completion) - #6080
Merged
Conversation
After submitting a workload, --watch follows the run's logs to completion and exits with the run's outcome, reusing the same Bricklens-with-MLflow-fallback pipeline as `air logs`. - Text mode: prints "Submitted run", the dashboard link, and "Monitoring run and streaming logs...", then streams the logs. - JSON mode: emits a SUBMITTED event with the run id, a STATUS event on each lifecycle transition, the streamed LOG/ALERT events, and a closing terminal-status envelope (SUCCESS/FAILED/CANCELED) — matching the Python CLI's --watch JSONL contract. - Without --watch, the plain submit path now prints a tip about --watch. - STATUS events are watch-scoped (opt-in via logRequest.onStatusChange), so the merged `air logs` output is unchanged. --dry-run still takes precedence over --watch (nothing is submitted or streamed). Co-authored-by: Isaac
Collaborator
Integration test reportCommit: e74adb3
16 interesting tests: 8 FAIL, 4 KNOWN, 4 SKIP
|
maggiewang-db
left a comment
Contributor
There was a problem hiding this comment.
Can you do a manual test with the fallback mlflow path?
| // envelope after streaming. Mirrors the Python CLI's --watch JSONL contract. | ||
| out := cmd.OutOrStdout() | ||
| printSubmittedEvent(out, runIDStr, dashboardURL) | ||
| terminalStatus := "FAILED" |
Contributor
There was a problem hiding this comment.
Isaac claims this terminalStatus is never updated in the fallback mlflow code path. Can you double check?
Contributor
Author
There was a problem hiding this comment.
fixed, thanks for the catch!
The closing --watch JSON envelope derived its status from the onStatusChange callback, which only fires on the Bricklens path. On the MLflow fallback (or an active run that terminates without a Bricklens-observed transition) the status stayed at its "FAILED" default, mislabeling a successful run. Derive the terminal status from the run's actual state (re-resolved after streaming) instead, correct regardless of which backend served the logs. Add a regression test that forces the MLflow fallback and asserts the envelope reports SUCCESS. Co-authored-by: Isaac
maggiewang-db
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes & Why
After submitting a workload, --watch follows the run's logs to completion and exits with the run's outcome, reusing the same Bricklens-with-MLflow-fallback pipeline as
air logs.air logsoutput is unchanged.--dry-run still takes precedence over --watch (nothing is submitted or streamed).
Tests
Unit tests (experimental/air/cmd/)
Acceptance tests (acceptance/experimental/air/)
Manual verification:

Properly monitors and outputs logs from runs on manual test: