Skip to content

fix(environments): warn when heartbeat signals shutdown on first beat - #1786

Open
priyanka25aug wants to merge 1 commit into
anthropics:mainfrom
priyanka25aug:fix/heartbeat-first-beat-diagnostic
Open

fix(environments): warn when heartbeat signals shutdown on first beat#1786
priyanka25aug wants to merge 1 commit into
anthropics:mainfrom
priyanka25aug:fix/heartbeat-first-beat-diagnostic

Conversation

@priyanka25aug

Copy link
Copy Markdown

Problem

When using ant beta:worker poll --on-work spawn.sh, if spawn.sh exits immediately after launching a background container — e.g. kubectl apply or docker run -d — the poller releases the work-item lease before the spawned worker connects. The first heartbeat then returns state: stopping, _heartbeat_loop silently calls stop.set() and returns, and the session is permanently stuck at stop_reason: requires_action with no output to explain why.

This is the issue described in #1779: the failure is deterministic once you understand the cause, but the SDK gives the user nothing to go on — no warning, no pointer to the blocking requirement.

Fix

src/anthropic/lib/environments/_worker.py — the hand-written carve-out, not Stainless-generated code.

1. _heartbeat_loop: detect and warn on first-beat shutdown

Add a heartbeat_count counter. When state: stopping / lease_extended: false arrives on the very first successful heartbeat, emit log.warning with an explicit message:

  • names the root cause (spawn script exited before worker connected)
  • gives the concrete fix (docker run foreground vs -d, kubectl wait vs kubectl apply)
  • links to the self-hosted environments docs

All subsequent shutdowns continue to use log.info (normal end-of-session).

2. handle_item docstring: add a Note: section

Documents the blocking requirement inline where users reading the API are most likely to encounter it, complementing any docs-side updates.

Testing

The change is a logging-only diagnostic path; existing behaviour for normal session teardown (heartbeat_count > 1) is unchanged. The new log.warning branch fires only when heartbeat_count == 1 at shutdown, which is the exact condition that previously produced a silent hang.

Fixes #1779

@priyanka25aug
priyanka25aug requested a review from a team as a code owner July 28, 2026 21:49
@priyanka25aug

Copy link
Copy Markdown
Author

@craigie-ant @raghu-giatec — tagging you both since this directly addresses the issue you reported and diagnosed. @craigie-ant, given you've already traced the root cause on the issue, would appreciate your review when you have a moment — happy to adjust anything that doesn't fit the codebase conventions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant