Skip to content

fix(ipc): close the session as JOB_SHUTDOWN when the job shuts down - #8039

Merged
chenghao-mou merged 1 commit into
livekit:mainfrom
regalmoix:fix/job-shutdown-close-reason
Nov 9, 2026
Merged

chenghao-mou merged 1 commit into
livekit:mainfrom
regalmoix:fix/job-shutdown-close-reason

Conversation

@regalmoix

Copy link
Copy Markdown
Contributor

When a job shuts down with the AgentSession still open, the session's close event reports CloseReason.USER_INITIATED. That covers worker drain or migration, SIGTERM, ctx.shutdown(), a room disconnect without a participant event, and an entrypoint crash. So the close event (including the one recorded in the session report) and lk.close_reason all blame the user for a shutdown the job started.

_shutdown_job closes the primary session through the public aclose() (added in #3541; #5875 only bounded it with a timeout), and aclose() always uses USER_INITIATED. AgentSession.start() registers a job shutdown callback that closes with JOB_SHUTDOWN (#2398), but callbacks run later. By then the session is already closed, and only the first close counts.

Fix

_shutdown_job now closes with session._aclose_impl(reason=CloseReason.JOB_SHUTDOWN). That's the same call start() already registers, so it doesn't change the public API.

Testing

  • New test_thread_job_shutdown_closes_session_as_job_shutdown (thread executor, in-process): a fake session records the close reason. On main it fails with [USER_INITIATED] == [JOB_SHUTDOWN].
  • The _HangingSession fake follows the call change, and its test still passes.
  • tests/test_ipc.py and uv run pytest --unit --audio_eot pass, ruff is clean, and make type-check reports no new errors.

Why no regression

  • The session closes at the same point, with the same timeout. Only the reason changes.
  • Sessions already closed keep their reason, because this close is then a no-op. Examples: PARTICIPANT_DISCONNECTED from room I/O, ERROR, and session.shutdown(), which stays USER_INITIATED.
  • No test or consumer in the repo branches on USER_INITIATED at job shutdown.
  • A ctx.shutdown() from user code is still a job shutdown, as the JOB_SHUTDOWN callback from automatically close agent session when participant disconnected #2398 already treats it. The lk.shutdown.user_initiated span attribute still records who started it.
  • A user subclass overriding aclose() is no longer called at this point. That matches the existing JOB_SHUTDOWN callback, which already goes through _aclose_impl.

_shutdown_job closed the primary AgentSession through the public
aclose(), which always uses USER_INITIATED, before the JOB_SHUTDOWN
shutdown callback AgentSession.start() registers could run. Drains,
migrations, ctx.shutdown(), room disconnects and entrypoint crashes all
reported USER_INITIATED. Close with JOB_SHUTDOWN instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@regalmoix
regalmoix requested a review from a team as a code owner October 21, 2026 18:37
@regalmoix

Copy link
Copy Markdown
Contributor Author

@longcw @davidzhao — you both own this file (#5875, #5741 and #7872, #7131 touch exactly this path). Mind taking a look when you get a moment? Independent of #8028 / #8030.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@chenghao-mou chenghao-mou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for the PR

@chenghao-mou
chenghao-mou merged commit 7fe59c3 into livekit:main Nov 9, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants