[ashwinsreedhar28]

Summary

The CUDA initialization and GPU benchmark fitness checks allocate on the device inside the handler process. For a handler that has imported torch but not yet initialized CUDA (vLLM AsyncLLM/LLM built in-process, before the engine starts), that allocation creates the process's first context, and vLLM then has to spawn its EngineCore instead of fork:

WARNING [system_utils.py:157] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. ... Reasons: CUDA is initialized

The spawned EngineCore re-imports the engine and torch from scratch, 10.6-11.5 s per cold start on vLLM v0.30.0 (less on v0.32.0, whose import is cheaper). Fixes #657.

This change skips both checks when torch.cuda.is_initialized() is False, checked before torch.cuda.is_available() because its default cudaGetDeviceCount path also initializes the driver and poisons later forks (PyTorch CUDA notes). A handler that already created a context gets the checks exactly as before; the gpu_test binary, where it runs, still tests device allocation. It is the rule from #632 ("never add a torch import to cold start") extended to "never initialize CUDA".

Not affected either way: runpod-workers/worker-vllm (runs vllm serve in a subprocess, no torch in the handler, so these checks already skip) and SGLang (forces spawn itself).

Measurement

Runpod Serverless, runpod 1.12.0, vLLM v0.30.0, Qwen3-8B bf16, RTX 4090, network volume, compile cache warm. Per-phase wall time recorded inside the worker, three cold starts each. The second column disables all auto system checks with RUNPOD_SKIP_AUTO_SYSTEM_CHECKS=true; it bounds what this PR can recover and attributes it, since the only phase that changes is the EngineCore re-import. This PR itself was not measured on Serverless (the remaining checks don't touch torch); it keeps them on.

	all auto checks on	RUNPOD_SKIP_AUTO_SYSTEM_CHECKS=true
EngineCore start method	spawn	fork
import vllm in EngineCore	10.6-11.5 s	none
engine ready, median	63.3 s	52.3 s
weight load, profile, graph capture	same	same

Raw data and runner: https://github.com/ashwinsreedhar28/vllm-coldstart/tree/main/bench/results (configs A and D).

Tests

Two tests in test_system_checks.py: with torch imported and no context, neither check calls is_available() or touches the device (device_count, synchronize, zeros, randn, matmul not called). On main both fail (the initialization check on device_count, the benchmark on randn); with this change both pass. Existing tests are unchanged: their MagicMock torch reports is_initialized() truthy, so the checks run as before.

pytest tests/test_serverless/test_modules/test_fitness/test_system_checks.py: 39 passed.

Notes

• CuPy path unchanged: CuPy has no is_initialized() equivalent, and the engines this affects use torch.
• Alternative considered: running the two checks in a subprocess. That would restore the ~1.5 s torch import perf(serverless): skip torch fitness checks unless loaded #632 removed (in the child) for a check the gpu_test binary already covers, so skipping is the smaller change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KGxXNtPM12ALm9YXqHhiY4

[Copilot]

🟢 Approval recommended

No active review issues were identified for the change that prevents PyTorch fitness checks from initializing CUDA.

No open issues 🎉

What changed in this PR

This PR addresses #657 by preventing PyTorch fitness checks from initializing CUDA in the handler process, preserving the option for in-process vLLM engines to fork rather than spawn.

Changes:

• Skip CUDA initialization and GPU benchmark checks when PyTorch has no initialized CUDA context, before calling is_available().
• Add regression tests verifying that skipped checks do not probe CUDA availability or perform device operations.
• Document the new guard, extending #632’s cold-start optimization.

File	Description
tests/​test_serverless/​test_modules/​test_fitness/​test_system_checks.py	Adds regression tests for skipping both checks without a CUDA context.
runpod/​_health/​system.py	Adds a shared CUDA-state guard to both PyTorch checks and updates documentation.

_{Review effort: Balanced}