SGLang v0.5.20 upgrade checklist: CUDA 12, response store, TTFT

The build lane decides whether you pin

The trade-off is binary: keep the current build lane or move the build, then pay the flag and benchmark cost. On ROCm, SGLang v0.5.20 reduced GLM-5.2 load time at tensor parallelism 4 on four MI355X GPUs from 505.7 seconds to 40.4 seconds. SGLang v0.5.19 was the final release with CUDA 12 -cu12x artifacts, and SGLang v0.5.20 ended that lane. The arithmetic: a blocked build lane means pin or migrate; an open build lane means a five-point checklist. The work is the image build, the API contract, and the benchmark run.

Check the image tag before touching the server. A fleet that depends on a narrow build lane cannot assume the upgrade keeps the same path. The safe move is to pin the current build for that lane, or to validate the new lane in a staging cluster first. Shared build matrices need an explicit pin in the deployment manifest so the upgrade does not drift into production by accident.

Response storage changes the API contract

SGLang v0.5.20 changes /v1/responses so it no longer keeps earlier outputs in memory unless the server is launched with --enable-response-store; without that flag, lookups, response chaining, and background calls fail with 400, and PD deployments cannot enable the store. Response chaining needs the flag before rollout. Without chaining, the default leaves the API surface smaller. Test the exact call path before rollout. The client code should fail loudly when the store is off, because a silent fallback can hide a broken integration until traffic is high enough to matter.

Disaggregated serving tightens the constraint. When the deployment shape cannot enable the store, the response endpoint is not a drop-in replacement for the old behavior. Plan the client side around the calls that will return an error, and keep the pinned build available for the slice of traffic that still needs chaining. Record the client behavior in the release note, because the next incident will be easier to debug when the expected error is already written down.

Run the five-point checklist before flipping the fleet

Run these in order. Each maps to a breaking change or a number you need to re-measure. The items cover the build path, the API contract, removed flags, the cache benchmark, and the accelerator load path.

• Pin CUDA 12: keep the -cu12x build path or migrate it before upgrading.
• Re-enable response storage: set --enable-response-store for the affected /v1/responses call paths.
• Strip prefill CP v1 flags: remove the old v1 options. SGLang v0.5.20 dropped the prefill context parallelism v1 runtime and its command-line options, leaving the strategy-based approach as the sole prefill CP route and blocking prefill CP on HIP, NPU, and MUSA until ported.
• Benchmark the unified radix tree: re-measure token hit rate and mean TTFT on the traffic shape you run. In the reported shared-system-prompt case, SGLang v0.5.20's unified radix tree branching-point caching raised DeepSeek-V4-Flash token hit rate from 43.8% to 60.8% and lowered mean time-to-first-token from 1.57 seconds to 1.07 seconds.
• Validate ROCm load time: re-measure load time on your ROCm fleet and compare it against the pinned build.

The scheduler adds another number: SGLang v0.5.20 introduced an HRRN scheduling policy, a queueing rule, whose mean time-to-first-token was 69% lower than first-come-first-served in a GLM-5.2 production trace. Bursty traffic needs a reproduction run before the result is trusted. Steady traffic should rest on the build lane and the cache benchmark.

Use the numbers as a gate, not a rumor

Run the list in staging before you touch production. Keep the pinned build available for rollback. Keep the canary small and the rollback path warm. Capture the same trace shape you use in production, then compare first-token latency, cache hit rate, and load time. A blocked build lane calls for pinning. Cache and scheduler wins in the trace support an upgrade. A flat trace supports keeping the pin and revisiting later.

Keep the checks separate: a build-lane pin is a supply-chain decision, and a cache win is a performance signal. The fleet should move only when the staging trace matches the production shape and the rollback path is still available. Write the decision in the release note. Name the trace, the rollback command, and the owner.