Run SGLang v0.5.19: upgrade for cross-node MoE, gate beam search

Upgrade only when decode or attention is the bottleneck

You are deciding whether SGLang v0.5.19 is a serving upgrade, an eval feature, or a config trap. The trade-off is narrow: cross-node MoE decode and Lean attention earn the upgrade, while beam search stays a correctness path. The arithmetic is the gate: on MI355X, the reported Lean kernel gain reaches at most 1.52x throughput and at most 3.62x lower inter-token latency. If neither workload applies, the release is mostly churn.

DeepEP v2 ElasticBuffer is selected with --moe-a2a-backend deepep_v2 for FP8 DeepSeek-V3/V4 and Qwen3-MoE; the fixed-size buffers let MoE decode run under CUDA graphs across nodes. The payoff is real when decode is the slow part and the graph boundary is the tax.

SGLang v0.5.19 became available on 5 September 2026, with 786 merged pull requests from about 214 contributors. Volume is not the deciding factor. Decide whether decode or attention is the bottleneck.

The cost is the compatibility surface. A release that changes decode, attention, and request semantics is a routing decision.

v0.5.18, released around 22 August 2026, already added --startup-weight-load-mode overlap; weight loading and CUDA graph capture now run in parallel rather than one after the other. The v0.5.19 gains are separate from that cold-start path.

On MI300X and MI355X, Lean attention ships on by default; set SGLANG_DISABLE_LEAN_ATTENTION=1 to turn it off. The default matters when attention is the bottleneck.

Keep beam_width behind a correctness gate

v0.5.19 exposes beam search with the beam_width request parameter, and the response contains the n best sequences instead of one sampled sequence. The shape fits eval and reranking, where the ranked set is the deliverable.

The caller must accept multiple sequences, score them, and choose one. If the endpoint wants one answer, the extra sequences are overhead.

For measurement, the ranked set is the output; for reranking, it is the input to the next stage. The compatibility wall is the reason it stays out of general serving.

Run the three-gate check before rollout

Each gate is a one-minute check. Run the gates before you change the flag. The first two keep beam search out of paths that cannot mix with it. The last one scopes the upgrade to the workloads that earn it.

• Use beam_width only where n-best output is a correctness requirement, such as eval or reranking.
• Beam_width is off-limits on paths that need speculative decoding, disaggregation, DP attention, or HiCache.
• Upgrade for DeepEP v2 ElasticBuffer or Lean attention only if cross-node MoE decode or MI300X/MI355X throughput and latency gains apply.

Treat the gates as a rollout decision. Beam search belongs only where the ranked set is the answer, not on a chat endpoint that expects one sampled sequence. The excluded features make the serving path a config check, not a debate. The upgrade stays scoped to the workloads that actually use the two wins: if your model is not the matching MoE workload, the decode win is not yours, and if your hardware is not the matching attention target, the attention win is not yours.

The checklist is also a rollback plan. If a gate fails, the fix is to move the workload, not to weaken the gate. Keep the serving path predictable.

Before you upgrade, snapshot the request mix. Mark which endpoints need one sequence and which need ranked sets. Then run the checklist. During rollout, keep beam search behind a separate endpoint or a separate model route. A separate route lets you compare latency and quality without mixing traffic. Keep the baseline visible during rollout.

If a request fails the second gate, do not try to make it pass by disabling one feature at a time. The incompatibility is the feature, and tuning around it is wasted effort. If the checklist passes, upgrade. If it does not, stay on the current release and revisit when your workload changes.