Upgrade vLLM 0.29 without breaking your fleet: test MRV2, fallbacks, and queue limits first
A shared vLLM stack needs a pre-upgrade test matrix for MRV2 support, fallback triggers, admission controls, and removed architectures before the old runner disappears.
You are about to point a shared vLLM fleet at 0.29.0, and the first thing to check is which runner each pod actually uses. vLLM 0.29.0 makes Model Runner V2 the default path for all models, while Model Runner V1 is deprecated and slated for removal in v0.32. The release contains 594 commits from 277 contributors, including 91 new contributors. A shared fleet has a short window to prove every model, flag, and queue path before the fallback disappears. If some endpoints fall back, the affected share is the fallback count divided by total endpoints, and that share runs a different execution path. The trade-off is runner coverage versus removal risk: the new runner is default, but the old runner is leaving.
The default runner forces a fleet-wide test
The runner change forces a fleet-wide test. It changes the execution path for the whole fleet and how the fleet absorbs load. For a shared stack, a config change can alter latency, memory, and queue behavior across tenants. The fallback share can carry different queue priorities, memory limits, and batch behavior. The test matrix must separate model support, feature support, and admission behavior.
Fallbacks hide the upgrade
When a pod keeps an unsupported capability, it can look healthy while it runs on the old path. The stated plan is to close those Model Runner V2 capability gaps in 2-3 weeks. For a fleet, treat it as a compatibility audit rather than a routine rollout. A single pod may be fine, but a fleet with mixed runners can show uneven tail latency and memory pressure.
A fallback is useful for a single pod and dangerous for a fleet. It lets a deployment keep a feature while the rest of the fleet moves forward. That creates different memory profiles, batch behaviors, and failure modes. Your monitoring must show which runner is active, not just that requests are succeeding. If the log says the old runner is active, treat that as a finding. Use the gap-closure plan to set a retest date. Those flags are easy to leave on by accident.
Admission controls and removed architectures change behavior without crashing
The release also adds admission controls and drops model architectures, which changes shared-stack behavior without always crashing. Admission controls are where the rejection policy lives. Queue behavior decides whether overload becomes latency or rejections. A model that ran before can fail at load time. Memory sizing and batch-size settings need a recheck after the runner changes.
The upgrade checklist is ordered by severity. Inventory first, because a hard stop is a hard stop. Fallback checks second, because a hidden path can look healthy. Admission and queue checks last, because they change behavior without crashing.
- Inventory every model architecture and runner flag in the fleet.
- Confirm whether each deployment uses Model Runner V2 or falls back to Model Runner V1.
- Turn off or isolate capabilities that Model Runner V2 does not cover yet, because a pod can still use Model Runner V1 when it enables them, including custom logits processors, sequence parallelism, dual-batch overlap, elastic expert parallelism, and some speculative decoding paths.
- Benchmark tensor-parallel all-reduce and prefix caching on Model Runner V2.
- Set admission control flags and queue limits, then verify rejected requests.
- Validate max_num_batched_tokens and KV cache sizing.
Do not treat a successful startup as a pass. A model can load, serve a few requests, and still hit an admission rejection under load. Compare the canary against the old version for the same traffic mix, model mix, and tenant queue mix, then check rejected requests, queue depth, tail latency, and memory headroom. Record the active runner, flags, and queue depth in the canary notes. Keep the old runner visible in metrics, logs, and pod labels during the canary. Stop on a fallback and decide whether that feature is required. Plan a model swap or a different backend on a hard stop. Tune the flag on a rejection before the fleet sees it.