TensorRT-LLM 1.2 Migration Checklist

If your service still calls LLM(backend='tensorrt'), the call raises a ValueError. The arithmetic is subtraction: TensorRT-LLM 1.2 eliminated the separate TensorRT offline compilation path, moved to a PyTorch-native execution model, dropped the TensorRT execution path, and left PyTorch as the only execution backend. It no longer installs the tensorrt pip dependency. TensorRT-LLM 1.2 landed in July 2026. Expect fewer moving parts and a forced rewrite of the serving path.

The compile step is gone, not deprecated

TensorRT-LLM 1.2 removed the trtllm-build, trtllm-refit, and trtllm-prune command-line tools. Build scripts that expect a separate engine artifact now have nothing to call. The old flow assumed a staged pipeline: prepare the model, then compile it into a serving engine. That assumption breaks. A build pipeline that still waits for an engine file fails before the model serves.

Treat this as an inference engine upgrade, not a version bump. The old engine file was a deployable object. The new path changes what the service expects at startup. Mark engine files in the artifact registry as deprecated. Rewrite release notes that still say compile first. The serving stack now behaves like a model-serving runtime.

TensorRT-LLM 1.2 removed the model-specific convert_checkpoint.py scripts. Checkpoint conversion steps need to be replaced with the new execution path. Model onboarding docs, internal wikis, and runbooks may still describe a step that no longer exists. Treat those docs as stale until someone rewrites them against the new execution model.

The backend string now raises an error

Code makes the failure explicit. It turns a silent misconfiguration into a startup error. Smoke tests should fail fast, not after a long deploy. Switch the code to the PyTorch-native path. Treat the ValueError as a release gate.

Make that error a test, not an incident. Add a startup check that rejects the old backend string before the service binds. Add a config lint rule that flags the removed value. Shared base images should carry the check so every downstream service inherits it. The failure should be visible in the deploy log, not in a customer-facing error page.

Audit every backend string in the config store before rollout. Backend selection often lives in feature flags, environment variables, or service manifests. Each of those can hide the old string. A grep across the repo starts the audit, but the config store holds production truth.

The dependency list shrinks

The dependency list changes. That changes image builds, lockfiles, and supply-chain reviews. Separate any tensorrt pin in the base image from the serving stack. The serving stack should not carry a dependency it no longer uses.

Update lockfiles before you update the service. Trace the layer that adds tensorrt when the image scanner reports it as present. Correct the manifest when the bill-of-materials process still lists it. A stale dependency should not remain in the bill of materials. Handle it as a supply-chain cleanup, not just a code change.

Expect labor, not license fees: audit time, test time, and rollback planning. The scope is clear, but it touches the parts of the stack that usually change only when a release forces a rewrite. Budget for the audit before the rollout.

Run the pre-upgrade audit

Use the list below as a pre-upgrade pass. Each item runs quickly and maps to a removed piece of the old workflow. Run it to catch the old path before it reaches production.

• Search configs and code for backend='tensorrt'; replace the call with the PyTorch-native path.
• Remove trtllm-build, trtllm-refit, and trtllm-prune from build scripts and pipelines.
• Drop tensorrt from pinned dependencies and image layers.
• Delete or reroute model-specific convert_checkpoint.py steps.
• Run a smoke request against the new path and confirm a response.

The list stays short because the removals are explicit. Prove the replacement serves your traffic with acceptable latency and error rates on a canary before you widen the rollout. Keep the old image available for rollback until the new path has handled real traffic.

Keep the validation gate narrow. Check that the service starts, that the model loads, that a request returns, and that logs show the expected execution path. Send a small slice of traffic to a canary lane first. Without a canary lane, run a manual load test against the canary before you promote the image. Do not try to prove the old path is dead. Prove the new path is safe.