Fit a 124B vision MoE into a two-GPU agent stack: the Ling-3.0-flash-VL serving checklist

The trade-off is memory, not math

A vision MoE fits a two-GPU agent stack when the resident weights fit the GPU class and the active compute stays small. Ling-3.0-flash-VL has 124B total parameters and 5.5B activated parameters. The resident-to-active ratio is about 22.5x.

The arithmetic comes first: you buy VRAM for the full model, but pay compute for a small active slice. It is an open weights vision model from InclusionAI, the model maker, under the MIT license. For screenshot or GUI-agent workloads, the serving flags decide whether the model is production-viable.

Pick the precision/TP pair before the loop

SGLang, the serving engine, gives the recipe matrix. The matrix is the first thing to pin down because the checkpoint class sets the tensor-parallel width. A wrong pair can make the server start, then fail under load. The agent loop should not discover that after launch.

The BF16 recipe is sized for KV-cache headroom. That headroom is what lets the server keep a live context window open. If the weights consume all available memory, the agent loop will stall on a long screenshot. The quantized path changes the width, so the hardware class and the checkpoint precision must be checked together.

The checklist is the launch gate

Run these before the agent loop. Each item is a quick check.

• Set the precision/TP pair: FP8 at TP2 on H100/H200, FP4 or INT4 at TP1, or BF16 at TP4 on 288GB-class or 141GB-class GPUs, or TP8 on H100.
• Confirm the parser: use the ling3 reasoning and tool-call parser, via auto detection or explicit flags.
• Set the 256K YaRN context, a long-context scaling method: --context-length 262144, the rope_scaling override, and SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1.
• Check the context ceiling: native config is 131,072 tokens, with YaRN extension up to 1M tokens.
• Anchor self-hosted cost against OpenRouter: $0.060 per million input, $0.180 per million output, $0.012 cache read.

The parser item exists because screenshot agents fail silently when tool calls are misparsed. The context item exists because long UI history can exceed the native window. The cost item exists because self-hosting only wins when amortized GPU spend stays below the API anchor.

Parser checks are cheap. A missing tool-call parser can let the model produce a plausible answer while the harness never sees the action. For a GUI agent, that is the failure mode you want to catch before a screenshot. The context check is the same idea: the server must accept the window the agent will actually use.

Costs are a ceiling, not a target

The API anchor is a sanity check, not a procurement target. If your self-hosted unit cost is above it, you are paying for convenience, not scale. Cache reads matter when the agent resends the same UI state. If your harness can reuse cached prefixes, the effective input cost drops. The model license keeps the weights portable, but the serving flags are the real integration surface.

Use the API anchor to test your own math. Put your GPU amortization, power, and ops time into the same per-million-token shape. If the self-hosted number is lower, the model is worth serving. If it is higher, the model is a convenience tier, not a cost win. Keep the API anchor visible in the shared team runbook.

Screenshot agents are unforgiving of silent integration errors. The model may look healthy while the harness loses the action. The context can fail only after the history crosses the server limit. The cost can shift when cache behavior changes. The checklist exists to catch those states before they become production incidents.

Before a screenshot, verify the server log shows the parser resolved correctly. If it did not, the model may still answer, but the agent harness will drop tool calls. Check the context flag in the process environment, not just the client request. A client can ask for a long window, but the server must allow it. For a GUI agent, the window is the difference between a single action and a multi-step repair.