Build a multimodal cost model for image, audio, and video
Price image, audio, and video as separate token streams to set cost, first-packet latency, and concurrency budgets in production.
A multimodal request is a stack of media streams, not one blended token price. The trade-off is capability against unit economics. IBM describes a multimodal LLM as a large language model that processes and reasons across text, images, and audio. Alibaba's Qwen3-Omni API prices image and video input at $0.46 per 1M tokens, audio input at $2.21 per 1M tokens, and the audio portion of text-plus-audio output at $8.76 per 1M tokens, while the text portion is free. A multimodal LLM is production-ready only when each stream is costed, latency-budgeted, and throttled separately.
Qwen3-Omni accepts text, image, audio, and video inputs, yet generates only text and audio outputs. If the product promises video, route it to a separate generator; Kling AI's Video 2.6 currently offers 5-second and 10-second video outputs in English and Chinese. The ledger should show that external stream as its own row.
Build the ledger before you set a budget
The ledger is the unit of control for a multimodal request. Each row is one modality stream in one request, and the fields are the ones you need to compare it with a text-only request. A text-only baseline is the same request priced with text tokens only; the ledger makes the delta visible.
- Input modality: the media type entering the request.
- Output modality: the media type leaving the model, plus any external generator.
- Tokens per request for that modality.
- Price per 1M tokens for that modality.
- First-packet latency for that output modality.
- Real-time factor under load.
- Concurrency ceiling for that stream.
Done means a table where a request can be costed, latency-budgeted, and throttled without opening the provider dashboard or a spreadsheet tab. The table should be small enough to read in a review and specific enough to drive admission control. Keep the fields in the serving layer, not a spreadsheet, so limits can reject a request before it burns GPU time. That keeps the ledger from becoming a post-mortem document after launch.
Split the request into modality streams. Record input modality, output modality, and tokens per request for each stream. A request log should show an image question, an audio transcript, and a video clip as separate rows, not one blended payload.
Price each stream with provider rates. Copy the provider's per-1M-token rates into the ledger as a price column next to each modality, so image token pricing is not inferred from text. The column should be per stream, not per request, because a single request can mix cheap image tokens with expensive audio output. Averaging image, audio, and video into one blended token price hides the audio-output penalty and the video first-packet latency.
Budget latency by first packet and real-time factor. Set a first-packet target for each output modality. Qwen3-Omni has theoretical first-packet latency of 234 milliseconds for audio and 547 milliseconds for video, and remains under one real-time factor with multiple concurrent requests. The budget should separate audio from video, not one end-to-end number.
Cap concurrency from the real-time factor. Use it as the ceiling for that stream. The queue should not turn a fast first packet into a slow stream.
Route outputs that the model cannot produce. Check output support before promising media. A fallback path should not bill the LLM for video it cannot emit.
Keep the ledger honest under load
In model serving, the ledger becomes the control plane. It shows which modality drives cost, which stream drives tail latency, and where concurrency erodes the first-packet budget. Keep the rows separate so mixed requests expose each contribution in the unit economics.
Review the ledger with the same cadence you use for capacity planning. A modality mix can change the cost of a request without changing the model, the latency of a first packet, or the concurrency ceiling.