Summary
API clients can supply arbitrary mm_processor_kwargs per request. vLLM
forwards these values through the multimodal rendering and processing pipeline
to Hugging Face and custom image/video processor constructors and calls.
Resource-sensitive values such as image size, patch size, patch count, crop
count, or frame count can therefore be controlled by a remote client.
For processors that upscale inputs, a single request can force an allocation
large enough to terminate the API server or worker with an out-of-memory
condition.
Affected Components
The per-request field was introduced in commit
041e29471671a3d43afedb6a749421d56ef4d3e7 and first shipped in v0.7.3.
The generic forwarding path remains present in current main.
The Isaac path is a concrete high-amplification example:
ChatCompletionRequest.mm_processor_kwargs
OnlineRenderer request extras
MMProcessorInputs.hf_processor_mm_kwargs
BaseMultiModalProcessor.info.get_hf_processor(**hf_processor_mm_kwargs)
IsaacProcessingInfo.get_image_processor(**kwargs)
IsaacImageProcessor(**kwargs)
process_vision_for_patches()
Equivalent constructor forwarding exists in InternVL-derived processors,
H2OVL, Step3VL, NemotronVL, and other multimodal processors. The issue is at
the shared remote-input boundary and is not limited to a fixed model list.
Impact
- Availability: one request can exhaust CPU or GPU memory and terminate a
shared service process.
- Scope: all users of the affected service instance lose availability.
- Authentication: vLLM's API key is optional, so deployments without an
external authentication layer are remotely reachable without privileges.
Remediation
Reject non-empty per-request mm_processor_kwargs at every API-server request
schema by default. Operators serving only trusted clients can explicitly opt
in to the existing override behavior. Server-level --mm-processor-kwargs and
offline LLM overrides remain available.
This shared boundary covers current and future multimodal processors instead
of maintaining a model-specific list or a denylist that can miss custom
processor arguments.
A patch for this was merged here: vllm-project/vllm#58830
References
Summary
API clients can supply arbitrary
mm_processor_kwargsper request. vLLMforwards these values through the multimodal rendering and processing pipeline
to Hugging Face and custom image/video processor constructors and calls.
Resource-sensitive values such as image size, patch size, patch count, crop
count, or frame count can therefore be controlled by a remote client.
For processors that upscale inputs, a single request can force an allocation
large enough to terminate the API server or worker with an out-of-memory
condition.
Affected Components
The per-request field was introduced in commit
041e29471671a3d43afedb6a749421d56ef4d3e7and first shipped in v0.7.3.The generic forwarding path remains present in current
main.The Isaac path is a concrete high-amplification example:
ChatCompletionRequest.mm_processor_kwargsOnlineRendererrequest extrasMMProcessorInputs.hf_processor_mm_kwargsBaseMultiModalProcessor.info.get_hf_processor(**hf_processor_mm_kwargs)IsaacProcessingInfo.get_image_processor(**kwargs)IsaacImageProcessor(**kwargs)process_vision_for_patches()Equivalent constructor forwarding exists in InternVL-derived processors,
H2OVL, Step3VL, NemotronVL, and other multimodal processors. The issue is at
the shared remote-input boundary and is not limited to a fixed model list.
Impact
shared service process.
external authentication layer are remotely reachable without privileges.
Remediation
Reject non-empty per-request
mm_processor_kwargsat every API-server requestschema by default. Operators serving only trusted clients can explicitly opt
in to the existing override behavior. Server-level
--mm-processor-kwargsandoffline
LLMoverrides remain available.This shared boundary covers current and future multimodal processors instead
of maintaining a model-specific list or a denylist that can miss custom
processor arguments.
A patch for this was merged here: vllm-project/vllm#58830
References