Overview概览 Method方法 Live training step训练步骤演示 Results结果 Checkpoints模型权重 Citation引用

Region-Level Policy Optimization
for Fine-grained MLLM Perception 面向细粒度多模态感知的
区域级策略优化

Vision-RL²: train the RoI predictor with region-level reinforcement learning from a frozen MLLM reader Vision-RL²：用冻结的多模态大模型作为“阅读器”，以区域级强化学习训练 RoI 预测器

Yuheng Shi¹, Xiaohuan Pei¹, Minjing Dong², Chang Xu¹

¹University of Sydney ²City University of Hong Kong ¹悉尼大学 ²香港城市大学

Preprint 预印本

arXiv Code代码 🤗 Models模型

Overview概览

Fine-grained perception in multimodal large language models (MLLMs) is usually bought with resolution, and every extra visual token is paid twice, in the vision encoder and in the language model. We start from a measurement: the two operations behind fine-grained perception, localizing the region of interest (RoI) and recognizing its content, do not need the same resolution. In a controlled diagnostic on ZoomBench, localization tolerates roughly 3–4× stronger token compression than recognition. That asymmetry motivates localizing from a coarse view and spending resolution only on the selected evidence, which puts the burden on the RoI predictor.

多模态大模型（MLLM）的细粒度感知通常靠“堆分辨率”换来，而每一个额外的视觉 token 都要在视觉编码器和语言模型里各付一次代价。我们从一个测量出发：细粒度感知背后的两步操作，定位感兴趣区域（RoI）和识别其内容，对分辨率的需求并不相同。在 ZoomBench 上的受控诊断中，定位能承受比识别高出约 3–4 倍的 token 压缩。这一不对称性意味着可以在粗视图上定位，只把分辨率花在被选中的证据上，于是压力落到了 RoI 预测器身上。

Existing predictors trade one thing for another. Decoding box coordinates with the MLLM can be trained from answers but costs a full model pass per query. A lightweight proposal network distilled from the model's attention (SD-RPN) is fast but inherits the noise of its attention targets and is never checked against the answer. Because the RoI carries no annotation and reaches the answer only through a discrete region choice, we optimize the proposal network with region-level reinforcement learning: coherent regions are actions, and a frozen MLLM reader scores each one by how its removal changes the answer likelihood. A subtractive objective prunes distracting proposals, an additive objective recovers missed evidence, and only the small predictor is updated. At inference, a sparse visual encoding magnifies the evidence and drops background tokens.

现有的预测器各有取舍。让 MLLM 直接解码框坐标可以由答案端到端训练，但每个问题都要付出一次完整的模型前向；从模型注意力蒸馏出的轻量提议网络（SD-RPN）速度快，却继承了注意力目标的噪声，而且从未用答案来校验。由于 RoI 没有标注、又只能通过离散的区域选择影响答案，我们用区域级强化学习来优化提议网络：把连通区域当作动作，用冻结的 MLLM 阅读器衡量“去掉这个区域后答案似然变化多少”。减法目标剪掉干扰提议，加法目标找回遗漏证据，只更新小预测器。推理时，稀疏视觉编码放大证据并丢弃背景 token。

Teaser: localization vs recognition compression tolerance, token efficiency, and method overview

(a) The box is still re-predicted correctly at 25× token reduction, while reading the crop already fails at 9×. (b) Population survival under scene vs. crop compression. (c) Coarse localization with sparse reading matches the dense SD-RPN pipeline with 4.2× fewer visual tokens (Qwen3.5-4B). (d) Vision-RL²: the RPN proposes regions, a frozen reader scores each region's contribution to the answer, and the reward updates the RPN. (a) token 压缩 25 倍时框仍能被正确重预测，而读裁剪区在 9 倍压缩时已经失败。(b) 分别压缩“场景”与“裁剪区”时的样本存活率。 (c) 粗定位加稀疏读取，用少 4.2 倍的视觉 token 达到稠密 SD-RPN 流程的精度（Qwen3.5-4B）。(d) Vision-RL²：RPN 提出区域，冻结阅读器为每个区域评分，奖励更新 RPN。

3–4×

more compression tolerated by localization than recognition定位比识别多承受的 token 压缩倍数

+3.0

six-benchmark average from region-level RL alone (Qwen3.5-4B)仅靠区域级 RL 带来的六基准平均提升（Qwen3.5-4B）

4.2×

fewer visual tokens at the same accuracy as SD-RPN与 SD-RPN 同精度下节省的视觉 token 倍数

80.1

main-table average of Vision-RL² 9B, ahead of Gemini-3.1-Pro and Vision-OPD-9BVision-RL² 9B 的主表平均分，高于 Gemini-3.1-Pro 与 Vision-OPD-9B

Method方法

Regions as actions, a frozen reader as the critic区域即动作，冻结阅读器即评判

The predictor is SD-RPN: three trainable blocks attached after block K of the frozen MLLM, producing a dense RoI map P_θ from the last prefilling token in a single pass. Vision-RL² keeps this interface and replaces its training signal. The map is smoothed, binarized at a peak-relative threshold, and split into connected components; the top-K components are the regions R₁…R_K. A mask renders a set of retained regions into a masked image, and the functional score h_φ(M) is the log-odds of the gold answer under a frozen reader (the MLLM itself) that sees only the masked image. A region's contribution is the leave-one-out drop Δ_k = h_φ(M) − h_φ(M ⊖ R_k), clipped to ±5, and n regions cost only n+1 reader passes.

预测器是 SD-RPN：在冻结 MLLM 的第 K 层之后接三个可训练模块，由最后一个 prefill token 一次前向得到稠密 RoI 图 P_θ。 Vision-RL² 保留这一接口，只替换训练信号。RoI 图经平滑、按峰值相对阈值二值化、再拆分为连通分量，取前 K 个分量作为区域 R₁…R_K。掩码把保留的区域集合渲染成掩码图像，功能得分 h_φ(M) 是冻结阅读器（即 MLLM 本身）只看掩码图像时，标准答案的对数几率。一个区域的贡献是留一法的下降量 Δ_k = h_φ(M) − h_φ(M ⊖ R_k)（裁剪到 ±5）， n 个区域只需 n+1 次阅读器前向。

Overview of region-level RL: regions, masked images, reader scores, margin, advantages

One training step. Predicted regions R₁…R₄ and control regions are extracted from the RoI map; the frozen reader scores each action's masked image; contributions Δ_k and the margin b form the advantages and the policy loss. 一个训练步骤。从 RoI 图中提取预测区域 R₁…R₄ 与对照区域；冻结阅读器为每个动作的掩码图像评分；贡献 Δ_k 与阈值 b 构成优势与策略损失。

Subtractive group减法组

Each predicted region is removed in turn. Small likelihood changes happen even for irrelevant removals, so two control regions grown in low-evidence areas set a sample-specific noise margin b. A region whose contribution beats b receives negative credit for its removal and is preserved; otherwise its confidence is lowered. Removals compete under a softmax removal policy, so credit for removing one region reinforces keeping the others.

依次移除每个预测区域。即使移除的是无关区域，似然也会有小幅波动，因此在低证据区生长两个对照区域，得到样本特定的噪声阈值 b。贡献超过 b 的区域，其“移除”动作获得负优势从而被保留；否则降低其置信度。各移除动作在 softmax 移除策略下相互竞争，移除某一区域的信用会反过来强化保留其余区域。

Additive group加法组

Removal cannot recover evidence the policy never proposed. Frozen response-to-image attention maps from six MLLM layers propose supplementary regions outside the prediction. Each is scored by its leave-one-out contribution against the augmented mask; a positive contribution raises the region in the RoI map through its mean inclusion log-likelihood, a negative one suppresses it. No margin is needed here, because exclusion is already the zero-gradient default.

移除无法找回策略从未提出的证据。来自 MLLM 六个层的冻结“回答到图像”注意力图，在预测区域之外提出补充区域。每个补充区域按其相对增广掩码的留一贡献评分：贡献为正则通过区域内的平均包含对数似然抬高 RoI 图，为负则压制。这里不需要阈值，因为“不包含”本就是零梯度的默认状态。

Anchors and objective锚定项与总目标

Both policy losses are scaled by a detached attainability weight (samples the reader cannot answer under any mask contribute little), a KL anchor to the initial SD-RPN keeps the map from drifting, and a BCE anchor handles single-region samples. Only the predictor is trained; the MLLM stays frozen. Inference is unchanged: one answer-free RoI prediction before decoding.

两个策略损失都乘以一个不回传梯度的“可达性”权重（阅读器在任何掩码下都答不出的样本贡献很小），对初始 SD-RPN 的 KL 锚定项防止 RoI 图漂移，BCE 锚定项处理只有单个区域的样本。只训练预测器，MLLM 保持冻结。推理不变：解码前只做一次无需答案的 RoI 预测。

Sparse visual encoding稀疏视觉编码

At inference the trained predictor enables a simple principle: spend the visual-token budget on evidence, not on area. We take the bounding box of the predicted foreground, measure its foreground occupancy ρ_fg, and re-encode the crop at zoom η = min(√(1/ρ_fg), η_max), keeping only the tokens inside the foreground. Position embeddings are assigned on the full crop grid before background tokens are dropped, so the sparse token set is positionally indistinguishable from a dense encoding of the same crop, while the evidence is seen at η× finer resolution under the same budget. Source tokens are not re-encoded; their KV cache is reused.

推理时，训练好的预测器让一个简单原则成为可能：把视觉 token 预算花在证据上，而不是花在面积上。取预测前景的外接框，测量其前景占比 ρ_fg，以缩放倍率 η = min(√(1/ρ_fg), η_max) 重新编码裁剪区，只保留前景内的 token。位置编码在丢弃背景 token 之前按完整裁剪网格分配，因此稀疏 token 集在位置上与同一裁剪区的稠密编码无法区分，而证据在相同预算下以 η 倍更精细的分辨率被看到。源图 token 不重新编码，复用其 KV 缓存。

A dense crop spends the budget uniformly over the bbox; the sparse crop encodes only foreground tokens at η× finer resolution. 稠密裁剪把预算均匀地花在外接框上；稀疏裁剪只编码前景 token，且分辨率高出 η 倍。

One training step, with real numbers一个真实的训练步骤

The overview figure is static; this is the same procedure run on real training samples with the released code and the SD-RPN initialization of Qwen3.5-4B. Every number below comes from the frozen reader on that sample. Step through the six stages, or let it play.

概览图是静态的；这里用发布的代码和 Qwen3.5-4B 的 SD-RPN 初始化，在真实训练样本上跑同一套流程。下面的每个数字都来自该样本上的冻结阅读器。可以逐步查看六个阶段，也可以自动播放。

Loading cases…正在加载样例…

Results结果

Main comparison主结果对比

All methods share a 16,384 source-image token limit, produce free-form responses, and are scored by rule-based parsing with an LLM judge for the remaining cases. Base-model and Vision-OPD / ZwZ rows are re-evaluated on released weights; other rows are quoted from their publications.

所有方法共享 16,384 的源图 token 上限，自由生成回答，先用规则解析、剩余情况交给 LLM 评判。基础模型与 Vision-OPD / ZwZ 行基于公开权重复测，其余行引自原论文。

Model模型	Size	V* Bench	ZoomBench	HR-Bench 4K	HR-Bench 8K	MME-RW EN	MME-RW CN	Average平均
Large-scale open / closed-source models大规模开源 / 闭源模型
GPT-5.4	—	77.0	52.7	84.0	77.9	74.2	70.9	72.8
Gemini-3.1-Pro	—	88.0	61.2	89.6	86.9	76.5	73.3	79.3
Qwen3-VL-Instruct	235B	91.1	56.1	86.1	80.4	71.7	69.0	75.8
Qwen3.5	397B	88.0	57.2	89.4	85.5	74.8	69.8	77.4
Kimi-K2.6	1T	88.5	53.1	81.9	78.0	69.2	66.1	72.8
Qwen2.5-VL based基于 Qwen2.5-VL
DeepEyes	7B	85.9	46.5	75.1	72.6	64.1	64.1	68.1
Thyme	7B	82.2	45.1	77.0	72.0	64.8	64.6	67.6
DeepEyesV2	7B	81.7	45.0	77.9	73.8	64.9	65.1	68.0
ZwZ	7B	86.9	55.6	75.9	72.4	65.0	63.5	69.9
Vision-RL² (ours)	7B	91.6	59.8	78.8	75.0	62.2	58.7	71.0
Qwen3-VL based基于 Qwen3-VL
Qwen3-VL-Instruct	8B	84.8	43.0	79.6	75.3	63.2	64.6	68.4
ZwZ	8B	90.6	58.0	84.4	81.6	69.9	69.2	75.6
P2R	4B	93.2	–	81.9	80.5	–	–	–
P2R	8B	93.7	–	81.5	82.6	–	–	–
Qwen3.5 based基于 Qwen3.5
Qwen3.5	4B	85.9	51.5	83.6	80.1	59.1	60.6	70.1
Qwen3.5	9B	83.8	54.9	84.9	83.5	72.5	67.9	74.6
Vision-OPD	4B	90.6	59.5	82.0	79.1	74.2	70.6	76.0
Vision-OPD	9B	90.6	65.1	87.1	85.6	73.2	70.3	78.7
Vision-RL² (ours)	4B	91.1	65.1	84.3	80.3	65.8	65.3	75.3
Vision-RL² (ours)	9B	95.3	68.4	86.8	86.1	73.4	70.6	80.1
Gemma-4 based (encoder-free)基于 Gemma-4（无视觉编码器）
Gemma-4-it	12B	72.8	46.5	75.5	67.5	65.2	52.4	63.3
SD-RPN	12B	78.0	57.0	82.4	77.5	65.7	53.1	69.0
Vision-RL² (ours)	12B	82.2	60.7	85.0	79.4	67.6	61.6	72.8

A key distinction is the trainable parameter budget: DeepEyes, ZwZ, P2R, and Vision-OPD fine-tune the full MLLM, whereas Vision-RL² updates only the small attached predictor. The 9B model still attains the highest average in the table, with the best V* Bench and ZoomBench scores overall.

一个关键区别是可训练参数量：DeepEyes、ZwZ、P2R 与 Vision-OPD 都微调整个 MLLM，而 Vision-RL² 只更新附加的小预测器。即便如此，9B 模型仍取得表中最高的平均分，并在 V* Bench 与 ZoomBench 上全表最佳。

Accuracy versus tokens and latency精度 vs. token 数与延迟

Under the training-aligned short-answer protocol, our model lies above the base model, SD-RPN, and Vision-OPD at every source-image token limit from 576 to 4,096. At the 576 limit it exceeds the base model at 4,096 by more than three points with about a quarter of the tokens, and it matches SD-RPN at 4,096 with 4.2× fewer visual tokens (4B). The routing head adds three blocks on the answer call's own prefill and routes in 31–49 ms, roughly 30× cheaper than attention routing or coordinate decoding.

在与训练对齐的短答案协议下，我们的模型在 576 到 4,096 的每个源图 token 上限都高于基础模型、SD-RPN 与 Vision-OPD。576 上限时以约四分之一的 token 超过基础模型在 4,096 上限的精度三分以上，并以少 4.2 倍的视觉 token 追平 SD-RPN 在 4,096 时的精度（4B）。路由头只在回答调用自身的 prefill 上增加三个模块，31–49 ms 完成路由，比注意力路由或坐标解码便宜约 30 倍。

Six-benchmark average vs visual tokens and latency

Six-benchmark average versus visual tokens per sample (left of each pair) and end-to-end latency (right), Qwen3.5-4B and 9B.六基准平均分 vs. 每样本视觉 token 数（每组左图）与端到端延迟（右图），Qwen3.5-4B 与 9B。

Latency decomposition of routing interfaces

Latency decomposition of two-stage routing at the 4,096-token limit; accuracy above each bar.4,096 token 上限下两阶段路由的延迟分解；柱顶为精度。

Against visual token pruning (VisionZip, DART) on Qwen2.5-VL-7B: suite-average accuracy vs. retained tokens.与视觉 token 剪枝方法（VisionZip、DART）在 Qwen2.5-VL-7B 上的对比：套件平均精度 vs. 保留 token 数。

Qualitative comparison定性对比

RoI predictions of SD-RPN and Vision-RL² on cases where their answers diverge. Dimmed regions are the discarded tokens. SD-RPN scatters probability over irrelevant regions or covers the evidence only partially; after region-level RL the predictor keeps exactly the evidence the answer depends on.在两者答案不同的样例上，SD-RPN 与 Vision-RL² 的 RoI 预测。变暗区域为被丢弃的 token。SD-RPN 把概率散布到无关区域或只覆盖部分证据；经区域级 RL 后，预测器恰好保留答案所依赖的证据。

Checkpoints and code模型权重与代码

Training and evaluation code is released in the VisionRL2 repository: SD-RPN online pseudo-label training, the region-level RL recipe for all four backbones (Qwen3.5-4B / 9B, Qwen2.5-VL-7B and the encoder-free Gemma-4-12B), and the two evaluation protocols. Model weights and the RL pool will be released on Hugging Face.

训练与评估代码发布在 VisionRL2 仓库：SD-RPN 在线伪标签训练、四个主干（Qwen3.5-4B / 9B、Qwen2.5-VL-7B 以及无视觉编码器的 Gemma-4-12B）的区域级 RL 配方，以及两套评估协议。模型权重与 RL 训练池将在 Hugging Face 发布。

Backbone主干	Twig	Trainable params可训练参数	Main-table avg.主表平均	SD-RPN	Vision-RL²
Qwen3.5-VL-4B	K = 21, T = 3	predictor only仅预测器	75.3	coming soon即将发布	VisionRL2-Qwen3.5-4B
Qwen3.5-VL-9B	K = 21, T = 3	predictor only仅预测器	80.1	coming soon即将发布	VisionRL2-Qwen3.5-9B
Qwen2.5-VL-7B	K = 18, T = 3	predictor only仅预测器	71.0	coming soon即将发布	VisionRL2-Qwen2.5-VL-7B
Gemma-4-12B-it (encoder-free)（无视觉编码器）	K = 27, T = 3	predictor only仅预测器	72.8	coming soon即将发布	VisionRL2-Gemma-4-12B

Citation引用

@article{shi2026visionrl2,
  title   = {Region-Level Policy Optimization for Fine-grained MLLM Perception},
  author  = {Shi, Yuheng and Pei, Xiaohuan and Dong, Minjing and Xu, Chang},
  journal = {arXiv preprint},
  year    = {2026}
}

Region-Level Policy Optimizationfor Fine-grained MLLM Perception 面向细粒度多模态感知的区域级策略优化