REKI: Registering Knowledge in VLAs

REKI：在 VLA 中登记知识

Juliette Marrie (Kyutai) · Paul Pacaud (Inria) · Adrien Ramanana Rahary (École nationale des ponts et chaussées) · Shizhe Chen (Inria) · Moritz Böhle (Kyutai)

Juliette Marrie（Kyutai）· Paul Pacaud（Inria）· Adrien Ramanana Rahary（法国国立路桥学校，École nationale des ponts et chaussées）· Shizhe Chen（Inria）· Moritz Böhle（Kyutai）

（译注：本对照依据 arXiv:2610.22178v1 的 HTML 版本。附录中几张在 HTML 里被压平的表格，按 HTML 的单元格结构重排为 Markdown 表格，数值未改。）

Abstract

摘要

We introduce REKI, a 2.4B vision-language-action model (VLA) pretrained exclusively on public data that sets a new Pareto frontier between performance and model size: on LIBERO-Plus, RoboTwin Clean2Rand and RoboCasa365, it is outperformed only by models about twice its size. REKI consists of a vision–language model (VLM) followed by a flow-matching action expert, and is pretrained on a curated mixture of public robot, egocentric human and vision–language data. Additionally, we endow it with a memory that gives the policy access to its recent past at negligible cost: a few register tokens in the action expert summarise the output of the VLM at each frame and persist in the expert’s cache. This memory improves success in simulation and on real robots, and makes oscillations between poses about four times rarer. REKI uses a simple model design and training paradigm, and is optimised to be efficient at training and inference. We release the full code across data curation, training, inference and evaluation, deployment, and all pretrained and finetuned model weights: github.com/kyutai-labs/reki.

我们推出 REKI，一个只用公开数据预训练的 24 亿参数视觉—语言—动作模型（vision-language-action model, VLA），它在性能与模型规模之间确立了新的帕累托前沿：在 LIBERO-Plus、RoboTwin Clean2Rand 和 RoboCasa365 上，只有规模约为它两倍的模型能胜过它。REKI 由一个视觉—语言模型（vision–language model, VLM）和其后的流匹配（flow matching）动作专家组成，在精选的公开机器人数据、第一人称人类数据和视觉—语言数据的混合上预训练。此外，我们给它加了一种记忆，让策略以可以忽略的代价获取自己最近的过去：动作专家中的少量寄存器词元（register token）在每一帧概括 VLM 的输出，并保留在专家的缓存中。这种记忆在仿真和真实机器人上都提高了成功率，并使在两个姿态之间来回振荡的情况减少到约四分之一。REKI 采用简单的模型设计和训练范式，并针对训练和推理效率做了优化。我们开放了涵盖数据整理、训练、推理与评测、部署的全部代码，以及所有预训练和微调后的模型权重：github.com/kyutai-labs/reki。

1 Introduction

1 引言

Vision-language-action models (VLAs) have become an effective paradigm for language-conditioned robot control. A pretrained vision–language model (VLM) encodes camera views and instruction, based on which a smaller action expert predicts the next actions (Black et al., 2024; Physical Intelligence et al., 2025; Bjorck et al., 2025; Shukor et al., 2025; Yuan et al., 2026; Ma et al., 2026).

视觉—语言—动作模型（VLA）已经成为以语言为条件的机器人控制的有效范式。一个预训练的视觉—语言模型（VLM）对相机画面和指令进行编码，一个较小的动作专家据此预测接下来的动作（Black et al., 2024; Physical Intelligence et al., 2025; Bjorck et al., 2025; Shukor et al., 2025; Yuan et al., 2026; Ma et al., 2026）。

The strongest VLAs today are pretrained partly on proprietary data, and neither these data nor the pretraining pipelines are released (Physical Intelligence et al., 2025; Yuan et al., 2026), making it hard to reproduce their results. Since each new model changes architecture, data and training recipe at once, gains cannot be traced back to a single choice. With REKI, we release the full pipeline, from data curation to deployment. REKI has a simple design, is pretrained on public data only and sets a new Pareto frontier between performance and model size. It pairs a VLM with a flow-matching action expert, and a memory mechanism gives the policy access to its recent past.

当今最强的 VLA 部分使用专有数据预训练，而这些数据和预训练流程都没有公开（Physical Intelligence et al., 2025; Yuan et al., 2026），结果因此难以复现。每个新模型同时改变架构、数据和训练配方，收益无法归因到某一项选择上。借助 REKI，我们开放了从数据整理到部署的完整流程。REKI 设计简单，只用公开数据预训练，在性能与模型规模之间确立了新的帕累托前沿。它把一个 VLM 与一个流匹配动作专家组合起来，并通过一种记忆机制让策略获取自己最近的过去。

Preserving past context can improve control (Octo Model Team et al., 2024; Liu et al., 2025; Chi et al., 2025), yet most VLAs predict actions from the current observation alone (Kim et al., 2024; Black et al., 2024; Physical Intelligence et al., 2025; Bjorck et al., 2025; Shukor et al., 2025; Lee et al., 2025; Qu et al., 2025; Liu et al., 2026). Existing ways of incorporating such context, however, often introduce substantial overhead (Fig. 2): feeding past frames to the VLM adds their image tokens to every VLM pass (Jang et al., 2025; Yuan et al., 2026), recurrent methods require sequential training (Cherepanov et al., 2026; Jiang et al., 2026), and others introduce additional, dedicated memory modules to the policy (Shi et al., 2026; Koo et al., 2026).

保留过去的上下文可以改善控制（Octo Model Team et al., 2024; Liu et al., 2025; Chi et al., 2025），但大多数 VLA 只根据当前观测预测动作（Kim et al., 2024; Black et al., 2024; Physical Intelligence et al., 2025; Bjorck et al., 2025; Shukor et al., 2025; Lee et al., 2025; Qu et al., 2025; Liu et al., 2026）。而现有引入这类上下文的方法往往带来可观的额外开销（图 2）：把过去的帧送入 VLM，会让每次 VLM 前向都多出这些帧的图像词元（Jang et al., 2025; Yuan et al., 2026）；循环方法需要顺序训练（Cherepanov et al., 2026; Jiang et al., 2026）；还有一些方法给策略加上额外的专用记忆模块（Shi et al., 2026; Koo et al., 2026）。

REKI summarises past context through a small set of registers (Darcet et al., 2024) in the action expert. The registers of each frame are cached across chunks (Fig. 3). Each frame is thus encoded only once and no separate module, sequential unroll during training or test-time update mechanisms are necessary. This yields clear gains at almost no overhead, and our 2.4B model REKI ranks first among VLAs pretrained on public data on LIBERO-Plus, RoboTwin Clean2Rand and RoboCasa365 (Fig. 1), and generalises to real robots.

REKI 通过动作专家中的一小组寄存器（Darcet et al., 2024）来概括过去的上下文。每一帧的寄存器在各个动作块之间被缓存下来（图 3）。这样每一帧只编码一次，不需要单独的模块，不需要训练时的顺序展开，也不需要测试时的更新机制。这以几乎为零的开销带来了明显收益：我们的 24 亿参数模型 REKI 在 LIBERO-Plus、RoboTwin Clean2Rand 和 RoboCasa365 上，在用公开数据预训练的 VLA 中排名第一（图 1），并能推广到真实机器人上。

Our ablations show that memory consistently improves performance, with the most pronounced gains for the first past frame (Fig. 5). Beyond task success, we find that memory stabilises action generation: without memory, the policy often oscillates between poses across action chunks, while with memory, such oscillations are about four times rarer (Fig. 5).

消融实验表明，记忆能稳定地提升性能，其中第一个过去帧带来的收益最明显（图 5）。除了任务成功率，我们还发现记忆让动作生成更稳定：没有记忆时，策略常常在相邻动作块之间于两个姿态间来回振荡；有了记忆，这种振荡减少到约四分之一（图 5）。

Our contributions are thus three-fold: (i) We introduce REKI, a 2.4B-parameter VLA pretrained on public data, which sets a new Pareto frontier between performance and model size. (ii) We introduce register tokens in the action expert to act as a memory, which can be cached efficiently, and analyse how this reduces the failure modes of the memoryless policy. We hope REKI serves as a strong and accessible baseline for future VLA research. To foster VLA research, we release the model weights and our optimised training and inference code, from data curation to deployment on a real robot.

因此，我们的贡献有三方面：(i) 我们推出 REKI，一个用公开数据预训练的 24 亿参数 VLA，它在性能与模型规模之间确立了新的帕累托前沿。(ii) 我们在动作专家中引入寄存器词元作为记忆，它可以高效缓存，并分析了它如何减少无记忆策略的失败模式。我们希望 REKI 能成为未来 VLA 研究的一个强大且易于使用的基线。为了促进 VLA 研究，我们开放了模型权重以及经过优化的训练和推理代码，从数据整理一直到真实机器人上的部署。

（译注：原文称“三方面”，但只列出了 (i) 和 (ii)，此处照原文翻译。）

Figure 1: Success rate against model size. REKI sets a new Pareto frontier between performance and model size: it scores highest on LIBERO-Plus, and is only outperformed by models about twice its size on RoboTwin Clean2Rand and RoboCasa365. We report full results in Fig. 4.

图 1：成功率与模型规模。 REKI 在性能与模型规模之间确立了新的帕累托前沿：它在 LIBERO-Plus 上得分最高，在 RoboTwin Clean2Rand 和 RoboCasa365 上只被规模约为它两倍的模型超过。完整结果见图 4。

2 Related work

2 相关工作

Open pretraining. Many VLAs release their weights, and often their finetuning code, but not the code or the exact data mixture used for pretraining (Physical Intelligence, 2025; Bjorck et al., 2025; Liu et al., 2026; Zheng et al., 2025; Chen et al., 2026). The strongest VLAs are pretrained partly on proprietary data, collected (Physical Intelligence et al., 2025; Wang et al., 2026a; Liu et al., 2026) or synthesised (Yuan et al., 2026), and several have no public weights (Physical Intelligence et al., 2026; Wang et al., 2026a; Yuan et al., 2026). OpenVLA, Octo and RDT-1B released their full pipeline on public data such as Open X-Embodiment (Kim et al., 2024; Octo Model Team et al., 2024; Liu et al., 2025), followed by SmolVLA, MolmoAct and MolmoAct2, which also releases bimanual data collected by its authors (Shukor et al., 2025; Lee et al., 2025; Fang et al., 2026). These releases have helped the research community pretrain models again, change their data or design, and compare them under the same conditions. Models pretrained on larger in-house data, with more recent design choices, have nonetheless kept a lead on most benchmarks. Building on this line of work, REKI is pretrained on a broad mixture of public robot, egocentric human and vision–language data, and released in full, from data curation to deployment on a real robot. It reaches the best results among VLAs pretrained on public data on three benchmarks (Section 6.1), and is optimised for fast inference (Section 6.3).

开放预训练。 许多 VLA 公开了权重，往往也公开了微调代码，但没有公开预训练所用的代码或确切的数据配比（Physical Intelligence, 2025; Bjorck et al., 2025; Liu et al., 2026; Zheng et al., 2025; Chen et al., 2026）。最强的 VLA 部分使用专有数据预训练，这些数据有的是采集的（Physical Intelligence et al., 2025; Wang et al., 2026a; Liu et al., 2026），有的是合成的（Yuan et al., 2026），其中几个模型没有公开权重（Physical Intelligence et al., 2026; Wang et al., 2026a; Yuan et al., 2026）。OpenVLA、Octo 和 RDT-1B 基于 Open X-Embodiment 等公开数据开放了完整流程（Kim et al., 2024; Octo Model Team et al., 2024; Liu et al., 2025），随后有 SmolVLA、MolmoAct 和 MolmoAct2，其中 MolmoAct2 还公开了作者自己采集的双臂数据（Shukor et al., 2025; Lee et al., 2025; Fang et al., 2026）。这些开放让研究社区可以重新预训练模型、更换数据或设计，并在相同条件下进行比较。不过，用更大规模内部数据、采用更新设计预训练的模型，在大多数基准上仍保持领先。REKI 延续这一路线，在广泛的公开机器人数据、第一人称人类数据和视觉—语言数据混合上预训练，并从数据整理到真实机器人部署完整开放。它在三个基准上取得了用公开数据预训练的 VLA 中的最好结果（第 6.1 节），并针对快速推理做了优化（第 6.3 节）。

Memory in VLAs. Most of these models act on the current observation alone, yet past context helps a policy identify the task state and recover from failed attempts (Shi et al., 2026; Torne et al., 2026; Gao et al., 2026), and methods differ in where they pay for it (Fig. 2). The most direct approach stacks or compresses past frames in the VLM (Yuan et al., 2026; Torne et al., 2026; Physical Intelligence et al., 2026; Jang et al., 2025). Its cost then grows with the window, at least in the vision encoder (Koo et al., 2026). A recurrent state (Cherepanov et al., 2026; Xiao et al., 2026) or fast weights (Jiang et al., 2026) keep the input short, but must be unrolled step by step in training. Others store past features or keyframes in an extra module (Shi et al., 2026; Koo et al., 2026; Mark et al., 2026; Sridhar et al., 2026). Like $\mu$VLA and HAMLET, REKI carries the past in a few learnable tokens, but as registers (Darcet et al., 2024) in the action expert. The VLM encodes each frame once, a training window is one batched pass, and at inference memory adds under 1% to the prediction time of a model without memory ($K\mkern 1.25mu{=}\mkern 1.25mu1$, Section A.1). Without memory, a policy can also switch between modes at each new action chunk (Agarwal et al., 2026; Lian et al., 2026), which is attributed to perceptual aliasing in reinforcement learning (Whitehead & Ballard, 1991; Singh et al., 1994; McCallum, 1996). In REKI, memory makes such switches about four times rarer (Section 6.2), without the extra policy queries of temporal ensembling or bidirectional decoding (Zhao et al., 2023b; Liu et al., 2024b).

VLA 中的记忆。 这些模型大多只根据当前观测行动，但过去的上下文能帮助策略识别任务进展到哪一步，并从失败的尝试中恢复（Shi et al., 2026; Torne et al., 2026; Gao et al., 2026），各种方法的区别在于这份代价付在哪里（图 2）。最直接的做法是在 VLM 中堆叠或压缩过去的帧（Yuan et al., 2026; Torne et al., 2026; Physical Intelligence et al., 2026; Jang et al., 2025）。这样代价会随窗口增长，至少在视觉编码器中如此（Koo et al., 2026）。循环状态（Cherepanov et al., 2026; Xiao et al., 2026）或快速权重（fast weights）（Jiang et al., 2026）让输入保持简短，但训练时必须逐步展开。还有一些方法把过去的特征或关键帧存进额外的模块（Shi et al., 2026; Koo et al., 2026; Mark et al., 2026; Sridhar et al., 2026）。与 $\mu$VLA 和 HAMLET 一样，REKI 用少量可学习的词元承载过去，但这些词元是动作专家中的寄存器（Darcet et al., 2024）。VLM 对每一帧只编码一次，一个训练窗口就是一次批处理前向；推理时，记忆让预测时间比没有记忆的模型（$K\mkern 1.25mu{=}\mkern 1.25mu1$，A.1 节）增加不到 1%。没有记忆时，策略还可能在每个新的动作块切换模式（Agarwal et al., 2026; Lian et al., 2026），在强化学习中这被归因于感知混淆（perceptual aliasing）（Whitehead & Ballard, 1991; Singh et al., 1994; McCallum, 1996）。在 REKI 中，记忆使这种切换减少到约四分之一（第 6.2 节），而且不需要时间集成（temporal ensembling）或双向解码那样额外调用策略（Zhao et al., 2023b; Liu et al., 2024b）。

Figure 2: How VLAs carry the past. Each panel shows one training window of three frames, with a loss under each frame that receives one. (a) Past frames enter the VLM with the current one, in full (Yuan et al., 2026), compressed (Torne et al., 2026; Physical Intelligence et al., 2026) or from a cache (Jang et al., 2025). The VLM takes the whole window as input, and only the last frame is supervised. (b) A state is written at each step and read at the next, in the backbone (Cherepanov et al., 2026; Xiao et al., 2026) or as fast weights in the head (Jiang et al., 2026). Each step needs the output of the previous one, so the window is unrolled step by step. (c) Each frame is encoded on its own, and an added module mixes the memory tokens of past frames into the prediction for the current one. HAMLET (Koo et al., 2026) supervises the last frame only, and MemoryVLA (Shi et al., 2026) every frame, with the bank entries stored without gradient. (d) In REKI, the memory tokens are registers in the action expert. Their keys and values are kept across frames and read by the self-attention already present. The window is one batched pass, and every frame receives a loss.

图 2：VLA 如何承载过去。 每个子图展示一个包含三帧的训练窗口，接受损失的帧下方画出了损失。(a) 过去的帧与当前帧一起进入 VLM，可以是完整的（Yuan et al., 2026）、压缩的（Torne et al., 2026; Physical Intelligence et al., 2026）或来自缓存的（Jang et al., 2025）。VLM 以整个窗口为输入，只有最后一帧受监督。(b) 每一步写入一个状态，下一步读取，状态可以在主干中（Cherepanov et al., 2026; Xiao et al., 2026），也可以是头部的快速权重（Jiang et al., 2026）。每一步都需要上一步的输出，因此窗口要逐步展开。(c) 每一帧单独编码，一个额外模块把过去各帧的记忆词元混入当前帧的预测。HAMLET（Koo et al., 2026）只监督最后一帧，MemoryVLA（Shi et al., 2026）监督每一帧，记忆库条目存储时不带梯度。(d) 在 REKI 中，记忆词元是动作专家中的寄存器。它们的键和值跨帧保留，由本来就有的自注意力读取。窗口是一次批处理前向，每一帧都接受损失。

3 Background

3 背景

Problem formulation. At each decision step $t$ we predict the chunk $\bm{A}_{t}\in\mathbb{R}^{H\times d_{a}}$ of the next $H$ actions of an embodied system, conditioned on the current context $c_{t}$ with

问题表述。 在每个决策步 $t$，我们预测具身系统接下来 $H$ 个动作组成的动作块（chunk）$\bm{A}_{t}\in\mathbb{R}^{H\times d_{a}}$，其条件为当前上下文 $c_{t}$，其中

$$c_{t}=(z_{t},\,o_{t},\,\bm{s}_{t}), \tag{1}$$

which gathers a text prompt $z_{t}$, camera observations $o_{t}$ and the current robot state $\bm{s}_{t}\in\mathbb{R}^{d_{s}}$. More generally, a policy may condition on every context observed so far, $c_{1:t}$.

它包含文本提示 $z_{t}$、相机观测 $o_{t}$ 和当前机器人状态 $\bm{s}_{t}\in\mathbb{R}^{d_{s}}$。更一般地，策略可以以迄今观测到的所有上下文 $c_{1:t}$ 为条件。

The camera observations $o_{t}=\{o_{t}^{1},\dots,o_{t}^{V}\}$ gather the $V$ available views, static or embodied. The robot state $\bm{s}_{t}$ is expressed as absolute joint angles or an absolute end-effector pose, and the actions as absolute joint angles or delta end-effector poses and yield the next $H$ states at a given control frequency. Action and state dimensions ($d_{a}$, $d_{s}$) vary across embodiments and control modes.

相机观测 $o_{t}=\{o_{t}^{1},\dots,o_{t}^{V}\}$ 汇集了 $V$ 个可用视角，可以是固定的，也可以是装在机器人身上的。机器人状态 $\bm{s}_{t}$ 用绝对关节角或绝对末端执行器位姿表示；动作用绝对关节角或末端执行器位姿增量表示，并在给定控制频率下得到接下来的 $H$ 个状态。动作和状态的维度（$d_{a}$、$d_{s}$）随具身形态和控制模式而不同。

3.1 Diffusion policy

3.1 扩散策略

Following prior work (Chi et al., 2025; Black et al., 2024), the action expert generates the chunk by flow matching (Lipman et al., 2023). Write $\bm{A}=\bm{A}_{t}$ for the chunk generated at step $t$ and $c=c_{1:t}$ for the contexts observed up to it; the index $t$ is fixed in this subsection and omitted. Given noise $\bm{\epsilon}\sim\mathcal{N}({\bm{0}},{\bm{I}})$ and a flow time $\tau\sim p(\tau)$, the interpolant $\bm{A}^{\tau}=(1-\tau)\,\bm{A}+\tau\,\bm{\epsilon}$ moves from the clean chunk at $\tau=0$ to noise at $\tau=1$, and the expert $v_{\theta}$ is trained to regress its constant velocity $\bm{\epsilon}-\bm{A}$,

沿用先前的工作（Chi et al., 2025; Black et al., 2024），动作专家用流匹配（Lipman et al., 2023）生成动作块。记 $\bm{A}=\bm{A}_{t}$ 为第 $t$ 步生成的动作块，$c=c_{1:t}$ 为截至此时观测到的上下文；本小节中索引 $t$ 固定，省略不写。给定噪声 $\bm{\epsilon}\sim\mathcal{N}({\bm{0}},{\bm{I}})$ 和流时间 $\tau\sim p(\tau)$，插值 $\bm{A}^{\tau}=(1-\tau)\,\bm{A}+\tau\,\bm{\epsilon}$ 从 $\tau=0$ 时的干净动作块移动到 $\tau=1$ 时的噪声，专家 $v_{\theta}$ 被训练去回归它的恒定速度 $\bm{\epsilon}-\bm{A}$：

$$\ell_{\mathrm{fm}}(\theta;\,\bm{A},c)=\mathbb{E}_{\tau,\bm{\epsilon}}\!\left[\bigl\lVert v_{\theta}(\bm{A}^{\tau},\tau\mid c)-(\bm{\epsilon}-\bm{A})\bigr\rVert_{2}^{2}\right]. \tag{2}$$

The squared error is averaged over the valid steps and action dimensions of the chunk, so that every embodiment has the same weight whatever its number of action dimensions. The action loss is the expectation of this term over the dataset $\mathcal{D}$ of (context, chunk) pairs,

平方误差在动作块的有效步和动作维度上取平均，这样无论动作维度有多少，每种具身形态的权重都相同。动作损失是该项在（上下文，动作块）对组成的数据集 $\mathcal{D}$ 上的期望：

$$\mathcal{L}_{\mathrm{act}}(\theta)=\mathbb{E}_{(c,\bm{A})\sim\mathcal{D}}\!\left[\ell_{\mathrm{fm}}(\theta;\,\bm{A},c)\right]. \tag{3}$$

At inference, $\bm{A}^{1}\sim\mathcal{N}({\bm{0}},{\bm{I}})$ is integrated from $\tau=1$ to 0 in ten Euler steps.

推理时，从 $\bm{A}^{1}\sim\mathcal{N}({\bm{0}},{\bm{I}})$ 出发，用十个欧拉步从 $\tau=1$ 积分到 0。

3.2 Conditioning

3.2 条件输入

The action expert is conditioned on the camera observations, the text instruction and the robot state, through two separate paths. The observations and the instruction are variable-length token sequences. An encoder embeds them into an encoded context

动作专家通过两条独立的路径，以相机观测、文本指令和机器人状态为条件。观测和指令是变长的词元序列。一个编码器把它们嵌入为编码上下文

$$\bm{E}_{t}=\mathrm{Enc}_{\phi}(z_{t},\,o_{t})\in\mathbb{R}^{L_{t}\times d}, \tag{4}$$

whose tokens the expert reads as keys and values in its cross-attention blocks. The state is a single low-dimensional vector. It is pooled into one state embedding $\bm{g}_{t}=\mathrm{Pool}_{\psi}(\bm{s}_{t})\in\mathbb{R}^{d_{e}}$, which modulates the expert through its adaptive normalisation (Peebles & Xie, 2023), alongside the flow-time embedding. Since $\bm{g}_{t}$ does not depend on $\tau$, it is computed once per frame for all Euler steps.

专家在其交叉注意力模块中把这些词元当作键和值来读取。状态是一个低维向量，被汇聚为一个状态嵌入 $\bm{g}_{t}=\mathrm{Pool}_{\psi}(\bm{s}_{t})\in\mathbb{R}^{d_{e}}$，与流时间嵌入一起，通过自适应归一化（Peebles & Xie, 2023）调制专家。由于 $\bm{g}_{t}$ 不依赖于 $\tau$，它每帧只计算一次，供所有欧拉步使用。

Encoded context. Following recent practice (Yuan et al., 2026; Yang et al., 2026), $\bm{E}_{t}$ is the final-layer hidden states of a VLM, read by cross-attention. This keeps the encoder and the action expert independent, and is reported to match or outperform alternatives such as processing the encoded context jointly with the noisy chunk by self-attention (Wang et al., 2026a), or attending the backbone at single or all layers (Black et al., 2024; Physical Intelligence et al., 2025; Shukor et al., 2025; Bjorck et al., 2025). We use Qwen3.5-2B (Qwen Team, 2026) and pack the prompt and the camera views into a single stream (Appendix Fig. 9). The prompt $z_{t}$ carries the language instruction, the embodiment, the control mode, the control frequency and a binned episode length (Appendix A). The embodiment, control-frequency and episode-length fields are dropped with probability 0.15, 0.2 and 0.2 during training. Each view $o_{t}^{v}$ is preceded by a learned camera-role token (side, overhead, wrist left, wrist right) and embedded by the VLM’s visual encoder at native resolution. Views above $448^{2}$ pixels are first downscaled with their aspect ratio preserved. Views are shuffled and randomly dropped during training.

编码上下文。 沿用近来的做法（Yuan et al., 2026; Yang et al., 2026），$\bm{E}_{t}$ 取 VLM 最后一层的隐藏状态，由交叉注意力读取。这让编码器和动作专家保持相互独立；据报道，这种做法与其他方案效果相当或更好，例如用自注意力把编码上下文与带噪动作块一起处理（Wang et al., 2026a），或在单层或所有层上关注主干（Black et al., 2024; Physical Intelligence et al., 2025; Shukor et al., 2025; Bjorck et al., 2025）。我们使用 Qwen3.5-2B（Qwen Team, 2026），把提示和各相机画面打包成一条序列（附录图 9）。提示 $z_{t}$ 包含语言指令、具身形态、控制模式、控制频率和一个分档的回合长度（附录 A）。训练时，具身形态、控制频率和回合长度字段分别以 0.15、0.2 和 0.2 的概率被丢弃。每个视角 $o_{t}^{v}$ 前面加一个学习得到的相机角色词元（侧面、俯视、左腕、右腕），并由 VLM 的视觉编码器以原生分辨率嵌入。超过 $448^{2}$ 像素的画面先在保持宽高比的前提下缩小。训练时，各视角的顺序被打乱，并随机丢弃。

State embedding. $\mathrm{Pool}_{\psi}$ is one linear layer with a GELU per state layout, followed by a shared zero-initialised projection at the expert’s width. Its output modulates the expert through the adaptive normalisation. The state is dropped with probability 0.2 during training.

状态嵌入。 $\mathrm{Pool}_{\psi}$ 对每种状态布局使用一个带 GELU 的线性层，之后接一个共享的、零初始化的投影，投影到专家的宽度。它的输出通过自适应归一化调制专家。训练时，状态以 0.2 的概率被丢弃。

4 REKI model

4 REKI 模型

Building on the policy of Section 3, REKI gives the action expert a memory of prior frames through registers (Section 4.1), which are trained end to end with the flow-matching objective and cached at inference (Section 4.2).

在第 3 节策略的基础上，REKI 通过寄存器让动作专家拥有对先前帧的记忆（第 4.1 节）。寄存器用流匹配目标端到端训练，推理时被缓存（第 4.2 节）。

4.1 Preserving past context

4.1 保留过去的上下文

Standard VLAs condition on the current context $c_{t}$ alone. We give the expert access to the preceding frames through $n_{r}$ memory tokens per frame, implemented as registers (Darcet et al., 2024). The registers of frame $t$ cross-attend the encoded prompt and views of that frame, self-attend the registers of the earlier frames, and persist as keys and values in the expert’s cache, implemented as a rolling window over the $K-1$ preceding frames with $K\mkern 1.25mu{=}\mkern 1.25mu3$ (Section 4.2). Since they do not attend the action tokens and skip the adaptive normalisation, they depend only on the observations and can be shared across frames and denoising steps.

标准 VLA 只以当前上下文 $c_{t}$ 为条件。我们通过每帧 $n_{r}$ 个记忆词元让专家能够访问之前的帧，这些词元以寄存器（Darcet et al., 2024）的形式实现。第 $t$ 帧的寄存器对该帧经过编码的提示和画面做交叉注意力，对更早各帧的寄存器做自注意力，并作为键和值保留在专家的缓存中；缓存实现为覆盖前 $K-1$ 帧的滚动窗口，其中 $K\mkern 1.25mu{=}\mkern 1.25mu3$（第 4.2 节）。由于寄存器不关注动作词元，也跳过自适应归一化，它们只依赖于观测，因此可以在各帧和各去噪步之间共享。

Cost and consistency. The memory is added without changing the encoder, which still processes one frame at a time, so the encoder cost is that of a memoryless policy. During training, the frames are encoded in a single batched pass with no recurrence to unroll, and each frame is supervised on its own chunk (Cherepanov et al., 2026; Jiang et al., 2026), so a window of $K$ frames yields $K$ training signals rather than one (Fig. 2).

代价与一致性。 记忆的加入不改变编码器，编码器仍然一次处理一帧，所以编码器的代价与无记忆策略相同。训练时，各帧在一次批处理前向中编码，没有需要展开的循环，每一帧都用自己的动作块单独监督（Cherepanov et al., 2026; Jiang et al., 2026），因此一个 $K$ 帧的窗口产生 $K$ 个训练信号，而不是一个（图 2）。

Registers. As in Qwen-RobotManip (Yuan et al., 2026), $n_{r}$ learnable register tokens $\bm{M}\in\mathbb{R}^{n_{r}\times d_{e}}$ are prepended to the noisy actions in the expert’s stream, so that frame $t$ enters it as $\bm{X}_{t}=[\,\bm{M}\,;\,\bm{A}_{t}^{\tau}\,]$, with the actions linearly projected to the expert’s width $d_{e}$; we write $\bm{M}_{t}$ for the register tokens of frame $t$. While Qwen-RobotManip discards the registers after each forward pass, we keep their keys and values across frames, giving the action expert access to past observations.

寄存器。 与 Qwen-RobotManip（Yuan et al., 2026）一样，$n_{r}$ 个可学习的寄存器词元 $\bm{M}\in\mathbb{R}^{n_{r}\times d_{e}}$ 被放在专家序列中带噪动作的前面，于是第 $t$ 帧以 $\bm{X}_{t}=[\,\bm{M}\,;\,\bm{A}_{t}^{\tau}\,]$ 的形式进入专家，其中动作被线性投影到专家宽度 $d_{e}$；我们把第 $t$ 帧的寄存器词元记为 $\bm{M}_{t}$。Qwen-RobotManip 在每次前向之后丢弃寄存器，而我们跨帧保留它们的键和值，让动作专家能访问过去的观测。

Attention. Every token of $\bm{X}_{t}$ cross-attends $\bm{E}_{t}$. In self-attention, writing $\mathrm{Attn}(\bm{Q};\,\bm{V})$ for attention with queries $\bm{Q}$ over the tokens of $\bm{V}$ as keys and values,

注意力。 $\bm{X}_{t}$ 的每个词元都对 $\bm{E}_{t}$ 做交叉注意力。在自注意力中，记 $\mathrm{Attn}(\bm{Q};\,\bm{V})$ 为以 $\bm{Q}$ 为查询、以 $\bm{V}$ 中的词元为键和值的注意力，则

$$\bm{M}_{t}\leftarrow\mathrm{Attn}\bigl(\bm{M}_{t};\,[\bm{M}_{1};\dots;\bm{M}_{t}]\bigr),\qquad\bm{A}_{t}^{\tau}\leftarrow\mathrm{Attn}\bigl(\bm{A}_{t}^{\tau};\,[\bm{M}_{1};\dots;\bm{M}_{t};\,\bm{A}_{t}^{\tau}]\bigr). \tag{5}$$

The history thus enters the velocity field through the attention support alone (Fig. 3b).

于是，历史只通过注意力的作用范围进入速度场（图 3b）。

Positions. Self-attention positions are 2D, the frame index and the slot within the frame, encoded by the temporal and spatial axes of the VLM’s multimodal rotary embedding (Appendix A).

位置。 自注意力的位置是二维的：帧索引和帧内槽位，分别用 VLM 多模态旋转位置编码的时间轴和空间轴编码（附录 A）。

Figure 3: REKI. (a) Camera views and instruction are encoded as context $\bm{E}_{t}$, which every expert block reads by cross-attention. The state embedding $\bm{g}_{t}$ and the flow time modulate the actions through adaptive normalisation. The noisy chunk enters the expert’s stream, and the flow loss compares the predicted velocity with its target. The register keys and values of past frames are kept in a cache and read by self-attention, which is the memory of Section 4.1. (b) One of the ten blocks, holding $n_{r}\mkern 1.25mu{=}\mkern 1.25mu32$ registers and the action tokens of the current frame. Arrows run from a query to what it reads. Action tokens read the current and past registers and themselves, and registers never read them.

(a) REKI architecture. (b) One expert block.

图 3：REKI。 (a) 相机画面和指令被编码为上下文 $\bm{E}_{t}$，每个专家模块都通过交叉注意力读取它。状态嵌入 $\bm{g}_{t}$ 和流时间通过自适应归一化调制动作。带噪动作块进入专家序列，流损失把预测的速度与其目标比较。过去各帧的寄存器键和值保存在缓存中，由自注意力读取，这就是第 4.1 节的记忆。(b) 十个模块之一，包含 $n_{r}\mkern 1.25mu{=}\mkern 1.25mu32$ 个寄存器和当前帧的动作词元。箭头从查询指向它读取的对象。动作词元读取当前和过去的寄存器以及它们自己，而寄存器从不读取动作词元。

(a) REKI 架构。(b) 一个专家模块。

4.2 Training and inference

4.2 训练与推理

Objective. A training sample is a window of $K$ frames from one episode, taken at a 1 s stride and indexed from 1, the oldest, to $K$, the most recent. At the start of an episode, a window holds fewer frames. Each frame $t$ has its own chunk $\bm{A}_{t}$, and its contexts $c_{1:t}$ cover the frames up to it, without past states. The loss averages the flow-matching loss of Eq. 2 over the chunks of all frames, each conditioned on the contexts up to its frame,

目标。 一个训练样本是同一回合中以 1 秒为间隔取出的 $K$ 帧组成的窗口，从最早的 1 到最新的 $K$ 编号。在回合开始处，窗口中的帧数较少。每一帧 $t$ 有自己的动作块 $\bm{A}_{t}$，其上下文 $c_{1:t}$ 覆盖截至该帧的所有帧，但不含过去的状态。损失对所有帧的动作块求式 2 的流匹配损失的平均，每个动作块以截至其所在帧的上下文为条件：

$$\mathcal{L}_{\mathrm{act}}(\theta)=\mathbb{E}_{(c_{1:K},\,\bm{A}_{1:K})\,\sim\,\mathcal{D}}\left[\frac{1}{K}\sum_{t=1}^{K}\ell_{\mathrm{fm}}\!\left(\theta;\,\bm{A}_{t},\,c_{1:t}\right)\right]. \tag{6}$$

Since every later frame of the window reads the registers of a frame, these registers receive gradient from all of their losses. At pretraining, the LM head of the VLM is also trained to predict the subtask when one is annotated and the answers of the vision–language data, with a cross-entropy loss of weight 0.1 (Appendix Fig. 9).

由于窗口中每个更晚的帧都会读取某一帧的寄存器，这些寄存器从所有这些损失中获得梯度。预训练时，VLM 的语言模型头也被训练去预测子任务（在有标注时）以及视觉—语言数据的答案，使用权重为 0.1 的交叉熵损失（附录图 9）。

Multi-draw variance reduction. The loss of Eq. 2 is an expectation over the flow time and the noise, and a single $(\tau,\bm{\epsilon})$ draw per frame gives a noisy estimate of it. An extra draw costs one pass of the expert, whereas the context and the registers it reads cost one pass of the encoder, which dominates the cost of a training step. As in Yuan et al. (2026) and Allshire et al. (2026), we therefore estimate the expectation from eight independent draws per frame. The eight noisy chunks share the encoded context and the registers of their frame, so the encoder runs once for all of them. The registers of frame $s$ thus receive gradient from the eight draws of every frame from $s$ to $K$.

多次抽样降低方差。 式 2 的损失是对流时间和噪声的期望，每帧只抽一组 $(\tau,\bm{\epsilon})$ 得到的估计噪声很大。多抽一次只需要多跑一次专家，而它读取的上下文和寄存器要花一次编码器前向，编码器前向在一个训练步的代价中占主导。因此，与 Yuan et al. (2026) 和 Allshire et al. (2026) 一样，我们每帧独立抽八次来估计这个期望。八个带噪动作块共享其所在帧的编码上下文和寄存器，所以编码器对它们只运行一次。于是第 $s$ 帧的寄存器从第 $s$ 帧到第 $K$ 帧每一帧的八次抽样中获得梯度。

Implementation. The frames of a window are packed into a single batch and kept independent in the backbone by cu_seqlens boundaries. In the expert, the self-attention of Eq. 5 is computed by one variable-length FlashAttention call. For this, the keys and values that each group of queries reads are gathered into one sequence: the register tokens $\bm{M}_{1:t}$ for the registers of frame $t$, and $\bm{M}_{1:t}$ followed by the action tokens of one noisy chunk for the queries of that chunk (Appendix Fig. 9(b)).

实现。 一个窗口的各帧被打包进同一个批次，在主干中用 cu_seqlens 边界保持相互独立。在专家中，式 5 的自注意力通过一次变长 FlashAttention 调用计算。为此，每组查询读取的键和值被收集为一条序列：对第 $t$ 帧的寄存器来说是寄存器词元 $\bm{M}_{1:t}$；对某个带噪动作块的查询来说，是 $\bm{M}_{1:t}$ 后接该动作块的动作词元（附录图 9(b)）。

Inference. At inference, each frame needs one encoder pass over the current observation and ten Euler steps on the action tokens. The registers do not read the action tokens and skip adaptive normalisation (Section 4.1), so neither they nor the context depend on the flow time $\tau$ or on the state. The registers of the current frame are thus computed together with the action tokens in the first Euler step, and their keys and values are reused at the nine others. The register keys and values are also kept for the frames that follow, $(K-1)\,n_{r}$ pairs per expert block. The cache is a rolling window, so the registers of the oldest frame are dropped when those of a new frame enter (Appendix Fig. 10). At training, each window of $K$ frames is processed in isolation. At inference, the cached registers were themselves computed with the registers of earlier frames, so information can persist beyond the window. The VLM prefill and the expert are each compiled into a single CUDA graph. With two $256{\times}256$ images and a chunk of ten actions, REKI predicts a chunk in 21.8 ms on one H100 at $K\mkern 1.25mu{=}\mkern 1.25mu3$, the lowest latency among the VLAs compared in Section 6.3. In the measurement of Fig. 5, the VLM prefill takes 9.5 ms and the Euler steps 8.1 ms, and reading the cached registers adds 0.1 ms.

推理。 推理时，每一帧需要对当前观测做一次编码器前向，并在动作词元上做十个欧拉步。寄存器不读取动作词元，也跳过自适应归一化（第 4.1 节），所以它们和上下文都不依赖流时间 $\tau$ 或状态。因此当前帧的寄存器在第一个欧拉步中与动作词元一起计算，其键和值在其余九步中复用。寄存器的键和值还会为后续各帧保留，每个专家模块 $(K-1)\,n_{r}$ 对。缓存是一个滚动窗口，新一帧的寄存器进入时，最早一帧的寄存器被丢弃（附录图 10）。训练时，每个 $K$ 帧的窗口单独处理。推理时，缓存的寄存器本身是借助更早各帧的寄存器计算出来的，所以信息可以保留到窗口之外。VLM 预填充和专家各自被编译为单个 CUDA 图。输入两张 $256{\times}256$ 图像、动作块含十个动作时，REKI 在 $K\mkern 1.25mu{=}\mkern 1.25mu3$ 下在一块 H100 上用 21.8 毫秒预测一个动作块，是第 6.3 节所比较 VLA 中延迟最低的。在图 5 的测量中，VLM 预填充耗时 9.5 毫秒，欧拉步 8.1 毫秒，读取缓存的寄存器增加 0.1 毫秒。

5 Experimental setup

5 实验设置

5.1 Pretraining data

5.1 预训练数据

Robot manipulation. We train on a broad mixture of public teleoperation datasets: RoboSet (Bharadhwaj et al., 2024), Galaxea (Jiang et al., 2025), AgiBot World 2026 (AgiBot World Team, 2026), RoboCOIN (Wu et al., 2025b), RoboMIND (Wu et al., 2025a), RDT-1B (Liu et al., 2025), DROID (Khazatsky et al., 2024), BridgeData V2 (Walke et al., 2023), and twelve datasets of Open X-Embodiment (Open X-Embodiment Collaboration, 2024) including RT-1 (Brohan et al., 2023) and BC-Z (Jang et al., 2022) (Section B.1), complemented by motion-planned simulation from InternData-A1 (Tian et al., 2026) and GR00T-X-Embodiment-Sim (Bjorck et al., 2025) and bimanual YAM demonstrations from ABC (Allshire et al., 2026). These span single-arm, bimanual, mobile, and humanoid platforms across tabletop, in-the-wild, and loco-manipulation settings (Appendix Table 1). Action and state dimensions, control conventions and units vary across sources, and a robot may appear under several conventions. Following Qwen-VLA (Wang et al., 2026a), we keep each dataset’s native control convention rather than remapping to a shared action space, normalise each action dimension to $[-1,1]$ by per-dataset quantiles, and zero-pad actions to a common width under a loss mask. States are absolute, actions are absolute joint positions or delta end-effector poses, and rotations are harmonised to one Euler parameterisation. The prompt and the per-layout state projections of Section 3.2 tell the policy which convention a sample follows. In the spirit of Qwen-RobotManip’s multi-stage filtering (Yuan et al., 2026), we curate every source before training: we keep only the degrees of freedom the policy commands, discard static and near-zero-variance episodes, and correct per-dataset sign conventions. Section B.2 gives the curation stages and the action representation.

机器人操作。 我们在广泛的公开遥操作数据集混合上训练：RoboSet（Bharadhwaj et al., 2024）、Galaxea（Jiang et al., 2025）、AgiBot World 2026（AgiBot World Team, 2026）、RoboCOIN（Wu et al., 2025b）、RoboMIND（Wu et al., 2025a）、RDT-1B（Liu et al., 2025）、DROID（Khazatsky et al., 2024）、BridgeData V2（Walke et al., 2023），以及 Open X-Embodiment（Open X-Embodiment Collaboration, 2024）中的十二个数据集，包括 RT-1（Brohan et al., 2023）和 BC-Z（Jang et al., 2022）（B.1 节）；此外补充了来自 InternData-A1（Tian et al., 2026）和 GR00T-X-Embodiment-Sim（Bjorck et al., 2025）的运动规划仿真数据，以及来自 ABC（Allshire et al., 2026）的 YAM 双臂示范。这些数据涵盖单臂、双臂、移动和人形平台，场景包括桌面、真实野外环境和移动操作（附录表 1）。动作和状态的维度、控制约定和单位因来源而异，同一个机器人也可能以多种约定出现。沿用 Qwen-VLA（Wang et al., 2026a）的做法，我们保留每个数据集原有的控制约定，而不是重新映射到一个共享的动作空间；按每个数据集的分位数把每个动作维度归一化到 $[-1,1]$，并在损失掩码下把动作零填充到统一宽度。状态是绝对量，动作是绝对关节位置或末端执行器位姿增量，旋转统一为一种欧拉角参数化。第 3.2 节的提示和按布局区分的状态投影告诉策略一个样本遵循哪种约定。参照 Qwen-RobotManip 的多阶段过滤（Yuan et al., 2026），我们在训练前整理每个数据源：只保留策略控制的自由度，丢弃静止和方差接近零的回合，并纠正各数据集的符号约定。B.2 节给出了整理的各个阶段和动作表示。

Egocentric human manipulation. We add three egocentric human manipulation datasets, EgoDex (Hoque et al., 2026), EgoVerse (Punamiya et al., 2026) and Xperience (Ropedia, 2026), recorded from head-mounted or hand-held cameras. They contribute ${\sim}350$k episodes and 170M frames. The action target follows Qwen-VLA’s 32D bimanual layout (Wang et al., 2026a), wrist motion and hand articulation per hand, here with the wrist expressed in the camera frame; no proprioceptive state is used, and each is resampled to the pace of robot teleoperation (Yuan et al., 2026). Section B.4 gives the sources, target definition and resampling.

第一人称人类操作。 我们加入了三个第一人称人类操作数据集：EgoDex（Hoque et al., 2026）、EgoVerse（Punamiya et al., 2026）和 Xperience（Ropedia, 2026），用头戴或手持相机录制。它们贡献了约 35 万个回合、1.7 亿帧。动作目标沿用 Qwen-VLA 的 32 维双手布局（Wang et al., 2026a），即每只手的腕部运动和手部关节姿态，这里的腕部在相机坐标系中表示；不使用本体感知状态，每个数据集都重采样到机器人遥操作的节奏（Yuan et al., 2026）。B.4 节给出了数据来源、目标定义和重采样方法。

Vision–language data. We train on the FineVision (Wiedmann et al., 2025) and LLaVA-OneVision-1.5 (An et al., 2025) datasets most relevant for spatial reasoning (Section B.5), as well as targeted spatial-grounding and pointing corpora: RoboPoint (Yuan et al., 2024), RefSpatial (Zhou et al., 2025), and PixMo (Deitke et al., 2025). This auxiliary supervision preserves the backbone’s perception and spatial reasoning, and is mixed in at a small ratio with the robot and egocentric data.

视觉—语言数据。 我们在 FineVision（Wiedmann et al., 2025）和 LLaVA-OneVision-1.5（An et al., 2025）中与空间推理最相关的数据集（B.5 节）上训练，另外加上专门的空间定位和指点语料：RoboPoint（Yuan et al., 2024）、RefSpatial（Zhou et al., 2025）和 PixMo（Deitke et al., 2025）。这种辅助监督保持了主干的感知和空间推理能力，以较小的比例与机器人数据和第一人称数据混合。

5.2 Models

5.2 模型

REKI pairs the Qwen3.5-2B vision–language backbone with an action expert of 10 blocks of width 768, 32 registers per frame and a memory window of $K\mkern 1.25mu{=}\mkern 1.25mu3$ frames. It is pretrained on the mixture above for 200k steps on 32 H100 GPUs with a global batch of 2,880 samples, then finetuned on each benchmark separately. The backbone is trained at $0.1\times$ the learning rate of the expert, and 10% of the training samples are vision–language data. Finetuning steps, node counts and the number of evaluation rollouts per task are in Appendix Table 3, the remaining hyperparameters in Appendix Table 2. Rows marked scratch skip the pretraining and finetune the initialised model directly on the benchmark.[^1]

REKI 把 Qwen3.5-2B 视觉—语言主干与一个动作专家组合起来，专家有 10 个宽度为 768 的模块，每帧 32 个寄存器，记忆窗口为 $K\mkern 1.25mu{=}\mkern 1.25mu3$ 帧。它在上述混合数据上用 32 块 H100 GPU、全局批大小 2,880 个样本预训练 20 万步，然后在每个基准上分别微调。主干的学习率是专家的 $0.1\times$，10% 的训练样本是视觉—语言数据。微调步数、节点数和每个任务的评测次数见附录表 3，其余超参数见附录表 2。标为 scratch 的行跳过预训练，直接在基准上微调初始化的模型。[^1]

[^1] Baseline numbers are taken from published results (Appendix Table 7). The training budgets published for these baselines are in Appendix Table 9. Models in italics in the tables are not released at the time of writing.

[^1] 基线数字取自已发表的结果（附录表 7）。这些基线公布的训练预算见附录表 9。表中斜体的模型在撰写本文时尚未公开。

5.3 Visual data augmentation

5.3 视觉数据增强

We apply data augmentation to make the model robust to visual variations: colour jittering, mild random cropping, random camera dropout and shuffled camera order. When finetuning on RoboTwin, RoboCasa365 and RoboCasa-GR1, we add a mirroring augmentation, which extends the horizontal flip common in image classification to robot data. With probability 0.5, a sample is replaced by its left-right mirror image. The images are flipped, the state and the actions are reflected through the same plane, the arms of a two-arm robot are swapped, and the prompt marks the sample as mirrored (Section C.2). The augmentations and other training settings, such as the state dropout rate, have a large effect on robustness: on LIBERO-Plus, disabling camera dropout or the image augmentations lowers the success under camera-viewpoint changes by more than 20 points, and disabling state dropout lowers the success under changes of the initial robot state by 8.1 points (Section D.2).

我们使用数据增强让模型对视觉变化保持鲁棒：颜色抖动、轻微随机裁剪、随机丢弃相机和打乱相机顺序。在 RoboTwin、RoboCasa365 和 RoboCasa-GR1 上微调时，我们加入镜像增强，把图像分类中常见的水平翻转推广到机器人数据上。样本以 0.5 的概率被替换为其左右镜像：图像翻转，状态和动作关于同一平面做反射，双臂机器人的两只手臂互换，提示中标记该样本为镜像（C.2 节）。这些增强和其他训练设置（例如状态丢弃率）对鲁棒性影响很大：在 LIBERO-Plus 上，关闭相机丢弃或图像增强会使相机视角变化下的成功率下降超过 20 个百分点，关闭状态丢弃会使机器人初始状态变化下的成功率下降 8.1 个百分点（D.2 节）。

6 Experiments

6 实验

6.1 Results

6.1 结果

Benchmarks. LIBERO (Liu et al., 2023a) has four suites of ten single-arm tabletop tasks, and LIBERO-Plus (Fei et al., 2025) perturbs its 40 tasks along seven dimensions, from the camera viewpoint and the initial robot state to lighting and object layout. The models are finetuned on the four suites and evaluated on both. RoboTwin 2.0 (Chen et al., 2025b) has 50 tasks on the dual-arm AgileX ALOHA, in clean scenes and in scenes with a randomised background, lighting, clutter and table height. For the out-of-distribution evaluation (Clean2Rand; Yuan et al., 2026), the models are finetuned on clean scenes only and evaluated under six conditions: on clean scenes (Easy), under each randomisation alone and under all four together (Hard). The Clean2Rand score is the mean over these six conditions. RoboCasa365 (Nasiriany et al., 2026) has kitchen tasks for a Franka arm on a mobile base. The models are trained on the human demonstrations of its 300 pretraining tasks and evaluated on 50 target tasks: 18 atomic tasks, 16 composite tasks seen in training and 16 composite tasks unseen in training. RoboCasa-GR1 (Bjorck et al., 2025) has 24 tabletop tasks for the Fourier GR-1 humanoid, with 1000 demonstrations per task. We compare mainly with VLAs of similar complexity, which pair a VLM with an action expert, such as $\pi_{0.5}$, GR00T, ABot-M0 and InternVLA-A1.5. As a gauge of the current upper bound on these benchmarks, we also report larger models pretrained on in-house data, Qwen-RobotManip, RLDX-1 and Xiaomi-Robotics-1, and the world-action model ABot-M0.5, built on a video generation model. They are trained under different conditions, with more or proprietary data or a video model, so they are a reference rather than a like-for-like comparison. Figure 4 summarises the results across benchmarks, and Appendix Tables 4, 5 and 6 list them in full.

基准。 LIBERO（Liu et al., 2023a）有四套、每套十个单臂桌面任务；LIBERO-Plus（Fei et al., 2025）沿七个维度扰动这 40 个任务，从相机视角、机器人初始状态到光照和物体布局。模型在这四套任务上微调，在两个基准上评测。RoboTwin 2.0（Chen et al., 2025b）有 50 个双臂 AgileX ALOHA 任务，场景分为干净场景，以及背景、光照、杂物和桌面高度随机化的场景。在分布外评测（Clean2Rand；Yuan et al., 2026）中，模型只在干净场景上微调，在六种条件下评测：干净场景（Easy）、每种随机化单独施加，以及四种随机化同时施加（Hard）。Clean2Rand 得分是这六种条件的平均。RoboCasa365（Nasiriany et al., 2026）是在移动底座上的 Franka 机械臂执行的厨房任务。模型在其 300 个预训练任务的人类示范上训练，在 50 个目标任务上评测：18 个原子任务、16 个训练中见过的组合任务和 16 个训练中没见过的组合任务。RoboCasa-GR1（Bjorck et al., 2025）有 24 个傅利叶 GR-1 人形机器人的桌面任务，每个任务 1000 条示范。我们主要与复杂度相近、把 VLM 与动作专家组合的 VLA 比较，例如 $\pi_{0.5}$、GR00T、ABot-M0 和 InternVLA-A1.5。为了衡量这些基准目前的上限，我们也报告了用内部数据预训练的更大模型 Qwen-RobotManip、RLDX-1 和 Xiaomi-Robotics-1，以及建立在视频生成模型之上的世界—动作模型 ABot-M0.5。它们的训练条件不同，使用了更多数据、专有数据或视频模型，因此只作为参照，而不是同等条件下的比较。图 4 汇总了各基准的结果，附录表 4、5 和 6 列出了完整结果。

Figure 4: Benchmark results. Out of distribution, on LIBERO-Plus (a) and RoboTwin Clean2Rand (c), REKI leads the compared VLAs pretrained on public data; in distribution, on RoboTwin (b), RoboCasa365 (d) and RoboCasa-GR1 (e), it is competitive with larger models. Hatched bars mark models pretrained on in-house data and outlined bars world-action models. Each bar gives the overall score and the ticks the subset scores. Full results for (a) to (d) are in Appendix Tables 4, 5 and 6.

图 4：基准结果。 在分布外的 LIBERO-Plus (a) 和 RoboTwin Clean2Rand (c) 上，REKI 在所比较的用公开数据预训练的 VLA 中领先；在分布内的 RoboTwin (b)、RoboCasa365 (d) 和 RoboCasa-GR1 (e) 上，它与更大的模型不相上下。带斜线的柱表示用内部数据预训练的模型，描边柱表示世界—动作模型。每根柱给出总分，刻度标记给出各子集得分。(a) 至 (d) 的完整结果见附录表 4、5 和 6。

Out of distribution. REKI stands out most when the test conditions differ from training. On LIBERO-Plus and Clean2Rand, it ranks first among the VLAs pretrained on public data, despite having the fewest parameters (2.4B, Figs. 4 and 4). It leads the next of these models by 5.6 points on LIBERO-Plus (InternVLA-A1.5, 84.8) and by 10.1 points on Clean2Rand (ABot-M0, 55.7). On the composite tasks of RoboCasa365 unseen in training, it is on par with Qwen-RobotManip (14.1 against 14.9) and behind Xiaomi-Robotics-1 (32.1), both larger and pretrained on in-house data.

分布外。 当测试条件与训练不同时，REKI 最为突出。在 LIBERO-Plus 和 Clean2Rand 上，它在用公开数据预训练的 VLA 中排名第一，尽管参数最少（24 亿，图 4 和图 4）。它在 LIBERO-Plus 上领先这类模型中的第二名 5.6 个百分点（InternVLA-A1.5，84.8），在 Clean2Rand 上领先 10.1 个百分点（ABot-M0，55.7）。在 RoboCasa365 训练中没见过的组合任务上，它与 Qwen-RobotManip 持平（14.1 对 14.9），落后于 Xiaomi-Robotics-1（32.1），后两者规模都更大，并用内部数据预训练。

In distribution. When trained and evaluated on the same tasks, REKI is competitive with larger models. On RoboCasa365, it reaches 38.4, ahead of the larger RLDX-1 (36.0) and Qwen-RobotManip (35.9), both pretrained on in-house data (Fig. 4), and on RoboCasa-GR1 it reaches 58.6, on par with the larger RLDX-1 (58.7) and ABot-M0 (58.3) (Fig. 4). On LIBERO, it reaches 97.5, where most models lie between 97.6 and 99.4. REKI is weakest on RoboTwin, where it reaches 85.9 and 84.6, below the similarly sized InternVLA-A1.5 (93.3 / 93.0), which is supervised by an additional video generation model during training and uses a more complex, multi-stage pretraining (Fig. 4).

分布内。 在相同任务上训练和评测时，REKI 能与更大的模型竞争。在 RoboCasa365 上，它达到 38.4，领先于同样用内部数据预训练、规模更大的 RLDX-1（36.0）和 Qwen-RobotManip（35.9）（图 4）；在 RoboCasa-GR1 上，它达到 58.6，与更大的 RLDX-1（58.7）和 ABot-M0（58.3）持平（图 4）。在 LIBERO 上，它达到 97.5，而多数模型在 97.6 到 99.4 之间。REKI 在 RoboTwin 上最弱，得分为 85.9 和 84.6，低于规模相近的 InternVLA-A1.5（93.3 / 93.0）；后者在训练中额外受一个视频生成模型监督，并使用更复杂的多阶段预训练（图 4）。

Pretraining gains. Trained from scratch on each benchmark, REKI reaches 52.0 on Clean2Rand, on par with Qwen-RobotManip (52.3) and ahead of StarVLA (36.6), which both have larger VLMs (4.5B and 4.4B). REKI is ahead of both under the Hard condition (26.3 against 22.6 and 10.6). On LIBERO-Plus it leads with 86.1, ahead of Qwen-RobotManip (78.3) and StarVLA (75.0). Pretraining on public data benefits REKI most out of distribution, and in particular under the RoboTwin randomisations (Appendix Tables 5 and 4). In distribution, it adds 2.9 and 0.8 points on RoboTwin and 1.0 on LIBERO. Out of distribution, it adds 13.8 points on Clean2Rand and 4.3 on LIBERO-Plus, and the gain grows where REKI without pretraining fails most, with 30.3 points under clutter and 22.4 under the Hard condition. On LIBERO-Plus, the gain is also largest under perturbed robot states (11.7).

预训练带来的收益。 在每个基准上从头训练时，REKI 在 Clean2Rand 上达到 52.0，与 Qwen-RobotManip（52.3）持平，领先于 StarVLA（36.6），而后两者的 VLM 都更大（45 亿和 44 亿）。在 Hard 条件下，REKI 领先两者（26.3 对 22.6 和 10.6）。在 LIBERO-Plus 上，它以 86.1 领先，高于 Qwen-RobotManip（78.3）和 StarVLA（75.0）。用公开数据预训练对 REKI 在分布外帮助最大，尤其是在 RoboTwin 的各种随机化下（附录表 5 和 4）。在分布内，预训练在 RoboTwin 上带来 2.9 和 0.8 个百分点，在 LIBERO 上带来 1.0 个百分点。在分布外，它在 Clean2Rand 上带来 13.8 个百分点，在 LIBERO-Plus 上带来 4.3 个百分点；而且未经预训练的 REKI 失败越多的地方，收益越大：杂物条件下提升 30.3 个百分点，Hard 条件下提升 22.4 个百分点。在 LIBERO-Plus 上，机器人状态受扰动时收益也最大（11.7）。

Additional data or supervision. REKI is outperformed by several larger and more recent models that add data or supervision to the vanilla recipe, with large in-house data collections (Xiaomi Robotics Team, 2026) or video generation models (Song et al., 2026; Zhang et al., 2026). Similarly, InternVLA-A1.5 uses the same VLM as REKI, with an additional video generation model, a larger action expert (24 layers vs 10 for REKI) and a multi-stage pretraining, and is ahead in distribution on RoboTwin (93.3 vs 85.9). Out of distribution, it is behind on LIBERO-Plus (84.8 vs 90.4). Qwen-RobotManip, pretrained with additional synthetic data, gains more from pretraining than REKI (16.2 vs 13.8 points on Clean2Rand). REKI reaches its lead out of distribution without such additions, and we expect it to benefit similarly from additional data, be it real (Xiaomi Robotics Team, 2026), synthetic (Yuan et al., 2026) or generated by a world model (Kim et al., 2026; GigaBrain Team et al., 2026).

额外的数据或监督。 REKI 被几个规模更大、更新的模型超过，它们在基本配方之上增加了数据或监督，例如大规模内部数据采集（Xiaomi Robotics Team, 2026）或视频生成模型（Song et al., 2026; Zhang et al., 2026）。类似地，InternVLA-A1.5 与 REKI 使用同一个 VLM，但另加了一个视频生成模型、一个更大的动作专家（24 层，REKI 为 10 层）和多阶段预训练，在 RoboTwin 分布内领先（93.3 对 85.9）。在分布外，它在 LIBERO-Plus 上落后（84.8 对 90.4）。用额外合成数据预训练的 Qwen-RobotManip，从预训练中得到的收益比 REKI 更多（Clean2Rand 上 16.2 对 13.8 个百分点）。REKI 不靠这些额外手段就在分布外取得领先，我们预计它也能从额外数据中得到类似的收益，无论是真实数据（Xiaomi Robotics Team, 2026）、合成数据（Yuan et al., 2026），还是世界模型生成的数据（Kim et al., 2026; GigaBrain Team et al., 2026）。

6.2 Ablation

6.2 消融实验

Figure 5: Memory window, oscillation and inference cost. REKI trained with $K\mkern 1.25mu{=}\mkern 1.25mu3$ and evaluated with $K\mkern 1.25mu{=}\mkern 1.25mu3,2,1$ ($3\to 3$, $3\to 2$, $3\to 1$), and a memoryless model ($1\to 1$). (a) On RoboCasa365, the black line gives the average. (b) RoboTwin Clean episodes in which an arm reverses its motion at three or more consecutive chunks (50 tasks, 20 matched seeds each). (c) Two-arm tasks move both arms in at least 80% of the successful episodes with memory, one-arm tasks a single arm. The three other tasks are left out. (d) On one H100. The numbers give the difference to $K\mkern 1.25mu{=}\mkern 1.25mu1$ in ms.

图 5：记忆窗口、振荡与推理代价。 以 $K\mkern 1.25mu{=}\mkern 1.25mu3$ 训练、分别以 $K\mkern 1.25mu{=}\mkern 1.25mu3,2,1$ 评测的 REKI（$3\to 3$、$3\to 2$、$3\to 1$），以及无记忆模型（$1\to 1$）。(a) RoboCasa365 上，黑线表示平均值。(b) RoboTwin Clean 中某只手臂在连续三个或更多动作块上反转运动方向的回合（50 个任务，每个任务 20 个匹配种子）。(c) 双臂任务指在有记忆的成功回合中至少 80% 都移动了两只手臂的任务，单臂任务指只移动一只手臂的任务。其余三个任务未计入。(d) 在一块 H100 上测得。数字为与 $K\mkern 1.25mu{=}\mkern 1.25mu1$ 相比的差值，单位毫秒。

Memory window. Figure 5 varies the memory window of REKI at finetuning and at evaluation, on LIBERO-Plus after finetuning on LIBERO, on RoboTwin Clean and on RoboCasa365. Finetuned and evaluated with three frames ($3\to 3$), REKI reaches 90.4 on LIBERO-Plus, 73.4 on RoboTwin and 38.4 on RoboCasa365, against 88.9, 67.9 and 33.0 without memory ($1\to 1$). Most of the gain already comes with the first past frame ($3\to 2$: 90.1, 73.6 and 39.3). As every frame of a training window is supervised (Eq. 6), the model finetuned with memory also learns to act without it. Evaluated without memory ($3\to 1$), it performs close to the model finetuned at $K\mkern 1.25mu{=}\mkern 1.25mu1$ (88.9, 67.2 and 34.9), so the gain comes mainly from reading the past frames at inference, not from training on windows.

记忆窗口。 图 5 在微调和评测时改变 REKI 的记忆窗口，评测基准为在 LIBERO 上微调后的 LIBERO-Plus、RoboTwin Clean 和 RoboCasa365。以三帧微调和评测时（$3\to 3$），REKI 在 LIBERO-Plus 上达到 90.4，在 RoboTwin 上 73.4，在 RoboCasa365 上 38.4；没有记忆时（$1\to 1$）分别为 88.9、67.9 和 33.0。大部分收益在加入第一个过去帧时就已获得（$3\to 2$：90.1、73.6 和 39.3）。由于训练窗口中的每一帧都受监督（式 6），用记忆微调的模型也学会了在没有记忆时行动。在没有记忆的条件下评测时（$3\to 1$），它的表现接近以 $K\mkern 1.25mu{=}\mkern 1.25mu1$ 微调的模型（88.9、67.2 和 34.9），因此收益主要来自推理时读取过去的帧，而不是来自在窗口上训练。

Oscillation. For each arm, we compare how it moves during one chunk with how it moved during the previous one, ignoring small movements. Figure 5 counts the episodes in which an arm moves in roughly the opposite direction at three or more consecutive chunks, that is, it goes back and forth at least twice (Fig. 7). We set this criterion on 16 of the 50 tasks and confirmed the effect of memory on the other 34. Fewer than 5% of these episodes succeed. With the same weights, removing the past frames at evaluation raises their share from 4.1% to 14.8%, and one past frame already brings it down to 4.6%. Trained without memory, REKI oscillates in 18.6% of the episodes, and in half of those that fail. The difference is largest in two-arm tasks (Fig. 5).

振荡。 对每只手臂，我们把它在一个动作块内的运动与它在上一个动作块内的运动作比较，忽略小幅运动。图 5 统计了某只手臂在连续三个或更多动作块上朝大致相反方向运动（也就是至少来回两次，图 7）的回合数。我们在 50 个任务中的 16 个上确定了这一判据，并在其余 34 个任务上确认了记忆的效果。这类回合中成功的不到 5%。使用相同权重时，评测时去掉过去的帧会使这类回合的占比从 4.1% 升到 14.8%，而一个过去帧就能把它降到 4.6%。在没有记忆的情况下训练时，REKI 在 18.6% 的回合中出现振荡，在失败回合中占一半。双臂任务中的差异最大（图 5）。

Why memory helps. A policy without memory maps the current observation to the next chunk. If two poses A and B each look like demonstration states that continue towards the other, the chunk predicted at A leads to B, and the chunk predicted at B leads back to A. We attribute the oscillation to this form of perceptual aliasing, under which memoryless policies are known to fail (Whitehead & Ballard, 1991; Singh et al., 1994; McCallum, 1996). A short history resolves the ambiguity, since a window that has just come from B does not match demonstrations that move towards B. In two-arm tasks, the most frequent cycle is an arm swap, in which one arm reaches while the other retracts (65 against 1 of 1000 episodes without and with memory, top row of Fig. 7). It typically starts a few seconds after a release. Memory does not rescue all these episodes: where the policy without memory oscillates, the policy with memory succeeds on 45% of the seeds.

记忆为什么有用。 没有记忆的策略把当前观测映射到下一个动作块。如果两个姿态 A 和 B 看起来都像是朝着对方继续运动的示范状态，那么在 A 预测的动作块会把手臂带到 B，在 B 预测的动作块又会把它带回 A。我们把这种振荡归因于这种形式的感知混淆，已知无记忆策略在感知混淆下会失败（Whitehead & Ballard, 1991; Singh et al., 1994; McCallum, 1996）。一段短的历史就能消除歧义，因为刚从 B 过来的窗口与朝 B 运动的示范对不上。在双臂任务中，最常见的循环是手臂交替：一只手臂伸出，另一只缩回（无记忆和有记忆时，1000 个回合中分别出现 65 次和 1 次，图 7 上排）。它通常在一次松开物体后几秒钟出现。记忆并不能挽救所有这些回合：在无记忆策略出现振荡的种子上，有记忆策略的成功率为 45%。

6.3 Inference latency

6.3 推理延迟

Figure 6: Inference latency (H100), from the raw observation to the actions of one chunk, with LIBERO inputs, without compilation (left) and compiled (right). Dashed bars are compiled code that we built. ABot-M0.5 runs with its RoboCasa365 weights, the only ones released, and with its own LIBERO setting, which takes two $128{\times}128$ images.

图 6：推理延迟（H100），从原始观测到一个动作块的动作，使用 LIBERO 输入，左为未编译，右为编译后。虚线柱是我们自行构建的编译代码。ABot-M0.5 使用其 RoboCasa365 权重（唯一公开的权重）和它自己的 LIBERO 设置运行，输入为两张 $128{\times}128$ 图像。

Protocol. We measure the latency of one call, from the raw observation to the unnormalised actions of one chunk, including preprocessing and the transfers between host and device. This latency bounds how quickly a policy can react to a new observation and how often it can replan, whatever the number of actions it predicts. In practice, policies replan well before the end of a long chunk, typically in under a second, since executing more actions open loop makes them slower to react (Black et al., 2024; Chi et al., 2025; Liu et al., 2024b). Each model predicts one chunk from two $256{\times}256$ images ($128{\times}128$ for ABot-M0.5), the robot state and a fixed prompt, at batch size one, with its released checkpoint and inference code. Each value in Fig. 6 is the median over 320 chunks (80 for ABot-M0.5) and three repetitions, each on its own GPU of a node reserved for the timing (Section D.4).

测量方法。 我们测量一次调用的延迟，从原始观测到一个动作块的未归一化动作，包括预处理以及主机与设备之间的数据传输。无论策略预测多少个动作，这一延迟都决定了它对新观测的反应速度和重新规划的频率上限。实际中，策略远在一个长动作块执行完之前就重新规划，通常不到一秒，因为开环执行更多动作会让反应变慢（Black et al., 2024; Chi et al., 2025; Liu et al., 2024b）。每个模型以批大小 1，用其公开的检查点和推理代码，根据两张 $256{\times}256$ 图像（ABot-M0.5 为 $128{\times}128$）、机器人状态和一个固定提示预测一个动作块。图 6 中的每个值是 320 个动作块（ABot-M0.5 为 80 个）和三次重复的中位数，每次重复在专门预留用于计时的节点上的一块独立 GPU 上运行（D.4 节）。

Compilation. Most of the compared models are released without a compiled path, or with one that can be further optimised. Without compilation, all models are slow (Fig. 6, left): REKI, $\pi_{0.5}$ with openpi (Physical Intelligence, 2025) and ABot-M0 (Yang et al., 2026) take 118 to 139 ms per chunk, InternVLA-A1.5 (Ma et al., 2026) 352 ms, and X-VLA (0.9B; Zheng et al., 2025) 51 ms. In most of these runs, the GPU is busy less than half of the time, so compiling the prediction, which removes most of the host time, makes it two to seven times faster (Fig. 6, right). We compile REKI entirely into CUDA graphs, including its memory (Section A.1). For a fair comparison, we also compile X-VLA, ABot-M0, InternVLA-A1.5 and ABot-M0.5 (Chen et al., 2026) without changing their computation, and report both their released path and ours.

编译。 所比较的模型大多公开时没有编译路径，或者有一条仍可进一步优化的编译路径。不编译时，所有模型都很慢（图 6 左）：REKI、使用 openpi 的 $\pi_{0.5}$（Physical Intelligence, 2025）和 ABot-M0（Yang et al., 2026）每个动作块需要 118 到 139 毫秒，InternVLA-A1.5（Ma et al., 2026）需要 352 毫秒，X-VLA（9 亿参数；Zheng et al., 2025）需要 51 毫秒。在这些运行的大多数中，GPU 有一半以上的时间处于空闲，所以编译预测过程（它消除了大部分主机端时间）能带来二到七倍的加速（图 6 右）。我们把 REKI 完全编译为 CUDA 图，包括它的记忆（A.1 节）。为了公平比较，我们也在不改变计算的前提下编译了 X-VLA、ABot-M0、InternVLA-A1.5 和 ABot-M0.5（Chen et al., 2026），并同时报告它们公开的路径和我们的路径。

Results. With compilation, REKI is the fastest of the compared models (Fig. 6, right): it predicts a chunk in 21.8 ms, against 25.5 ms for X-VLA, 33.8 ms for $\pi_{0.5}$ with openpi, 46.1 ms for ABot-M0 and 51.3 ms for InternVLA-A1.5. Its GPU time per chunk, 16.5 ms, is on par with that of X-VLA (16.1 ms) and below that of the other models, which need at least 25.3 ms. InternVLA-A1.5 uses the same VLM as REKI, and the difference lies in its action expert, which runs at each of the ten Euler steps: with 0.5B parameters in 24 layers against 0.2B in 10, its Euler steps take 34.9 ms against 8.2 ms for REKI (Appendix Table 10). ABot-M0.5 generates future video latents before it predicts actions, and takes 2.7 s per chunk without compilation and 1.4 s with the CUDA graphs that we built.

结果。 编译后，REKI 是所比较模型中最快的（图 6 右）：它用 21.8 毫秒预测一个动作块，X-VLA 为 25.5 毫秒，使用 openpi 的 $\pi_{0.5}$ 为 33.8 毫秒，ABot-M0 为 46.1 毫秒，InternVLA-A1.5 为 51.3 毫秒。它每个动作块的 GPU 时间为 16.5 毫秒，与 X-VLA（16.1 毫秒）相当，低于其他模型，后者至少需要 25.3 毫秒。InternVLA-A1.5 与 REKI 使用同一个 VLM，差别在于它的动作专家，专家在十个欧拉步中的每一步都要运行：它有 5 亿参数、24 层，而 REKI 是 2 亿参数、10 层，它的欧拉步耗时 34.9 毫秒，REKI 为 8.2 毫秒（附录表 10）。ABot-M0.5 在预测动作之前先生成未来的视频隐变量，每个动作块不编译时需要 2.7 秒，用我们构建的 CUDA 图后需要 1.4 秒。

7 Real robot experiments

7 真实机器人实验

Figure 7: Oscillation without memory. Two RoboTwin rollouts of REKI trained without memory, stacking three blocks. Frames are one chunk apart (1 s), and A and B mark the two poses the arm alternates between. In the top row, the two arms swap at every chunk, one reaching while the other retracts. In the bottom row, a single arm moves back and forth. The blocks do not move in either row.

图 7：没有记忆时的振荡。 无记忆训练的 REKI 在 RoboTwin 中堆叠三个方块的两次执行。相邻画面相隔一个动作块（1 秒），A 和 B 标出手臂来回切换的两个姿态。上排中，两只手臂在每个动作块都互换角色，一只伸出时另一只缩回。下排中，一只手臂来回移动。两排中方块都没有移动。

Experimental setup. We use a 6-DoF UR5 robotic arm with a Weiss WSG50 gripper and UMI-style parallel jaws. The action loop runs at 15 Hz. The robot operates over a 1 m${\times}$1 m tabletop workspace observed by a fixed Orbbec Femto Mega RGB camera, see Fig. 8, left. Observations include an RGB image, and a 7D absolute joint state. Actions are specified as 7D absolute joint state.

实验设置。 我们使用一台 6 自由度 UR5 机械臂，配 Weiss WSG50 夹爪和 UMI 式平行夹指。动作循环以 15 Hz 运行。机器人在 1 m${\times}$1 m 的桌面工作区内操作，由一台固定的 Orbbec Femto Mega RGB 相机观测，见图 8 左。观测包括一张 RGB 图像和 7 维绝对关节状态。动作也以 7 维绝对关节状态给出。

Tasks and protocol. We evaluate performance across five manipulation tasks, and report task success rates and partial subtask success rates. Tasks span diverse action primitives, including pick, place, stack, and sweep. The tasks and partial success scores are defined in Appendix E. We collect 50 training episodes per task with a Meta Quest controller, and evaluate on 10 test episodes per task, with randomised initial object placements and distractors in each episode.

任务与评测方法。 我们在五个操作任务上评测性能，报告任务成功率和部分子任务成功率。任务涵盖多种动作基元，包括抓取、放置、堆叠和清扫。任务和部分成功的计分见附录 E。我们用 Meta Quest 手柄为每个任务采集 50 条训练回合，每个任务在 10 个测试回合上评测，每个回合中物体的初始摆放和干扰物都是随机的。

Baselines. We finetune the pretrained checkpoints of REKI and $\pi_{0.5}$ on the five tasks for 3,000 optimiser steps, with a global batch of 360 samples. Both models use a learning rate of $5\times 10^{-5}$, held constant after warmup. The action chunk has 50 steps (3.3 s) for $\pi_{0.5}$, fixed by its pretraining, and 30 steps (2 s) for REKI. In both models, only the last four vision encoder blocks are trained. The backbone is trained at $0.1\times$ the learning rate of the action expert. REKI is finetuned with a memory window of $K\mkern 1.25mu{=}\mkern 1.25mu3$ and without memory ($K\mkern 1.25mu{=}\mkern 1.25mu1$). $\pi_{0.5}$ has no memory.

基线。 我们把 REKI 和 $\pi_{0.5}$ 的预训练检查点在五个任务上微调 3,000 个优化器步，全局批大小为 360 个样本。两个模型都使用 $5\times 10^{-5}$ 的学习率，预热后保持不变。动作块长度对 $\pi_{0.5}$ 是 50 步（3.3 秒），由其预训练决定；对 REKI 是 30 步（2 秒）。两个模型都只训练视觉编码器的最后四个模块。主干的学习率是动作专家的 $0.1\times$。REKI 分别以 $K\mkern 1.25mu{=}\mkern 1.25mu3$ 的记忆窗口和无记忆（$K\mkern 1.25mu{=}\mkern 1.25mu1$）微调。$\pi_{0.5}$ 没有记忆。

Results. Across the five tasks, REKI outperforms $\pi_{0.5}$ by 7 to 37.5 percentage points (Fig. 8, right). Removing the memory leaves performance on the shorter-horizon tasks mainly unchanged (65 against 65 and 57.5 against 57.5), but consistently degrades it on the longer-horizon tasks, where REKI outperforms its memoryless variant by 12 to 17.5 percentage points, supporting that memory matters most when tasks require tracking progress over many steps. For qualitative examples, see Fig. 12.

结果。 在五个任务上，REKI 比 $\pi_{0.5}$ 高出 7 到 37.5 个百分点（图 8 右）。去掉记忆对较短时程任务的表现基本没有影响（65 对 65，57.5 对 57.5），但在较长时程任务上一致地降低表现，在这些任务上 REKI 比其无记忆版本高 12 到 17.5 个百分点，这支持了一个看法：当任务需要在多步中追踪进度时，记忆最为重要。定性示例见图 12。

8 Discussion

8 讨论

Figure 8: Real-robot experiments. Left: the UR5 arm with a WSG50 gripper, observed by a fixed Orbbec Femto Mega RGB camera, with the task objects and distractors. Right: task score of REKI with and without memory and of $\pi_{0.5}$ (Physical Intelligence et al., 2025) on two shorter- and three longer-horizon tasks, over 10 episodes each. Solid bars give full successes, hatched bars partial credit.

图 8：真实机器人实验。 左：配 WSG50 夹爪的 UR5 机械臂，由一台固定的 Orbbec Femto Mega RGB 相机观测，图中有任务物体和干扰物。右：有记忆和无记忆的 REKI 以及 $\pi_{0.5}$（Physical Intelligence et al., 2025）在两个较短时程和三个较长时程任务上的任务得分，每个任务 10 个回合。实心柱为完全成功，斜线柱为部分得分。

A vanilla VLA. REKI follows a standard VLA recipe: a VLM’s last hidden states are read by a flow-matching action expert, pretrained in a single stage on public robot, egocentric human and vision–language data, and then finetuned on each benchmark. It uses no proprietary data (Yuan et al., 2026; Kim et al., 2026, cf.), no world model (Chen et al., 2026, cf.) and no additional training stages. Even so, it can compete with, and often outperform, VLAs of similar size pretrained on in-house data, and it leads the compared VLAs pretrained on public data out of distribution (Fig. 4), where pretraining adds 13.8 points on Clean2Rand and 4.3 on LIBERO-Plus (Section 6.1). Many of the strongest entries on current leaderboards take the other route: they are world-action models built on video generation models (Chen et al., 2026; Wang et al., 2026b; Zhang et al., 2026; Song et al., 2026) or VLAs pretrained on large proprietary data (Xiaomi Robotics Team, 2026; GigaBrain Team et al., 2026). These models set the current state of the art, but they can be costly to run (Section 6.3). Their pretraining is also hard to reproduce, which makes it difficult to separate the effects of data scale, model size, modelling choices and additional supervision such as world modelling. We intend REKI to serve as an open reference on which each of these factors can be tested in isolation.

一个朴素的 VLA。 REKI 遵循标准的 VLA 配方：VLM 的最后隐藏状态由一个流匹配动作专家读取，在公开机器人数据、第一人称人类数据和视觉—语言数据上单阶段预训练，再在每个基准上微调。它不使用专有数据（参见 Yuan et al., 2026; Kim et al., 2026），不使用世界模型（参见 Chen et al., 2026），也没有额外的训练阶段。即便如此，它仍能与用内部数据预训练、规模相近的 VLA 竞争，而且往往更好；在分布外，它在所比较的用公开数据预训练的 VLA 中领先（图 4），预训练在 Clean2Rand 上带来 13.8 个百分点，在 LIBERO-Plus 上带来 4.3 个百分点（第 6.1 节）。当前排行榜上许多最强的模型走的是另一条路：它们是建立在视频生成模型上的世界—动作模型（Chen et al., 2026; Wang et al., 2026b; Zhang et al., 2026; Song et al., 2026），或是用大规模专有数据预训练的 VLA（Xiaomi Robotics Team, 2026; GigaBrain Team et al., 2026）。这些模型代表了当前最好水平，但运行成本可能很高（第 6.3 节）。它们的预训练也难以复现，这让人很难区分数据规模、模型规模、建模选择以及世界建模之类额外监督各自的作用。我们希望 REKI 能作为一个开放的参照，在它上面可以单独检验这些因素中的每一个。

An efficient open baseline. We release REKI with its full pipeline, from data curation to deployment, and the checkpoints of every benchmark, so that new architectures, data mixtures or training objectives can be pretrained and compared against it under fixed conditions. The simplicity of the recipe and the efficiency of the code keep such comparisons affordable: finetuning REKI takes 8 H100 GPU-hours on our real-robot tasks and 40 to 400 on the benchmarks (Tables 11 and 3). The memory adds no cost at training (Section 4.2) and less than 1% of the inference time, so REKI is an efficient starting point for studying memory in embodied AI. In the released code, REKI runs entirely as CUDA graphs, memory included, and predicts an action chunk from the LIBERO inputs in 21.8 ms on one H100. It stays the fastest of the compared models even when their released code is compiled (Section 6.3).

一个高效的开放基线。 我们连同完整流程（从数据整理到部署）以及每个基准的检查点一起开放 REKI，让新的架构、数据配比或训练目标能在固定条件下预训练，并与它比较。配方的简单和代码的高效让这样的比较负担得起：在我们的真实机器人任务上微调 REKI 需要 8 个 H100 GPU 小时，在各基准上需要 40 到 400 个（表 11 和表 3）。记忆在训练时不增加代价（第 4.2 节），推理时增加的时间不到 1%，所以 REKI 是研究具身智能中记忆问题的一个高效起点。在开放的代码中，REKI 完全以 CUDA 图运行，记忆也包括在内，在一块 H100 上根据 LIBERO 输入预测一个动作块需要 21.8 毫秒。即使其他模型的公开代码也经过编译，它仍是所比较模型中最快的（第 6.3 节）。

Why inference speed matters. Inference speed matters not only at deployment, where it bounds how quickly a policy can react (Section 6.3), but also for training. VLAs are increasingly post-trained with reinforcement learning, in simulation (Li et al., 2025; Chen et al., 2025a; Zang et al., 2026) or in learned world models (Zhu et al., 2025; Sharma et al., 2026), where the policy is queried for every action chunk of every rollout, so that its inference time is multiplied across the whole training run. At 21.8 ms per chunk, REKI offers a strong open starting point for such post-training and lowers the barrier to it, a step towards making it accessible beyond the few groups with large compute budgets.

推理速度为什么重要。 推理速度不仅在部署时重要（它决定策略的反应速度上限，第 6.3 节），对训练也很重要。VLA 越来越多地用强化学习做后训练，可以在仿真中（Li et al., 2025; Chen et al., 2025a; Zang et al., 2026），也可以在学到的世界模型中（Zhu et al., 2025; Sharma et al., 2026）；在这些过程中，每次执行的每个动作块都要调用一次策略，所以它的推理时间会在整个训练过程中成倍累积。REKI 每个动作块 21.8 毫秒，为这类后训练提供了一个强大的开放起点，降低了门槛，让它不再只属于少数拥有大量算力的团队。

Memory. In REKI, a few register tokens in the action expert summarise each frame and stay in the expert’s cache, so the VLM still sees each frame only once. Even this short memory helps: with three frames, success rises by 1.5 to 5.5 points on LIBERO-Plus, RoboTwin and RoboCasa365, and by 12 to 17.5 points on the longer real-robot tasks. Most of the gain comes from a single past frame, which in particular reduces repetitive oscillations between two poses, suggesting that a little temporal context is enough to tell apart observations that look alike (Section 6.2). A memory of a few seconds therefore appears sufficient for most tasks in standard benchmarks. Whether the registers can also carry the much longer context of tasks that depend on events far in the past remains to be tested.

记忆。 在 REKI 中，动作专家里的少量寄存器词元概括每一帧，并保留在专家的缓存中，所以 VLM 对每一帧仍然只看一次。即使是这样短的记忆也有帮助：用三帧时，成功率在 LIBERO-Plus、RoboTwin 和 RoboCasa365 上提高 1.5 到 5.5 个百分点，在较长时程的真实机器人任务上提高 12 到 17.5 个百分点。大部分收益来自单个过去帧，它尤其减少了在两个姿态之间的反复振荡，这说明一点时间上下文就足以区分看起来相似的观测（第 6.2 节）。因此，对于标准基准中的大多数任务，几秒钟的记忆似乎就够了。至于寄存器能否承载依赖于很久以前事件的任务所需的更长上下文，还有待检验。

References

参考文献

Acharya et al. (2019) Manoj Acharya, Kushal Kafle, and Christopher Kanan. TallyQA: Answering complex counting questions. In Proc. AAAI Conf. Artif. Intell. , 2019.

Agarwal et al. (2026) Abhinav Agarwal, Adam Wei, Taylan Kargin, Michael Zeng, Cole Becker, Arif Kerem Dayı, Pablo Parrilo, Asuman Ozdaglar, and Russ Tedrake. Training and evaluating diffusion policies with long context lengths. arXiv:2606.16447 , 2026.

AgiBot World Team (2026) AgiBot World Team. AgiBot World 2026. Hugging Face dataset, 2026. URL https://huggingface.co/datasets/agibot-world/AgiBotWorld2026 .

Allshire et al. (2026) Arthur Allshire, Himanshu Gaurav Singh, Ritvik Singh, Adam Rashid, Hongsuk Choi, David McAllister, Justin Yu, Yiyuan Chen, Huang Huang, Pieter Abbeel, et al. Scalable behavior cloning with open data, training, and evaluation. arXiv:2606.27375 , 2026.

An et al. (2025) Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, et al. LLaVA-OneVision-1.5: Fully open framework for democratized multimodal training. arXiv:2509.23661 , 2025.

Bharadhwaj et al. (2024) Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, and Vikash Kumar. RoboAgent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In Proc. IEEE Int. Conf. Robot. Autom. , 2024.

Biten et al. (2019) Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marçal Rusinol, Ernest Valveny, C. V. Jawahar, and Dimosthenis Karatzas. Scene text visual question answering. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2019.

Bjorck et al. (2025) Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, et al. GR00T N1: An open foundation model for generalist humanoid robots. arXiv:2503.14734 , 2025.

Black et al. (2024) Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π 0 \pi_{0} : A vision-language-action flow model for general robot control. arXiv:2410.24164 , 2024.

Brohan et al. (2023) Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems , 2023.

Cadene et al. (2024) Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, Steven Palma, Pepijn Kooijmans, Michel Aractingi, Mustafa Shukor, Dana Aubakirova, Martino Russi, Francesco Capuano, Caroline Pascal, Jade Choghari, Khalil Meftah, Maxime Ellerbach, Jess Moss, and Thomas Wolf. LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch. https://github.com/huggingface/lerobot , 2024.

Carter (2024) Jimmy Carter. TextOCR-GPT4V. Hugging Face dataset, 2024. URL https://huggingface.co/datasets/jimmycarter/textocr-gpt4v .

Chen et al. (2024a) Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. ALLaVA: Harnessing GPT4V-synthesized data for lite vision-language models. arXiv:2402.11684 , 2024a.

Chen et al. (2025a) Kang Chen, Zhihao Liu, Tonghe Zhang, Zhen Guo, Si Xu, Hao Lin, Hongzhi Zang, Xiang Li, Quanlu Zhang, Zhaofei Yu, et al. π rl \pi_{\texttt{rl}} : Online RL fine-tuning for flow-based vision-language-action models. arXiv:2510.25889 , 2025a.

Chen et al. (2024b) Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. ShareGPT4V: Improving large multi-modal models with better captions. In Proc. Eur. Conf. Comput. Vis. , 2024b.

Chen et al. (2026) Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, et al. ABot-M0.5: Unified mobility-and-manipulation world action model. arXiv:2607.00678 , 2026.

Chen et al. (2025b) Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv:2506.18088 , 2025b.

Cherepanov et al. (2026) Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky, Aydar Bulatov, Artem Pshenitsyn, Yuri Kuratov, Alexey Skrynnik, Aleksandr I Panov, and Alexey K Kovalev. μ \mu VLA: On recurrent memory for partially observable manipulation in VLA models. arXiv:2606.12497 , 2026.

Chi et al. (2025) Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. Int. J. Robot. Res. , 44(10-11):1684–1704, 2025.

Chng et al. (2019) Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, and Errui Ding. ICDAR2019 robust reading challenge on arbitrary-shaped text (RRC-ArT). In Proc. Int. Conf. Doc. Anal. Recognit. , 2019.

Darcet et al. (2024) Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In Int. Conf. Learn. Represent. , 2024.

Deitke et al. (2025) Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and PixMo: Open weights and open data for state-of-the-art vision-language models. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2025.

Fang et al. (2026) Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, et al. MolmoAct2: Action reasoning models for real-world deployment. arXiv:2605.02881 , 2026.

Fei et al. (2025) Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, et al. LIBERO-Plus: In-depth robustness analysis of vision-language-action models. arXiv:2510.13626 , 2025.

Gao et al. (2026) Yihuai Gao, Jeff Jinyun Liu, Shuang Li, and Shuran Song. Gated memory policy: In-context memorization and adaptation. arXiv:2604.18933 , 2026.

GigaBrain Team et al. (2026) GigaBrain Team, Angen Ye, Axiang Sun, Can Jin, Chenxi Cheng, Chong Shi, Dengke Shang, Dingqian Zhang, Guan Huang, Guangqiang Wang, et al. GigaBrain-0.7: Scaling embodied foundation models to emergent capabilities with a three-system architecture. arXiv:2608.15875 , 2026.

Goyal et al. (2017) Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2017.

Hessel et al. (2022) Jack Hessel, Jena D. Hwang, Jae Sung Park, Rowan Zellers, Chandra Bhagavatula, Anna Rohrbach, Kate Saenko, and Yejin Choi. The abduction of Sherlock Holmes: A dataset for visual abductive reasoning. In Proc. Eur. Conf. Comput. Vis. , 2022.

Hoque et al. (2026) Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. EgoDex: Learning dexterous manipulation from large-scale egocentric video. In Int. Conf. Learn. Represent. , 2026.

Hudson & Manning (2019) Drew A. Hudson and Christopher D. Manning. GQA: A new dataset for real-world visual reasoning and compositional question answering. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2019.

Jang et al. (2022) Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. BC-Z: Zero-shot task generalization with robotic imitation learning. In Proc. Conf. Robot Learn. , pp. 991–1002, 2022.

Jang et al. (2025) Huiwon Jang, Sihyun Yu, Heeseung Kwon, Hojin Jeon, Younggyo Seo, and Jinwoo Shin. ContextVLA: Vision-language-action model with amortized multi-frame context. arXiv:2510.04246 , 2025.

Jhamtani & Berg-Kirkpatrick (2018) Harsh Jhamtani and Taylor Berg-Kirkpatrick. Learning to describe differences between pairs of similar images. In Proc. Conf. Empir. Methods Nat. Lang. Process. , 2018.

Jiang et al. (2025) Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, and Hang Zhao. Galaxea open-world dataset and G0 dual-system VLA model. arXiv:2509.00576 , 2025.

Jiang et al. (2026) Yunfan Jiang, Yevgen Chebotar, Ruijie Zheng, Fengyuan Hu, Yunhao Ge, Jimmy Wu, Tianyuan Dai, Scott Reed, Li Fei-Fei, Yuke Zhu, et al. RoboTTT: Context scaling for robot policies. arXiv:2607.15275 , 2026.

Karakus (2024) Hazal Karakus. MSCOCO-ControlNet-Canny-Less-Colors. Hugging Face dataset, 2024. URL https://huggingface.co/datasets/hazal-karakus/mscoco-controlnet-canny-less-colors .

Khazatsky et al. (2024) Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. In Robotics: Science and Systems , 2024.

Kim et al. (2026) Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsung Jang, Daewon Choi, Dongsu Han, et al. RLDX-1 technical report. arXiv:2605.03269 , 2026.

Kim et al. (2024) Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. OpenVLA: An open-source vision-language-action model. arXiv:2406.09246 , 2024.

Koo et al. (2026) Myungkyu Koo, Daewon Choi, Taeyoung Kim, Kyungmin Lee, Changyeon Kim, Younggyo Seo, and Jinwoo Shin. HAMLET: Switch your vision-language-action model into a history-aware policy. In Int. Conf. Learn. Represent. , 2026.

LAION (2023) LAION. GPT4V-Dataset. Hugging Face dataset, 2023. URL https://huggingface.co/datasets/laion/gpt4v-dataset .

Laurençon et al. (2024) Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? In Adv. Neural Inf. Process. Syst. , 2024.

Lee et al. (2025) Jason Lee, Jiafei Duan, Haoquan Fang, Yuquan Deng, Shuo Liu, Boyang Li, Bohan Fang, Jieyu Zhang, Yi Ru Wang, Sangho Lee, et al. MolmoAct: Action reasoning models that can reason in space. arXiv:2508.07917 , 2025.

Lerner et al. (2022) Paul Lerner, Olivier Ferret, Camille Guinaudeau, Hervé Le Borgne, Romaric Besançon, José G. Moreno, and Jesús Lovón Melgarejo. ViQuAE, a dataset for knowledge-based visual question answering about named entities. In Proc. Int. ACM SIGIR Conf. Res. Dev. Inf. Retr. , 2022.

Li et al. (2025) Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Zhaohui Yang, Kaiyan Zhang, Xuekai Zhu, Yuchen Zhang, Tianxing Chen, Ganqu Cui, et al. SimpleVLA-RL: Scaling VLA training via reinforcement learning. arXiv:2509.09674 , 2025.

Li et al. (2024) Xiaotong Li, Fan Zhang, Haiwen Diao, Yueze Wang, Xinlong Wang, and Lingyu Duan. DenseFusion-1M: Merging vision experts for comprehensive multimodal perception. In Adv. Neural Inf. Process. Syst. , 2024.

Li et al. (2023) Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan L. Yuille. Super-CLEVR: A virtual benchmark to diagnose domain robustness in visual reasoning. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2023.

Lian et al. (2026) Shijie Lian, Bin Yu, Xiaopeng Lin, Zhaolong Shen, Laurence Tianruo Yang, Yurun Jin, Haishan Liu, Changti Wu, Hang Yuan, Cong Huang, and Kai Chen. IntentVLA: Short-horizon intent modeling for aliased robot manipulation. arXiv:2605.14712 , 2026.

Lindström & Abraham (2022) Adam Dahlgren Lindström and Savitha Sam Abraham. CLEVR-Math: A dataset for compositional language, visual and mathematical reasoning. arXiv:2208.05358 , 2022.

Lipman et al. (2023) Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In Int. Conf. Learn. Represent. , 2023.

Liu et al. (2023a) Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Adv. Neural Inf. Process. Syst. , 2023a.

Liu et al. (2023b) Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Trans. Assoc. Comput. Linguist. , 11:635–651, 2023b.

Liu et al. (2024a) Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In Int. Conf. Learn. Represent. , 2024a.

Liu et al. (2023c) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Adv. Neural Inf. Process. Syst. , 2023c.

Liu et al. (2025) Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-1B: A diffusion foundation model for bimanual manipulation. In Int. Conf. Learn. Represent. , 2025.

Liu et al. (2026) Songming Liu, Bangguo Li, Kai Ma, Lingxuan Wu, Hengkai Tan, Xiao Ouyang, Hang Su, and Jun Zhu. RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization. arXiv:2602.03310 , 2026.

Liu et al. (2024b) Yuejiang Liu, Jubayer Ibn Hamid, Annie Xie, Yoonho Lee, Maximilian Du, and Chelsea Finn. Bidirectional decoding: Improving action chunking via guided test-time sampling. arXiv:2408.17355 , 2024b.

Lu et al. (2022) Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Adv. Neural Inf. Process. Syst. , 2022.

Lu et al. (2024) Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, and Bill Yuchen Lin. WildVision: Evaluating vision-language models in the wild with human preferences. In Adv. Neural Inf. Process. Syst. , 2024.

Luo et al. (2026) Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-H0.7: A latent world-action model from egocentric videos. arXiv:2605.00078 , 2026.

Luo et al. (2024) Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Minzheng Wang, Pengpeng Zeng, Lianli Gao, et al. MMEvol: Empowering multimodal large language models with Evol-Instruct. arXiv:2409.05840 , 2024.

Ma et al. (2026) Haoxiang Ma, Junhao Cai, Xiaoxu Xu, Hao Li, Yuyin Yang, Yang Tian, Jiafei Cao, Hongrui Zhu, Zherui Qiu, Zhaxizhuoma, et al. InternVLA-A1.5: Unifying understanding, latent foresight, and action for compositional generalization. arXiv:2607.04988 , 2026.

Mark et al. (2026) Max Sobol Mark, Jacky Liang, Maria Attarian, Chuyuan Fu, Debidatta Dwibedi, Dhruv Shah, and Aviral Kumar. BPP: Long-context robot imitation learning by focusing on key history frames. arXiv:2602.15010 , 2026.

McCallum (1996) Andrew Kachites McCallum. Reinforcement Learning with Selective Perception and Hidden State . PhD thesis, University of Rochester, 1996.

Mishra et al. (2019) Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. OCR-VQA: Visual question answering by reading text in images. In Proc. Int. Conf. Doc. Anal. Recognit. , 2019.

Nasiriany et al. (2026) Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. RoboCasa365: A large-scale simulation framework for training and benchmarking generalist robots. In Int. Conf. Learn. Represent. , 2026. arXiv:2603.04356.

Octo Model Team et al. (2024) Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. In Robotics: Science and Systems , 2024.

Open X-Embodiment Collaboration (2024) Open X-Embodiment Collaboration. Open X-Embodiment: Robotic learning datasets and RT-X models. In Proc. IEEE Int. Conf. Robot. Autom. , 2024.

OpenGVLab (2024) OpenGVLab. ShareGPT-4o, 2024. URL https://sharegpt4o.github.io/ .

Peebles & Xie (2023) William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2023.

Physical Intelligence (2025) Physical Intelligence. openpi: Open-source models and packages for robotics. https://github.com/Physical-Intelligence/openpi , 2025.

Physical Intelligence et al. (2025) Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π 0.5 \pi_{0.5} : A vision-language-action model with open-world generalization. arXiv:2504.16054 , 2025.

Physical Intelligence et al. (2026) Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, George Bokinsky, Shihao Cao, Thomas Charbonnier, et al. π 0.7 \pi_{0.7} : A steerable generalist robotic foundation model with emergent capabilities. arXiv:2604.15483 , 2026.

Pi et al. (2024) Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, and Tong Zhang. Image textualization: An automatic framework for creating accurate and detailed image descriptions. arXiv:2406.07502 , 2024.

Pont-Tuset et al. (2020) Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, and Vittorio Ferrari. Connecting vision and language with localized narratives. In Proc. Eur. Conf. Comput. Vis. , 2020.

Punamiya et al. (2026) Ryan Punamiya, Simar Kareer, Zeyi Liu, Josh Citron, Ri-Zhao Qiu, Xiongyi Cai, Alexey Gavryushin, Jiaqi Chen, Davide Liconti, Lawrence Y. Zhu, et al. EgoVerse: An egocentric human dataset for robot learning from around the world. arXiv:2604.07607 , 2026.

Qu et al. (2025) Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, Jiayuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. SpatialVLA: Exploring spatial representations for visual-language-action model. In Robotics: Science and Systems , 2025.

Quattoni & Torralba (2009) Ariadna Quattoni and Antonio Torralba. Recognizing indoor scenes. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2009.

Qwen Team (2026) Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https://qwen.ai/blog?id=qwen3.5 .

Ren et al. (2015) Mengye Ren, Ryan Kiros, and Richard Zemel. Exploring models and data for image question answering. In Adv. Neural Inf. Process. Syst. , 2015.

Ropedia (2026) Ropedia. Xperience-10M: A large-scale egocentric multimodal dataset of human experience. Hugging Face dataset, 2026. URL https://huggingface.co/datasets/ropedia-ai/xperience-10m .

Schwenk et al. (2022) Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-OKVQA: A benchmark for visual question answering using world knowledge. In Proc. Eur. Conf. Comput. Vis. , 2022.

Shao et al. (2019) Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2019.

Sharma et al. (2026) Ansh Kumar Sharma, Yixiang Sun, Ninghao Lu, Yunzhe Zhang, Jiarao Liu, and Sherry Yang. World-Gymnast: Training robots with reinforcement learning in a world model. arXiv:2602.02454 , 2026.

Shi et al. (2026) Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. MemoryVLA: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. In Int. Conf. Learn. Represent. , 2026.

Shridhar et al. (2021) Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning. In Int. Conf. Learn. Represent. , 2021.

Shukor et al. (2025) Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics. arXiv:2506.01844 , 2025.

Sidorov et al. (2020) Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. TextCaps: A dataset for image captioning with reading comprehension. In Proc. Eur. Conf. Comput. Vis. , 2020.

Sima et al. (2024) Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. DriveLM: Driving with graph visual question answering. In Proc. Eur. Conf. Comput. Vis. , 2024.

Siméoni et al. (2025) Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. DINOv3. arXiv:2508.10104 , 2025.

Singh et al. (2019) Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards VQA models that can read. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2019.

Singh et al. (1994) Satinder P. Singh, Tommi Jaakkola, and Michael I. Jordan. Learning without state-estimation in partially observable Markovian decision processes. In Proc. Int. Conf. Mach. Learn. , pp. 284–292, 1994.

Song et al. (2026) Wenxuan Song, Jiayi Chen, Jingbo Wang, Shuai Zhou, Xicheng Gong, Zehua Fan, Ziyang Zhou, Junwu E, Haodong Yan, Fuhao Li, et al. UniWAM: Unified world-action model. arXiv:2610.02054 , 2026.

Sridhar et al. (2026) Ajay Sridhar, Jennifer Pan, Satvik Sharma, and Chelsea Finn. MemER: Scaling up memory for robot control via experience retrieval. In Int. Conf. Learn. Represent. , 2026.

StarVLA Community (2026) StarVLA Community. StarVLA: A Lego-like codebase for vision-language-action model developing. arXiv:2604.05014 , 2026.

Suhr et al. (2017) Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi. A corpus of natural language for visual reasoning. In Proc. Annu. Meet. Assoc. Comput. Linguist. , 2017.

Tian et al. (2026) Yang Tian, Yuyin Yang, Yiman Xie, Zetao Cai, Xu Shi, Ning Gao, Hangxu Liu, Xuekun Jiang, Zherui Qiu, Feng Yuan, et al. InternData-A1: Pioneering high-fidelity synthetic data for pre-training generalist policy. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2026.

Tong et al. (2024) Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal LLMs. In Adv. Neural Inf. Process. Syst. , 2024.

Torne et al. (2026) Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, Allen Z Ren, Haohuan Wang, Jiaming Tang, Kyle Stachowicz, et al. MEM: Multi-scale embodied memory for vision language action models. arXiv:2603.03596 , 2026.

Tu et al. (2023) Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, and Cihang Xie. How many unicorns are in this image? A safety evaluation benchmark for vision LLMs. arXiv:2311.16101 , 2023.

Veit et al. (2016) Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. COCO-Text: Dataset and benchmark for text detection and recognition in natural images. arXiv:1601.07140 , 2016.

Walke et al. (2023) Homer Rich Walke, Kevin Black, Tony Z. Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. BridgeData V2: A dataset for robot learning at scale. In Proc. Conf. Robot Learn. , pp. 1723–1736, 2023.

Wang et al. (2023) Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. To see is to believe: Prompting GPT-4V for better visual instruction tuning. arXiv:2311.07574 , 2023.

Wang et al. (2026a) Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, et al. Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments. arXiv:2605.30280 , 2026a.

Wang et al. (2026b) Tianheng Wang, Zhou Xie, Heng Jia, Jianhua Xu, Tong Zhang, and Kaicheng Yu. An action is worth one patch: Unified world-action modeling with PatchWAM. arXiv:2609.25961 , 2026b.

Wang et al. (2020) Xinyu Wang, Yuliang Liu, Chunhua Shen, Chun Chet Ng, Canjie Luo, Lianwen Jin, Chee Seng Chan, Anton van den Hengel, and Liangwei Wang. On the general value of evidence, and bilingual scene-text visual question answering. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2020.

Weyand et al. (2020) Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google Landmarks Dataset v2: A large-scale benchmark for instance-level recognition and retrieval. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2020.

Whitehead & Ballard (1991) Steven D. Whitehead and Dana H. Ballard. Learning to perceive and act by trial and error. Machine Learning , 7(1):45–83, 1991.

Wiedmann et al. (2025) Luis Wiedmann, Orr Zohar, Amir Mahla, Xiaohan Wang, Rui Li, Thibaud Frere, Leandro von Werra, Aritra Roy Gosthipaty, and Andrés Marafioti. FineVision: Open data is all you need. arXiv:2510.17269, 2025.

Wu et al. (2025a) Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation. In Robotics: Science and Systems , 2025a.

Wu et al. (2025b) Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, et al. RoboCOIN: An open-sourced bimanual robotic data COllection for INtegrated manipulation. arXiv:2511.17441 , 2025b.

Xiao et al. (2026) Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, and Xiaoyuan Yu. AVA-VLA: Improving vision-language-action models with active visual attention. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2026.

Xiaomi Robotics Team (2026) Xiaomi Robotics Team. Xiaomi-Robotics-1: Scaling vision-language-action models with over 100K hours of real-world trajectories. arXiv:2607.15330 , 2026.

Xu et al. (2025) Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. LLaVA-CoT: Let vision language models reason step-by-step. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2025.

Xu et al. (2024) Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, and Lifu Huang. Vision-Flan: Scaling human-labeled tasks in visual instruction tuning. arXiv:2402.11690 , 2024.

Yang et al. (2019) Kaiyu Yang, Olga Russakovsky, and Jia Deng. SpatialSense: An adversarially crowdsourced benchmark for spatial relation recognition. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2019.

Yang et al. (2026) Yandan Yang, Shuang Zeng, Tong Lin, Xinyuan Chang, Dekang Qi, Junjin Xiao, Haoyun Liu, Ronghan Chen, Yuzhi Chen, Dongjie Huo, et al. ABot-M0: VLA foundation model for robotic manipulation with action manifold learning. arXiv:2602.11236 , 2026.

Yuan et al. (2026) Haoqi Yuan, Zhixuan Liang, Anzhe Chen, Ye Wang, Haoyang Li, Pei Lin, Yiyang Huang, Zixing Lei, Tong Zhang, Jiazhao Zhang, et al. Qwen-RobotManip technical report: Alignment unlocks scale for robotic manipulation foundation models. arXiv:2606.17846 , 2026.

Yuan et al. (2024) Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. RoboPoint: A vision-language model for spatial affordance prediction in robotics. In Proc. Conf. Robot Learn. , 2024.

Zang et al. (2026) Hongzhi Zang, Mingjie Wei, Si Xu, Yongji Wu, Zhen Guo, Yuanqing Wang, Hao Lin, Peihong Wang, Liangzhi Shi, Yuqing Xie, et al. RLinf-VLA: A unified and efficient framework for reinforcement learning of vision-language-action models. In Robotics: Science and Systems , 2026.

Zhai et al. (2023) Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2023.

Zhang et al. (2026) Fei Zhang, Zhaochong An, Duncan Frost, Yikai Wang, Pengfei Liu, Ya Zhang, Michal Drozdzal, and Amir Bar. World action modeling with progressive visual planning. arXiv:2610.02508 , 2026.

Zhao et al. (2023a) Bo Zhao, Boya Wu, Muyang He, and Tiejun Huang. SVIT: Scaling up visual instruction tuning. arXiv:2307.04087 , 2023a.

Zhao et al. (2023b) Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems , 2023b.

Zheng et al. (2025) Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, Ya-Qin Zhang, Jiangmiao Pang, Jingjing Liu, Tai Wang, and Xianyuan Zhan. X-VLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv:2510.10274 , 2025.

Zhong et al. (2026) Linqing Zhong, Yi Liu, Yifei Wei, Ziyu Xiong, Maoqing Yao, Si Liu, and Guanghui Ren. ACoT-VLA: Action chain-of-thought for vision-language-action models. arXiv:2601.11404 , 2026.

Zhou et al. (2025) Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, and Shanghang Zhang. RoboRefer: Towards spatial referring with reasoning in vision-language models for robotics. In Adv. Neural Inf. Process. Syst. , 2025.

Zhu et al. (2025) Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. WMPO: World model-based policy optimization for vision-language-action models. arXiv:2511.09515 , 2025.

Zhu et al. (2016) Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. Visual7W: Grounded question answering in images. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. , 2016.

Appendix

附录

Appendix A Model details

附录 A 模型细节

Figure 9: (a) The VLM encoder. It feeds the expert of Fig. 3 the encoded context. A vision–language model encodes the views and the prompt; its final-layer hidden states are the encoded context and its LM head predicts the subtask. (b) The registers and noisy action tokens of a frame cross-attend the context of that frame and self-attend the registers of the $K$ frames up to it, so the past reaches the current chunk only through the registers, and only these are kept for later frames.

(a) The VLM encoder. (b) Attention pattern, $K\mkern 1.25mu{=}\mkern 1.25mu3$.

图 9：(a) VLM 编码器。它为图 3 中的专家提供编码上下文。视觉—语言模型对各视角画面和提示进行编码；其最后一层隐藏状态就是编码上下文，其语言模型头预测子任务。(b) 一帧的寄存器和带噪动作词元对该帧的上下文做交叉注意力，对截至该帧的 $K$ 帧寄存器做自注意力，所以过去只能通过寄存器到达当前动作块，也只有寄存器会为后续各帧保留。

(a) VLM 编码器。(b) 注意力模式，$K\mkern 1.25mu{=}\mkern 1.25mu3$。

Figure 9 shows how the prompt and the camera views are packed into one stream for the VLM of Section 3.

图 9 展示了第 3 节中如何把提示和相机画面打包成一条序列送入 VLM。

Prompt. The prompt $z_{t}$ of Section 3.2 is a structured user turn carrying one field: value per line:

提示。 第 3.2 节的提示 $z_{t}$ 是一个结构化的用户轮次，每行一个“字段: 值”：

embodiment: gr1
instruction: grasp the mug
speed: 2
fps: 20
control mode: absolute joint

The speed field is the length of the episode in frames divided by 100 and rounded, so that a smaller value marks a shorter demonstration. It is left out at evaluation. Episodes of ABC (Allshire et al., 2026) that are labelled with their outcome also carry a success field.

speed 字段是回合的帧数除以 100 后取整，因此数值越小，示范越短。评测时不包含该字段。ABC（Allshire et al., 2026）中标注了结果的回合还带有一个 success 字段。

Positions. In the expert’s self-attention, register $i$ of a frame sits at slot $i$ and action step $j$ at slot $n_{r}+j$, so that the registers of each frame carry their own temporal encoding. The frame and slot axes reuse the temporal and spatial axes of the VLM’s multimodal rotary embedding.

位置。 在专家的自注意力中，一帧的第 $i$ 个寄存器位于槽位 $i$，第 $j$ 个动作步位于槽位 $n_{r}+j$，因此每一帧的寄存器都带有自己的时间编码。帧轴和槽位轴复用 VLM 多模态旋转位置编码的时间轴和空间轴。

A.1 Inference cost and caching

A.1 推理代价与缓存

Figure 10: Register cache at inference ($K\mkern 1.25mu{=}\mkern 1.25mu3$). (a) In the first Euler step of a chunk, the VLM encodes the current frame $t$. The registers of frame $t$ read its context and the cached registers of frames $t{-}1$ and $t{-}2$. The action tokens read the context and the registers of the three frames; past frames are not encoded again. (b) In the $N{-}1$ other Euler steps, only the action tokens are updated, and they read the context and the register keys and values from the cache. The registers do not read the action tokens and skip adaptive normalisation, so they do not depend on the flow time and are computed once per frame. When the next frame enters, the registers of frame $t{-}2$ are dropped.

图 10：推理时的寄存器缓存（$K\mkern 1.25mu{=}\mkern 1.25mu3$）。(a) 在一个动作块的第一个欧拉步中，VLM 对当前帧 $t$ 进行编码。第 $t$ 帧的寄存器读取它的上下文以及第 $t{-}1$ 帧和第 $t{-}2$ 帧缓存的寄存器。动作词元读取上下文和这三帧的寄存器；过去的帧不再重新编码。(b) 在其余 $N{-}1$ 个欧拉步中，只更新动作词元，它们从缓存中读取上下文以及寄存器的键和值。寄存器不读取动作词元，也跳过自适应归一化，所以它们不依赖流时间，每帧只计算一次。下一帧进入时，第 $t{-}2$ 帧的寄存器被丢弃。

What is cached. One chunk is an encoder pass, then $N$ Euler steps over the action tokens, the first of which also computes the register tokens of the current frame (Fig. 10). Only the noisy actions change from one Euler step to the next, so four quantities are reused across the $N$ steps.

缓存什么。 一个动作块包括一次编码器前向，然后在动作词元上做 $N$ 个欧拉步，其中第一步还计算当前帧的寄存器词元（图 10）。从一个欧拉步到下一个欧拉步，只有带噪动作在变，所以有四个量在 $N$ 步中复用。

1. The encoded context $\bm{E}_{t}$ of Eq. 4.
2. Its projection to keys and values, one pair per expert block.
3. The keys and values of the register tokens $\bm{M}_{t}$ of the current frame, one pair per expert block.
4. The register keys and values of the $K-1$ preceding frames, carried over from earlier chunks.

1. 式 4 的编码上下文 $\bm{E}_{t}$。
2. 它到键和值的投影，每个专家模块一对。
3. 当前帧寄存器词元 $\bm{M}_{t}$ 的键和值，每个专家模块一对。
4. 前 $K-1$ 帧的寄存器键和值，从之前的动作块沿用下来。

The first three are built the same way at training and at inference, while the fourth is not: the first registers of a window hold memory from earlier registers at inference, and none at training.

前三项在训练和推理时以相同方式构建，第四项则不同：推理时，窗口中最早的寄存器带有来自更早寄存器的记忆，训练时则没有。

What it costs. Figure 5 gives the measurement. On one H100, a chunk takes 22.1 ms without registers, 22.6 ms at $K\mkern 1.25mu{=}\mkern 1.25mu1$ and 22.8 ms at $K\mkern 1.25mu{=}\mkern 1.25mu3$, the window used for training and evaluation. Even at $K\mkern 1.25mu{=}\mkern 1.25mu32$, a chunk takes only 24.6 ms. The cache adds about 2 MiB of GPU memory per past frame, 4 MiB at $K\mkern 1.25mu{=}\mkern 1.25mu3$, on a policy of 4.7 GiB.

代价多大。 图 5 给出了测量结果。在一块 H100 上，一个动作块在没有寄存器时耗时 22.1 毫秒，$K\mkern 1.25mu{=}\mkern 1.25mu1$ 时 22.6 毫秒，$K\mkern 1.25mu{=}\mkern 1.25mu3$（训练和评测所用的窗口）时 22.8 毫秒。即使在 $K\mkern 1.25mu{=}\mkern 1.25mu32$ 时，一个动作块也只需 24.6 毫秒。缓存每个过去帧大约增加 2 MiB 显存，$K\mkern 1.25mu{=}\mkern 1.25mu3$ 时为 4 MiB，而策略本身占 4.7 GiB。

Protocol. Every measurement is taken on one H100, with two $256{\times}256$ cameras, a chunk of $10$ actions and $N\mkern 1.25mu{=}\mkern 1.25mu10$ Euler steps, in bf16, with the expert and the VLM prefill run as CUDA graphs. Each value is the median of two interleaved repetitions of 200 chunks. The split by phase comes from a separate pass with a synchronisation after each phase, scaled to the median total. The model is trained and evaluated at $K\mkern 1.25mu{=}\mkern 1.25mu3$. The longer windows are used only for these measurements.

测量方法。 所有测量都在一块 H100 上进行，使用两个 $256{\times}256$ 相机、含 $10$ 个动作的动作块和 $N\mkern 1.25mu{=}\mkern 1.25mu10$ 个欧拉步，精度为 bf16，专家和 VLM 预填充以 CUDA 图运行。每个值是两次交错重复、每次 200 个动作块的中位数。分阶段的耗时来自单独一次在每个阶段后同步的运行，并按总耗时的中位数缩放。模型以 $K\mkern 1.25mu{=}\mkern 1.25mu3$ 训练和评测，更长的窗口只用于这些测量。

Appendix B Data

附录 B 数据

B.1 Robot embodiments

B.1 机器人具身形态

Table 1 lists the robots of the pretraining mixture with their action conventions and source datasets. From Open X-Embodiment, the mixture uses twelve datasets: Taco Play, Stanford Hydra, NYU Franka Play, BC-Z, RT-1, Furniture Bench, Austin Sailor, Austin BUDS, UT Austin MUTEX, Berkeley Cable Routing, VIOLA and Berkeley RPT. The joint-delta actions of Berkeley RPT and RoboSet are converted to end-effector deltas by forward kinematics. A robot appears under several conventions when its datasets command it differently, joint angles in one and end-effector deltas in another. The state projection of Section 3.2 is selected by state layout and control convention, and the action statistics of Section 5.1 are per dataset, so no source is remapped to another’s convention. A state layout is the ordered list of state channels a dataset records, for example the joint angles and gripper opening of each arm, and fixes the input width $d_{s}$ of its projection.

表 1 列出了预训练混合数据中的机器人及其动作约定和来源数据集。混合数据使用了 Open X-Embodiment 中的十二个数据集：Taco Play、Stanford Hydra、NYU Franka Play、BC-Z、RT-1、Furniture Bench、Austin Sailor、Austin BUDS、UT Austin MUTEX、Berkeley Cable Routing、VIOLA 和 Berkeley RPT。Berkeley RPT 和 RoboSet 的关节增量动作通过正向运动学转换为末端执行器增量。当一个机器人的不同数据集以不同方式控制它时，例如一个用关节角、另一个用末端执行器增量，它就以多种约定出现。第 3.2 节的状态投影按状态布局和控制约定选择，第 5.1 节的动作统计量按数据集计算，所以没有任何数据源被重新映射到另一个数据源的约定。状态布局是一个数据集记录的状态通道的有序列表，例如每只手臂的关节角和夹爪开合度，它决定了对应投影的输入宽度 $d_{s}$。

Robot	Action type	Sources
Single-arm
WidowX	$\Delta$EE + G	(Walke et al., 2023)
Google Robot	$\Delta$EE + G	(Brohan et al., 2023; Jang et al., 2022)
Franka Panda	$\Delta$EE + G; J + G	(Open X-Embodiment Collaboration, 2024; Bharadhwaj et al., 2024; Wu et al., 2025a; Khazatsky et al., 2024; Tian et al., 2026)
UR5e	J + G	(Wu et al., 2025a)
Dual-arm
AgileX Cobot Magic	J + G	(Wu et al., 2025b; Wu et al., 2025a; Liu et al., 2025)
ALOHA	J + G	(Tian et al., 2026)
AgiBot G1	J + G	(Tian et al., 2026)
AgiBot G2A	J + G + W	(AgiBot World Team, 2026)
Galaxea R1 Lite	J + G + W + B; J + G	(Jiang et al., 2025; Wu et al., 2025b)
Dual Franka FR3	J + G	(Wu et al., 2025a)
AgiBot A2D	J + G	(Wu et al., 2025b)
Galbot G1	J + G	(Wu et al., 2025b)
RealMan RMC-AIDA-L	J + G	(Wu et al., 2025b)
AI2 AlphaBot 2	J + G	(Wu et al., 2025b)
YAM	J + G	(Allshire et al., 2026)
Bimanual Panda	$\Delta$EE + G; $\Delta$EE + DH	(Bjorck et al., 2025)
Fourier GR-1	J + DH	(Bjorck et al., 2025)
TienKung	J + G; J + DH	(Wu et al., 2025a)
AIRBOT MMK2	J + DH	(Wu et al., 2025b)
Unitree G1	J + DH	(Wu et al., 2025b)
Kuavo	J + DH	(Wu et al., 2025b)
Human
Real Human	$\Delta$EE + eigengrasp	(Hoque et al., 2026; Punamiya et al., 2026; Ropedia, 2026)

Table 1: Robot embodiments in the training mixture. “EE” = end-effector pose; “J” = joint angles; “$\Delta$” = delta (relative) commands, otherwise absolute; “G” = parallel gripper; “DH” = dexterous hand; “W” = waist/torso; “B” = mobile base. A robot may appear under several conventions across its source datasets.

机器人	动作类型	来源
单臂
WidowX	$\Delta$EE + G	(Walke et al., 2023)
Google Robot	$\Delta$EE + G	(Brohan et al., 2023; Jang et al., 2022)
Franka Panda	$\Delta$EE + G; J + G	(Open X-Embodiment Collaboration, 2024; Bharadhwaj et al., 2024; Wu et al., 2025a; Khazatsky et al., 2024; Tian et al., 2026)
UR5e	J + G	(Wu et al., 2025a)
双臂
AgileX Cobot Magic	J + G	(Wu et al., 2025b; Wu et al., 2025a; Liu et al., 2025)
ALOHA	J + G	(Tian et al., 2026)
AgiBot G1	J + G	(Tian et al., 2026)
AgiBot G2A	J + G + W	(AgiBot World Team, 2026)
Galaxea R1 Lite	J + G + W + B; J + G	(Jiang et al., 2025; Wu et al., 2025b)
Dual Franka FR3	J + G	(Wu et al., 2025a)
AgiBot A2D	J + G	(Wu et al., 2025b)
Galbot G1	J + G	(Wu et al., 2025b)
RealMan RMC-AIDA-L	J + G	(Wu et al., 2025b)
AI2 AlphaBot 2	J + G	(Wu et al., 2025b)
YAM	J + G	(Allshire et al., 2026)
Bimanual Panda	$\Delta$EE + G; $\Delta$EE + DH	(Bjorck et al., 2025)
Fourier GR-1	J + DH	(Bjorck et al., 2025)
TienKung	J + G; J + DH	(Wu et al., 2025a)
AIRBOT MMK2	J + DH	(Wu et al., 2025b)
Unitree G1	J + DH	(Wu et al., 2025b)
Kuavo	J + DH	(Wu et al., 2025b)
人类
真人	$\Delta$EE + eigengrasp	(Hoque et al., 2026; Punamiya et al., 2026; Ropedia, 2026)

表 1：训练混合数据中的机器人具身形态。 “EE”= 末端执行器位姿；“J”= 关节角；“$\Delta$”= 增量（相对）指令，否则为绝对量；“G”= 平行夹爪；“DH”= 灵巧手；“W”= 腰部/躯干；“B”= 移动底座。同一机器人在不同来源数据集中可能以多种约定出现。（译注：表中 eigengrasp 为“本征抓取”，即手指关节配置的主成分，见 B.4 节。）

B.2 Data curation

B.2 数据整理

Aggregating data across embodiments, simulators, and collection pipelines introduces heterogeneous noise: sensor outliers, temporal state-action misalignment, and inconsistent conventions across datasets sharing a robot.

跨具身形态、仿真器和采集流程汇总数据会引入各种噪声：传感器离群值、状态与动作在时间上的错位，以及共用同一机器人的数据集之间不一致的约定。

• Channel selection. We keep only the degrees of freedom the policy commands, dropping derived or uncontrolled channels: forward-kinematic end-effector poses, joint velocities, efforts, IMU readings, and telepresence-parked head, waist, and leg joints.
• Degenerate episodes. We discard static and near-zero-variance recordings, including episodes whose follower arm never moves, a failure mode Qwen-RobotManip reports for ${\sim}81\%$ of RoboMIND’s UR subset.
• Sudden jumps. We fit a smooth trend to each channel with median and Savitzky–Golay filters, and flag a frame when its residual and its acceleration or jerk exceed a per-channel threshold, the larger of 0.05 in raw units and four times the pooled 95th percentile. Grippers are exempt. Whole episodes are removed only from InternData-A1, where simulator collisions cause the jumps (6.3% of its episodes).
• Action representation. Rotations are harmonised to one Euler parameterisation; other conventions are kept as the source provides them. Each action dimension is mapped to $[-1,1]$ from its per-dataset 1st and 99th percentiles (Wang et al., 2026a), which removes scale differences across embodiments while preserving relative motion within each source.
• Convention correction. We fix per-dataset sign conventions that would otherwise break cross-dataset consistency (e.g., a sign-flipped elbow joint in InternData); differing scales and encodings are then absorbed by the per-dataset normalisation.

• 通道选择。 我们只保留策略控制的自由度，去掉派生的或不受控的通道：正向运动学得到的末端执行器位姿、关节速度、力矩、IMU 读数，以及遥操作时停放不动的头部、腰部和腿部关节。
• 退化回合。 我们丢弃静止和方差接近零的录制，包括从动臂始终不动的回合；Qwen-RobotManip 报告，RoboMIND 的 UR 子集中${\sim}81\%$ 的回合存在这种问题。
• 突跳。 我们用中值滤波和 Savitzky–Golay 滤波为每个通道拟合平滑趋势；当某帧的残差以及加速度或加加速度（jerk）超过该通道的阈值时将其标记，阈值取原始单位下的 0.05 与汇总 95 分位数的四倍中较大者。夹爪不做此检查。只有 InternData-A1 会整回合删除，因为其中的突跳由仿真器碰撞引起（占其回合的 6.3%）。
• 动作表示。 旋转统一为一种欧拉角参数化；其他约定保持数据源原样。每个动作维度按其所在数据集的第 1 和第 99 百分位数映射到 $[-1,1]$（Wang et al., 2026a），这消除了不同具身形态之间的尺度差异，同时保留每个数据源内部的相对运动。
• 约定纠正。 我们修正会破坏跨数据集一致性的各数据集符号约定（例如 InternData 中符号相反的肘关节）；不同的尺度和编码随后由按数据集的归一化吸收。

B.3 Sampling weights

B.3 采样权重

During pretraining, each dataset $d$ is sampled with probability proportional to $F_{d}^{0.5}\,L_{d}^{0.5}\,V_{d}^{0.35}$ (Table 2). $F_{d}$ is its number of frames, and $L_{d}$ and $V_{d}$ are its language and vision diversity, each divided by its median over the datasets. Datasets from the same source are grouped before weighting.

预训练期间，每个数据集 $d$ 的采样概率与 $F_{d}^{0.5}\,L_{d}^{0.5}\,V_{d}^{0.35}$ 成正比（表 2）。$F_{d}$ 是它的帧数，$L_{d}$ 和 $V_{d}$ 是它的语言多样性和视觉多样性，各自除以所有数据集上的中位数。来自同一来源的数据集在加权前先合并。

Both diversities are the effective rank of a set of L2-normalised embeddings, $2^{H(p)}$ with $p_{i}=\sigma_{i}^{2}/\sum_{j}\sigma_{j}^{2}$, where $\sigma_{i}$ are the singular values of the embedding matrix. The embeddings are not centred, so that a dataset with two distinct instructions does not collapse to rank one. For the language diversity, the rows are the SigLIP (Zhai et al., 2023) text embeddings of the distinct episode task strings, each scaled by the square root of the fraction of frames it covers. For the vision diversity, the rows are the DINOv3 (Siméoni et al., 2025) class tokens of every 15th frame of the main external camera, at most 50,000 frames per dataset.

两种多样性都是一组经 L2 归一化的嵌入的有效秩 $2^{H(p)}$，其中 $p_{i}=\sigma_{i}^{2}/\sum_{j}\sigma_{j}^{2}$，$\sigma_{i}$ 是嵌入矩阵的奇异值。嵌入不做中心化，这样一个只有两条不同指令的数据集不会塌缩为秩一。对语言多样性，矩阵的行是各条不同回合任务字符串的 SigLIP（Zhai et al., 2023）文本嵌入，每条乘以它所覆盖帧数比例的平方根。对视觉多样性，矩阵的行是主外部相机每隔 15 帧取一帧的 DINOv3（Siméoni et al., 2025）类别词元，每个数据集最多 50,000 帧。

The RoboCOIN and InternData-A1 collections are further weighted by 0.5, and the weights are capped so that no dataset is seen more than four times.

RoboCOIN 和 InternData-A1 两个数据集合再额外乘以 0.5 的权重，并对权重设置上限，使任何数据集被看到的次数不超过四遍。

B.4 Egocentric data

B.4 第一人称数据

Sources. The three egocentric sources differ in capture conditions: EgoDex is seated tabletop recordings on Apple Vision Pro, EgoVerse head-mounted rigs, and Xperience in-the-wild moving-camera footage. Frame rates range from 20 to 30 Hz and are left unaltered; the differences are absorbed by the per-dataset quantile normalisation of Section 5.1.

来源。 三个第一人称数据源的采集条件不同：EgoDex 是用 Apple Vision Pro 录制的坐姿桌面场景，EgoVerse 使用头戴设备，Xperience 是真实野外环境中相机移动的画面。帧率在 20 到 30 Hz 之间，保持不变；差异由第 5.1 节按数据集的分位数归一化吸收。

Action target. We follow the layout of Qwen-VLA’s egocentric target (Wang et al., 2026a): per hand, a 6-dimensional wrist motion as translation and axis-angle rotation, and 10 articulation coefficients, 32 dimensions in total. Unlike Qwen-VLA, which expresses the wrist relative to its own initial pose, we express it in the observation camera frame through the rigid transform $T_{\mathrm{cam}}^{-1}T_{\mathrm{wrist}}$. The articulation coefficients are the first 10 principal components (“eigengrasps”) of the finger-joint configuration under a PCA basis shared across our three sources. The target is defined relative to the frame that opens each action chunk, so it is composed at load time from the stored world-frame wrist poses and per-frame camera poses rather than precomputed.

动作目标。 我们沿用 Qwen-VLA 第一人称目标的布局（Wang et al., 2026a）：每只手有一个 6 维腕部运动（平移和轴角旋转）以及 10 个关节姿态系数，共 32 维。Qwen-VLA 把腕部表示为相对其自身初始位姿的量，而我们通过刚体变换 $T_{\mathrm{cam}}^{-1}T_{\mathrm{wrist}}$ 把它表示在观测相机坐标系中。关节姿态系数是手指关节配置在三个数据源共享的 PCA 基下的前 10 个主成分（“本征抓取”，eigengrasps）。目标相对于每个动作块起始的那一帧定义，所以它在加载时由存储的世界坐标系腕部位姿和逐帧相机位姿组合得到，而不是预先计算好的。

State and pacing. The egocentric data have no proprioceptive state, and the policy receives a learned null state in its place. Human hands move faster than a teleoperated robot, so the human actions are slowed down in time (Yuan et al., 2026). Waypoint $i$ of a chunk is the hand pose at time $i\rho/f$ after the observation, where $f$ is the frame rate of the recording and $\rho\leq 1$ a per-source pace (Table 2). The pose is interpolated between the two nearest frames. A one-second chunk at $\rho=0.25$ thus holds a quarter of a second of human motion.

状态与节奏。 第一人称数据没有本体感知状态，策略接收一个学习得到的空状态代替。人手的移动比遥操作机器人快，所以人类动作在时间上被放慢（Yuan et al., 2026）。动作块中的第 $i$ 个路径点是观测之后 $i\rho/f$ 时刻的手部位姿，其中 $f$ 是录制的帧率，$\rho\leq 1$ 是每个数据源的节奏系数（表 2）。位姿在最近的两帧之间插值。因此在 $\rho=0.25$ 时，一个一秒的动作块包含四分之一秒的人类动作。

B.5 Vision–language data

B.5 视觉—语言数据

REKI is co-trained on vision–language data at 10% of the training samples (Table 2). The pool is every FineVision (Wiedmann et al., 2025) dataset except the text-only ones, plus the LLaVA-OneVision-1.5 (An et al., 2025) datasets that FineVision does not already contain, ${\sim}20.2$M samples. We keep the datasets covering the tasks most relevant to manipulation: natural-scene objects and attributes, spatial relations, counting, grounding, captioning, scene text, and knowledge and commonsense VQA. Documents, charts, tables, mathematics, code, transcription, GUI grounding, memes and puzzles are dropped. 60 datasets remain, 7.64M samples or 37.8% of the pool. One question–answer pair is drawn per sample and sampling is uniform over samples, so exposure is proportional to the size of a dataset.

REKI 在视觉—语言数据上联合训练，占训练样本的 10%（表 2）。数据池包括 FineVision（Wiedmann et al., 2025）中除纯文本以外的所有数据集，加上 FineVision 尚未包含的 LLaVA-OneVision-1.5（An et al., 2025）数据集，${\sim}20.2$M 个样本。我们保留覆盖与操作任务最相关内容的数据集：自然场景中的物体与属性、空间关系、计数、定位、图像描述、场景文字，以及知识和常识类视觉问答。文档、图表、表格、数学、代码、转录、GUI 定位、表情包和谜题类数据被去掉。剩下 60 个数据集，共 764 万个样本，占数据池的 37.8%。每个样本抽取一个问答对，在样本上均匀采样，所以一个数据集被看到的程度与其规模成正比。

The 51 kept from FineVision are a_okvqa (Schwenk et al., 2022), alfworldgpt (Shridhar et al., 2021), allava_laion (Chen et al., 2024a), allava_vflan (Chen et al., 2024a), aokvqa (Schwenk et al., 2022), art (Chng et al., 2019), cambrian(filtered)_processed (Tong et al., 2024), clevr (Lindström & Abraham, 2022), clevr_math (Lindström & Abraham, 2022), coco_colors (Karakus, 2024), cocoqa (Ren et al., 2015), cocotext (Veit et al., 2016), densefusion_1m (Li et al., 2024), drivelm (Sima et al., 2024), est_vqa (Wang et al., 2020), google_landmarks (Weyand et al., 2020), image_textualization(filtered) (Pi et al., 2024), indoor_qa (Quattoni & Torralba, 2009), laion_gpt4v (LAION, 2023), LLaVA_Instruct_150K (Liu et al., 2023c), lnqa (Pont-Tuset et al., 2020), localized_narratives (Pont-Tuset et al., 2020), lrv_normal(filtered) (Liu et al., 2024a), lvis_instruct4v (Wang et al., 2023), mimic_cgd (Laurençon et al., 2024), mmevol (Luo et al., 2024), nlvr2 (Suhr et al., 2017), objects365_qa (Shao et al., 2019), ocrvqa (Mishra et al., 2019), oodvqa (Tu et al., 2023), scienceqa (Lu et al., 2022), scienceqa(nona_context) (Lu et al., 2022), sharegpt4o (OpenGVLab, 2024), sharegpt4v(coco) (Chen et al., 2024b), sharegpt4v(knowledge) (Chen et al., 2024b), sharegpt4v(llava) (Chen et al., 2024b), sharegpt4v(sam) (Chen et al., 2024b), sketchyvqa (Tu et al., 2023), spatialsense (Yang et al., 2019), spot_the_diff (Jhamtani & Berg-Kirkpatrick, 2018), st_vqa (Biten et al., 2019), super_clevr(mathv360k) (Li et al., 2023), tallyqa (Acharya et al., 2019), textcaps (Sidorov et al., 2020), textocr(gpt4v) (Carter, 2024), textvqa (Singh et al., 2019), vision_flan(filtered) (Xu et al., 2024), visual7w (Zhu et al., 2016), vqav2 (Goyal et al., 2017), vsr (Liu et al., 2023b), and wildvision (Lu et al., 2024). The nine kept from LLaVA-OneVision-1.5 are gqa (Hudson & Manning, 2019), llava_cot_100k (Xu et al., 2025), llava_wild (Liu et al., 2023c), llrv_gpt4v (Liu et al., 2024a), sherlock (Hessel et al., 2022), svit (Zhao et al., 2023a), viquae (Lerner et al., 2022), vision_oritented, and visual_chat.

从 FineVision 中保留的 51 个数据集为：a_okvqa (Schwenk et al., 2022)、alfworldgpt (Shridhar et al., 2021)、allava_laion (Chen et al., 2024a)、allava_vflan (Chen et al., 2024a)、aokvqa (Schwenk et al., 2022)、art (Chng et al., 2019)、cambrian(filtered)_processed (Tong et al., 2024)、clevr (Lindström & Abraham, 2022)、clevr_math (Lindström & Abraham, 2022)、coco_colors (Karakus, 2024)、cocoqa (Ren et al., 2015)、cocotext (Veit et al., 2016)、densefusion_1m (Li et al., 2024)、drivelm (Sima et al., 2024)、est_vqa (Wang et al., 2020)、google_landmarks (Weyand et al., 2020)、image_textualization(filtered) (Pi et al., 2024)、indoor_qa (Quattoni & Torralba, 2009)、laion_gpt4v (LAION, 2023)、LLaVA_Instruct_150K (Liu et al., 2023c)、lnqa (Pont-Tuset et al., 2020)、localized_narratives (Pont-Tuset et al., 2020)、lrv_normal(filtered) (Liu et al., 2024a)、lvis_instruct4v (Wang et al., 2023)、mimic_cgd (Laurençon et al., 2024)、mmevol (Luo et al., 2024)、nlvr2 (Suhr et al., 2017)、objects365_qa (Shao et al., 2019)、ocrvqa (Mishra et al., 2019)、oodvqa (Tu et al., 2023)、scienceqa (Lu et al., 2022)、scienceqa(nona_context) (Lu et al., 2022)、sharegpt4o (OpenGVLab, 2024)、sharegpt4v(coco) (Chen et al., 2024b)、sharegpt4v(knowledge) (Chen et al., 2024b)、sharegpt4v(llava) (Chen et al., 2024b)、sharegpt4v(sam) (Chen et al., 2024b)、sketchyvqa (Tu et al., 2023)、spatialsense (Yang et al., 2019)、spot_the_diff (Jhamtani & Berg-Kirkpatrick, 2018)、st_vqa (Biten et al., 2019)、super_clevr(mathv360k) (Li et al., 2023)、tallyqa (Acharya et al., 2019)、textcaps (Sidorov et al., 2020)、textocr(gpt4v) (Carter, 2024)、textvqa (Singh et al., 2019)、vision_flan(filtered) (Xu et al., 2024)、visual7w (Zhu et al., 2016)、vqav2 (Goyal et al., 2017)、vsr (Liu et al., 2023b) 和 wildvision (Lu et al., 2024)。从 LLaVA-OneVision-1.5 中保留的 9 个数据集为：gqa (Hudson & Manning, 2019)、llava_cot_100k (Xu et al., 2025)、llava_wild (Liu et al., 2023c)、llrv_gpt4v (Liu et al., 2024a)、sherlock (Hessel et al., 2022)、svit (Zhao et al., 2023a)、viquae (Lerner et al., 2022)、vision_oritented 和 visual_chat。

To these we add the object- and free-space-reference splits of RoboPoint (Yuan et al., 2024), the 2D and 3D splits of RefSpatial (Zhou et al., 2025), and PixMo-Points (Deitke et al., 2025) rewritten as pointing and counting questions, all with coordinates on a $[0,1000]$ grid.

在此基础上，我们加入 RoboPoint（Yuan et al., 2024）的物体指代和自由空间指代子集、RefSpatial（Zhou et al., 2025）的 2D 和 3D 子集，以及改写为指点和计数问题的 PixMo-Points（Deitke et al., 2025），坐标都在 $[0,1000]$ 网格上。

Appendix C Training configuration

附录 C 训练配置

Table 2 lists the training configuration. Table 3 gives the node count, the number of steps and the settings that differ between pretraining and the five finetuning runs. Settings absent from Table 3 are those of Table 2 in every run.

表 2 列出了训练配置。表 3 给出了预训练和五次微调运行的节点数、步数，以及各次运行之间不同的设置。表 3 中未列出的设置在每次运行中都与表 2 相同。

Setting	Value
Encoder
Vision–language backbone	Qwen3.5-2B (18 gated-DeltaNet + 6 full-attention layers)
Image resolution	native, up to $448{\times}448$
Expert conditioning	cross-attention to the final-layer backbone states in every expert block
Trainable components	backbone at a reduced LR (Table 3), last $4$ vision blocks, expert
Vision–language co-training	$10\%$: FineVision (subset), LLaVA-OneVision-1.5 (subset), RoboPoint, RefSpatial, PixMo
Text loss (subtask, vision–language data)	cross-entropy, weight $0.1$, pretraining only
Prompt-field dropout	$0.15$ (embodiment), $0.2$ (control frequency, speed)
Action expert
Size	$10$ blocks, $768$ hidden, $12$ heads, $3072$ MLP
Precision	bfloat16, flow heads in fp32
Proprioception	per-layout linear layer with GELU, shared zero-initialised projection, summed into the adaptive-normalisation conditioning
Proprioception dropout	$0.2$
Registers per frame	$32$
Window	$K=3$ frames at a $1$ s stride
Action generation
Action chunk	$1$ s ($\mathrm{round}(\text{control freq.})$ steps)
Objective	flow matching, DoF-balanced
Time-step distribution	Beta
Flow draws per forward pass	$8$
Data
Robot corpora	OXE, DROID, AgiBot, RoboMIND, Galaxea, RoboSet, RDT, RoboCOIN, GR00T, InternData, ABC
Egocentric human video	EgoDex, EgoVerse, Xperience
Sampling weight	$\propto\{\#\text{frames}\}^{0.5}\,\{\text{language diversity}\}^{0.5}\,\{\text{vision diversity}\}^{0.35}$ (Section B.3)
Normalisation	quantile (1st / 99th percentile)
Action-speed alignment	EgoDex $0.6$, EgoVerse $0.25$, Xperience $1$ (unchanged)
Vision & augmentation
Camera dropout	$0.15$ per camera in pretraining, $0.2$ in finetuning; at least one camera kept
Camera order	shuffled
Colour jitter	brightness, contrast, saturation $\times\,\mathcal{U}(0.8,1.2)$, drawn for each frame of the window and shared by all cameras of the frame
Random crop	overhead and side cameras, scale $\mathcal{U}(0.95,1)$, one box per camera for the window; pretraining and LIBERO only
Optimisation
Learning rate	expert: $1{\times}10^{-4}$, VLM: $1{\times}10^{-5}$ to $3{\times}10^{-5}$ (Table 3)
Weight decay	$0$
Gradient clipping	$1.0$
Schedule	constant with warmup ($1{,}000$ steps when finetuning the pretrained model, $5{,}000$ otherwise), then decay over the last $5{,}000$ steps (finetuning only)
Weight averaging	EMA, decay $0.999$
Gradient checkpointing	✓

Table 2: Training configuration.

设置	取值
编码器
视觉—语言主干	Qwen3.5-2B （18 个门控 DeltaNet 层 + 6 个全注意力层）
图像分辨率	原生分辨率，最大 $448{\times}448$
专家条件输入	每个专家模块对主干最后一层状态做交叉注意力
可训练组件	主干（降低学习率，表 3）、最后 $4$ 个视觉模块、专家
视觉—语言联合训练	$10\%$: FineVision（子集）、LLaVA-OneVision-1.5（子集）, RoboPoint, RefSpatial, PixMo
文本损失（子任务、视觉—语言数据）	交叉熵，权重 $0.1$，仅预训练
提示字段丢弃	$0.15$（具身形态），$0.2$（控制频率、speed）
动作专家
规模	$10$ 个模块，隐藏维度 $768$，$12$ 个头，MLP 维度 $3072$
精度	bfloat16，流输出头用 fp32
本体感知	每种布局一个带 GELU 的线性层，共享的零初始化投影，加到自适应归一化的条件输入上
本体感知丢弃	$0.2$
每帧寄存器数	$32$
窗口	$K=3$ 帧，间隔 $1$ 秒
动作生成
动作块	$1$ 秒（$\mathrm{round}(\text{control freq.})$ 步）
目标	流匹配，按自由度平衡
时间步分布	Beta
每次前向的流抽样次数	$8$
数据
机器人语料	OXE, DROID, AgiBot, RoboMIND, Galaxea, RoboSet, RDT, RoboCOIN, GR00T, InternData, ABC
第一人称人类视频	EgoDex, EgoVerse, Xperience
采样权重	$\propto\{\#\text{frames}\}^{0.5}\,\{\text{language diversity}\}^{0.5}\,\{\text{vision diversity}\}^{0.35}$ （B.3 节）
归一化	分位数（第 1 / 第 99 百分位数）
动作速度对齐	EgoDex $0.6$，EgoVerse $0.25$，Xperience $1$（不变）
视觉与增强
相机丢弃	预训练中每个相机 $0.15$，微调中 $0.2$；至少保留一个相机
相机顺序	打乱
颜色抖动	亮度、对比度、饱和度 $\times\,\mathcal{U}(0.8,1.2)$，窗口中每帧单独抽取，同一帧的所有相机共用
随机裁剪	俯视和侧面相机，比例 $\mathcal{U}(0.95,1)$，整个窗口每个相机一个裁剪框；仅预训练和 LIBERO
优化
学习率	专家：$1{\times}10^{-4}$，VLM：$1{\times}10^{-5}$ 至 $3{\times}10^{-5}$（表 3）
权重衰减	$0$
梯度截断	$1.0$
调度	预热后保持常数（微调预训练模型时预热 $1{,}000$ 步，其他情况 $5{,}000$ 步），然后在最后 $5{,}000$ 步衰减（仅微调）
权重平均	EMA，衰减 $0.999$
梯度检查点	✓

表 2：训练配置。

	Pretraining	LIBERO	RoboTwin Clean	RoboTwin Clean+Rand	RoboCasa365	RoboCasa-GR1
Nodes ($8\times$H100)	4	1	1	3	4	3
Steps	200k	15k	20k	25k	25k	25k
Wall-clock time (h)	176	5.2	10.9	14.2	12.5	8.3
GPU-hours	5,632	41	87	341	401	199
Backbone LR ($\times$ expert LR)	0.1	0.3	0.2	0.3	0.3	0.2
Sampling weights $\propto\sqrt{\#\text{frames}}$	✓∗	✗	✗	✓	✗	✗
Overhead-camera dropout	✗	✓	✗	✗	✗	✗
Mirroring (Section 5.3)	✗	✗	✓	✓	✓	✓
Evaluation rollouts per task	–	50	20	20	50	50

Table 3: Per-run setup of REKI for pretraining and the five finetuning runs, and the number of evaluation rollouts per task. The two RoboTwin runs are finetuned on the Clean demonstrations alone (Clean2Rand) and on Clean and Randomised together (in-distribution). ∗Multiplied by the language and vision diversity terms of Section B.3.

	预训练	LIBERO	RoboTwin Clean	RoboTwin Clean+Rand	RoboCasa365	RoboCasa-GR1
节点数（$8\times$H100）	4	1	1	3	4	3
步数	200k	15k	20k	25k	25k	25k
墙钟时间（小时）	176	5.2	10.9	14.2	12.5	8.3
GPU 小时	5,632	41	87	341	401	199
主干学习率（$\times$ 专家学习率）	0.1	0.3	0.2	0.3	0.3	0.2
采样权重 $\propto\sqrt{\#\text{frames}}$	✓∗	✗	✗	✓	✗	✗
俯视相机丢弃	✗	✓	✗	✗	✗	✗
镜像（第 5.3 节）	✗	✗	✓	✓	✓	✓
每任务评测次数	–	50	20	20	50	50

表 3：REKI 各次运行的设置，包括预训练和五次微调，以及每个任务的评测次数。两次 RoboTwin 运行分别只在 Clean 示范上微调（Clean2Rand），以及在 Clean 和随机化示范上一起微调（分布内）。∗再乘以 B.3 节的语言和视觉多样性项。

C.1 Camera selection

C.1 相机选择

In pretraining, two rules remove cameras from a sample after the random camera dropout. When two or more side cameras remain, one of them is kept at random and the others are removed. On a two-arm robot with both wrist cameras present, the wrist camera of a static arm is removed. An arm is static when, over the action chunk, every action dimension of the arm varies by less than $0.1$ of its standard deviation in the dataset. Both rules keep at least one camera other than the overhead camera. Finetuning uses neither rule. Random camera dropout never removes the overhead camera, except in LIBERO finetuning (Table 3).

预训练中，在随机相机丢弃之后，还有两条规则会从样本中去掉相机。当剩下两个或更多侧面相机时，随机保留其中一个，去掉其余的。在双臂机器人上，如果两个腕部相机都在，静止手臂的腕部相机会被去掉。若在一个动作块内，某只手臂的每个动作维度的变化都小于该维度在数据集中标准差的 $0.1$，就认为这只手臂是静止的。两条规则都至少保留一个俯视相机以外的相机。微调时两条规则都不用。随机相机丢弃从不去掉俯视相机，LIBERO 微调除外（表 3）。

C.2 Mirroring

C.2 镜像

A sample is mirrored with probability $0.5$ in the finetuning runs on RoboTwin, RoboCasa365 and RoboCasa-GR1 (Table 3). The images are flipped horizontally, and the state and the actions are reflected through the same plane. On a two-arm robot, the two arms and their wrist cameras are swapped. The prompt of a mirrored sample starts with [MIRROR WORLD], and its state goes through a separate projection with separate normalisation statistics. The policy is evaluated in the unmirrored world only.

在 RoboTwin、RoboCasa365 和 RoboCasa-GR1 的微调中，样本以 $0.5$ 的概率被镜像（表 3）。图像水平翻转，状态和动作关于同一平面反射。在双臂机器人上，两只手臂及其腕部相机互换。镜像样本的提示以 [MIRROR WORLD] 开头，其状态经过一个单独的投影，使用单独的归一化统计量。策略只在未镜像的世界中评测。

Appendix D Full benchmark results

附录 D 完整基准结果

Tables 4 and 5 give the per-condition results summarised in Fig. 4, together with the in-distribution results on LIBERO and RoboTwin. Each LIBERO-Plus column perturbs one dimension: the camera viewpoint, the initial robot state, the language instruction, the lighting, the background texture, the sensor noise or the object layout. The Overall column is the success rate over all 10,030 perturbed episodes, so each dimension is weighted by its number of episodes (Fei et al., 2025). For the in-distribution RoboTwin results, the models are finetuned on clean and randomised scenes together and evaluated on each (Clean, Rand). For Clean2Rand, the models are finetuned on clean scenes only and evaluated on clean scenes (Easy), under a randomised background, lighting, clutter or table height applied alone, and under all four together (Hard). The baseline numbers are taken from published results, and Table 7 gives the source of each one. The Clean2Rand row of Qwen-RobotManip (Yuan et al., 2026) is its joint-space variant. The LIBERO-Plus results of $\pi_{0}$ and $\pi_{0.5}$ are from the zero-shot block of ACoT-VLA (Zhong et al., 2026), which evaluated the released checkpoints. ACoT-VLA is initialised from $\pi_{0.5}$, so it is grouped with the models pretrained on in-house data. The LIBERO-Plus results of StarVLA are those of its Qwen3-VL-OFT model in the StarVLA repository. For Being-H0.7 and ABot-M0.5, the LIBERO-Plus Overall is pooled from the seven published dimensions, because the published total does not match them. The RoboCasa365 leaderboard is at https://robocasa.ai/leaderboard. On RoboCasa-GR1, the result of GR00T-N1.5 is that of the released checkpoint in the Isaac-GR00T repository (https://github.com/NVIDIA/Isaac-GR00T, branch n1.5-release), and the result of $\pi_{0.5}$ is the one reported by StarVLA, which states no source for it. Parameter counts include the action expert, are rounded from exact counts and are read off the released checkpoints where available, including those of StarVLA and RLDX-1 (Kim et al., 2026). Qwen-RobotManip states no expert size; our expert follows its design, so we count 0.2B for it.

表 4 和表 5 给出了图 4 所汇总结果的各条件明细，以及 LIBERO 和 RoboTwin 上的分布内结果。LIBERO-Plus 的每一列扰动一个维度：相机视角、机器人初始状态、语言指令、光照、背景纹理、传感器噪声或物体布局。Overall 列是全部 10,030 个受扰动回合上的成功率，因此每个维度按其回合数加权（Fei et al., 2025）。对 RoboTwin 的分布内结果，模型在干净场景和随机化场景上一起微调，分别在两者上评测（Clean、Rand）。对 Clean2Rand，模型只在干净场景上微调，在干净场景（Easy）、单独施加的随机背景、光照、杂物或桌面高度，以及四者同时施加（Hard）的条件下评测。基线数字取自已发表的结果，表 7 给出了每个数字的来源。Qwen-RobotManip（Yuan et al., 2026）的 Clean2Rand 一行是其关节空间版本。$\pi_{0}$ 和 $\pi_{0.5}$ 的 LIBERO-Plus 结果取自 ACoT-VLA（Zhong et al., 2026）的零样本部分，该文评测了公开的检查点。ACoT-VLA 由 $\pi_{0.5}$ 初始化，所以归入用内部数据预训练的模型。StarVLA 的 LIBERO-Plus 结果是 StarVLA 代码库中其 Qwen3-VL-OFT 模型的结果。对 Being-H0.7 和 ABot-M0.5，LIBERO-Plus 的 Overall 由已发表的七个维度汇总得到，因为已发表的总分与这些维度对不上。RoboCasa365 排行榜见 https://robocasa.ai/leaderboard。在 RoboCasa-GR1 上，GR00T-N1.5 的结果是 Isaac-GR00T 代码库（https://github.com/NVIDIA/Isaac-GR00T，n1.5-release 分支）中公开检查点的结果，$\pi_{0.5}$ 的结果是 StarVLA 报告的，StarVLA 没有说明其来源。参数量包括动作专家，由精确数值四舍五入得到，在有公开检查点时从检查点读取，StarVLA 和 RLDX-1（Kim et al., 2026）也是如此。Qwen-RobotManip 没有说明专家的规模；我们的专家沿用其设计，所以按 2 亿计。

Method	#Params VLM	#Params Expert	LIBERO	Camera	Robot	Language	Light	Backgd	Noise	Layout	Overall
No pretraining
StarVLA (StarVLA Community, 2026)	4.4B	0.1B	98.0	47.0	60.1	87.0	96.3	95.3	73.1	79.2	75.0
Qwen-RobotManip-scratch (Yuan et al., 2026)	4.5B	0.2B	98.2	70.4	44.9	88.1	95.8	95.5	84.4	79.1	78.3
REKI (ours) scratch	2.2B	0.2B	96.5	94.4	63.9	80.7	97.5	96.0	96.0	79.2	86.1
With pretraining – in-house data
$\pi_{0}$ (Black et al., 2024)	2.9B	0.3B	94.4	61.0	40.8	63.5	89.3	84.1	80.1	76.4	69.4
$\pi_{0.5}$ (Physical Intelligence et al., 2025)	2.9B	0.4B	97.6	75.8	79.4	83.3	95.5	95.0	89.6	87.0	85.7
ACoT-VLA (Zhong et al., 2026)	2.9B	0.9B	98.5	72.6	82.6	87.5	97.7	96.5	87.8	88.1	86.6
Being-H0.7 (Luo et al., 2026)	3B	–	99.2	82.0	59.0	82.8	97.8	90.0	93.5	88.5	84.0†
Qwen-RobotManip (Yuan et al., 2026)	4.5B	0.2B	99.1	87.2	75.5	85.6	96.6	97.7	97.7	87.3	89.0
RLDX-1 (Kim et al., 2026)	5.3B	1.6B	97.8	75.0	91.8	92.2	97.1	95.5	80.3	80.7	86.7
With pretraining – public data only
ABot-M0 (Yang et al., 2026)	4.4B	0.2B	98.6	60.4	67.9	86.4	96.2	91.6	86.4	82.6	80.5
ABot-M0.5 (Chen et al., 2026)	5B	–	99.4	70.5	87.4	88.6	94.0	89.7	75.5	85.2	83.7†
InternVLA-A1.5 (Ma et al., 2026)	2.2B	0.5B	98.9	83.1	55.1	86.9	96.4	98.2	95.6	85.2	84.8
REKI (ours)	2.2B	0.2B	97.5	93.2	75.6	88.5	97.7	98.4	98.6	84.7	90.4

Table 4: LIBERO and LIBERO-Plus results. All models are finetuned on LIBERO; LIBERO-Plus evaluates out-of-distribution robustness across seven perturbation dimensions. Italic: model not released at the time of writing. Bold: best in each column within each block, italic rows excluded. †Pooled from the seven reported dimensions, which the published total does not match.

方法	参数量 VLM	参数量 专家	LIBERO	相机	机器人	语言	光照	背景	噪声	布局	总体
无预训练
StarVLA (StarVLA Community, 2026)	4.4B	0.1B	98.0	47.0	60.1	87.0	96.3	95.3	73.1	79.2	75.0
Qwen-RobotManip-scratch (Yuan et al., 2026)	4.5B	0.2B	98.2	70.4	44.9	88.1	95.8	95.5	84.4	79.1	78.3
REKI（本文） scratch	2.2B	0.2B	96.5	94.4	63.9	80.7	97.5	96.0	96.0	79.2	86.1
有预训练——内部数据
$\pi_{0}$ (Black et al., 2024)	2.9B	0.3B	94.4	61.0	40.8	63.5	89.3	84.1	80.1	76.4	69.4
$\pi_{0.5}$ (Physical Intelligence et al., 2025)	2.9B	0.4B	97.6	75.8	79.4	83.3	95.5	95.0	89.6	87.0	85.7
ACoT-VLA (Zhong et al., 2026)	2.9B	0.9B	98.5	72.6	82.6	87.5	97.7	96.5	87.8	88.1	86.6
Being-H0.7 (Luo et al., 2026)	3B	–	99.2	82.0	59.0	82.8	97.8	90.0	93.5	88.5	84.0†
Qwen-RobotManip (Yuan et al., 2026)	4.5B	0.2B	99.1	87.2	75.5	85.6	96.6	97.7	97.7	87.3	89.0
RLDX-1 (Kim et al., 2026)	5.3B	1.6B	97.8	75.0	91.8	92.2	97.1	95.5	80.3	80.7	86.7
有预训练——仅公开数据
ABot-M0 (Yang et al., 2026)	4.4B	0.2B	98.6	60.4	67.9	86.4	96.2	91.6	86.4	82.6	80.5
ABot-M0.5 (Chen et al., 2026)	5B	–	99.4	70.5	87.4	88.6	94.0	89.7	75.5	85.2	83.7†
InternVLA-A1.5 (Ma et al., 2026)	2.2B	0.5B	98.9	83.1	55.1	86.9	96.4	98.2	95.6	85.2	84.8
REKI（本文）	2.2B	0.2B	97.5	93.2	75.6	88.5	97.7	98.4	98.6	84.7	90.4

表 4：LIBERO 和 LIBERO-Plus 结果。 所有模型都在 LIBERO 上微调；LIBERO-Plus 在七个扰动维度上评测分布外鲁棒性（表中“相机”至“总体”各列均为 LIBERO-Plus）。斜体：撰写本文时尚未公开的模型。粗体：每个分组中各列的最佳值，斜体行不参与比较。†由已报告的七个维度汇总得到，与已发表的总分不一致。

Method	#Params VLM	#Params Expert	RoboTwin Clean	RoboTwin Rand	Clean2Rand Easy	Backgd	Light	Clutter	Height	Hard
No pretraining
StarVLA (StarVLA Community, 2026)	4.4B	0.1B	88.2	88.3	58.1	27.1	50.9	24.2	48.4	10.6
Qwen-RobotManip-scratch (Yuan et al., 2026)	4.5B	0.2B	88.7	88.4	71.6	60.6	70.7	24.6	63.6	22.6
REKI (ours) scratch	2.2B	0.2B	83.0	83.8	67.1	62.4	68.2	27.5	60.7	26.3
With pretraining – in-house data
$\pi_{0}$ (Black et al., 2024)	2.9B	0.3B	65.9	58.4	–	–	–	–	–	–
$\pi_{0.5}$ (Physical Intelligence et al., 2025)	2.9B	0.4B	82.7	76.8	73.1	67.0	69.2	57.9	67.6	47.9
GR00T-N1.7 (Bjorck et al., 2025)	1.5B	1.6B	–	–	43.6	40.4	41.9	27.1	39.0	20.7
Being-H0.7 (Luo et al., 2026)	3B	–	90.2	89.6	–	–	–	–	–	–
Qwen-RobotManip (Yuan et al., 2026)	4.5B	0.2B	93.4	92.5	73.2	74.6	68.4	61.3	71.0	62.6
With pretraining – public data only
ABot-M0 (Yang et al., 2026)	4.4B	0.2B	86.1	85.1	70.7	56.5	68.8	46.0	56.3	36.0
ABot-M0.5 (Chen et al., 2026)	5B	–	94.0	94.2	–	–	–	–	–	–
InternVLA-A1.5 (Ma et al., 2026)	2.2B	0.5B	93.3	93.0	–	–	–	–	–	–
REKI (ours)	2.2B	0.2B	85.9	84.6	73.4	71.3	73.5	57.8	69.9	48.7

Table 5: RoboTwin results. Left: in-distribution results for models finetuned on clean and randomised scenes. Right: out-of-distribution evaluation on RoboTwin Clean2Rand, where models are finetuned on the Clean dataset only and tested under various environment randomisations. Italic: model not released at the time of writing. Bold: best in each column within each block, italic rows excluded.

方法	参数量 VLM	参数量 专家	RoboTwin Clean	RoboTwin Rand	Clean2Rand Easy	背景	光照	杂物	高度	Hard
无预训练
StarVLA (StarVLA Community, 2026)	4.4B	0.1B	88.2	88.3	58.1	27.1	50.9	24.2	48.4	10.6
Qwen-RobotManip-scratch (Yuan et al., 2026)	4.5B	0.2B	88.7	88.4	71.6	60.6	70.7	24.6	63.6	22.6
REKI（本文） scratch	2.2B	0.2B	83.0	83.8	67.1	62.4	68.2	27.5	60.7	26.3
有预训练——内部数据
$\pi_{0}$ (Black et al., 2024)	2.9B	0.3B	65.9	58.4	–	–	–	–	–	–
$\pi_{0.5}$ (Physical Intelligence et al., 2025)	2.9B	0.4B	82.7	76.8	73.1	67.0	69.2	57.9	67.6	47.9
GR00T-N1.7 (Bjorck et al., 2025)	1.5B	1.6B	–	–	43.6	40.4	41.9	27.1	39.0	20.7
Being-H0.7 (Luo et al., 2026)	3B	–	90.2	89.6	–	–	–	–	–	–
Qwen-RobotManip (Yuan et al., 2026)	4.5B	0.2B	93.4	92.5	73.2	74.6	68.4	61.3	71.0	62.6
有预训练——仅公开数据
ABot-M0 (Yang et al., 2026)	4.4B	0.2B	86.1	85.1	70.7	56.5	68.8	46.0	56.3	36.0
ABot-M0.5 (Chen et al., 2026)	5B	–	94.0	94.2	–	–	–	–	–	–
InternVLA-A1.5 (Ma et al., 2026)	2.2B	0.5B	93.3	93.0	–	–	–	–	–	–
REKI（本文）	2.2B	0.2B	85.9	84.6	73.4	71.3	73.5	57.8	69.9	48.7

表 5：RoboTwin 结果。 左（RoboTwin Clean、Rand 两列）：在干净场景和随机化场景上微调的模型的分布内结果。右（Clean2Rand 各列）：RoboTwin Clean2Rand 上的分布外评测，模型只在 Clean 数据集上微调，在各种环境随机化下测试。斜体：撰写本文时尚未公开的模型。粗体：每个分组中各列的最佳值，斜体行不参与比较。

Method	#Params	Atomic	Composite Seen	Composite Unseen	Avg
Xiaomi-Robotics-1∗ (Xiaomi Robotics Team, 2026)	5.1B	80.2	57.1	32.1	57.4
ABot-M0.5 (Chen et al., 2026)	5B	75.9	38.3	2.7	40.4
RLDX-1∗ (Kim et al., 2026)	6.9B	67.6	27.9	8.5	36.0
Qwen-RobotManip∗ (Yuan et al., 2026)	4.7B	68.6	20.1	14.9	35.9
GR00T-N1.5∗ (Bjorck et al., 2025)	2.4B	50.7	14.8	2.7	23.9
GR00T-N1.6∗ (Bjorck et al., 2025)	3.0B	51.1	9.4	1.7	21.9
$\pi_{0.5}$∗ (Physical Intelligence et al., 2025)	3.3B	39.6	7.1	1.2	16.9
REKI (ours)	2.4B	65.8	32.0	14.1	38.4

Table 6: RoboCasa365 results behind Fig. 4, on the 50 target tasks of the multi-task setting. Hatched in Fig. 4, and marked $\ast$ here: pretrained on in-house data. Italic: model not released at the time of writing.

方法	参数量	原子任务	组合任务（见过）	组合任务（未见过）	平均
Xiaomi-Robotics-1∗ (Xiaomi Robotics Team, 2026)	5.1B	80.2	57.1	32.1	57.4
ABot-M0.5 (Chen et al., 2026)	5B	75.9	38.3	2.7	40.4
RLDX-1∗ (Kim et al., 2026)	6.9B	67.6	27.9	8.5	36.0
Qwen-RobotManip∗ (Yuan et al., 2026)	4.7B	68.6	20.1	14.9	35.9
GR00T-N1.5∗ (Bjorck et al., 2025)	2.4B	50.7	14.8	2.7	23.9
GR00T-N1.6∗ (Bjorck et al., 2025)	3.0B	51.1	9.4	1.7	21.9
$\pi_{0.5}$∗ (Physical Intelligence et al., 2025)	3.3B	39.6	7.1	1.2	16.9
REKI（本文）	2.4B	65.8	32.0	14.1	38.4

表 6：图 4 背后的 RoboCasa365 结果，基于多任务设置下的 50 个目标任务。图 4 中带斜线、此处标 $\ast$ 的模型：用内部数据预训练。斜体：撰写本文时尚未公开的模型。

Method	LIBERO	LIBERO-Plus	RoboTwin Clean, Rand	RoboTwin Clean2Rand	RoboCasa365	RoboCasa-GR1
No pretraining
StarVLA	Qwen-RobotManip	repository	checkpoint card	Qwen-RobotManip	–	paper
Qwen-RobotManip-scratch	paper	paper	paper	paper	–	–
With pretraining – in-house data
$\pi_{0}$	Qwen-RobotManip	ACoT-VLA	InternVLA-A1.5	–	–	–
$\pi_{0.5}$	Qwen-RobotManip	ACoT-VLA	Qwen-RobotManip	Qwen-RobotManip	leaderboard	StarVLA
ACoT-VLA	paper	paper	–	–	–	–
GR00T-N1.5	–	–	–	–	leaderboard	repository
GR00T-N1.6	–	–	–	–	leaderboard	–
GR00T-N1.7	–	–	–	Qwen-RobotManip	–	–
Being-H0.7	Qwen-RobotManip	Qwen-RobotManip	paper	–	–	paper
Qwen-RobotManip	paper	paper	paper	paper	paper	–
RLDX-1	paper	paper	–	–	leaderboard	paper
Xiaomi-Robotics-1	–	–	–	–	leaderboard	–
With pretraining – public data only
ABot-M0	paper	paper	paper	Qwen-RobotManip	–	paper
ABot-M0.5	paper	paper	paper	–	paper	–
InternVLA-A1.5	paper	paper	paper	–	–	–

Table 7: Source of each baseline result in Tables 4, 5 and 6 and Fig. 4. Paper: the paper of the model. The name of another model: the paper of that model, which reports the result. Checkpoint card: the card of the released checkpoint. Repository: the per-task table of the released checkpoint in the README of the repository of the model. Leaderboard: the RoboCasa365 leaderboard. –: no published result.

方法	LIBERO	LIBERO-Plus	RoboTwin Clean, Rand	RoboTwin Clean2Rand	RoboCasa365	RoboCasa-GR1
无预训练
StarVLA	Qwen-RobotManip	代码库	检查点卡片	Qwen-RobotManip	–	论文
Qwen-RobotManip-scratch	论文	论文	论文	论文	–	–
有预训练——内部数据
$\pi_{0}$	Qwen-RobotManip	ACoT-VLA	InternVLA-A1.5	–	–	–
$\pi_{0.5}$	Qwen-RobotManip	ACoT-VLA	Qwen-RobotManip	Qwen-RobotManip	排行榜	StarVLA
ACoT-VLA	论文	论文	–	–	–	–
GR00T-N1.5	–	–	–	–	排行榜	代码库
GR00T-N1.6	–	–	–	–	排行榜	–
GR00T-N1.7	–	–	–	Qwen-RobotManip	–	–
Being-H0.7	Qwen-RobotManip	Qwen-RobotManip	论文	–	–	论文
Qwen-RobotManip	论文	论文	论文	论文	论文	–
RLDX-1	论文	论文	–	–	排行榜	论文
Xiaomi-Robotics-1	–	–	–	–	排行榜	–
有预训练——仅公开数据
ABot-M0	论文	论文	论文	Qwen-RobotManip	–	论文
ABot-M0.5	论文	论文	论文	–	论文	–
InternVLA-A1.5	论文	论文	论文	–	–	–

表 7：表 4、5、6 和图 4 中每个基线结果的来源。论文：该模型自己的论文。另一个模型的名称：报告该结果的那个模型的论文。检查点卡片：公开检查点的说明卡片。代码库：该模型代码库 README 中公开检查点的逐任务表格。排行榜：RoboCasa365 排行榜。–：没有已发表的结果。