HOIST: Supplementary material

Abstract

This document provides the HOIST training algorithm, additional placement and payload-motion results, and implementation and hardware details.

HOIST Algorithm

Algorithm S1: HOIST Training

1. Inputs: VR teleoperation demonstrations $\mathcal{D}$, pretrained VLA policy $\pi_\text{VLA}$ (GR00T N1.6), whole-body controller $f_\text{WBC}$ (SONIC, frozen throughout)

2. Output: Refined hoisting policy $(\pi_\text{VLA}^*, \mu_\theta^*, f_\text{WBC})$

3. Phase 1 — Supervised Finetuning

4. Collect VR teleoperation demonstrations with multimodal observations: $\tau_i = \bigl(\{o_t, u_t^\text{demo}, a_t^\text{motor}\}_{t=1}^{T_i},\, y_i\bigr)$

5. Each observation $o_t$ contains ego RGB/depth, side-view RGB, proprioception, ego velocity, navigation command, and language goal $y_i$

6. Convert teleoperation commands into VLA action chunks $\Delta U_t^\text{demo} = [\Delta u_t,\dots,\Delta u_{t+H-1}]$ with horizon $H{=}24$

7. Initialize from the pretrained GR00T N1.6 VLA and attach encoders for newly introduced modalities (depth, ego velocity, navigation command)

8. Supervise the flow-matching action expert to generate demonstration chunks conditioned on VLM features and robot state, using 4 denoise steps

9. Finetune the final 4 VLM layers, the action expert, and the new-modality encoders on $\mathcal{D}$; keep the remaining VLM backbone and $f_\text{WBC}$ frozen $\Rightarrow$ obtain $\pi_\text{VLA}^*$

10. Freeze $\pi_\text{VLA}^*$ and $f_\text{WBC}$; cache VLM features for rollout-level RL refinement

11. Phase 2 — Iterative Batched RL Refinement

12. Initialize noise-steering SAC actor-critic $(\mu_\theta, Q_\phi)$

13. For iteration $k = 1,2,\dots,K$:

    1. Collect $N=10$ autonomous rollouts using $(\pi^*_{\mathrm{VLA}}, \mu_\theta, f_{\mathrm{WBC}})$

    2. The actor deterministically predicts a replacement initial-noise tensor $$a_t^{\mathrm{RL}} = \mu_\theta(o_t), \quad a_t^{\mathrm{RL}} \in \mathbb{R}^{24 \times 25}$$

    3. Here, 24 is the action-chunk horizon, and 25 is the per-step planner-command dimension covering head target, left/right end-effector targets, navigation command, and base-height command

    4. Replace the flow-matching action expert’s initial noise with the scaled actor output: $$\epsilon_t^{\mathrm{init}} \leftarrow s_k \, a_t^{\mathrm{RL}}$$

    5. Execute the VLA action chunk through the frozen whole-body controller $f_{\mathrm{WBC}}$

    6. For each rollout $\tau_i$:

       1. Observe terminal payload pose $(\hat{x}, \hat{y}, \hat{\psi})$ and target $(x^*, y^*, \psi^*)$

       2. Compute terminal errors: $$\Delta x = \hat{x}-x^*, \quad \Delta y = \hat{y}-y^*, \quad \Delta \psi = \hat{\psi}-\psi^*$$

       3. Compute terminal reward: $$r_T = \exp\!\left(-\left[ 1.5\!\left(\tfrac{\Delta x}{100\,\text{mm}}\right)^{\!2} + 1.0\!\left(\tfrac{\Delta y}{80\,\text{mm}}\right)^{\!2} + 1.0\!\left(\tfrac{|\Delta\psi|}{30^\circ}\right)^{\!2} \right]\right)$$

    7. Convert rollouts into an offline replay buffer $\mathcal{B}_{0:kN}$

    8. Train/update $(\mu_\theta, Q_\phi)$ with offline SAC on $\mathcal{B}_{0:kN}$ for fixed gradient steps

    9. Resume from previous RL checkpoint for the next stage

14. Return refined hoisting policy $(\pi^*_{\mathrm{VLA}}, \mu^*_\theta, f_{\mathrm{WBC}})$

Design rationale.

During RL refinement, the VLA policy $\pi_\text{VLA}^*$ and WBC $f_\text{WBC}$ remain frozen; only the actor-critic $(\mu_\theta, Q_\phi)$ is updated. This preserves the imitation-learned visuomotor prior while restricting exploration to the VLA initial-noise space. Each batch of autonomous rollouts is converted into an offline replay buffer, and RL policy is updated with SAC using terminal task rewards. Thus, the method is not tabula-rasa RL, but rollout-level refinement of the frozen VLA flow-matching action-generation trajectory through a compact noise-steering module.

Additional Placement Results

Tables S1 and S2 report terminal placement errors in simulation and on the real platform, respectively. HOIST denotes VLA-50 with iterative batched RL refinement; the rollout count indicates the total RL rollouts used.

Simulation

All three HOIST variants substantially reduce $|\Delta x|$ relative to VLA-50. Performance is not strictly monotonic in simulation: HOIST (+20 RL) achieves the lowest mean Manhattan error (2.54 cm), while HOIST (+30 RL) shows a slight increase (6.25 cm). We attribute this to evaluation variance over a small number of trials; on the real platform the mean Manhattan error decreases monotonically (Table S2).

Real Platform

On the real platform, HOIST reduces mean Manhattan error from 9.28 cm (VLA-50) to 6.38 cm (+30 RL), indicating improved translational placement. However, yaw remains high-variance: +20 RL improves mean yaw relative to VLA-50, whereas +30 RL has larger mean yaw error. We therefore interpret the real-platform gains primarily as improvements in translational placement and Manhattan error, rather than as monotonic orientation refinement.

IMU Residual Motion Analysis

IMU data were collected in simulation only. We attach a 6-axis IMU to the payload and record accelerometer and gyroscope readings throughout each trial. Acceleration is gravity-compensated using the payload orientation quaternion and reported as RMS over active frames (frames where gravity-compensated acceleration magnitude $> 0.5\,\text{m/s}^2$ or angular velocity magnitude $> 0.01\,\text{rad/s}$).

Interpretation.

Compared with VLA-50, the HOIST variants generally reduce gyroscope RMS, with the strongest reduction at +20 RL. Acceleration metrics and gyroscope peaks are not uniformly improved across RL stages, so we interpret the IMU results as evidence for reduced angular RMS in some refined policies rather than a uniform reduction of all residual-motion metrics. Human teleoperation exhibits higher accelerometer readings, while gyroscope responses overlap with or are exceeded by some learned-policy settings.

Implementation Details and Hyperparameters

VLA Supervised Finetuning

RL Rollout Refinement

Hardware and Sensing