EXIMO: VLM-Guided Exploration of VLA Policies

Summary

EXIMO is an efficient algorithm for fine-tuning vision-language-action (VLA) policies to learn new tasks quickly. It operates in three stages: explore, imitate, and optimize. During exploration, a vision-language model (VLM) acts as a planner, breaking down long-horizon tasks into shorter subtasks for the VLA, and together they collect an orchestrated dataset. During imitation, the VLA is fine-tuned on this data. Finally, residual off-policy reinforcement learning further optimizes the policy. Experiments show EXIMO significantly outperforms existing approaches in sample efficiency and final performance.