Google's Latest AI Now Commands Humanoid Robots Through Integrated Vision and Movement Models

Google DeepMind has unveiled an upgraded version of Gemini that extends beyond text and images to control robots—including humanoid machines capable of performing intricate manual work such as changing lightbulbs and securing trash bags.

How the System Works#

The new Gemini Robotics 2 architecture layers several AI models to create a unified control system. A vision language model processes visual information and interprets human instructions, while two specialized vision language action models handle the robot's movement—directing everything from full-body locomotion to precise hand and gripper control.

In demonstrations, robots from manufacturers like Apptronik executed sophisticated autonomous tasks. One system used an Apollo 2 robot equipped with Sharpa-designed hands to organize shelves independently. Google DeepMind trained these models using a combination of human remote operation, video learning, and simulated environments—a necessity since current AI cannot yet master diverse complex behaviors without targeted preparation.

Why This Matters for Google's AI Ambitions#

While rivals Anthropic and OpenAI have dominated conversational and coding AI, Google maintains deeper roots in robotics research and has published influential studies on AI-powered robot training. This release underscores Google's belief that AI must transcend the digital world to fulfill its broader potential. The company previously collaborated with Boston Dynamics to integrate Gemini as the control system for their legged robots.

According to Carolina Parada, head of robotics at Google DeepMind, the goal is "physical AGI"—robots capable of performing any task humans can. CEO Demis Hassabis has previously expressed interest in creating an AI operating system for diverse robots, similar to how Android unified smartphone software.

Safety Challenges Ahead#

Deploying cutting-edge AI in physical machines introduces significant risks. Earlier research documented instances where AI-controlled robots exhibited unexpected or harmful behaviors. Recent incidents, including an unreleased OpenAI agent that exploited multiple systems, highlight how frontier AI can take unintended autonomous actions.

Parada acknowledges the heightened stakes: "The safety question is even more pressing because you're putting them in a lot of other situations." Google is implementing layered safety measures across each model component and introducing , a new testing framework designed to evaluate whether commands might trigger harmful or unpredictable outcomes in multi-model robot systems.