Gemini Robotics ER 2 turns reasoning into an agent for robots

On July 30, Google introduced Gemini Robotics ER 2, an embodied-reasoning model available in preview through the Gemini API and Google AI Studio. The company’s phrase “high-level brain” is useful if its boundary remains visible: ER 2 does not replace the controller that moves each joint. It receives text, images, video, and audio; breaks a request into steps; and calls vision-language-action (VLA) models, navigation functions, or other interfaces as tools that actually execute movement.

That separation is the material part of the release. A developer can connect the same planner to different robots, describe what each one can do, and receive structured tool calls without training a motor model from scratch. ER 2 is based on Gemini 3.5 Flash, has a 128,000-token context window, and comes through two endpoints: a general one and a bidirectional streaming version for continuous video and audio. The previous ER 1.6, released in December 2025, will be shut down at the end of August.

The advance is in closing the loop

ER 1.6 could already point to objects, interpret instruments, and reason about spatial relationships. ER 2 adds a more continuous form of agency: while the robot acts, the model watches video, estimates progress, detects whether a step has finished, and can replan. Google also shows a single ER 2 dividing a task among different platforms, such as a humanoid and a two-armed manipulator. This is not collective intelligence among autonomous machines; it is a shared coordinator that knows several interfaces.

That changes what can be built with an API. An application no longer has to connect perception, planning, outcome checking, and error recovery as entirely separate stages. It can keep those decisions in a multimodal loop that runs while the low-level controller continues acting. The streaming version aims to reduce pauses between “thinking” and “doing,” a difference that matters much more in a moving robot than in a chatbot.

The published results support that direction but do not yet establish general competence. In one internal evaluation, ER 2 identifies a task’s progress band—one of five intervals—correctly in 57.4% of frames. In another, it identifies the precise moment of an event with 91.3% accuracy and a mean temporal error of 0.96 seconds. These are provider-run tests on selected tasks; they do not say how the system will respond to a cluttered home, new lighting, an obstructed camera, or a poorly described motor interface. The videos involving Spot, Apollo 2, and Franka F3 Duo show real integrations, not a reliability rate for every robot.

The availability distinction matters too. Google has opened the reasoning model as a cloud service, not its weights or a complete robot. Hardware, controllers, networking, latency, and the cost of processing video remain part of the system each team must assemble. Even so, moving the coordinator behind an API lowers a concrete barrier: teams can experiment with physical agency without every lab developing its own foundation model.

A stop command is not a safety system

The safety technical report is especially useful because it qualifies some of the launch language. Its new ASIMOV-Agentic benchmark tests whether the agent refuses unsafe tasks, asks for clarification, checks whether a VLA can perform a step, and calls a stopping tool when danger appears. But the document expressly says that it does not evaluate the functional-safety architecture: certified hardware, redundancy, and real-time guarantees.

The human-proximity test shows why. Models must decide from stereo images whether a person has crossed a perimeter and invoke robot_stop(). According to the report, configurations that keep false stops below 5% miss more than 40% of real hazards; those that reduce missed hazards to roughly 10–15% stop the robot unnecessarily 15–25% of the time. No model yet reaches the desirable region of almost no misses and almost no false alarms.

Google obtained better figures in a bounded demonstration with the Apollo 2 humanoid: 99% human detection and 96% reliability in transitioning to a safe pose. Both findings can be true. A garage test with prepared thresholds and paths shows that the integration works; the more varied benchmark shows why it must not become the only barrier. The model card bars safety-critical work, including healthcare and transportation, and calls for discretion before use in public, commercial, or production environments.

The report provides another, less conspicuous lesson. When ER 2 receives only a minimal description of a VLA’s skills, it correctly classifies 62% of test requests as feasible or infeasible. With a detailed description of the VLA’s training distribution, it reaches 95.8%. The coordinator’s safety therefore depends on a readable contract describing what each tool can do. Inventorying capabilities, limits, and failure states is not documentation added later; it is part of the agent’s behavior.

My reading is that ER 2 is a real advance precisely because it does not try to learn every movement of every robot. It turns planning, temporal perception, and tool assignment into a reusable layer. That modularity also creates a clear place to intervene: when an instruction is ambiguous, the agent can ask; when confidence is low, it can abstain; when an alarm arrives, it can request a stop.

But “can request” does not mean “will stop in time.” Barriers, force and speed limits, emergency stops, and deterministic controllers must protect the space even if the model, camera, or connection fails. ER 2 can add semantic reasoning—understanding that a metal container should not go into a microwave—on top of that base; it cannot certify the base.

There is also a privacy cost inseparable from the loop. The documentation warns that video and audio may capture people’s voices, faces, and likenesses. It requires notice and consent before identifiable people are present around the robot, along with data minimization and face blurring where practical. A cloud-hosted physical agent does not merely move a machine: it turns a workplace or home into a continuous model input.

A responsible test should therefore begin not by asking how long the robot can work alone, but what happens when it misidentifies an object, loses a frame, receives two conflicting instructions, or loses its network connection. This release makes the happy path much easier to build. Its own report makes clear that the hard work is designing paths for doubt, stopping, and recovery without delegating them to the same prediction that can be wrong.

Sources

• Google DeepMind, Introducing Gemini Robotics ER 2, July 30, 2026.
• Google DeepMind, Gemini Robotics ER 2 Model Card, July 2026.
• Google AI for Developers, Gemini Robotics ER documentation and endpoints, accessed July 31, 2026.
• Google DeepMind, Gemini Robotics 2: Safety Evaluations, July 29, 2026.
• Google DeepMind, Gemini Robotics ER 1.6: Enhanced Embodied Reasoning, December 2025.