Former Meta researchers launch visual AI model to guide factory robots

Two researchers who previously worked at Meta's AI division have started Perceptron, a company dedicated to bringing AI systems into physical industrial spaces. This week they unveiled Isaac 0.5, a new vision model engineered to enable robots to work autonomously on the factory floor and in warehouse settings.

What Isaac 0.5 Does#

The model tackles a central challenge in robotics: helping machines both perceive their surroundings and make decisions about how to act. Isaac 0.5 equips vision-guided robots with the capability to work through complex, multi-step tasks in real environments. Beyond guiding robots in real-time, the system can also analyze video footage captured by those machines to extract actionable insights about warehouse or manufacturing operations.

The startup released the model as open-weight, meaning researchers and developers can examine its underlying parameters and training methodology.

The Problem It Solves#

Current industrial AI solutions force an uncomfortable tradeoff: either adopt general-purpose models that require significant cloud computing resources for each robot, or use narrow models that excel at specific tasks but lack flexibility. Perceptron positions Isaac 0.5 as a third way—a flexible, general-purpose system that adapts to different environments rather than being hardwired for one repetitive process.

To illustrate the gap, consider a package-sorting robot. Such a machine must complete numerous subtasks: read labels, calculate spatial relationships between boxes, select which item to grab, and sequence its movements efficiently. Existing tools handle many of these individually, but few integrate them into a coherent, adaptable workflow.

Training at Scale#

Building a model capable of such versatility requires immense volumes of diverse video data. Perceptron fed Isaac 0.5 approximately one million hours of general video footage to teach the system to recognize diverse environments and scenarios. The team supplemented this with egocentric video—first-person perspectives captured via GoPro or wearable cameras showing humans performing physical tasks—and UMI video, which records repetitive human motions to teach robots movement patterns.

The founders declined to reveal specific data sources but confirmed internally constructing "petabyte-scale datasets spanning images, text, video, and robotic trajectories."