Robots are learning how to replace human hands, by watching the world through human eyes.
Perceptron, a startup founded in November 2024 by Armen Aghajanyan and Akshat Shrivastava -- both former research scientists at Meta's Fundamental AI Research (FAIR) division -- just released Isaac 0.5, and the training approach is the interesting part, per TechCrunch's reporting.
How It Actually Learns
"The company also relied heavily on what is known as ego video, video captured, typically through a GoPro or a wearable camera, from the perspective of a person completing a physical task, as well as UMI video, which are similarly used to teach AI systems movements by recording repetitive human actions." That's on top of a genuinely massive base: Perceptron built what it calls petabyte-scale datasets, training the model on roughly one million hours of general video to teach it to recognize settings, objects, and scenarios, before layering the more targeted ego and UMI footage on top to teach specific physical movements.
The Problem They Say Nobody's Solved
"Physical AI today forces a false choice: generalist foundation models that need multiple dedicated cloud GPUs for every instance, or narrow models that handle perception or control, but never both." Isaac 0.5 is pitched as trying to close that gap -- a single model meant to both understand a scene and act on it, without needing a separate specialized system for each half of the job.
Why Even Simple Tasks Are Hard
Co-founder Akshat Shrivastava used package sorting as the example, framing it as: "Imagine there's a robot being deployed to sort packages right now. What are the tasks it would need to do?" His answer: "A robot would first have to read the label on the package, do some spatial analysis to understand where the boxes are, and decide which one to pick up" -- a sequence that sounds trivial to a person but requires several distinct kinds of perception and reasoning chained together correctly, every time, for a machine.
The Founders' Own Confidence
"'Nothing like this really exists out there,' said [Armen] Aghajanyan. 'We're really excited about it.'"
Where the Money's Coming From
Isaac 0.5 is being released as an open-weight model, meaning anyone can inspect its parameters and training materials. Perceptron previously raised $16 million in 2024 from Bessemer Venture Partners, The Explorer Fund, and SmartGateVC, and is reportedly closing an additional funding round now, with its sights set on manufacturing, logistics, warehousing, security, and even media and entertainment.