Roboticist Rodney Brooks, founder of iRobot, on why the current push to train humanoid robots with video is doomed to fail. Sensory input is so important to how humans learn and execute dexterous tasks, and yet we lack the technology to capture the sense of touch and proprioception. Not only that, but the success in end-to-end learning upon which the hype is predicated (speech to text, image classification and LLMs) is built on models for how humans process speech, vision and language inputs; and we lack similar understanding of how humans/animals process touch and proprioception.

Source: Why Today’s Humanoids Won’t Learn Dexterity – Rodney Brooks