Brain Waves Could Be the Next Breakthrough for Physical AI Data Scarcity

Encord, a company that builds data tooling for training AI models, is experimenting with brain wave sensors to address the critical shortage of physical training data for robots. In a warehouse in San Leandro, California, Encord pilots—employees who collect robotic training data—wear headsets equipped with sensors from German neurotech startup Zander Labs that measure brain activity.

The pilot, Andrew Ceja, wears the headset while playing Jenga, carefully removing blocks. The goal is to create a dataset that captures not just visual information but also mental states such as errors, intent, and surprise. Lucas Gehrke, a Zander neuroscientist overseeing the work, explains that the level of brain activity during tasks can indicate when models need to exert more effort.

Vineeth Velmurugan, Encord’s head of robot learning and a veteran of OpenAI’s robot lab and Berkshire Grey, calls this the “bleeding edge” of solving the robotics data bottleneck. He notes that unlike large language models, which were trained on vast amounts of text from the internet, physical training data for robots is scarce and hard to come by. According to Velmurugan, a dataset roughly five times the size of YouTube’s video corpus is needed to make significant progress.

Encord originally helped companies annotate data for machine vision applications, but as clients moved to end-to-end learning for robotic manipulation, the company pivoted to manufacturing training data. Currently, Encord collects egocentric video from factory workers and uses leader-follower robotic arm rigs to generate data on specific skills like pouring coffee or stacking chips. The company is also testing forearm sensors that detect electrical signals in muscles to build 3D models of hand positions.

Data is annotated with physical descriptions like “right hand tightens bolt,” which Velmurugan estimates is 100 times more valuable than raw video for training specific tasks, though it costs 20 times more to produce. The high cost of generating physical data is a key challenge, as it cannot be simply scraped from the web like text. Velmurugan believes Encord’s unique position—working with multiple robotics companies—gives it insight into which data techniques are gaining traction industry-wide.