The research company co-founded and chaired by Elon Musk used two separate neural networks to develop its one-shot imitation learning system.

The first, a vision network, analyzes an image from the robot’s camera to determine the location of objects in reality (in OpenAI’s video example, these objects are blocks of wood on a table). The network is able to do this despite never having seen the actual table or blocks before. Instead, the researchers trained it using hundreds of thousands of simulated images, each featuring various permutations of lighting, textures, and objects.

The second, an imitation network, determines the intent of a task it observes a human demonstrating via a virtual simulation. It then imitates the task in the real-world setting. Again, this network was trained on thousands of virtual demonstrations, but none that took place in reality.

Source: https://futurism.com/elon-musk-just-unveiled-breakthrough-ai-research-heres-what-your-need-to-know/