Science One prompt

Robot demos turned into a MuJoCo simulation from one prompt

Given only multi-view RGB video and robot actions by lingxiao guo, a robotics master's student at CMU, GPT-6 calibrated the cameras, built the object assets, identified the physics parameters, ran MuJoCo and rendered the result in Blender.

View on X
957likes
104reposts
593saves
162Kviews
Made with
GPT-6, MuJoCo, Blender
Role
Robotics master's student, CMU
Posted
lingxiao guo@Lingxiao234

GPT-6 does real2sim end to end from robot demonstrations.

I gave it only multi-view RGB and robot actions, and one simple prompt.

It calibrated the cameras, built the object assets, performed physics system-ID, ran MuJoCo, and rendered in Blender.

View on X
Still from the video

More in Science

A ChatGPT-designed schizophrenia drug, synthesized in a garage labScience@DouglasYaoDY15K1.1K6.8K
A Napoleonic general's cipher, unread for 217 years, broken in 6 hoursScience@CarterWChurch12K1.7K6.2K
All 15,973 semigroups of order 6 classified by AI agents in LeanScience@nasqret843141251
A conjecture open since 2004 is now a theorem, checked in LeanScience@LechMazur2073396