New England Patriots quarterback Tom Brady has often credited his success to spending countless hours studying his opponent’s movements on film. This understanding of movement is necessary for all living species, whether it’s figuring out the best angle for throwing a ball, or perceiving the motion of predators and prey. But simple videos can’t actually give us the full picture.
That’s because traditional videos and photos for studying motion are two-dimensional, and don’t show us the underlying 3-D structure of the person or subject of interest. Without the full geometry, we can’t inspect the small and subtle movements that help us move faster or make sense of the precision needed to perfect our athletic form.
Recently, though, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have come up with a way to get a better handle on this understanding of complex motion.
The new system uses an algorithm that can take 2-D videos and turn them into 3-D-printed “motion sculptures” that show how a human body moves through space.
In addition to being an intriguing aesthetic visualization of shape and time, the “MoSculp” system could enable a much more detailed study of motion for professional athletes, dancers, or anyone who wants to improve their physical skills.
“Imagine you have a video of Roger Federer serving a ball in a tennis match, and a video of yourself learning tennis,” says PhD student Xiuming Zhang SM '18, lead author of a new paper about the system. “You could then build motion sculptures of both scenarios to compare them and more comprehensively study where you need to improve.”
Because motion sculptures are 3-D, users can use a computer interface to navigate around the structures and see them from different viewpoints, revealing motion-related information inaccessible from the original viewpoint.
Zhang wrote the paper alongside MIT professors of electrical engineering and computer science William Freeman PhD '92 and Stefanie Mueller, PhD student Jiajun Wu SM '16, Google researchers Qiurui He and Tali Dekel, as well as former CSAIL PhD students Andrew Owens SM '13, PhD '16 and Tianfan Xue PhD '17.
