Researchers from the Laboratory of Artificial Intelligence of the Massachusetts Institute of Technology presented MoSculp – a program that can do 3D sculptures of the motion of an object based on video clips. The algorithm time-by-picture analyzes the motion video and recreates its three-dimensional shape, visualizing the movement in the form of waves. The article with the work description is published on arXiv.
Analyzing complex motion of objects on static photographs is very difficult, so instead, other methods are used: slow motion or a variety of photography. For example, the method of chronophotography allows you to visualize the movement of an object with pictures taken with small gaps: on the finished image you can see the whole process of movement. Nevertheless, all methods used for visualization of motion have an important limitation: they allow us to consider it only in two-dimensional space. For a thorough assessment of complex motility, however, movement should be assessed in volume.
A group of developers and designers led by Xiuming Zhang from MIT came up with a way to solve this problem. The motion detection algorithm and its subsequent three-dimensional visualization first marks a two-dimensional object on the video through all the frames. Then he recreates the three-dimensional shape of the object based on all frames, creating his “skeleton.” Then this “skeleton” is used to visualize the motion on the video, as a result of which the movement takes shape.
In the program, as noted by the creators, you can select the desired zone of motion (for example, the entire body, only the hands or only the legs), and also choose its texture: visualize the movement in wood or skin. The resulting sculpture can be rendered into a three-dimensional image. To look at other examples of MoSculp’s work, as well as download and try the program yourself, you can go to the lab’s website.
The sculptures made with MoSculp can also be printed on a 3D printer. You can read more about this technology with our 3D printing section .