You can play for people from any video.
Researchers at Facebook’s artificial intelligence lab created the Vid2Play neural network. She cuts people out of commercials and turns them into playable characters that can be controlled.
The algorithm generates new shots with the hero of the video, while the video can have an arbitrary background, and the system effectively displays the person in motion.
For this, the team used two neural networks: Pose2Pose marked the pose of the person and cut movements, and Pose2Frame placed the character and the surrounding environment on a new “layer”. As a result, the hero can be controlled by any simplest control, such as a keyboard or joystick.
To train the system used three videos with a length of 5 to 8 minutes. One was devoted to playing tennis in the street, the second to a man swinging a sword indoors, and the third to a walking person.
Each neural network solves a computational problem that has never been completely solved before, paving the way for video games with realistic graphics. In addition, controlled characters from YouTube videos can find a place in virtual measures and augmented realities.