Science

This Drone Creates a Real-Life Video Game View

The rugged, easily mounted GoPro camera changed the cinematography of action sports and house-parties forever, but it seems that was just the tip of the iceberg. The classic over-the-shoulder view in so many video games is now achievable IRL via a drone programmed to follow a subject while filming.

At MIT’s EmTech Digital conference on Tuesday in San Francisco, Skydio CEO Adam Bry revealed footage of the company’s forthcoming consumer drone for autonomous cinematography.

To do its work, this drone has to overcome a number of challenges. It has to accurately identify and track its user, so it can keep them in view, but also maintain total, real-time awareness of its surroundings so it can avoid crashing into obstacles. During all this, it’s attempting to create a good shot — keep the subject near the center of the frame, show the area ahead in the subject’s path.

In the demonstration, subjects would often try to avoid the drone, ducking behind obstacles or leading it toward a high obstacle that will get in its path. When a runner’s oath took him under a low-hanging branch, the drone went around, kept the runner in frame, then quickly caught up and got back into its preferred position.

“The system we have to today is more advanced than we could have anticipated when we started this three years ago,” Bry said in the talk. “Certainly… within that time-frame.”

The system uses more traditional machine vision techniques than modern deep learning approaches, but with relatively simple image texture analysis and cameras facing in all directions, the drone is able to sense its world down the “centimeter scale” in real-time.

This drone will be a Godsend for people looking to immortalize their latest panoramic hike or bike ride. There’s no word on the drone’s maximum speed, but it’s unlikely it could keep up with a snowboarder at full clip, at least for now. But the possible applications go on — sports like rock climbing could benefit from cinematic video, on a level with the climber, all the way up. It’s good to provide these sorts of tools to people since, at the end of the day, they will simply find a way to capture their various adventures, no matter what.

But, of course, there’s no reason this has to be used only for sports. Right now, the drone is being pitched to capture mobile outdoor activity, but there’s no reason it couldn’t be adjusted to follow an actor during a movie scene, for example, or to point not a camera but a light in their direction. Drones cameras could capture the Superbowl with a level of intimacy never before possible — but they could also accidentally block the game-winning field goal kick.

Computational cinematography” is a quickly emerging field, and this first, high octane consumer entry shows that if anything is going stop the automation of camera work, speed and ruggedness will not be the things to do it.