Ever since Epic Games opened its Unreal Engine 4 technology to the world, new use cases have been coming in fast and furious. The latest example of just how far real-time video game engine technology has come will be on display at GDC 2017 this week. Epic Games partnered with visual effects and creative content studio The Mill to shoot a new Chevrolet video that utilizes UE4 technology and an augmented reality Blackbird motion tracking vehicle.
That electric car is the brainchild of The Mill. And before the top-secret vehicle was revealed, Epic Games CTO Kim Libreri got an early look under the hood.
“This high-tech car can run any performance envelope of any car, and it has all the electronic equipment that you need to be able to track it, so you know where it was relative to the camera car, and also generates the light and reflection information that you need to be able to light a computer-generated car,” Libreri said.
Libreri met with Vince Baertsoen, the head of R&D at The Mill, last year in Germany at FMX 2016. At the time, the one challenge Baertsoen had was that the director filming the car was still seeing the Blackbird and the transformation into whatever car they wanted it to become occurred in post production. The Holy Grail was for those shooting the sequence to see the final version of the vehicle in real-time.
At GDC, Epic is showcasing a 2017 Camaro ZL1 in a race against the Chevy FNR concept car, except the vehicles are actually photorealistic pixels running in real-time using UE4 technology. To prove the point, the ZL1 can be swapped out for a classic ’69 Camaro.
“Those cars are built like we would do a video game asset,” Libreri said. “Right now, it’s a specialized version of Unreal because we’ve just put the demo together, but these are features that are going to be available in regular Unreal. The only difference between this and a car that you would put in a video game is the amount of polygons in the car. We actually have a couple levels of detail to the car. The one that you see in the video is comprised of millions of polygons. We also have a low resolution version that would be a more normal game-level asset that would run on a PlayStation 4. The materials and lighting and most of the things you see in the video would run on a console in a more regular video game environment.”
These virtual vehicles were super-imposed on top of the Blackbird during a live action shoot on the Angeles Crest Highway in Los Angeles. The Blackbird uses professional grade AR, filming a 360 video from the center of the vehicle using Red Epic cameras. Everything that’s around the vehicle is filmed as if it was a panoramic 360 photography. And a spinning LiDAR scanner is scanning the surrounding environment.
“They take the output from these four cameras, stitch it into a panorama, and then beam it to Unreal Engine wirelessly,” Libreri explained. “And then we take that as lighting and reflection information that we can place on top of a car that they’ve tracked with a real-time tracking system developed by partner company Arraiy.”
Before joining Epic in 2014, Libreri spent 20 years working in the Hollywood visual effects industry at companies like Lucasfilm, Industrial Light & Magic, and Digital Domain. These vehicles are inserted into the live action compositing background plates, which are equivalent to the kinds of images ILM would use.
The director of the 60-second video is sitting inside a customized Mercedes ML that has a Russian Arm that can film the Blackbird from any angle. Inside the Mercedes, he can watch the UE4-generated vehicle in real-time and make filming adjustments on-the-fly. A PC running on a high-end consumer NVIDIA graphics card is set up inside of the Mercedes to transform the Blackbird into the Camaro vehicles.
“We’re using some pretty beefy hardware for the demo right now, but that hardware capability is going to be available in the cloud very, very shortly, so you’ll be able to run these kinds of graphics-on-demand projects from the cloud,” Libreri said.
In addition to handling the augmented reality, UE4 is also handling a lot of information simultaneously.
“Each of these shots is an individual shot like you would have in Premiere or Avid, where you can cut backwards and forwards, and trim, and add the audio tracks,” Libreri said. “It’s all running just like you were doing normal visual effects photography, but inside a game engine.”
The Mill officially revealed the Blackbird on stage at GDC during Epic’s keynote. And Chevy also used that event to debut the final version of the race, which offers a wow factor when the vehicles enter a tunnel and go all TRON-like to showcase the real-time visual effects UE4 opens up.
“At every GDC we like to do some project that not only blows people away and inspires them, but shows that together with a customer we take some of the best people on the planet using our technology and make our engine better,” Libreri said. “We do something that people thought was impossible, so that’s why we went to this next level.”
