Startup Uncorporeal Systems launched its Holographic Photography technology demo on SteamVR for HTC Vive that lets users step into a holographic, 3D panoramic photo and view it in true VR. This technology wasn’t the focus for the volumetric performance capture company. In fact, Sebastian Marino, CEO and co-founder of Uncorporeal, told Upload in an email that this demo wasn’t originally planned for public release.
“We’ve always been interested in comprehensively approaching volumetric video and light-field capture, but, this really caught us by surprise,” Marino said. “We found it incredibly fun to experiment with as we worked late into the evenings.”
From an innovation standpoint, Marino said it led the team to ask a lot of “what-if” questions: What if we point our cameras outward — as opposed to inward, which we do for our volumetric performance capture? Could we actually capture environments and add the same layers of depth to them? What algorithm changes would we need to make so this is possible? What if we completely solve this with software? What would happen if we used off-the-shelf consumer cameras? Mobile phone cameras?
“Then we made some improvements and started showing people, and they were saying ‘holy *^@#&!,’” Marino said. “That’s when we figured out we had something here.”
Unlike traditional photogrammetry, Holographic Photography takes a series of images (or videos) from a single location, like with the panorama mode on a phone or camera. Normally, this wouldn’t provide enough information for a 3D reconstruction, but Uncorporeal’s tools are said to be capable of higher precision than anything else Marino has seen available, so they are still able to provide the viewer with a strong sense of depth.
Marino said ordinary panoramic or stereo 360 captures only represent the parallax between your two eyes, but can’t represent how your view changes as you move your head. This is particularly obvious when you have nearby objects in the scene.
“Our technology allows the viewer enough movement to feel like they are really in the scene, but is still just as easy to capture as a normal panorama,” Marino said. “Other photogrammetry offerings can’t provide comparable depth information without using hundreds of photographs from different positions.”
The process of shooting the set of images that go into each of these holographic photos is somewhat similar to shooting for a stitched panorama. But, with a stitched panorama you want to minimize parallax in order to avoid artifacts and in this case you want to maximize parallax in order to allow the tech’s “solver” to determine the depths of different objects.
“Provided that you have a suitable camera and wide angle lens, and that you know a little bit about aperture and exposure settings, anyone can capture one of these,” Marino said. “The key, like with a panorama, is making sure you shoot enough photos, with sufficient overlap.”
Initially, Uncorporeal will be providing Holographic Photography to its professional customers to create photo-realistic environments, and Marino said they will very soon be able to offer the same technique for panoramic video. Eventually, Marino would like to offer it to general consumers too.
“We see the Holographic Photography technology as complementing our existing volumetric performance capture offerings,” Marino said. “It is easy and cheap to use for people who are already familiar with shooting 360-degree video, but is more aimed at generating seated rather than room-scale experiences. We see ourselves as a general reality capture company, rather than providing any single technology.”
Marino won a Science & Technology Academy Award in 2002 for the development of the ILM Creature Dynamics System, which is used to simulate the motion of cloth, hair and skin for computer-generated characters in feature film production. Several key members of the Uncorporeal team have Hollywood roots in the visual effects industry. They were very familiar with the challenges of recreating real people, places and things using digital methodologies.
“Many people have heard of the ‘uncanny valley,’ which is the tendency for recreations of humans to look unnatural as they approach (but don’t achieve) photo-realism,” Marino said. “There has also been a bit of a backlash against CGI in movies lately, in favor of practical or in-camera effects. These are all evidence of the same underlying fact, which is that our brains contain very sophisticated subsystems for analyzing things and judging whether they look real or not. From the outset, our goal has always been to capture and present reality, rather than to simulate or fabricate it from scratch. This is how we tackled volumetric performance capture that we are mostly known for, and it’s also how we are tackling this new area of location or environment capture.”
Marino sees a future far beyond the stereoscopic 3D films of today, which he believes are often disappointing because no matter how far forward it jumps out, the film remains constrained to a big rectangle on the wall in front of you. Barco has recently been trying to break free from that box by adding additional screens on either side, but Marino said Uncorporeal’s technology will ultimately offer filmmakers the ability to completely surround the audience in the story world, in a way that respects and obeys their side-to-side and front-to-back head movements.
Unlike the 360-degree panoramic films that people are making today, the viewer will feel transported and present in the locations where the story is unfolding.
Marino also sees this technology having many enterprise applications. As just one example, architecture visualization is already a big field of VR and given that architects already generate 3D virtual models, it makes perfect sense to create a fully virtual world that can be explored. But as those plans turn into construction sites, the model is no longer an accurate reflection of the project in development. This technology would allow the developer to provide regular updates to their clients, in the form of a virtual site inspection.
“From the initial feedback we’re getting, this technology could be used for tourism, commerce, education…just to name a few,” Marino said. “There are many use-cases we’re starting to explore with potential clients.”
Marino said the panoramic experiences that consumers have been able to try on Google Cardboard and Gear VR have primed the market for the kind of truly immersive experiences that only high-end HMDs can currently provide.
“Once positionally-tracked headsets become available to a wider audience, there is likely to be an explosion of demand for the type of truly spatial experiences that our technology can provide,” Marino said. “People are having a lot of fun with what is currently available, but the next steps towards photo-realistic, immersive experiences are going to be truly mind-blowing.”
John Gaudiosi is a freelance reporter with experience covering the world of videos game for multiple outlets, such as Fortune and [a]listdaily. Follow him on Twitter: @JohnGaudiosi.
