Today Oculus announced they have acquired Surreal Vision one of the top teams in the world working on computer vision technology. While this announcement likely has very little effect on the upcoming initial consumer version of the device, it could have a lot of implications moving forward – including perhaps tipping Oculus’ hand on creating ‘room scale VR’ that has been popularized by the HTC Vive in the future as well as plans for AR in the future.
Surreal Vision was founded by an all star group of three scientists – each of whom has helped to pioneer work in their field. Richard Newcombe is the inventor of KinectFusion and DynamicFusion as well as DTAM (Dense Tracking and Mapping) which he co-invented with Steven Lovegrove, one of Surreal’s other co-founders. Alongside them is Renato Salas-Moreno, the co-inventor of SLAM++ (which he invented with Newcombe) as well as the inventor of Dense Planar SLAM. All together the three make for a very impressive team.
In the official post on Oculus’ blog, the Surreal team shared some thoughts on the acquisition and what their future will be with Oculus:
“At Surreal Vision, we are overhauling state-of-the-art 3D scene reconstruction algorithms to provide a rich, up-to-date model of everything in the environment including people and their interactions with each other. We’re developing breakthrough techniques to capture, interpret, manage, analyse, and finally reproject in real-time a model of reality back to the user in a way that feels real, creating a new, mixed reality that brings together the virtual and real worlds.
Ultimately, these technologies will lead to VR and AR systems that can be used in any condition, day or night, indoors or outdoors. They will open the door to true telepresence, where people can visit anyone, anywhere.”
There are a couple takeaways from this statement, which can be read in full here. First of all, we can be sure that the team will be continuing to build on the work they have done in the past, as they mentioned “overhauling” the algorithms they already had. It seems like they will also have the time they need to work on this complex issue. The post mentions that the team will be working at Oculus’ Research facility in Redmond, Washington. This is likely the same research facility that Nate Mitchell teased at TechCrunch earlier this month, which is focused on solving problems that are perhaps ten years out.
The second big takeaway is that this appears to confirm that Oculus wants to not only revolutionize virtual reality, but wants to build augmented reality as well. The vision that the team painted, “these technologies will lead to VR and AR systems that can be used in any condition, day or night, indoors or outdoors,” suggests a lot more mobile experience, so it seems safe to say these technologies are being aimed at Mobile VR (i.e. the Gear VR) first. But it seems safe to say that Oculus is preparing for a world where both VR and AR exist at a mass market level – especially as their trajectories seemed aimed at intersection.
Among the most interesting things that the Surreal Vision team brings to the table is SLAM++, which we mentioned earlier. SLAM++ is an improvement on the traditional SLAM techniques, which uses a camera and a lot of software and algorithms to simultaneously map an environment and track the location within the environment. SLAM++ improves upon that tracking and mapping by localizing it to an object based level – meaning that the system can scan and track objects in the room – and then map them virtually in a game. The possibilities for this are ample.
One of the first things that comes to mind is an improved version of Valve’s Chaperone system, which uses Lighthouse tracking technology to detect walls and objects in a virtual space. This means that you can freely walk around in the room scale space provided by the HTC Vive and not worry about running into walls and tripping over the couch. The overlay is very simple, based on my experience, nothing more than a wireframe that fades in if you get too close (that could potentially change by the consumer release). If advanced and perfected, SLAM++ would allow you, for example, to have your hands and keyboard mapped into the game space in real time (which would make doing work in VR vastly easier). It would also be far more detailed than Chaperone which likely will be able to detect your desk but not necessarily all the objects on it.
This acquisition is Oculus’ fourth since joining Facebook last year – with three of the four: 13th Lab, Nimble and now Surreal Vision, focused on computer vision solutions. The pattern definitely suggests that Oculus is looking to eventually release a headset with inside out tracking and mapping solutions embedded in it – and one that will eventually merge the real and virtual worlds together. In a way the vision these hires paint is the yang to AR’s yin. Whereas we think of AR as bringing the digital world into the real, what this acquisition says is that Oculus wants to bring the real world into the virtual one. The worlds of augmented reality and virtual reality are continuously being pushed together and will continue to do so for the foreseeable future.
