This guest post is contributed by the creators behind a new augmented reality toolkit that brings AR to the Web on mobile phones.
The possibilities of augmented reality are countless and we haven’t even scratched the surface yet. We’ve been passionate about developing web-based AR for a long time. We would like AR to be useful for the world, to stop being a promise and become a reality. This project is the result of that strong desire to actually make it happen. It’s called AR.js.
What is AR.js?
AR.js is a solution for efficiently doing augmented reality on the web, available on github. Let’s take a detailed look at what it is:
- Very Fast: It runs efficiently even on mobile phones, 60 fps on my 2 year-old phone
- Web-based: It is a pure web solution, so no installation required. Full javascript based on three.js + jsartoolkit5
- Open Source: It is completely open source and free of charge
- Standards: It works on any phone with webgl and webrtc
The goal is to make it easy for people to do augmented reality; AR that can be easily experienced on today’s phones and easily designed using web technology. The AR.js project is about enabling those people. So now, anybody with a modern phone can enjoy open-source AR, free of charge, cross-platform and without installation.
How does this help?
AR.js is fast. This is the first time that we have an open-source web-based solution capable of doing 60fps on currently deployed phones.
Suddenly, most people have a phone capable of doing AR without installation. This was not possible before. You don’t have to wait for everybody to buy AR-enabled devices (such as Tango, Moverio…etc.) You already have one, in your pocket.
Standing on the shoulders of giants
AR.js would not have been possible without some inspiring projects that came before. It is thanks to the hard work of others, that we can today reach 60 frames per second AR. These projects helped make it possible:
- Three.js is a great library for creating 3d graphics on the web
- ARToolKit has been in development for years for augmented reality
- emscripten and asm.js allowed us to compile ARToolKit into JavaScript
- Chromium helped by being very fast
Only thanks to all of them, could we do our part: Optimizing performance from 5fps on high-end phones, to 60fps on 2 year-old phone.
How to try it on your mobile?
AR.js is based on standards and works on any phone with WebGL and WebRTC. It works on Android and Windows mobile. Unfortunately, it wont work on iOS devices at the moment. Safari doesn’t support WebRTC but Apple is currently working on it.
Follow these two steps to heck it out:
- Open this hiro marker image in your desktop browser
- Open this augmented reality web app in your phone browser, and point it to your screen
You are done! It will open a webpage which accesses phone’s camera, localize a hiro marker and add 3d on top of it.
Performance – What does fast mean?
We are still in the early stages of the project but here are some initial numbers to give you an idea:
- We reached 60fps running stable on a Nexus 6p.
- We’ve gotten some reports of it running around 50fps on a Sony Xperia Z2. This is a 2.5 year-old phone.
- Some other reports of it running around 50fps on an old Nexus 5 and 60fps on a Nexus 9. Note that Nexus 5 is a 3.5 year old phone.
- Last but not least, some more reports of it working on Windows mobile with Edge: 13fps on Lumia 950 for some, and 40-45fps on Lumia 930 for others. We are still amazed it runs on Windows phone.
Obviously, your mileage may vary. The performance you get will depends on three things: how heavy your 3D is, how you tune your parameters and the hardware that you are using.
What’s New?
Recently, we’ve been getting creative and working on developing new things with AR.js. One of them is playing around with shadows, syncing the position of virtual lights with reality for a more life-like finish:
We’ve been collaborating very closely with Fredrick Blomqvist. His input has had a great impact on AR.js innovation and we want to thank him. Together, we’ve been implementing refraction, giving the 3d a transparent/glassy effect. It ended up having a nice polished look. What do you guys think?
Refraction in Augmented Reality – Part of merging virtual objects in reality – it looks so nice! pic.twitter.com/vv06y7i2Si
— Jerome Etienne #ARjs (@jerome_etienne) March 6, 2017
Other crazy ideas we’ve been working on include a hole in the wall and a portal into another world. We want to take AR.js to new dimensions.
What’s next?
We did good on performance, but there is still a lot of room for optimisation. Using webworkers would increase CPU usage. Compiling in webassembly instead of asm.js should improve loading time and likely CPU performance. And obviously, we can still tweak more parameters.
We would like people to start experiencing augmented reality and getting comfortable with it. Just look at this PS Vita puzzle game in AR using a mirror and laser beam, or this genius AR Gameplay footage.
PS Vita has done a lot of imaginative gaming apps. You could theoretically do the same with AR.js running on normal phones, without the need to buy a new device.
Something else that you could do with AR.js is develop education apps with science related themes, like this Solar System or like this interactive chemistry app. We are excited to see what people will do with AR.js.
