Anyone that has experienced quality VR can’t help but get excited. For some, it is even a life-changing event when they realize that the technology can provide access to entirely new, compelling realities – sometimes even more compelling than the real world. A lot of the buzz is because the technology is still so new and nobody yet knows all the answers, thus there is plenty of room for exploration and discovery. However, I believe VR will always be exciting because we will never know all the answers. We don’t even come close to knowing all the answers for the real world, so how do we expect to someday know all the answers for multiple worlds that are not limited by the physical laws of reality? With VR, anything is possible and anything will always be possible.
There are many factors that go into a high-quality VR experience. One especially important factor is the interaction between user and the machine. It will become much more interesting as tracked hand-held controllers created by market leaders Oculus, HTC/Vive, Sony, and Sixense become widely available in 2016. Like those that have experienced high quality tracked head-mounted displays, anyone that has been fortunate enough to try a quality demo utilizing such hand interaction knows how these controllers take can take presence to an entirely new level.
So, if the possibilities in VR are so boundless, where do we start when it comes to designing quality interactions? One structured approach is to start with what I call interaction patterns and techniques.
An interaction pattern is a high-level interaction concept that can be used over and over again across different applications to achieve common user goals. An interaction technique is a more specific implementation of an interaction pattern. Similar interaction techniques are grouped under the same interaction pattern.
I looked at 100 or so VR interaction techniques, ranging from lost techniques used in the 1990s to techniques various startups have been exploring over the last 12 months. Then, as shown in the figure below, I organized these techniques into 16 interaction patterns that are grouped into 5 types of patterns. For example, the walking pattern (a form of viewpoint control) covers several walking interaction techniques that utilize the legs including real walking such as done with the HTC Vive, redirected walking (walk in a larger virtual space than available physical space) as used by The Void, and walking in place as implemented by Stompz (make leg/foot gestures while seated).
There is not time or space here to describe all of these patterns and techniques. However, let’s look at selection. Selection is the specification of one or more objects from a set in order to specify an object to which a command will be applied, to denote the beginning of a manipulation task, or to specify a target to travel toward. Selection of objects is not necessarily obvious in VR, especially when most objects are located at a distance from the user. The Hand Selection Pattern is perhaps the most common form of interaction when using tracked hand controllers. It works by direct object-touching that mimics real-world interaction — simply reach out the hand to intersect some virtual object and then select it (e.g., by pushing a button, making a fist, or uttering a voice command).
Implementation of hand selection can range from realistic hands (e.g., a high-quality embodied avatar with arms) to non-realistic hands (e.g., an abstract 3D cursor held in the hand with no representation of the hand or body). The go-go technique is a subset of a non-realistic or semi-realistic hand that expands upon a more straight forward implementation by enabling one to reach far beyond personal space. The virtual hand is directly mapped to the physical hand when within 2/3 of the full arm’s reach and when extended further, the hand “grows” in a nonlinear manner into the distance enabling the user to reach further into the environment. This technique enables closer objects to be selected (and manipulated) with greater accuracy while allowing further objects to be easily reached. Physical aspects of arm length and height have been found to be important for the go-go technique, so measuring arm length should be considered when using this technique. Measuring arm length can be done by simply asking the user to hold out the hands in front of the body at the start of the application. The go-go technique can be further extended, such as by providing rate control (i.e., velocity) options that enable infinite reach. Non-isomorphic hand rotations are similar but scale rotations instead of position for manipulation.
So given such a large number of interaction patterns and techniques, what is the best way to interact within VR? Like any great question, the answer is “it depends.” Each pattern or technique has strengths and weaknesses depending on application goals and the intended audience. The go-go technique described above works well when ease of reach is important but is usually not appropriate when training for real world tasks. Understanding distinctions and managing trade-offs is essential to creating high-quality VR interactive experiences. Instead of having a one solution fits all answer, designers who have a diverse understanding of different options, and when those options are most appropriate, will be able to innovate more quickly across a wide range of experiences. Such options are especially important now that a diverse set of quality input device options are starting to become more available.
Interaction patterns and techniques provide conceptual models to experiment with, suggestions and warnings of use, and starting points for innovative new designs. Although VR creators can benefit by having a library of options to choose from and a base to innovate upon, none of these interaction concepts are meant to be plug and play. There are numerous factors such as high-level goals, hardware constraints, and the target audience to consider. Quality VR design requires iteration more so than any other technology. Don’t assume that just because a technique does not work on its first implementation that it is necessarily a bad fit for your needs. Experiment, get feedback from users, and tweak the design until it works well or you can conclude that it is not appropriate for the project. Perhaps most importantly, learn from the experience so that you and your team build up VR expertise for future work so that when VR does finally go mainstream, you will be ready.
Like VR in general, we know some of the answers but we don’t come close to knowing all the answers when it comes to creating immersive interactions. And that is one reason why VR is and always will be so exciting.
Jason Jerald, Ph.D., is Co-Founder and Principal Consultant at NextGen Interactions which focuses on VR consulting and contracting services. If you enjoyed this article and would like to learn more about VR design, then take a look at his book “The VR Book: Human-Centered Design for Virtual Reality” at http://TheVRBook.net.
