Flashback: Virtual is Reality Again - Interface vs Interference Design

[photo: copyright cube3.com 2015]

Virtual reality is here to stay. It is showing up everywhere – from taking center stage at the Game Developers Conference (GDC) in San Francisco, California this year, to making a splash at SXSW in Austin, Texas. Practically everywhere you look, VR is sliding in to alter every industry it touches; but where did it all come from? Why is virtual reality proliferating now like never before?

In a series of interviews and re-publications, we venture into the past to see where this exciting virtual reality ride began. Today, we post an article originally written for a virtual reality magazine in the mid-90s that never made it to its 2nd issue. This was around the same time of a VR expo that occurred in New York City in 96. The author is named Larry Rosenthal of “cube3.com” who has been working with 3DVR tools since 1989. He has seen the many rises and falls of virtual reality over the last few decades.

This post is an analytical look at the fundamentals of design captured during one of the high points in virtual reality’s history. It has been re-published a few times after its inception in 1996; most recently right after Facebook acquired the nascent company known as Oculus Rift. This piece has not been edited in way to maintain it at its truest form. Enjoy!

Writer’s Note : Originally created in 1996, this article offers Designer’s a basic primer on 2D vs 3D Interface and Design issues. I’ve been asked to republish it 3 times online (every 6 or so years) as each “Virtual Reality or Virtual Worlds Press/Money Bubble arises. With yesterday’s Facebook purchase of Oculus Rift for 2 billion dollars, it seemed like the right time to republish again;) BTW — I’m Available.

It’s been 30 years since the GUI of the MACINTOSH was unleashed on the design community in 1984. It’s black and white “icons” led to the death of the “text-dos” interface and the birth of the 2D GUI interface. First used by the public on the Mac, then almost everywhere else with the introduction of “Windows”, the 2D GUI and its language of metaphors and iconography has become the standard for computer-user interface in today’s world.

The inherent strengths of the 2D GUI created a medium where the tool maker and tool user both learned to understand the media by “how to get around” and “what the story is “ and “how this thing is to work “ from the use of visual cues and relationships. These cues became so obviously more effective than the text-only interfaces that had been offered to do the same before as a text only interface.

Today, 30 years later, the interface, tools, and digital medium are again at a crossroads. Following the burst of activity that multi-user 3d virtual worlds and virtual object economies brought to the mainstream in 2006, Web3D/VR and real time 3d media has become poised to provide the immersive and game related interface to the mass page based metaphor internet.

At this time when the virtual worlds and realtime3d “fever” seems rebirthed and new models and functions for the 3d web/VR medium need to be discovered, it seems like a good time for designers to ask the questions about the 2D interface and communications, the capabilities and differences of a 3D interface and communications and how, why each does what they do and how and why and when a designer/developer now working with 3D communications and Web3D/VR should be aware of each disciplines strengths and weakness when utilizing each.

This article is a start to examine where and when 3D interfaces and communicative designs can or should be used:

The 2D HTML WWW

The rules and laws of 2D interface design and communications have with the GUI been implemented in mass and a community of computer users have learned to communicate and work via a 2D-interface metaphor. Pictures, words, pages, windows on the flat screen have been accepted as how to work on a computer. In 1994 the www exploded into the digital consciousness and it was 2D HTML and its pictures, words, click-able zones and 2d page metaphor of interface that “made the Internet” easy to navigate for everyone”. The discipline of “Graphic Design” over the last 70 years had taught everyone, the creator and audience alike, the principles of “understanding” 2D design and communications and the 2D html web used that understanding to succeed. Today millions are using the www and principles of 2D interface and communications are being used by trained designer/developers to create an understandable and usable tool and medium out of the WWW Internet.

The New 3D WWW

In 1994, a 3D version of the http protocol was developed. Named VRML, it would offer during the next few years the ability to create and distribute 3 dimensional information, models, and environments on the Internet as freely as the 2D html interface presented text and images. 3D communications and interfaces also follow basic principles many which flow from the established 2D principles. During the last few years the latest tools and services for designers that create Web3D have arrived and like the “PageMaker and Photoshop” designers before them did with 2D, designers are now flooding the internet with 3D avatars, worlds, spinning logos, objects and places/scenes. All of this “Web3D” content for the most part will be successful or not based on the content’s development and design according to the principles of 2D/3D interfaces and communications, and basic design. All of what developers/designers will and have placed on the WWW have been affected, knowingly or not by the content’s interface.

The principles of 2D and 3D interfaces and communications:

Web3D developers/designers should become familiar with the differences and similarities of 2D and 3D interfaces and communications. The basis of any “interface” as defined here, is to let the user communicate with the computer as transparently as possible in order to let the content be the message, not the effort involved in using the medium or tools. “Interface” is that layer between the user and the tool or task to be accomplished.

The “visual interface “ being discussed here can be considered the device to tell the “story of how to work this thing” with “this thing” being the screen based information presented on the computer. A 2D GUI conveys this information very effectively as the successful and growing 2D HTML www demonstrates but the 3D interface offers new elements and communications that effect the experience of the user differently and in ways that a 2D interface cannot.

For developers and designers to discuss the relative values of a 2D or 3D interface, a definition of basic design principles and elements must first be established. These Basic Design elements and principles have developed not over the last “digital” decades, but over the entire time of the existence of humans. They exist as a part of our “human condition” and because of this, they effect and control the “implementation” of any interface or communication, either 2D or 3D.

The Elements and Principles of design are complex and much has been written about them, for the purposes of this article, I provide the following (incomplete) list and brief descriptions so that the thoughts of how it relates to the difference between 2D and 3D and the benefits of each can be examined. I suggest that all Web3D developers and designers begin to learn more about these issues as they truly are the basis of successful 3D or 2D interface.

The Basic 2D Design Elements:

Line
Color
Shape
Direction

3D Design Elements Adds to 2D Elements:

Form
Texture

Modifying Elements: Each can exist as “animation” in 2D.

Time
Movement
Audio

The Principles of Design Order For Both 2D and 3D:

Dominance
Subdominance
Recessiveness
Scale
Position

2D is great, why do we need 3D on the web?

What 2D Interface and Communications do well:

The 2D HTML Interface and communication of today is primarily a product of text and image, although audio and animation are becoming more prevalent, the majority of the WWW is text and image. An effective 2D interface and communication is usually created by the balance and interaction of these elements and according to the principles of design above. These 2D design principles can create very effective interfaces and communications. Some of the most effective uses for 2D communications are:

Charts and Diagrams
Text
High Resolution imaging-photography

Why are these types of information ideal for 2D communications? Because analytical examinations are the best candidates for a 2D interface. Actions and Interactions that require thought and rational logic to be carried out seem to be well suited for the “focused” interface that the 2D offers. With no “world” to distract the interaction, the message in a flat 2D interface can be studied, examined and acted on. 2D interfaces can only offer a limited amount of information per “page or window” this is its strength as an “intellectual” type of interface experience.

Simplifying information in graphic 2D:

2D interfaces and communications are ideal to simplify and can “cartoon” visual information for easy comprehension by the user. This is why text and charts are so effective in 2D representations. Both only require clear line and shape to usually get their message across. If a developer or designer’s goal is to provide an interface that quickly “speaks” organizational image to a user than 2D interfaces contain all the elements to create successful solutions. The same can be said for “immediate call to action” type interfaces

In the real world, the traffic “STOP” Sign can be used as a good example of this type of effective 2D interface since the text, and shape and color create an “instantly recognized call to interaction” with the understanding user. Successful 2D interfaces on the computer utilize the same principles and the “caution, stop or bomb” icon message windows on a computer interface can best communicate their meanings the same way.

Photography simulating 3D reality:

High-resolution screens led to the ability to use high-resolution photography as an interface element. Photography as well as video or moving image can often be confused for the “3D” that an interface can be. Photography can capture detail and representation to such a high degree that as a 2D interface element it can offer such “depth of simulation” that, to many, a photograph of a street scene is more “real” on the screen than a 3D modeled navigational model. The detail of this 2D-interface element can be though its major problem in conveying the designer’s real intentions.

What if the street scene presented on a kiosk is meant to be a map to show walking directions between the buildings in a mall? Then the photographic details of the actual floor tile patterns of each buildings lobby may have no importance to the users main question posed to the kiosks interface: “Is my building to the left or right of where I’m standing?”. Photographic images and moving video can now look great on a high-resolution computer screen- they both offer an ability to simulate a 3D world within the 2D flat desktop metaphor. When aided by audio and movement, they offer many of the elements that 3D look to offer.

So the next questions that have to be asked are what does 3D really add to an interface?

What 3D Interface and Communications Add:

Immersion, more powerful than an Intellectual Engagement.

The addition of the “3rd” dimension to an interface brings with it the new elements as listed above and the often mentioned “Z-axis” Depth, to the 2D screen that is physically an object of the y- height and the x- width. The 2D interface that has developed over he last decade has adopted the “desktop” metaphor stating that although “pages” or windows can stack up over each other. There is always a “back surface” or “desktop” below providing a final border to the interface in the z-axis. This “final border” provided with it a device to simplify the usage of the computer and how much information could be presented to viewer at one time. The user would always be “above” or in front of the screen desktop and in the same “still” seated position. This addition of “depth” to the screen and the change from “desktop” to a “window onto an extended world” brings with it the major changes to interface and communications that 3D brings.

It is this “illusion” and “presence” that the user can now “ be in” an environment as opposed to just “viewing it from above” that provides the greatest challenges for developers and designers. This “immersion” factor requires that developers/designers provide an interface that is completely consistent since the user will be “thinking” that they are within this world and reality. This “illusion of immersion “ can be so much more effecting to the user than the “intellectual” involvement that 2D interfaces offer so well. They can become so effective that any element that seems out of place with the users’ understanding of the 3D interface, can destroy the illusion thus the interfaces usability.

If its a “real/virtual world” in the computer? What am I supposed to do? How does the “virtual world” work? And all of a sudden “I’m working in it?”

A “3D world” simulation brings with it two elements: those of Navigation and Emotional involvement. Although they can be part of a 2D interface, they are overwhelmingly more a part of creating a successful 3D interface.

We are beings of movement and motion within 3 dimensions. We walk from place to place in the physical 3D world. We navigate from position to position with our eyes in front of us. We see forward and rely on memory to tell us what we have walked past. We live in a 3D world of up/down/front back/right/left with each direction with its own outcome and new confrontations. All of this stimulation has evolved us into beings that “feel” and “react” based on the ever-changing environments and confrontations before us. We use devices like “interfaces” to “clean up” those confrontations and distractions to focus our attention and do a task.

Creating and developing a 2D interface allows us to immediately “cut out” the “backside” of our concerns in accomplishing a task. The real 3D world does not. The new 3D virtual worlds of interface available to the masses via web3D will undoubtedly be recreations of rooms, cities and spaces meant to “surround” the virtual presence of the user. The 3D spaces created will then require navigational devices within them to tell the user where and when they exist in this 3D world. The user will need to exist within the 3D space able to comprehend the space’s rules and laws that will very much be the users “reality” during their time “within” the interface. The users ability to understand and use the 3D world presented to them will be the mark of a “good” 3D interface.

Navigation as Interface element:

The mechanics of navigation for the web3D masses have become the immediate domain of the toolmakers. Each takes their “best shot” at how to “offer navigation in 3D space”. Most developers/designers will have to utilize one/all of their solutions during this early period of web3D integration into the WWW. Lets then discuss some questions about navigation that comes from within the world itself that’s offered by the developer/designer.

An Object’s Consistency in the World:

Inanimate objects around us do not move around and navigate as we do. A chair cannot instantly become a lamp, or grow an arm. Objects in the real world also have properties of mass and existence that define them as “real”. The digital world does not have such guarantees and offers such miracles. It is the application of such events and the status of the user to comprehend the change that is the major factor as an interface device. In a 2D interface the change of an object can occur and most likely be witnessed within the framed window. In a 3D virtual world the user can be looking 180 degrees in the opposite direction at the time of an object’s change. The challenge of the interface is to make the change known so that whatever purpose is served by the objects transformation, its “history” must be known by the user so that the object is useful for them.

Audio to suggest spatial relationships:

The ability to localize audio and effect is volume within the 3D world allows audio interface to become as important as any visual cue to understanding the world you’re within. Just as in the real world where you can “listen” to a conversation of the person you’re with in a crowded bar and only “hear” the sounds of the distant others around the bar, such audio localization’s can be an effective interface element to communicate to a user in the 3D world. Audio has been used by 2D interfaces to offer at best an “ambience” or “direct input feedback” to flat www interfaces, but audio as a localizing element to an object you cannot see (ex. behind you.) is a interface device that only the 3D medium offers.

The user’s effectiveness to alter the environment.

We affect and alter our real environment constantly. There are things we can do, like move a chair over to sit on, and things we cannot, like move a car over, sideways to fit into a too tight parking space. The same rules and laws are offered to exist with 3D web worlds. The user’s ability to create “cause and effect” will immediately place them in the world and define its rules. When a rule about an object is broken, consistency is lost and the user cannot be sure of an action’s reaction.

Time, Scale, and Distance as an Interface device:

When using a 2D interface the user is kept for almost all actions outside of the computer’s “clock” and is functioning within “real” time. The 2D interface is an element of the computer (object) on the desk in the users real world. Time may seem different as many experience when interacting with a computer but the rules of time are set by reality. Within the 3D interface elements like time can become altered and elements like distance that are associated with time can also be affected to behave “unreal”.

Scale also becomes a variable for a developer/designer to effect in relationship to distance and time. Is the castle far away or just small? Or both as you navigate closer? The manipulation of time and its relation to 3D space can become a navigational nightmare within the 3D interface. Our body size in the real world sets many of our notions of time, scale, and distance. It’s no coincidence that we measure distance in “feet”. We use our own mass and size to “fit” into our real world. Within a flat screen on our desktops 3D offers the world paradigm but to most at small scale (a screen that’s about 15 inches across☺. Interface designers working with 3D will have to confront the issue of scale, speed, and distances in a way never required in 2D windowed interfaces.

The Addition of EMOTION:

The developers/designers use of the 3D space and the objects within will when coupled with the discussed elements of navigation cause an emotional response within the user. Immersion within the (real) world will, by the nature of our being, cause an emotional response. Simulated 3D interfaces and worlds will offer the same responses. The “feelings” brought on by 3D immersion will be of an entirely different nature then those of the simple 2D interfaces immersion capabilities. The “roller coaster” IMAX film experience is very different than even the most well written and acted action movie presented on a normal movie screen. “Being there” within the world will effect our thoughts and feelings in a different way than the detached “windowed” 2D interface can offer. 3D interfaces offer the developer/designer the reaction of “I was there” to add too “I did/saw that” or “I read about that” when a user is recounting an interactive experience.

Conclusion:

Each new year will be another year “looking for the killer app” for web3D/VR media. Virtual Worlds, Avatars, Social Games MMO’s, Corporate Meetings, Virtual Objects Sales, etc. will all be in vogue for a few months at a time, But the differences between 2D and 3D media discussed in this article should, I hope, assist developer/designers in the choice to utilize one or the other media in their web projects. The 2D interface has always existed within the natural 3D world as a way to simplify and organize the information of world around us.

The 3D interface can simulate that world and offer the beginnings of it’s properties and our natural reactions and emotions to them. The capability to offer 3D interfaces to the WWW via all these web3D/VR technologies should offer designers and developers a way to offer communications and interfaces that can be used for certain applications more effectively than ever before. For Web3D/VR though to succeed as a widely used web media it must be used to its inherent interface strengths and not as just adding the latest technology to a web page or the internet.

In closing, always let “The task define the interface and the experience define the medium” The designer/developer should define the technologies required to create the interface needed to fulfill the task. Real time web3D/VR as one of those technologies offers the complexities of the 3D interface, with it the benefits and challenges of communicating within the 3D environment. When properly adopted, Web3D/VR can provide a unique interactive/immersive experience that may in fact become deeper than what the 2D Interface can and has for now provided, or they can truly place us in an almost inescapable world of “interference design.” Designer’s, every few years the next move is your’s.

[Source: Medium]

Author’s Update: April 2015

The above article and it’s reprints have stayed pretty much the same for the last 20 years. I wanted though to take this republishing to add some new thoughts and ideas based on what it think will be the single defining media difference between this current time of interest in VR and the others of the last 20 years.

Up until now almost all discussion of 3D VR has been based on realtime 3d computer graphics as the medium. Either polygonal or voxel based, the media of realtime VR has been “built 3d model” centric. That created the need for skilled designers to create the medium’s content and experiences. The article above is primarily based on traditional design ideas that have been taught for decades and which most professionals would have learned as they practiced Design over a career.

Now in 2015, the 360 degree motion video form has been introduced and is poised in my opinion to be the real factor in adoption of VR as a medium during the next few years. Although 360 still photographed panoramas were marketed as “VR”” since the 1990’s the addition of motion and 3d spatial audio truly makes them a compelling new medium. The debate over narratives vs. experiences and proof of which “defines” this new medium is sure to come. But a new medium requires mass usage to truly affect us and the ability of anyone with a phone and a built in 360 camera to record places and events and upload them to the global web/cloud will change the mediaverse we create and inhabit profoundly. Some of the issues in the article about the “factors” of Photography vs. 3D Modeling will now be relooked at in practice, and I’m sure some assumptions I made 20 years ago, will now be found wanting. That’s progress;)

What is of final note is the articles original intention to be a call for “Designer’s” to take note of the tools and medium they worked with. Today, especially now with “everyone” becoming VR “enabled” at the same basic time via social media platforms, the importance of Interface Design to a Medium , and to other people’s lives, has become paramount. This “Virtual Reality” is everyones responsibility.

About Larry Rosenthal /Cube3.com :

LarryR/Cube3 has over two decades of experience as a leader in the area of virtual worlds ,VR/AR, transmedia IP creation, social games, online 3D, UX/UI, and networked communities for both entertainment and corporate media uses. Over the years LarryR/Cube3 has worked with both media content makers and tool makers in all matters of finding customers and creating products, properties and services and then forming communities and ecosystems around them. Clients and Licensor’s of his original media content and designs include, Sony, CBS, AOL, Fox, Showtime, DC Comics, Ubisoft, Paramount and ABC.

LarryR/Cube3 has spoken at many industry events. Highlights included self created seminars at Apple Computers NYC, Siggraph Shows, Macworlds, and Digital Hollywood Panels. He created two of the original virtual reality user groups, the NYVRMLSIG and the SFWEB3D and works and projects of his have been written about in both print and online in periodicals such as WIRED, NEW MEDIA, Macworld, CNET, and Microsoft Press.

UploadVR’s “Flashback” series is an ongoing effort. We are looking for virtual reality pioneers who worked with VR in the 1990s and before. If you are one of those people or know someone who is, email Matthew Terndrup at terndrupm@gmail.com to arrange for an interview.

Read more Flashbacks: