Appendix 1

Detailed Taxonomy Discussion

Virtual Reality Systems- Fully Immersive

Fully Immersive Vehicle Simulator

Another very successful use for Virtual Reality training is in simulators. The Link flight simulator is the oldest example of this technology. It was invented to help reduce costly accidents while training pilots in the early years of aviation.(Unknown, P.3). Currently companies like Immersive Technologies, CMLabs and Xesa provide simulators for training merchant marine officers, crane and heavy equipment operators, pilots and for simulating automotive interiors and vehicle interfaces. Some simulators are simply a seat and controls with large display screens and custom computer controls and software. Other simulators add in six degree of freedom (6DOF) hydraulic/pneumatic or electric actuators to create physical motion profiles, similar to the real operator environment, as a means of increasing the sense of “presence” for the trainees. Such immersive simulation environments can approach or exceed the million dollar price but in comparison to using actual boats, aircraft or cranes, the economics are easily justified. They allow for training to encompass extreme conditions which may rarely occur in practice and yet the operator must be prepared to execute complex responses quickly in order to safely avoid severe consequences.

CAVE (Cave Automated Virtual Environments)

Cave Automated Virtual Environments are a technologically enhanced versions of Barker’s Panorama’s from the 18th century. (Boyle, 2013). Such immersive systems find continued use for architectural walk-throughs, design evaluations of large piping or engineering projects as well as simulation training for military, law enforcement and sports. Mechdyne and Virtra field exemplary CAVE or CAVE-like systems for commercial use while companies like TruGolf are successful with sports training systems. Their use may be jeopardized to some extent by the improvements in VR headsets and attendant lower costs. However established business cases and some advantages from allowing full use of natural senses and recreation of physical motion will keep CAVE systems viable for certain applications.

Tethered Headset VR

The Tethered Headset Personal Computer/Gaming Console Based category is the common baseline when consumers think of VR hardware. These rely on PCs or gaming consoles for the computationally intensive tasks of presenting rendered scenery to the users’ eyes in real time and in response to inputs from sensors indicating head position, head motion and eye gaze. The PC or console also interprets hand position via handheld controllers and controls any haptic feedback devices to stimulate the user’s tactile, thermal and other senses. The much publicized Oculus Rift is exemplary of this class of device. The HTC Vive, Sony PSVR and the PiMAX4K also represent typical tethered VR full immersive headsets. All require a powerful graphics processing PC or, in Sony’s case, a gaming console. These also require an “outside-in” tracking scheme of some form. These can be light emitting “lighthouses” or “towers” which are placed around the user in 2 or 3 locations and provide better full-body spacial tracking. This class of device represents the best experience for immersive gaming and 360° entertainment (video/animations) viewing. Beyond immersive gaming, which constitutes the early adopter market to a large degree, these systems continue to drive adaptation of existing entertainment, like movies, into new experience territory.

The gaming application dominates the market and for good reasons. VR’s ability to create “presence” and a full immersion experience make many classical video game genres much more engaging and realistic. The only real question is how soon traditional console and screen based games are displaced by VR gaming.

The use of VR as a new medium for pornography is also already well established. As with many new media the porn industry is innovating rapidly in its uses for immersive VR. VR headsets partnered with Internet connected sex toys, along with real time viewing, are some things already being fielded by early movers in this industry. (Borde, May 17, 2017). The dramatic growth in VR porn usage is reported in unflinching detail by industry data reference Pornhub Insights. A spike in activity on Christmas day 2017 followed by a general doubling of viewership rates year to date 2017 over same time span in 2016 may be a leading indicator for other VR genres. (Virtual reality porn, May 11, 2017).

One profession that has been and continues to apply immersive VR is the psychology and therapeutic mental health professions. Known as Virtual Reality Exposure Therapy, it has been found effective for people with phobias and Post Traumatic Stress Disorder (PTSD) challenges. (How virtual reality, 2017). In addition to use of the technology to help with past experience desensitization it has also been shown effective as a tool to train for future stressful situations. This training, known as “Virtual Reality Stress Inoculation Training” (VR-SIT) (Stez, 2008), has been applied to training combat medics in stress management and relaxation techniques. Immersive VR has demonstrated its ability to create psychological and physiological reactions during these training sessions similar to that of real world stress levels. The value is in being able to control the exposure incrementally and develop gradually the trainee’s ability to recognize and manage their reactions using breathing exercises and other proven stress management techniques.

Fully Immersive Mobile

This taxonomic category encompasses the affordable end of the immersive VR technology space and applies software and low priced optics to readily available smartphones to create Immersive VR headsets. The most successful leader in this category has been Samsung’s GearVR. (Worldwide Shipments, 2017) Google defined a new entry point in Mobile VR with the ultra-low-cost Google CardboardVR. This product defined the “giveaway” level of VR headsets with a simple folding cardboard housing and plastic Fresnel lenses that relied on a user’s existing smartphone. (Sanders, 2016)

Google’s more refined competitor to the Samsung GearVR is the Daydream platform. Both product and ecosystem, Daydream is general purpose a Mobile VR product supporting a range of smartphone models. The developer tools and environment are open for use by all in anticipation of establishing a standard format for content following the Daydream framework. Google intends to create an entry level but user friendly VR for mass market use in ways similar to the Android OS and Chrome browser products. Several manufacturers have already announced products and intent to support Daydream. Zeiss, the well-known lens and optical components company, provides the VR One for use with many smartphones and touts its compatibility with Cardboard and by extension Daydream. Zeiss is future-proofing their product by allowing use with AR compatible smartphones with an optionally transparent cover lens. Apple has not launched any mobile VR products yet but the rumors and behind the scenes activity (patents, etc) indicate their intent to join this market too. LG is marketing a rather novel take on mobile immersive VR with its LG VR One headset. This device connects via USB cable to the LG G5 Smartphone, has its own displays and IMU (Inertial Measurement Unit) but uses the processing power and Internet connectivity of the G5 Ito access among others, SteamVR content.

This particular taxonomic category has the probability to overlap with AR Mobile headsets when more smartphone companies introduce AR enabling features into their devices. Certainly it represents the most accessible path for experiencing VR and will be the most common VR experience accessible to consumers. Facebook and Google have both announced detailed strategies for evolving their social media platforms with VR features and content. This may well represent the mass market popularization for VR technology.

Augmented Reality

Augmented Reality Tablet Computer/Smartphone Based

Augmented Reality can be as simple as adding computer generated graphics and data on a real time camera view of a smartphone or tablet. The earliest renditions of this type of implementation relied on “markers” in the form of 2 D barcodes or pseudo-barcodes that would be recognizable in the camera image by the software and the 3 D graphic or data would appear at that location. (Craig, 2013, P. 17) Today’s technology can simplify this scheme with GPS data and camera image data, and smartphone IMU (inertial measurement unit) data combined to recognize a physical location and overlay the desired data or graphics to enhance the users experience of that physical, proximate reality.

This augmentation might take the form of directional arrows to help navigate through a busy airport terminal while on foot or finding a safe public restroom while sightseeing in an unfamiliar city. Instead of the blinking dot on the 2D map you would merely look at the scene with your camera on and see guiding arrows or other graphics to assist you.

A recently very high profile augmented reality success in this technology has been the Pokemon Go game. This global scavenger hunt sent game players searching the world to find Pokemon at various locations. The graphical 3D characters would appear as animated characters embedded in the real scene as viewed in real time through the smartphone or tablet computer screen. (www.pokemongo.com, Unkown)

The newest advance in this taxonomy node involves adding depth sensing Time of Flight camera technology in addition to traditional color image sensors. Lenovo’s Phab2 Pro was the first product to reach the market and supporting Google Project Tango’s specifications for AR applications. Its large size, at 6.4”, was panned by critics but the strength of its AR implementations was seen as remarkable in many reviews.(Lenovo Phab2, unkown) ASUS followed with its Zenphone AR, which included a motion tracking camera in addition to the Depth Sense camera. Lowes Home Improvement stores fielded an app for Project Tango phones that allows customers to measure rooms and place 3D versions of furniture or other products into the spaces available to check the fit and look prior to ordering the product. Wayfair, an online furniture and home fixtures retailer, has created its own similar app for Project Tango compatible smart phones. Occipital is marketing its Structure Sensor intended to turn any tablet into an AR capable device. This sensor attaches to a tablet and provides 3D object mapping of scenes to enable apps to recognize walls, floors, objects and surfaces in close proximity. (Occipital, unkown)

Augmented Reality Heads Up Display; Camera; Text/Graphic Display

This category represents many different versions of wearable products intended to augment a user’s experiences. One class of product is represented in the form of eyewear with Heads Up text or graphical displays directly project on the lens or as a virtual screen smartphone display. The Garmin Vara Vision and Recon Jet are exemplary of this class. Their target user is the runner, cyclist or other enthusiast who desires hands free, Heads Up Display of their preferred data. Monitoring speed, turn by turn navigation, heart rate, cadence, or other performance data is simple and visible at all times.

Google Glass was among the first Augmented Reality, Heads Up Displays and created a significant amount of popular attention when it was made available on a limited basis to developers. It was a short time after though that Google chose to discontinue their efforts amid negative reactions from some users and from the general public over the privacy issues raised by the prospect of Glass wearers taking surreptitious videos. A successor to Google Glass , Snap Spectacles, has had more popular appeal. This product is a wearable still and video camera in the form of sunglasses connected to a smartphone. Spectacles feature a small light around the camera lens to alert the public when videos or images are being captured. Spectacles are also much less expensive at $199 and with a simpler use case than Google Glass meant to address.

Sifilo X from Safilo group is a wearable product that connects to a smartphone. While not a true augmented reality display device this sunglass product monitors EEG/EOG/EMG brainwaves as a means for athletes and others to control stress response for maximum performance. It also includes IMU, temperature, pressure and UV sensors to monitor the wearer’s physical environment and provide audio cues and alerts.

Augmented Reality Tethered

This class of device brings many of the features of Virtual Reality headsets to devices with transparent screens or camera and depth sense equipped headsets. Microsoft’s Mixed Reality project aims to create business use cases for the technology. Hardware partners familiar from the Wintel ecosystem of desktop, laptop and tablet computing are busy releasing first generation hardware in support of Microsoft’s vision of Mixed Reality. Among these suppliers Acer, ASUS, Dell, HP and Lenovo have launched or announced their intent to launch headsets. Most feature 1440×1440 displays with 90° Fields of View (FoV) and Depth Sensing Cameras. The notable difference between Windows Mixed Reality and Tethered VR is the “Inside-Out” tracking scheme. Where most current Immersive VR headset require external markers or light towers to create realistic full body tracking the use of Depth Cameras and Motion Tracking Sensors on the Windows Mixed Reality compatible products uses on board sensors to achieve full motion tracking and also to sense hand held controllers or game/application props. While still a tethered device dependent on a personal computer to run games and software applications the “inside out” tracking simplifies room scale limited mobility interaction during game play or application use.

Microsoft’s targeting of business use case along with gaming addresses a much larger potential market willing and able to spend more on hardware and software in pursuit of productivity gains. Potential use cases include real time collaboration and coordination of work groups across the internet, design visualization, training for complex tasks, augmented troubleshooting and repair in the field. One potential victim of this technology may be the computer monitor manufacturers. Such device have no problem creating a virtual screen of large scale without taking up any desktop real estate.

Augmented Reality Mobile

This class of product is the spiritual successor to Ivan Sutherland’s Sword of Damocles, the earliest augmented reality head mounted system. Presently only Microsoft’s Hololens represents the most sophisticated version to be launched. Hololens creates realistic and lifelike holographic projections into the scene from a head mounted, fully self-contained device. The Hololens includes all processing hardware, batteries, sensors, displays and operator control interfaces. Engineering and packaging all of this into a light, fairly sleek design the Hololens establishes many new potential use cases that have been imagined but unattainable until now. Intel’s Project Alloy is purported to be targeting similar use cases and with similar hardware sophistication but no product is yet announced.

Hololens hardware includes an IMU, a Depth Camera, 4 “Environment Understanding Cameras, a 12MP photo/video camera, 4 microphones, 1 ambient light sensor. Its displays are described as HD 16:9 aspect ratio see through holographic lens waveguides. The ability to manage the many sensors and the displays is accomplished by a powerful, near supercomputer built into the headset. This tight coupling of sensing, displays and computational graphics processing provides a very low lag time between the viewer’s movements of eye gaze or body motion and the overlaid digital projections on the background IRL (In Real Life) elements.

Some other manufacturers are targeting similar functionality at a lower price point. The Vuzix Blade 3000, ODG R7/8/9 AR, Epson Moverio and Laforge Shima all fit into this taxonomic category.

Fully Immersive VR is fun and has addressed many exciting use cases but the AR Mobile devices represent a more practical device for use on an extended basis. Where fully immersed users can’t keep any touch with their surroundings AR users will still be able to see what’s going on around them and will be able to create applications that leverage the merger of digital data and graphics with IRL scenery and objects.