The Unfulfilled Promises of Virtual Reality

%26ldquo;Virtual reality holds the key to the evolution of the human mind%26rdquo; - Dr. LawrenceAngelo, a terrible film (1992) I was born in 1985, and thus grew up believing that by the mystical and far off Year 2000 I%26rsquo;d be testing my wits alongside Sherlock Holmes in a holodeck, plugging all sorts of electronic nonsense into my brain, and seeing the world through kickass bionic eyeballs. It%26rsquo;s now 2008, and aside from briefly experimenting with psychedelics, I%26rsquo;ve yet to experience anything compellingly realityish outside of the onein whichI pay parking tickets every month. Why? The term %26ldquo;virtual reality%26rdquo; was coined by dreadlocked theoristJaron Lanier(left) in the %26lsquo;80s, and quickly succumbed to much hype, theorization, and sci-fi speculation. %26ldquo;Home Reality Engines%26rdquo; were going to revolutionize our daily lives. We were going to turn into cats and prowl cyberspace, navigate file structures by zipping through miles of glowing electronic cabinets, and make sweet love to all sorts of virtual womens. It was all very poetic and exciting, and I believed every word of it, even the fictional ones. In a1988 interviewwith the Whole Earth Review, Lanier explained what it might be like to play an instrument inside of virtual reality: %26ldquo;With a saxophone you'll be able to play cities and dancing lights, and you'll be able to play the herding of buffalo's plains made of crystal, and you'll be able to play your own body and change yourself as you play the saxophone. You could become a comet in the sky one moment and then gradually unfold into a spider that's bigger than the planet that looks down at all your friends from high above.%26rdquo; If Lanier's vision existed, WiiMusic would be toast. So where did we go wrong? What happened to all of the grandiose speculation? Sure, I paid too much to be strapped into VR machines in %26lsquo;90s arcades, and current military simulations are quite advanced, but they%26rsquo;re far from the %26ldquo;Home Reality Engine,%26rdquo; and nothing at all like the cyberspace I envisioned as a fanciful child. I have yet to unfold into a spider bigger than the planet. As it happens, the fictions that inspired my fascination with VR may not be so farfetched %26ndash; and the following developments may eventually make Lanier%26rsquo;s crystal plains my new favorite place to pay parking tickets. David Cronenberg%26rsquo;s 1999 filmeXistenZdepicts an all-consuming form of virtual reality %26ndash; your typical %26ldquo;am I still in the game?%26rdquo; conundrum. It certainly isn%26rsquo;t the first instance of the concept, but it works well as an example. Two devices are presented within the film: the first is a morbid biological computer which is inserted into the spinal cord, and the second is a much more sterile device which sits atop the head, wirelessly manipulating thoughts. Only one is the %26ldquo;real%26rdquo; game, but I%26rsquo;ll leave the plot of the movie to the movie and try to avoid spoiling it any further (Netflix it, seriously). Above: eXistenZ's semi-erotic biological gaming device These fictional devices allow both for the player to control the game as he controls reality, with what feel like his physical limbs, as well as experience the game as if it were reality, with his own senses. Two technologies exist today which rudimentarily mimic the control concepts of the two technologies envisioned in eXistenZ. The first requires physical sensors to be placed within the brain (similar to the parasitic spinal cord device), and the second wirelessly reads brainwaves (a bit more appetizing). In 2005,Wired ran a storyabout a paralyzed man named Matthew Nagle, who, with the help of wired brain sensors and a refrigerator sized computer, was able to control a cursor, click on buttons, draw rough circles, and play Pong. (Right: Matt Nagle playing pong,Nature, 2006) The interface, called BrainGate, used 96 electrodes which were implanted into the part of Nagle%26rsquo;s brain which appeared to control his dominant hand. The system worked surprisingly well for the extent of the year-long experiment. Matt Nagle passed away in 2007, six years after the stabbing which rendered him a quadriplegic, and is remembered as a brave and selfless contributor to the advancement of the science. As is made apparent by Matt%26rsquo;s story, there are far more important applications of this technology aside from letting me pretend to pilot the Starship Enterprise. Nevertheless, the potential for videogames and mind control is clear, and gaming applications have already been developed. Of course, no one is going to stick electrodes in their brains to keep their hands free while they play WoW, so let%26rsquo;s look at the other option. Nagle%26rsquo;s experimental procedure tapped into brain signals directly related to movement. The result was effective, but it seems that similar (though somewhat less sophisticated) results can be achieved via non-invasive devices which read brainwaves through electrodes placed around the skull. Above: OCZ's "Neural Impulse Activator" Our own David Houghton played witha device created by OCZ,and was able to fairly rapidlyjump into a game of Unreal Tournament 3. The result isn%26rsquo;t quite the same as Nagle%26rsquo;s in-brain control - the OCZ device maps controls to different thought intensities. Think really hard to go right, think a bit less hard to go left, and so on. While it seems counterintuitive, consider how malleable the brain is %26ndash; once you receive feedback from a game that a certain brain pattern creates a certain result, replicating it should become easier and easier. David commented: %26ldquo;We used the mouse to aim, but everything else, including jumping and firing, was done entirely with our brains. Put simply, it felt like we were flying. The more we got a feel for the levels of thought needed, the faster, smoother, and more instinctive things got, until we eventually got back to the point we achieved in Pong, where we were hardly having to think about what we were doing. We were going Tetsuo on everyone%26rsquo;s asses and it felt great.%26rdquo; Sounds brilliant to me. If you%26rsquo;re dying to try it out, OCZ%26rsquo;s device isavailable now for around $160. Another company, Emotiv, has designeda similar neuroheadsetwhich also has the capability to record facial expressions (something Lanier predicted in the %26lsquo;08s). Emotiv%26rsquo;s headset isn%26rsquo;t available yet, but will eventually retail for $299. They may not provide as precise control as in-brain implants, but there%26rsquo;s certainly hope for improvement. So what about the flip side, computers controlling our sensations? To return to our initial example, eXistenZ isn%26rsquo;t just about being able to control a game with one%26rsquo;s mind %26ndash; it%26rsquo;s about the game manipulating all of the senses to completely immerse a player in a reality indistinguishable from any other. Donning a stereoscopic headset is not the same as having sensation piped directly into the brain. While we%26rsquo;re not looking at living in Cronenberg%26rsquo;s world any time soon,the potential exists, and is scientifically plausible. In fact, Sony owns a patent for a noninvasive method of implanting sensations into the brain via ultrasound. No, really,take a look. Sony stated a couple years ago that this is a %26ldquo;prophetic invention%26rdquo; and not something they are currently experimenting with,and some, like Nova Spivack,warn againt over hyping the concept,given that any actual application of it may be 100 years off. On the next page: Cyberspace%26hellip; ---*---*---*--- Page Break : page break 0 ---*---*---*--- Above: Cyberspace in 1982, ala TRON In 1991 Michael Benedikt gathered 50 attendees at the first International Conference on Cyberspace in Austin, Texas. Cyberspace was not, at the time, synonymous with the internet or the world wide web the way it often is now %26ndash; in 1992Bruce Sterlingdescribed it as the %26ldquo;place between phones,%26rdquo; and %26ldquo;a vast flowering electronic landscape.%26rdquo; Come 2003 and things didn%26rsquo;t quite pan out the way the early poets predicted. In aninterview with GAIN 2.0, Benedikt lamented: %26ldquo;%26hellip; cyberspace%26mdash;that wonderful, phantasmagoric three-dimensional alternative reality imagined by William Gibson%26mdash;was not actually shaping itself on-line as I and many others thought it surely would. What Mosaic, then Netscape, then Explorer delivered was mostly the content of your local drugstore newsstand, but worse: delivered more jerkily, more shallowly, and more resolutely two-dimensionally%26mdash;like paper flyers blown against the back of the computer screen%26hellip; To this day, only advanced intranet gamers have a foretaste of Gibsonian cyberspace: a real-time, shared, virtual space seamlessly mixing useful data, personal presence, and real-world, real-time connection.%26rdquo; Neal Stephenson didn%26rsquo;t coin the word %26ldquo;cyberspace%26rdquo; %26ndash; William Gibson and Neuromancer are credited with that %26ndash; but Stephenson%26rsquo;s novel, Snow Crash, is a stellar example of the %26ldquo;cyberpunk%26rdquo; vision. Snow Crash describes a virtual world that would make Benedikt proud (if it existed), and was a major factor in the popularization of the Sanskrit term %26ldquo;avatar.%26rdquo; Its hero, snarkily named Hiro Protagonist (now the Gamertag of J Allard), navigates a completely physically immersive virtual space ("The Metaverse%26rdquo;) which he can mold to his liking with his decidedly 1337 hacking skills. The internet today has grown to similar lengths, sans the immersive virtual space. Whether or not it%26rsquo;s impressive to the fathers of the cyberspace concept, the internet relays massive amounts of data, and has a very real effect on economics, public opinion, and day-to-day life. But gee Mr. Writer, what does it all mean for gaming? Ugh, Second Life. I didn%26rsquo;t want to, but I had to bring it up. It%26rsquo;s not that it%26rsquo;s the best MMOG out there, far from it, but its open-endedness, if clunky, makes it a good example of the cyberspace concept and its relation to virtual reality. Second Life is no Snow Crash Metaverse, but it doesapply many similar concepts. It%26rsquo;s a shared, malleable experience. And what is virtual reality, after all, if you%26rsquo;ve no one to share it with? In what %26ldquo;reality%26rdquo; are we surrounded by artificial characters?MMOGs like Second Life represent a major step toward the realization of the cyberpunk vision. As for the future, the number of MMOGs which currently exist or are in development is massive, and the genre%26rsquo;s growth doesn%26rsquo;t seem to be slowing. What began as text based D%26amp;D clones has evolved, and continues to evolve at aheavy clip. Enhance the graphics, physics processing and apply less rigid rule sets and something approaching the poetic visions of cyberspace begins to emerge. It may not be where the internet in general is heading (why complicate the ruthless efficiency of HTML?), but it is certainly a more fun and exciting way to interact online, and gamers seem to be the most willing to explore the possibilities laid out by the cyberpunk genre. On the next page: Augmented reality, and what to expect in the foreseeable future%26hellip; ---*---*---*--- Page Break : page break 1 ---*---*---*---  Tom Caudell coined the term “augmented reality” (AR) in 1990, but the concept of artificially modifying our perceptions of reality is very old. In a lose sense, any artistic endeavor can be considered an act of augmenting reality, especially those which manipulate perspective, such as trompe l'oeil (literally, "trick the eye") paintings. (Right: Pere Borrell del Caso, Escaping Criticism, 1874, National Gallery of Art, Washington D.C.) In the modern sense, augmented reality refers to the integration of real world and computer generated data. In this sense, the HUDs fighter pilots are accustomed to are forms of augmented reality. Simple HUDs are cool, but as a child, I was fairly certain that by now I’d see things the way robot vision was always portrayed in movies - grainy text would tick along the bottom of my vision, identifying well-muscled men as potential threats, three-dimensional grids would overlay everything for some reason, and the answers to all my homework assignments would materialize in front of me. Above: I can has neural-net processor? In the 1988 Whole Earth Review interview I mentioned earlier, Lanier suggested that after putting on our virtual reality gear, we would be presented with a virtual version of the room we were in. Our furniture would be replaced with virtual furniture (so we don’t bump into it), and new virtual elements would be spawned within the room. Ideally, one day I’ll put on my sunglasses and see virtual fish floating through downtown San Francisco, an arrow indicating the direction to my current destination, and an avatar representing a distant friend who has decided to chat with me. A lot has actually happened in the department of augmented reality since I was just a boy dreamer. The boring utilitarian applications aside (like dropping that white bar over the first down line in televised football games), gaming applications are being experimented with extensively, and will likely be one of the big driving forces for other augmented reality applications (aside from military research). Essentially, any game that superimposes virtual elements on video in real time can be considered an augmented reality game. The latest is You’re in the Movies, which dynamically cuts out the bodies of its players and places them into virtual scenes. Not exactly Minority Report, but it will do as an example. Above: OH MY GOD IT'S LIKE SHE'S IN A REAL MOVIE!  The much cooler applications involve headsets which overlay 3D models on real space. Unfortunately, the headsets are rather large at the moment, and generally have to be connected to bulky wearable computers (not exactly the magic sunglasses I envisioned). These headsets must be not only capable of calculating spatial geometry well enough to realistically superimpose computer generated 3D objects into real space, they must also track eye movement and have the processing power to render the objects in the first place, all in real time. Film studios get to spend months working on special effect scenes – these systems have milliseconds. (Right: A conceptualization of the Tinmith AR system, 2006) Clunky or not, the technology already exists. The barriers now are in regards to size and horsepower. To create effective consumer AR headsets, everything needs to be smaller. When we can pack the power of a couple PS3s into a pair of sunglasses, bundled with batteries capable of powering them, we can have all the augmented and virtual reality we want. Expect Japan to be first on the scene with virtual girlfriends. Of course, you won’t be able to touch that virtual girlfriend – she’s just a 3D model being projected into your vision. Star Trek wouldn’t have any of that, concocting a device which converts photons into physical matter. Sorry, it’s not so plausible. Certainly there have been creations that sort of look like holodecks, projecting 3D renders onto all six surfaces of a room, as well as holographic displays which no longer require stereoscopic headsets, but nothing that comes close to that whimsical TNG episode in which Picard and company are stuck in 1941 San Francisco.   It’s about, as I have already suggested, miniaturizing technology and increasing processor power. Consider all of the elements I have described packaged into one affordable consumer device – a sleek wearable stereoscopic headset which tracks head motion, eye motion, expressions, and neural activity. Add gloves which track hand motion and a highly open-ended MMOG and… You’ve got everything you need to navigate and interact with a virtual world. The next generation of neuralheadests should be able to take care of basic motion in space (without having to stand in a hamster ball), head and eye tracking will allow you to look around, and motion sensing force feedback gloves will allow you to interact with objects (guns, swords, BDSM whips) and menu systems. Add augmented reality functionality, and you can turn your office building into a war-zone, or your bedroom into a 1950’s bar, placing, as Lanier suggested, virtual objects over the real objects in your surroundings. Above: The technology is still far too expensive to make Home Reality Engines possible just yet. Even at the affordable end, the Headplay head mounted display costs about $500, and it's just a stereoscopic display - no head or eye tracking It’s no holodeck, but it’s perhaps even more advanced that what Lanier described in the ‘80s when he coined the term “virtual reality,” and all of the required elements already exist in rudimentary forms. The revolution will occur when VR gadgets are no longer designed as disparate peripherals intended to mesh with standard games, and are integrated into singular “VR consoles” for which triple-A game titles are specifically designed. Sega tried once with Sega VR, and that didn’t quite pan out. Neither did Nintendo’s Virtual Boy, but it was designed to exploit the '90s VR hype, not as a serious attempt to build a virtual reality console (the technology simply wasn't available). Above: Fail The hype surrounding virtual reality may have faded to an extent,  but with the popularity of MMOGs and the inevitable cooling-off of the "casual gaming" boom, I expect VR to regain some of its glamourous draw, and not just amongst speculative dreamers - console manufacturers will regain interest as well. I would be surprised if consumer devices of the grade I have described aren’t available within the next 10 years. Yeah, I know it has all been said before - predicting the future is hard - but if I'm right, you can be sure that I'll be one of the first to assume the body of an electric phantasm and take a swim in the binary oceans of The Metaverse. Hear more about this article in TalkRadar. Aug 6, 2008