Unhinging the display of Reality

Virtual Reality is a tidal force with companies such as Oculus, Sony, Samsung & Google creating the waves. Awestruck Developers are flocking to the technology, transporting you to worlds that could only exist in your dreams.

But even the most presence-inducing, GPU frying, hyper-realistic demo today is nothing compared to whats coming.

What we see today is the first steps in a long line of iterations, todays demo’s are akin to our dinky BASIC programs running on our commodore 64’s of yesteryear.

In a virtualized reality we will have to re-evaluate alot of the ideas that have always been with us in computing, and the display of user interfaces, my interest is in what virtual reality’s “compositing window manager” (the software that manages your GUI, lets you move programs around, minimize/maximize them, have them overlapping) will look like in the future.

Have you thought about how this will work? And I don’t mean virtual screens, I mean a composition manager for reality itself.

If we move several iterations into the future (10 years?) we can imagine looking back to the year 2014, and what we have is something akin to MS-DOS, you load a program and run only that program, probably from your 2d desktop, then you put your HMD on, after that switching realities is a problem, sure you can alt-tab to another game/experience, but its not a smooth transition, and it is very disorientating for the wearer.

The next logical progression is to create a sort of “Matrix-like” construct, or a multiverse, a blank slate 3d world that acts like a program launcher, this is something I explored with Secondsight, but at best this would be like Windows 1.0 and will not satisfy for long. Why?

Because we’re virtualizing reality here, and we will need to take full advantage of what that means - like windows gui, we should be able to overlay realities, display realities side by side, minimize them and even intermesh 3d objects from different realities. But it goes even deeper than that.

Reality is formed in your mind by your senses, therefore the real holy grail we are working towards, is the goal of virtualized senses. You can see the seeds of this with hijacking of our vision in head mounted displays and touch feedback by haptic controls. With the advances of synthetic biology - It will only take a few more iterations and we’ll be working on the directly on the sensory pathways themselves.

And we’ll need a composition manager for our senses, a composition manager that tells our minds what reality is. Like windows you’ll be able to partition off your senses in such new and profound ways.

Imagine for a moment, in objective reality, you are in a restaurant with your partner, wearing your favourite shoes, but you’ve partitioned off your feet, not to feel your socks and the confines of the shoe, but instead, a hot water foot bath program runs on this area. You’ve partitioned off your back to run 2 programs at the same time, one swedish massage and simultaneously a gentle back scratching program.

Back in objective reality you are facing a bland wall of the restaurant while your partner can see the open are of the dining experience, maybe they can share their perspective with you so you can both enjoy it - or maybe you prefer to run an amazonian forest.. or why not both intermixed?

We will have the ability to determine what Reality means and be able to experience many different sensations in unique & interesting ways. Virtualizing our senses is going to give us unprecedented control over our experience in life andnew horizons for communications between us.

Maybe this sounds exciting, maybe this sounds scary. Either way - this is the software that will determine your reality, at it will happen within our lifetime.