The future and impact of mixed reality

There is an enormous hype on everything you might call "immersive technologies" these days, that is virtual reality (VR), augmented reality (AR) and the artificial word invented by Microsoft mixed reality (MR). People outside tend to throw it all into one bucket: It's those weird people wearing even more weird things on their heads. And often this odd feeling may already nourish first reactions of rejection. I am not an exception here. For me personally, I find all those VR headsets frightening. I don't want to wear a black box on my head where I can't see the real world anymore. So I tend to shy away from VR generally. Last week I brought myself to wearing my first VR device: A simple DayDream together with a DayDream certified mobile. It was a blast, I really liked it. Still I prefer to see "what's around me" in the real world. 

This seems to be so important to me that I don't shy away from wearing the chunky HoloLens - because I can see real stuff through it. But it goes deeper. After I contributed to the first HoloLens repair app I see a bit clearer what works with MR and what doesn't. Wait a second: What was MR compared to AR again? I would put it like this: Where AR focusses more on giving extra information to this or that object in the real world, just like google glass did or any other mobile device AR app mixed reality is really about the big stuff: Big holograms and using the space you're in for a real interaction. Going around, looking behind things (e.g. holograms or real objects). There is no sharp border line between AR and MR. Funny enough many of the most impressive HoloLens apps don't use the room around at all. To be honest: You could learn about the different parts of a turbine very well in VR, too. Maybe you wouldn't move your body so much as in the most impressive Japan Airlines demonstration. I had a chat with one of the directors behind this app and he admitted you could have done it in VR as well.  

So why should you try to mix up real world and holograms? What's so cool about it? Chasing virtual Pokemon monsters? Definitely not. Well at least not for me. I'm an engineer, I'm this techie guy, I like to do real stuff in a real world where it's oily, dirty and loud (describes good music by the way, too). Of course there's the whole advertising industry waiting for us to be victims of a new visually enhanced world where we get flashy personalized commercials overlaid on our real world wherever we are – a horror scenario if you ask me. Still there's an enormous potential of this technology in a most sensible way: 

• Manipulation: All kinds of repair and maintenance scenarios in the industry, either on- or offline: Real world things you need to interact with are highlighted and extra information displayed. The way how to operate with certain parts is shown in "ghost mode" - a holographic animation shows how the real world object should be manipulated next. And it shows this on the real object, in the scale of the real object, making up for the most direct and realistic experience one could possibly imagine. Before/after scenarios assist during complex repairs to get things all wired up again.  
• Information: When learning things on a real machine it can be invaluable to get more information upon this or that part by simply pointing to it and getting all the information needed. This includes "which part can be found where" scenarios. So this makes up for all kinds of infotainment apps where learning on a real machine is applicable. Keep in mind it could make good sense to choose VR instead whenever accessing the real machine might be difficult.  
• Navigation: Know what comes next in a workflow. Know which parts of a machine have already been checked by you (or somebody else?) and which don't. Navigate inside rooms and find all kinds of virtual extra information stuck to the walls around you even when you return.  

So this all sounds cool. It is or can become reality NOW. The technology is there, even though it might look a bit clumsy right now. Anyway now is the time when to build the apps and scenarios.  

But is this already it? Pinpointing some extra information to the surrounding objects? I believe not. It's only the first tiny step on a journey yet to come. In my eyes the future is not about just displaying some shiny holograms here and there. It's more about connecting these visual effects with real world process data – e.g. time-variant variable data. So it's up to visualizing physical values such as pressure, temperature, fluid velocity etc. in a natural, yet impressive way. When manipulating a machine with pipes of hot fluids inside it might be very natural to see these pipes glowing and pulsating as well es getting visual and audio advice when getting too close to it. Visualizing currents in circuits might help to recognize unusual conditions earlier. Machines in production lines telling their overall status when just looking at them might become standard in future production lines. Visualizing wearing of machines and parts fed by big data analysis from the backend of the production lines could facilitate predictive maintenance in a very clever way making those data a direct experience onsite. So what it's really all about is – in my eyes – connecting data. The charming thing is that visualizing that data has become a modern adventure with lots of possibilities.  

We have broken outside the 2D-resricted way of ordinary computers or even mobile devices. We can now see things we couldn't see before (yes: They may literally be just around the corner). And by the way: It's arts doing a cool visualization. You recognize that it's no longer the usual developer folks teamed up in realizing holographic applications: All of a sudden you're teamed up with folks you would have found only in creative jobs such as the cinema or gameplay industry earlier. You line up with 3D artists and interaction designers. User centered design becomes part of "normal work" and is no longer restricted to some illusionary high-budget luxury software projects. It feels natural teaming up with all these creative heads and has made my working place a lot more inspiring and fun. This is where mixed reality becomes more than just a fancy visualization method: This is where it changes our lives.