It was early 2015 during Windows 10 Launch event when, Microsoft demonstrated the HoloLens headset, and along with it a whole new software platform, dubbed Windows Holographic, that was aligned with Windows 10, and underpinned the company’s AR ambitions.
Microsoft is unifying its virtual, augmented and mixed reality efforts around a single platform for shared experiences, officially under the label of Windows Mixed Reality.
If this vision comes to pass, Microsoft will have its mixed reality software running in headsets' price ranging from $300 to $3,000, offering both consumers and enterprises whole new interactive experiences in areas like gaming, industrial design, manufacturing, ecommerce, medicine, data visualization, and more.
Several companies have started using the term mixed reality to differentiate among the various points on the alternate reality spectrum.
On the simpler side, you have augmented reality monocular glasses and smartphone apps and games like Pokémon Go.
And on the other side you have more immersive experiences provided by virtual reality headsets like the Oculus Rift and the HTC Vive, as well as other, more affordable mobile wearables that rely on a smartphone screen, like the Samsung Gear VR.
Mixed reality is a blend of Augmented and Virtual Reality.
Think of this space, as the blending of real-world and virtual content into hybrid environments where physical and digital objects can coexist and interact.
Microsoft explains this in more detail in the new Windows Mixed Reality website that it has put up for developers, which also highlights the type of experiences that users can expect from the various types of hardware devices for this platform that are set to launch later this year.
The industry may be mired in confusing terminology at the moment, but Microsoft has already identified the points it plans to plug into in this continuously developing field.
Not just that, but it is moving full speed ahead in trying to make these plans come to fruition, in what many consider to the be the next frontier of computing. Underlying these ambitious plans is the Windows Mixed Reality platform that can be broken down into a number of different layers.
Although Microsoft was somewhat quiet when it came to providing details of its MR portal earlier on, the company has, in recent months, offered several insights into how it is building the technology infrastructure, and how it is working with OEMs to bring affordable headsets to the market.
The recently released Windows 10 Creators Update came with the Windows Mixed Reality portal built in, introducing support for mixed reality Windows apps on a range of different computing devices.
You'll now find yourself in a 3D space that can be navigated and interacted.
If you don't want to use Mixed Reality Portal and all Holographic stuff in Windows 10, you can choose to remove it.
Companies like Acer, Dell, HP, Lenovo and 3Glasses have already showcased their HMDs, or head-mounted displays, if you will, at CES 2017. General consumers are not yet able to purchase these headsets, which have actually been announced since October last year.
But the first of these, the Acer Windows Mixed Reality Development Edition headset is already shipping to developers, as is the development edition of HoloLens that has been on the market for more than a year now. If you’re willing to buy them, these developer edition devices offer interested people a ticket to this immersive and engaging new field.
But while these headsets may have started hogging the spotlight, the real star of the show is the Windows Mixed Reality platform that makes these experiences all possible. It is made up of a number of different layers.
At the top of the stack is the HMD, the head-mounted display itself. The layer below it is designed to handle input, middleware and APIs, application program interfaces, as well as mixed reality spatial perception. The final layer is made up of what Microsoft calls a shell, and this section also integrates cloud services like Xbox Live authentication and syncing, collaborative messaging via Skype, and more.
Starting with the top layer, the HMD come with what is called the Six Degrees of Freedom (6DoF), representing the level of 3D motion and tracking that is supported.
In simpler terms, this means that you will be able to move forward, backward, left, right, up or down, when wearing these headsets.
HoloLens, in particular, makes particularly good use of this, and supports what Microsoft has conveniently labeled inside-out 6DoF. What that means is that the device makes us of position tracking that combines data from multiple input sensors and cameras, along with a bunch of algorithms. All this makes for a very smooth experience.
For context, the smartphones VR headsets like Samsung Gear VR and Google Daydream View currently only support 3DoF, in other words allowing wearers to just pitch, yaw and roll. Oculus Rift and HTC Vive, however, have 6DoF position tracking, support for which Microsoft has included in the Windows 10 Creators Update.
The next layer is where all the magic happens.
We have the input layer to process all the visual, voice, gyroscopic and spatial input data that the mixed reality device collects. It is also possible to combine gestures, spatial sound and mapping here, too.
Then comes the part that contains middleware and API to connect with the rest of the software ecosystem. There are instructions for building holographic apps using Unity, the popular cross-platform game development engine, for instance.
Developers can also build mixed reality apps using the Microsoft DirectX runtime. As these headsets run any Universal Windows app, you can imagine the possibilities.
The final layer of the stack brings along a custom user interface, or the shell, which is also integrated with services like Xbox Live and Skype, as mentioned above.
The Acer headset, for example, lays out the operating system as rooms in a house, with the user being able to teleport into different rooms to launch various applications that are displays as either floating icons or tiles on a wall.
Apps and features like Photos, Movies & TV, the Edge web browser, or even the Start Menu are available here, and the experience also works with an Xbox controller.
These devices really are fully fledged Windows 10 computers for your head!
Microsoft may release a standard mixed reality hardware specification at some point in the future, but right now, the company is focused on the software stack and helping various partner companies engineer the coming wave of headsets.
For example, Microsoft has promised support for integrated graphics chips in mixed reality headsets via an update coming later this year.
Sure, they will probably only be limited to the recent 7th generation Intel Core series chips, but the point is that Microsoft wants to offer this mixed reality experience to users at all price points, and hardware of all capabilities.
The goal is to have mixed reality attainable for everyone. Any modern, $500 and up PC will be able to run the mixed reality platform, and will be fully compatible with the various virtual reality headsets that companies like Acer, Dell, HP and more are bringing to the market. Even the Xbox One console is set to receive support for the platform in 2018.
Which brings us to HoloLens, an untethered device that houses its own processor, graphic chip and display in what is currently a $3,000 package, primarily meant for developers and enterprise users.
Interestingly, despite the fact that Windows Mixed Reality is the software that powers the HoloLens, Microsoft continues to refer HoloLens and the other mixed reality headsets as separate thing. Which may hint that the company plans to position the enterprise-facing HoloLens as the more attractive option for mainstream business deployment.
End of March 2017 marked one year since Microsoft began shipping the HoloLens Development Edition in the United States. Since then, the company has expanded the availability of the device to regions like UK, Ireland, France, Germany, Australia, New Zealand, Japan, and soon, China. Over this period of time, more than 150 mixed reality apps have launched on the Windows Store.
In fact, the company also released the HoloLens Commercial Suite last August for $5,000.
The extra two grand getting buyers access to features like deeper mobile device management (MDM), with Microsoft Intune, along with Kiosk Mode that launches HoloLens directly into a specific app experience for use as an in-store demo in a retail environment. It also brings to the table BitLocker Drive Encryption, and access to virtual private network (VPN) protection. HoloLens remains an expensive proposition, but it is capable of doing what no other device on the market can.
And while the shiny new headsets from partner companies will draw the most attention, Microsoft’s mixed reality strategy goes far beyond this, as the software titan continues to refine the Windows Mixed Reality software platform for this whole new frontier of computing.
Exciting times, straight ahead!