Windows 10 Mixed Reality: Everything You Must Know

When Microsoft announced its intention to dive headfirst into the field of augmented reality, or AR as it is more commonly known as, at a Windows 10 event in early 2015, it took the world by surprise. The company demonstrated the HoloLens headset, and along with it a whole new software platform, dubbed Windows Holographic, that was aligned with Windows 10, and underpinned the company’s AR ambitions.

Less surprising was the announcement of renaming the platform two years later in March 2017.

Microsoft is unifying its virtual, augmented and mixed reality efforts around a single platform for shared experiences, officially under the label of Windows Mixed Reality.

The name may be different now, but the goal has not changed — the Redmond based technology titan wants to build an interoperable network of AR, VR and MR headsets from different manufacturers, all talking to one another, and capable of running Windows 10 Universal apps.

If this vision comes to pass, Microsoft will have its mixed reality software running in headsets ranging from $300 to $3,000, offering both consumers and enterprises whole new interactive experiences in areas like gaming, industrial design, manufacturing, ecommerce, medicine, data visualization, and more.

But before we get to the overview of what the company is planning, and what Windows Mixed Reality is all about, let’s take a quick look at what mixed reality actually is.

What is mixed reality?

Several companies have started using the term mixed reality to differentiate among the various points on the alternate reality spectrum. On the simper side, you have augmented reality monocular glasses and smartphone apps and games like Pokémon Go.

And on the other side you have more immersive experiences provided by virtual reality headsets like the Oculus Rift and the HTC Vive, as well as other, more affordable mobile wearables that rely on a smartphone screen, like the Samsung Gear VR.

Mixed reality is somewhere in the middle.

Think of this space, as the blending of real-world and virtual content into hybrid environments where physical and digital objects can coexist and interact. Microsoft explains this in more detail in the new Windows Mixed Reality website that it has put up for developers, which also highlights the type of experiences that users can expect from the various types of hardware devices for this platform that are set to launch later this year.

The industry may be mired in confusing terminology at the moment, but Microsoft has already identified the points it plans to plug into in this continuously developing field.

Not just that, but it is moving full speed ahead in trying to make these plans come to fruition, in what many consider to the be the next frontier of computing. Underlying these ambitious plans is the Windows Mixed Reality platform that can be broken down into a number of different layers.

The making of a mixed reality platform

Although Microsoft was somewhat quiet when it came to providing details of its MR platform earlier on, the company has, in recent months, offered several insights into how it is building the technology infrastructure, and how it is working with OEMs to bring affordable headsets to the market.

The recently released Windows 10 Creators Update came with the Windows Mixed Reality platform built in, introducing support for mixed reality Windows apps on a range of different computing devices.

And companies like Acer, Dell, HP, Lenovo and 3Glasses have already showcased their HMDs, or head-mounted displays, if you will, at CES 2017. General consumers are not yet able to purchase these headsets, which have actually been announced since October last year. But the first of these, the Acer Windows Mixed Reality Development Edition headset is already shipping to developers, as is the development edition of HoloLens that has been on the market for more than a year now. If you’re willing to buy them, these developer edition devices offer interested people a ticket to this immersive and engaging new field.

But while these headsets may have started hogging the spotlight, the real star of the show, however, is the Windows Mixed Reality platform that makes these experiences all possible. It is made up of a number of different layers.

At the top of the stack is the HMD, the head-mounted display itself. The layer below it is designed to handle input, middleware and APIs, application program interfaces, as well as mixed reality spatial perception. The final layer is made up of what Microsoft calls a shell, and this section also integrates cloud services like Xbox Live authentication and syncing, collaborative messaging via Skype, and more.

Layering it all up

Starting with the top layer, the HMD come with what is called the Six Degrees of Freedom (6DoF), representing the level of 3D motion and tracking that is supported. In simpler terms, this means that you will be able to move forward, backward, left, right, up or down, when wearing these headsets.

HoloLens, in particular, makes particularly good use of this, and supports what Microsoft has conveniently labeled inside-out 6DoF. What that means is that the device makes us of position tracking that combines data from multiple input sensors and cameras, along with a bunch of algorithms. All this makes for a very smooth experience.

For context, the smartphones VR headsets like Samsung Gear VR and Google Daydream View currently only support 3DoF, in other words allowing wearers to just pitch, yaw and roll. Oculus Rift and HTC Vive, however, have 6DoF position tracking, support for which Microsoft has included in the Windows 10 Creators Update.

The next layer is where all the magic happens.

We have the input layer to process all the visual, voice, gyroscopic and spatial input data that the mixed reality device collects. It is also possible to combine gestures, spatial sound and mapping here, too. Then comes the part that contains middleware and API to connect with the rest of the software ecosystem. There are instructions for building holographic apps using Unity, the popular cross-platform game development engine, for instance. Developers can also build mixed reality apps using the Microsoft DirectX runtime. As these headsets run any Universal Windows app, you can imagine the possibilities this provides

The final layer of the stack brings along a custom user interface, or the shell, which is also integrated with services like Xbox Live and Skype, as mentioned above. The Acer headset, for example, lays out the operating system as rooms in a house, with the user being able to teleport into different rooms to launch various applications that are displays as either floating icons or tiles on a wall. Apps and features like Photos, Movies & TV, the Edge web browser, or even the Start Menu are available here, and the experience also works with an Xbox controller.

These devices really are fully fledged Windows 10 computers for your head!

Setting the standards

Microsoft may release a standard mixed reality hardware specification at some point in the future, but right now, the company is focused on the software stack and helping various partner companies engineer the coming wave of headsets. Work on the Windows Mixed Reality platform, however, continues. For example, Microsoft has promised support for integrated graphics chips in mixed reality headsets via an update coming later this year.

Sure, they will probably only be limited to the recent 7th generation Intel Core series chips, but the point is that Microsoft wants to offer this mixed reality experience to users at all price points, and hardware of all capabilities. The goal is to have mixed reality attainable for everyone. Any modern, $500 and up PC will be able to run the mixed reality platform, and will be fully compatible with the various virtual reality headsets that companies like Acer, Dell, HP and more are bringing to the market. Even the Xbox One console is set to receive support for the platform in 2018.

Which brings us to HoloLens, an untethered device that houses its own processor, graphic chip and display in what is currently a $3,000 package, primarily meant for developers and enterprise users.

The HoloLens question

Interestingly, despite the fact that Windows Mixed Reality is the software that powers the HoloLens, Microsoft continues to refer HoloLens and the other mixed reality headsets as separate thing. Which may hint that the company plans to position the enterprise-facing HoloLens as the more attractive option for mainstream business deployment.

End of March 2017 marked one year since Microsoft began shipping the HoloLens Development Edition in the United States. Since then, the company has expanded the availability of the device to regions like UK, Ireland, France, Germany, Australia, New Zealand, Japan, and soon, China. Over this period of time, more than 150 mixed reality apps have launched on the Windows Store.

In fact, the company also released the HoloLens Commercial Suite last August for $5,000.

The extra two grand getting buyers access to features like deeper mobile device management (MDM), with Microsoft Intune, along with Kiosk Mode that launches HoloLens directly into a specific app experience for use as an in-store demo in a retail environment. It also brings to the table BitLocker Drive Encryption, and access to virtual private network (VPN) protection. HoloLens remains an expensive proposition, but it is capable of doing what no other device on the market can.

And while the shiny new headsets from partner companies will draw the most attention, Microsoft’s mixed reality strategy goes far beyond this, as the software titan continues to refine the Windows Mixed Reality software platform for this whole new frontier of computing.

Exciting times, straight ahead!

2 thoughts on “Windows 10 Mixed Reality: Everything You Must Know”

    • Hope you have enabled Mixed Reality Developer mode before trying it out.
      Please follow instructions given in article.


Leave a Comment