NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

Creators Update and the Windows Mixed Reality Portal/Simulator

I’ve been trying out the Creators Update (get it here) on an older Surface Pro 3 for quite a while but I haven’t given it a tonne of focus just yet but with the release I thought it was time to upgrade both my home PCs and my work PCs and I did that in a few spare hours the other day.

One of the things that the Creators Update brings with it is the Windows Mixed Reality Portal and that portal also comes with the Windows Mixed Reality Simulator.

I’ve got very familiar with running an app like the ‘Holograms’ app on my HoloLens and using it to position objects in the real world – here’s a screenshot of the Spaceman floating in mid-air inside a huge atrium at Manchester Metropolitan University the other month;

While I don’t have an immersive Windows Mixed Reality headset available to me today, the Creators Update makes it possible for me to make use of the Holograms app in an entirely virtual world shown to me via the Windows Mixed Reality Portal. So, if I run up the portal on my device and set it up in Developer Mode for Simulation as detailed in this document then I can use the simulator to see the Spaceman in a whole new ‘world’;

So now I have my HoloLens which blends virtual objects into my real world and I have the Mixed Reality Simulator which puts virtual objects into a virtual world projected onto a flat 2D screen and they’re running what looks like a very similar Holograms app

Can I Play RoboRaid?

My first thought was ‘Hey, maybe I can now go and play RoboRaid in this environment?’ but I don’t see that listed today in the Store that I access on my device whether from inside or outside of the Windows Mixed Reality Simulator.

That makes sense to me because I could see the RoboRaid developers perhaps needing to make some changes to their app in order to run inside of this environment rather than on HoloLens.

There are, clearly, differences between HoloLens hardware and immersive headset hardware with the grids on this page listing out different features including Gestures, Motion Controllers, Spatial Mapping and so it wouldn’t be surprising if an app like RoboRaid which makes extensive use of spatial mapping and gestures needed some tweaks to run on an immersive headset.

From looking into the Store and the docs, my curiosity was sparked enough to want to have a think about UWP device families, contracts, APIs for detecting headsets and so on and I wrote a few notes around those below based purely on my experiments in the debugger with the public bits. I have no ‘special’ knowledge here and, clearly, if I did then I wouldn’t have to conduct these types of experiments in a debugger to try and figure things out

What Device (Family) am I Running On?

I wondered what device family I was running on when I run an app inside of the Windows Mixed Reality Simulator on my PC. I think I know but I wanted to check.

If I write a basic ‘Hello World’ app that I run on the HoloLens emulator (Windows 14393) and the Mixed Reality Simulator (Windows 15063) choosing my platforms a little carefully;

then I can query the device family property (NB: this is rarely the right way to code for device specific code but it’s a cheap/cheerful thing to do here) then I see this from the Mixed Reality Simulator;

and this from the HoloLens emulator;

which lines up with what I’d expect – in the HoloLens case, the device is its own computer whereas in the other case it’s my PC that’s the computer. This was the output from this line of code;

            this.txtDeviceFamily.Text = AnalyticsInfo.VersionInfo.DeviceFamily;

Where Do My APIs Come From?

From the point of view of the UWP, this would suggest that when I’m running on the Mixed Reality Simulator I should have access to the Universal API Contract and the Desktop API Extensions.

What about on HoloLens? I would have access to the Universal API Contract alone – as far as I’m aware, there isn’t a set of ‘Mixed Reality’ extension APIs for the UWP and when I look into the folder on my machine I see;

although I’m not 100% certain that this is an exhaustive list but if I go and check the documentation then I see that APIs like these ones;

https://docs.microsoft.com/en-us/uwp/api/windows.graphics.holographic

reside in the Universal API contract and so have the potential to be called on any device family albeit with the developer understanding that they need to check for support prior to calling those APIs.

How then does a developer make a call to know whether they are running inside of a mixed reality headset environment?

Do I Have a Mixed Reality Headset?

I can check whether I have a mixed reality headset or not by making a call to the APIs IsSupported and IsAvailable on this HolographicSpace class.

The former tells me (from the docs) whether the OS supports headsets and the latter tells me whether one is attached to the PC and ready for rendering (post set-up by the user).

I was a bit puzzled because the docs seemed to suggest that the IsSupported API was present in 10586 and yet Lucian’s super-helpful plugin told me different;

and the code crashed on the HoloLens emulator running 14393 so that seemed to suggest that these APIs weren’t part of 14393.

This wouldn’t be a problem for me except that my HoloLens and emulator are on 14393 and so I ended up with code that looked like this for the moment;

        void UpdateHolographicHeadset()
        {
            var text = "Holographic not supported";

            if (Windows.Foundation.Metadata.ApiInformation.IsPropertyPresent(
                "Windows.Graphics.Holographic.HolographicSpace", "IsSupported") &&
                HolographicSpace.IsSupported)
            {
                text =
                    HolographicSpace.IsAvailable ?
                    "Holographic space available" :
                    "Holographic space not available";

                HolographicSpace.IsAvailableChanged += OnHeadsetAvailabilityChanged;
            }
            else if (AnalyticsInfo.VersionInfo.DeviceFamily == "Windows.Holographic")
            {
                text = "Holographic space supported on HoloLens";
            }
            this.txtHeadset.Text = text;
        }
        async void OnHeadsetAvailabilityChanged(object sender, object e)
        {
            await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
                () =>
                {
                    this.UpdateHolographicHeadset();
                }
            );
        }

and that seems to display reasonable results on the HoloLens emulator (14393);

and gives the same answer on my PC whether I launch the app inside/outside of the Mixed Reality Simulator;

and I could get this flag to change by switching simulation on/off and then running the app on my desktop;

although I’m not sure that I saw the app update dynamically from the “not available” –> “available” state when I switched Simulation back on but I’d have to test that again.

So, assuming that I was running everything on 15063 and could simply call these APIs, how would I differentiate between immersive/HoloLens headsets?

Immersive Headset?

There’s another flag to be tested on the HolographicDisplay class called IsOpaque which gives information about the type of display present. I could make a call to this (once again, factoring in the missing API on 14393);

     void UpdateDisplayType()
        {
            var displayType = "no display";

            if (Windows.Foundation.Metadata.ApiInformation.IsTypePresent(
                "Windows.Graphics.Holographic.HolographicDisplay"))
            {
                displayType =
                    (HolographicDisplay.GetDefault()?.IsOpaque == true) ?
                    "opaque lenses" : "transparent lenses";
            }
            else if (AnalyticsInfo.VersionInfo.DeviceFamily == "Windows.Holographic")
            {
                displayType = "transparent lenses";
            }
            this.txtDisplayType.Text = displayType;
        }

and on the Mixed Reality Simulator I see;

and I get the expected result on the HoloLens emulator;

but this could be a little confusing because I don’t actually have an immersive headset and I’m just running this app on a flat screen rather than with stereoscopic projection but the IsStereo flag can help me with that too;

        void UpdateStereoDisplay()
        {
            var stereoType = "unknown display";

            if (Windows.Foundation.Metadata.ApiInformation.IsTypePresent(
                "Windows.Graphics.Holographic.HolographicDisplay"))
            {
                stereoType = HolographicDisplay.GetDefault()?.IsStereo == true ?
                    "stereoscopic display" : "2D display";
            }
            this.txtStereoDisplay.Text = stereoType;
        }

but I don’t yet understand the output here because I somewhat expected the app inside of the simulator to say “2D” and it doesn’t seem to do that;

and it doesn’t when I run it purely on the desktop either;

so perhaps I need to think on that one a little bit more.

A Unity Example?

I wanted to see how this worked with a simple Unity example and so I made a ‘blank’ project with the HoloToolkit and just had a red cube and a wall and the idea is that when the user clicks on the cube, the red cube goes green.

I have this scene;

and then a script on the cube tries to handle a click to change the cube to be green and it also tries to hide the wall on headsets with transparent lenses;

using HoloToolkit.Unity.InputModule;
using UnityEngine;
using Windows.Graphics.Holographic;
using Windows.System.Profile;

public class CubeScript : MonoBehaviour, IInputClickHandler
{
    private void Awake()
    {
        var wall = GameObject.Find("Wall");

        if (wall != null)
        {
#if UNITY_UWP && !UNITY_EDITOR
            var opaque = true;

            if (Windows.Foundation.Metadata.ApiInformation.IsTypePresent(
                "Windows.Graphics.Holographic.HolographicDisplay"))
            {
                opaque = (HolographicDisplay.GetDefault()?.IsOpaque == true);
            }
            else if (AnalyticsInfo.VersionInfo.DeviceFamily == "Windows.Holographic")
            {
                opaque = false;
            }
            wall.SetActive(opaque);
#endif // UNITY_UWP
        }
    }
    public void OnInputClicked(InputClickedEventData eventData)
    {
        this.gameObject.GetComponent<Renderer>().material.color = Color.green;
    }
}

This seemed to work out fine on the HoloLens emulator (with the app manifest requesting a minimum platform of 14393, maximum of 15063 and targeting the Universal family);

with no wall which is what I was hoping for.

If I then deployed this to my local machine and ran it up inside of the Mixed Reality Portal using the simulator then I saw;

and so I get the wall and the cube running there although it is worth saying that I did experience some glitches (drivers perhaps?) around getting this to display in that sometimes it would display but on a few occasions I noticed the Mixed Reality Portal seemed to reset itself before I got to the Unity splash screen for my app.

Naturally, I’m sure there are better ways of doing what I’m doing here but I felt like I learned a few things and so thought I’d share as part of experimentation…it’s exciting to see these bits in the Creators Update and it’ll be even more exciting to get my hands on a headset and see what the experience is like there.

Don’t forget that you can sign up for news on those headsets here.