NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

On a recent project, I was trying to avoid having to write a simplified version of the control that the Holograms app offers on selected holograms for moving, rotating and scaling. This control here;

It’s essentially doing a job analogous to the resize handles that you get on a Window in the 2D world and I’d imagine that it took quite a bit of work to write, debug and get right.

I didn’t have time to try and reproduce it in full fidelity and so I really wanted to just re-use it because I knew that to write even a limited version would take me a few hours and, even then, it’d not be as good as the original.

I didn’t find such a control and so, between us, myself and Pete wrote one which ended up looking a little like this;

and it was pretty functional but it wasn’t as good as the one in the Holograms app and it took a bunch of time to write and debug.

However, just after it was completed someone inside of the company pointed me to a place where I could have taken a better control straight off-the-shelf and that’s what I wanted to write about in this post.

The Mixed Reality Design Labs

This came completely out-of-the-blue to me in that I hadn’t seen the Mixed Reality Design Labs before – here a quick screenshot from their home page on github;

I’m going to shorten “Mixed Reality Design Labs” to MRDL (rhymes with TURTLE) in this post because, otherwise, it gets too long.

Straight away I could see that on the home page are listed a number of very interesting sounding controls including these two, the Holobar and the Bounding Box.

 

and my first thought was as to whether those controls just draw the visuals or do they also implement the behaviour of scale/rotate/translate but, either way, they’d be saving me a tonne of work

I noticed also that this repo is linked from the mixed reality development documentation over here;

but it doesn’t look like it’s a simple 1:1 between the MRDL and this document page because (e.g.) the guidance here around ‘Text in Unity’ actually points into pieces from the HoloToolkit-Unity rather than controls within the MRDL.

By contrast, I think the other 4 sections listed here do point into the MRDL so it’s nearly a 1:1

Including the MRDL

I wanted to try some of these things out and so I spun up a blank Unity project, added the HoloToolkit-Unity and set it up for mixed reality development (i.e. set the project, scene, capability settings using the toolkit) before going to look at how I could start to make use of these pieces.

As there didn’t seem to be any Unity packages or similar in the repo, I figured it was just a matter of copying the MRDesignLabs_Unity folder into my project and I went with that and after Unity had seen those files I got a nice helpful dialog around the HoloLens MDL2 Symbols font;

and so I went with downloading and installing that but then I realised that I was perhaps meant to import that .ttf file as an asset into Unity here and then (using the helpful button on the dialog) assign it as the font for buttons as per below;

That was easy enough but I then dropped off down a bit of a black hole as there seems to be quite a lot in this project;

and I find this quite regularly with Unity – when someone delivers you a bunch of bits, how do you figure out which pieces they intended you to use directly and which bits are just supporting infrastructure for what they have built – i.e. there seems to be a lack of [private/public/internal] type visibility which would guide the user as to what was going on.

Rather than get bewildered though, I figured I’d go back to trying to solve my original problem – selecting, rotating, scaling, translating a 3D object…

Manipulations, Bounding Boxes, App Bars

I must admit, I got stuck pretty quickly.

Like the HoloToolkit, there are a lot of pieces here and it’s not easy to figure out how they all sit together and so I also opened up the Examples project and had a bit of a look at the scene called ManipulationGizmo_Examples which gave me more than enough clues to feel confident to play around in my own blank scene.

I started off by dragging out the HoloLens prefab which brings with it a tonne of pieces;

I did have a little dig around into where these pieces were coming from and what they might do but I’m not going to attempt to write that up here. What I think I noticed is that these pieces look to be largely independent rather than taking dependencies on the HoloToolkit-Unity but I may be wrong there as I haven’t spent a lot of time on it just yet.

Where I did focus in was on the Manipulation Manager which looked like the piece that I was going to need in order to get my scale/rotate/translate functionality working and it’s a pretty simple script which simply instantiates the ‘Bounding Box Prefab’ and the ‘App Bar Prefab’ and keeps them around as singletons for later use which seems to imply that only one object would have the bounding box wrapped around it at a time (i.e. a single selection model) which seems reasonable to me. I also noticed in that script that it takes a dependency on the Singleton<T> class from the HoloToolkit-Unity so that starts to go against my earlier thoughts around dependencies;

To try this out, I added a quick cube into my scene;

and (from the example scene) I’d figured that I needed to add a Bounding Box Target component to my cube in order to apply the bounding box behaviour;

I like all the options here but I left them as their default values for now and I ran up the code on my HoloLens using the Unity’s Holographic Remoting feature and, sure enough, when I can tap on the cube to select it I get the bounding box and I can then use manipulations in order to translate, rotate and scale matching the functionality that I asked for in the editor.

This screenshot comes from Unity displaying what’s happening remotely on the HoloLens and you can see the AppBar with its Done/Remove buttons and the grab handles which are facilitating scale/rotate/translate. What’s not visible is that the control has animations and highlights which make for a quality, production feel rather than a “Mike just hacked this together in a hurry” feel

I’m going to try and find some time to explore more of the MRDL and I’ll write some supplementary posts as/when those work out but if you’re building for Windows Mixed Reality then I think you should have an eye on this project as it has the potential to save you a tonne of work