Windows 10, UWP and Sphero–Bringing 2D UWP Demo Code to HoloLens

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

I made up a demo many months ago now for showing some of the universal nature of the Windows 10 UWP platform.

The idea of the demo was to show that the UWP was as device agnostic as possible whilst still allowing for device specialisation as necessary.

The demo was to code up an app from scratch to control a Sphero ball such that the demo app had similar functionality across PC, Phone and IoT but had slightly different UI for Phone and slightly different functionality for IoT.

I’ve used this demo in many places and you can find a recording of one earlier version of it over here on Channel9 from the UK Future Decoded event back in 2015;

image

although I think on that particular day I had a failure with the part of the demo that did a bit of AllJoyn but I’ve shown that working in a few other places so it does generally work out.

Over time, I threw lots of things into this demo such that I had an app that worked with keyboard, mouse, touch, speech, pen/ink, XBOX controller and also could be driven off simple and rotary switches attached to a Raspberry PI.

I’ve also delivered versions of this demo where the UI;

  1. Talks directly to the Sphero over Bluetooth.
  2. Talks over AllJoyn to another app which then talks to the Sphero over Bluetooth.
  3. Talks over an Azure event hub to another app which then talks to the Sphero over Bluetooth.

and had some fun with showing those different kinds of combinations.

In more recent times, I stopped using the demo as I felt that I was becoming “the guy with the Sphero demo” and there wasn’t much more that I could realistically add into it Smile

I also hit a bit of a technical snag in that code which had been quite reliable for me stopped working on the Windows 10 Anniversary Edition and I hadn’t taken the time to quite figure out what had broken.

That said, I had reached out to Twitter and had this discussion with @roadz about the problem perhaps coming down to the underlying Robotkit.dll that sat underneath my code.

image

and he’d done the legwork to figure some of this out and had a version of Robotkit in source form over on github here that might work better than the one that I’d been using.

Today, I picked up that code and have managed to get my demo back working on top of it although I had to make a minor, surgical hack to the piece of code around lines 66 of RobotProvider.cs;

image

For me, I find that when I debug this code, the Sphero that is paired with my PC does not seem to show up named as “Sphero” and so I had to take out this condition to get the code to work. It might be one of those “just on my PC” scenarios but making that one code change seemed to make it possible for me to layer my own little library back on top of the Robotkit.dll and get my Sphero functionality back up and running as it had done prior to the Anniversary Update.

It’s worth saying that the Robotkit.dll that I had was originally built for Windows 8.0 and so I’d been (pleasantly) surprised that it had survived all the way to Windows 10 and including being used on non-PC devices so I wasn’t too disappointed when it finally ran out of steam on me Smile

Recreating (some of) that Sphero Demo

I wanted to run through the steps to recreate some of that original Sphero demo so as to leave a basic interface which could control the Sphero using keyboard, mouse, touch, speech. For the moment, I haven’t included the pieces that I have shown in the past around using XBOX controllers, ink, AllJoyn and IoT switches to control the device although they aren’t too hard to add back in.

I recorded a quick demo of putting these pieces back together below starting from the code that @roadz had added to GitHub.

and that (demo!) code then gets committed back to GitHub here as you’ll see if you watch the video.

The intention of putting that back together though was to see if I could get that same code working on HoloLens.

On HoloLens…

Around the time that I was showing that original demo, I would often say “this works on PC, Phone, IoT and would work on other devices like HoloLens” but I didn’t have a device back then to actually try it out and so I finally got to take the exact same code and deploy it to the HoloLens as the screen capture below shows;

and then ran that to see if it worked with my Sphero as this video captured from the HoloLens shows;

and it works pretty well running as a 2D app but, equally, I could take this Sphero code and package it into a 3D/holographic app and perhaps come up with a more natural way of interacting with the Sphero.

That’s for a future post – for now, it’s good to have this code back running and to have tried it out on another device Smile