It’s been a while since I played with my Sphero device and so I thought I’d see if I could get some code up and running on Windows 10 UWP to talk to it.
It turned out to be a little more of a ‘journey’ than I’d hoped for but, nonetheless it was fun 
I last did any work with Sphero in this blog post where I wrote a Windows 8.1 application in HTML/JS on top of a C# library which itself wrapped a Sphero control library into a custom WinRT component such that it could be used from JS.
I thought I’d revisit something similar in Windows 10 but use C# and XAML for everything rather than bring in HTML/JS.
I went to look for a Sphero control library and ended up back here;
where I have to admit that it’s a bit sad to see that the WinRT library for talking to Sphero;
is still in a 0.9.0-preview variant built specifically for Windows 8.1 and it doesn’t look like anything much has changed with that library for around 2 years.
Bummer. I’d hope that perhaps this will change in the light of Windows 10 but, equally, I guess that I have to accept that they Sphero guys aren’t exactly sitting around doing nothing;
but how likely is it that this old Windows 8.1 library will be useful to me as a Windows 10 developer?
Let’s see!
Technically, it should be possible to make use of it and that should be true across various device families and so I thought I’d put something together to see if I could take this Windows 8.1 library and use it in a UWP app on Windows 10.
Putting the Basics Together on the PC
I paired my Sphero with my PC over bluetooth and I then took the RobotKit library that Sphero has on github and I build a quick Sphero control service out of it;
namespace BigWhiteBall.Services
{
using RobotKit;
using System;
using System.Threading.Tasks;
class SpheroService : ISpheroService
{
public event EventHandler SpheroConnectionChanged;
public SpheroService()
{
}
public void Drive(float speed)
{
this.speed = speed;
this.Roll();
}
public void Rotate(int angle)
{
this.angle = angle;
this.Roll();
}
void Roll()
{
// NB: using roll for both rotate and drive as SetHeading doesn't
// seem to work for me.
this.sphero.Roll(this.angle, this.speed);
}
bool HasSphero
{
get
{
return (this.sphero != null);
}
}
public bool HasConnectedSphero
{
get
{
bool connected = (this.HasSphero &&
(this.sphero.ConnectionState == ConnectionState.Connected));
return (connected);
}
}
RobotProvider Provider
{
get
{
return (RobotProvider.GetSharedProvider());
}
}
bool HasProvider
{
get
{
return (this.Provider != null);
}
}
public async Task<bool> ConnectToFirstSpheroAsync(TimeSpan timeout)
{
// I'm not being correct with respect to the timeout here as
// worst case scenario both of my async functions below could
// wait for 'timeout' so it could be a factor of 2 out.
if (!this.HasConnectedSphero)
{
if (!this.HasSphero)
{
await this.DiscoverFirstRobotAsync(timeout);
}
if (this.HasSphero)
{
await this.ConnectDiscoveredSpheroAsync(timeout);
}
}
return (this.HasConnectedSphero);
}
async Task DiscoverFirstRobotAsync(TimeSpan timeout)
{
TaskCompletionSource<bool> completionSource =
new TaskCompletionSource<bool>();
if (this.HasProvider && !this.HasSphero)
{
EventHandler<Robot> robotsHandler = null;
robotsHandler = (s, robot) =>
{
this.sphero = robot as Sphero;
this.sphero.OnConnectionStateChanged =
this.OnSpheroConnectionStateChanged;
completionSource.SetResult(true);
};
this.Provider.DiscoveredRobotEvent += robotsHandler;
this.Provider.FindRobots();
await Task.WhenAny(completionSource.Task, Task.Delay(timeout));
this.Provider.DiscoveredRobotEvent -= robotsHandler;
}
}
void OnSpheroConnectionStateChanged(Robot sender, ConnectionState state)
{
var handlers = this.SpheroConnectionChanged;
if ((state == ConnectionState.Disconnected) ||
(state == ConnectionState.Connected))
{
handlers(this, EventArgs.Empty);
}
if (state == ConnectionState.Connected)
{
this.sphero.SetBackLED(1.0f);
}
}
async Task ConnectDiscoveredSpheroAsync(TimeSpan timeout)
{
TaskCompletionSource<bool> completionSource =
new TaskCompletionSource<bool>();
if (this.HasProvider && this.HasSphero &&
!this.HasConnectedSphero)
{
EventHandler<Robot> robotsHandler = null;
robotsHandler = (s, robot) =>
{
completionSource.SetResult(true);
};
this.Provider.ConnectedRobotEvent += robotsHandler;
this.Provider.ConnectRobot(this.sphero);
await Task.WhenAny(completionSource.Task, Task.Delay(timeout));
this.Provider.ConnectedRobotEvent -= robotsHandler;
}
}
float speed;
int angle;
Sphero sphero;
}
}
and I extracted an interface out of that such that it could be injected into my application where needed from an IoC container (I’m using Autofac);
namespace BigWhiteBall.Services
{
using System;
using System.Threading.Tasks;
interface ISpheroService
{
bool HasConnectedSphero { get; }
event EventHandler SpheroConnectionChanged;
Task<bool> ConnectToFirstSpheroAsync(TimeSpan timeout);
void Rotate(int angle);
void Drive(float speed);
}
}
with that in place I built out a couple of view models to support a couple of screens worth of the most basic interaction. A main view model which keeps track of whether we have a sphero or not;
namespace BigWhiteBall.ViewModels
{
using BigWhiteBall.Services;
class MainUIControlViewModel : ViewModelBase
{
public MainUIControlViewModel(ISpheroService spheroService)
{
this.spheroService = spheroService;
this.spheroService.SpheroConnectionChanged += OnSpheroConnectionStateChanged;
}
public bool HasSphero
{
get
{
return (this.spheroService.HasConnectedSphero);
}
}
void OnSpheroConnectionStateChanged(object sender, System.EventArgs e)
{
base.OnPropertyChanged("HasSphero");
}
ISpheroService spheroService;
}
}
a ‘we are trying to detect sphero’ view model;
namespace BigWhiteBall.ViewModels
{
using BigWhiteBall.Services;
using Utility;
using System;
using System.Windows.Input;
class LookingForSpheroControlViewModel : ViewModelBase
{
public LookingForSpheroControlViewModel(ISpheroService spheroService)
{
this.spheroService = spheroService;
this.lookForSpheroCommand = new AlwaysExecuteCommand(this.LookForSphero);
this.LookForSphero();
}
public ICommand LookForSpheroCommand
{
get
{
return (this.lookForSpheroCommand);
}
}
public bool IsLookingForSphero
{
get
{
return (this.isLookingForSphero);
}
set
{
base.SetProperty(ref this.isLookingForSphero, value);
}
}
async void LookForSphero()
{
this.IsLookingForSphero = true;
await this.spheroService.ConnectToFirstSpheroAsync(
TimeSpan.FromSeconds(10));
this.IsLookingForSphero = false;
}
ISpheroService spheroService;
ICommand lookForSpheroCommand;
bool isLookingForSphero;
}
}
and a view model for actually driving the Sphero – note that the SpeedValue here is intended to be bound up to a control that ranges from 0 to 100 whereas the inner workings of this code divides that down to a range between 0.0 and 1.0;
namespace BigWhiteBall.ViewModels
{
using System;
using BigWhiteBall.Services;
class DriveSpheroControlViewModel : ViewModelBase
{
public DriveSpheroControlViewModel(ISpheroService spheroService)
{
this.spheroService = spheroService;
}
public float SpeedValue
{
get
{
return (this.speedValue);
}
set
{
base.SetProperty(ref this.speedValue, value);
this.DriveSphero();
}
}
public double RotationValue
{
get
{
return (this.rotationValue);
}
set
{
base.SetProperty(ref this.rotationValue, value);
this.RotateSphero();
}
}
void DriveSphero()
{
this.spheroService.Drive(this.speedValue / MAX_SLIDER_VALUE);
}
void RotateSphero()
{
this.spheroService.Rotate((int)this.rotationValue);
}
float speedValue;
static readonly float MAX_SLIDER_VALUE = 100.0f;
double rotationValue;
ISpheroService spheroService;
}
}
with those in place, I made a Universal app that does not use any Frame or any Page types and does not do any navigation but, instead, just hosts 3 user controls. My startup code (which doesn’t yet handle suspend/resume) simply builds up an Autofac container ( populating a simple view model locator object that I’ve defined in my App.xaml ) and sets the Content of the Window to be a new instance of a MainUIControl;
namespace BigWhiteBall
{
using ViewModels;
using Windows.ApplicationModel.Activation;
using Windows.UI.Xaml;
using Autofac;
using System.ComponentModel;
using Services;
sealed partial class App : Application
{
public App()
{
this.InitializeComponent();
}
protected override void OnLaunched(LaunchActivatedEventArgs e)
{
if (Window.Current.Content == null)
{
this.Locator.Build(builder =>
{
builder
.RegisterType<MainUIControlViewModel>()
.Named<INotifyPropertyChanged>(Constants.Values.MainViewModelName)
.As<INotifyPropertyChanged>();
builder.RegisterType<LookingForSpheroControlViewModel>()
.Named<INotifyPropertyChanged>(Constants.Values.LookingForSpheroViewModelName)
.As<INotifyPropertyChanged>();
builder.RegisterType<DriveSpheroControlViewModel>()
.Named<INotifyPropertyChanged>(Constants.Values.DriveSpheroViewModelName)
.As<INotifyPropertyChanged>();
builder
.RegisterInstance<ISpheroService>(new SpheroService());
});
Window.Current.Content = new MainUIControl();
}
Window.Current.Activate();
}
ViewModelLocator Locator
{
get
{
return (this.Resources["locator"] as ViewModelLocator);
}
}
}
}
and the MainUIControl is very simple – it just toggles the visibility of 2 other controls – one which is used when ‘looking’ for a Sphero and one which is used to drive the Sphero;
<UserControl
x:Class="BigWhiteBall.MainUIControl"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:BigWhiteBall"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
d:DesignHeight="300"
d:DesignWidth="400"
DataContext="{Binding Source={StaticResource locator},Path={Binding Source={StaticResource constants}, Path=MainViewModelXamlPathIndexer}}">
<Grid>
<Grid
Visibility="{Binding HasSphero, Converter={StaticResource boolVisibilityConverter},ConverterParameter=true}">
<local:LookingForSpheroControl />
</Grid>
<Grid
Visibility="{Binding HasSphero, Converter={StaticResource boolVisibilityConverter}}">
<local:DriveSpheroControl HorizontalAlignment="Center" VerticalAlignment="Center"/>
</Grid>
</Grid>
</UserControl>
and those two other controls are just data-bound onto the underlying view models and present a UI that looks like this. Firstly, we’re looking for a Sphero;
and then we might fail to find it;
and I made this screen ‘somewhat’ responsive in that if it gets a little bit too narrow then it resizes a few things;
and I did that using the now, well-established UWP trick of using AdaptiveTriggers which you can read about here.
Once a Sphero has been found, a second control comes into life which is about driving the Sphero. I’m not sure yet that the UI is quite visible enough here but this is how it currently stands;
As an aside, I really love how Sphero has high quality brand materials on this site and I wish that more companies did it – it would stop those horrible presentations where you see someone display a big, pixelated company logo that they’ve had to take from the web.
That UI is essentially just a Slider that has been re-templated somewhat on the left and an instance of my RadialValueSelector control that I built in this post.
I wanted to make this UI a bit responsive on the desktop and so I had it adjust itself at the point where the layout becomes portrait rather than landscape. By that, I mean that the Window width is becomes less than or greater than its height respectively.
In order to do that I used my own, custom trigger derived from StateTriggerBase which I called AspectRatioTrigger. I initially looked into the WindowsStateTriggers package that’s on NuGet and which I’d highly recommend but I think that the closest trigger in that package is one that makes decisions based on the orientation of the device whereas I wanted to make a decision based on the ‘orientation’ of the Window. So, I knocked up a trigger;
namespace BigWhiteBall.Utility
{
using Windows.UI.Core;
using Windows.UI.Xaml;
enum AspectRatio
{
None,
Portrait,
Landscape
}
class AspectRatioTrigger : StateTriggerBase
{
public static readonly DependencyProperty AspectRatioProperty =
DependencyProperty.Register(
"AspectRatio", typeof(AspectRatio), typeof(AspectRatioTrigger),
new PropertyMetadata(AspectRatio.None, OnAspectRatioChanged));
public AspectRatioTrigger()
{
}
public AspectRatio AspectRatio
{
get
{
return (AspectRatio)base.GetValue(AspectRatioProperty);
}
set
{
base.SetValue(AspectRatioProperty, value);
}
}
static void OnAspectRatioChanged(DependencyObject sender,
DependencyPropertyChangedEventArgs args)
{
AspectRatioTrigger trigger = (AspectRatioTrigger)sender;
if (trigger.window == null)
{
trigger.window = Window.Current;
trigger.window.SizeChanged += trigger.OnSizeChanged;
}
}
void OnSizeChanged(object sender,
WindowSizeChangedEventArgs e)
{
base.SetActive(
((this.AspectRatio == AspectRatio.Landscape) && (e.Size.Width > e.Size.Height)) ||
((this.AspectRatio == AspectRatio.Portrait) && (e.Size.Width <= e.Size.Height)));
}
Window window;
}
}
In playing with this very simple UI I made use of RelativePanel in a single place which is to deal with the transition between this;
where I am using a RelativePanel to tell the text that it should be positioned to the right of the slider and vertically aligned with its centre and this;
where I’ve used the Visual State Manager to change the RelativePanel properties on that text such that it is now positioned above the slider and aligned with its right hand edge.
However, I spent some time trying to use a RelativePanel for another aspect of the layout here and I came a bit unstuck so I thought I’d share.
I’ve got a simple layout which is highlighted here on the Live Visual Tree view from Visual Studio. In essence, there’s a grid on the left;
and a grid on the right;
and what I want to achieve is to have a landscape layout where each grid gets all of the available height and half of the available width whereas in my portrait layout I want the top grid to get an ‘auto’ proportion of the height and the lower grid to fill as per below;
I’m fairly familiar with achieving this kind of thing using Grid and playing around with rows, columns, row positions, column positions and row spans and column spans to achieve what I want but I haven’t yet figured out how I get a RelativePanel to give me a 50:50 split layout.
I’m not sure whether RelativePanel allows for that kind of layout. It almost feels like it could do with a RelativePanel.AlignLeftWithPanelCentre type property but maybe I’m either missing something about that panel or perhaps I should just accept that for this type of layout it makes sense to continue using Grids in the way that I’ve always done.
Regardless, while I haven’t shared every line of code above I hope you’ve got the idea as to what I’ve put together from a desktop perspective to provide basic speed/rotation control of a Sphero (you can do a lot more with Sphero) and at this point I really wanted to see how well/badly what I’d got worked on a Windows 10 Phone.
Moving to the Phone
Up until this point, all the bits that I’ve played with for Windows 10 Mobile I’ve done in an emulator but I didn’t think that was quite going to cut it here and so I went and dug out an old Lumia 920 which has a broken proximity sensor (making it very hard to end a phone call!) and I set about installing the Windows 10 Preview on it.
I’d say that it took around 1 day to get to the point where I had build 10512 installed onto this device. I had a number of problems including;
- Running out of storage space.
- Having updates repeatedly fail to install.
- Getting stuck in a loop where my phone would reboot, show cogs, fail with an unhappy face and then repeat.
In the end, I had to resort to the Windows Phone Recovery Tool in order to re-flash the phone, start the update process over again (without restoring any backup to the phone) and get it updated to Windows 10.
Ultimately, it worked out but I’m glad it wasn’t my main phone that I was experimenting with!
What was nice, though, is that from the point where the OS was installed, it took around 30 seconds to switch on developer mode;
and then to pair the device with my Sphero (having switched Bluetooth on);
and then to rebuild my app for ARM in Visual Studio and deploy it down to the device.
There’s an interesting thing going on here – my app is making use of that RobotKit.dll library that was built for Windows 8.1 around 2 years ago and (because it’s .NET based) I’m now deploying that down to a Windows 10 Phone and Visual Studio does its thing and it all nicely works.
Nice, no?
The other thing I’d say here is that because I’d already given some consideration to how my app sizes itself for different screens based both on width and on portrait/landscape I had an IJW – It Just Worked moment here on the phone which really impressed me.
Here’s some screenshots (as an aside, I can’t seem to get the ‘Project My Screen’ app working on Windows 10 at the moment and so these are proper screenshots from the phone);
A quick splash;
Looking for the ball;
In landscape;
Not finding him;
Finding him (the toast notification here comes from down in the RobotKit.dll library and I’ve yet to find a way of suppressing it and it’s going to come back to bite me in a big way later in the post);
Actually driving the ball;
and in landscape;
and so it all just ‘kind of works’ just like it did on the desktop and I thought it’d be worth trying to capture that in some kind of ‘home video’ and so that’s what I’m embedding below – excuse the very home-grown camera work!
Ok, so that’s all working on both the Windows ‘PC’ and the Windows ‘Mobile’ but I have another device on my desk running Windows 10 and that’s the Raspberry PI 2.
Moving to Windows IoT Core
In terms of moving this to Windows IoT Core, I hit a really interesting scenario which I thought I’d share. Firstly, I powered up my Raspberry PI 2 and went to its web admin UI in order to pair the device with the Sphero;
Then I switched Visual Studio to deploy to the Raspberry PI as a remote machine and that all worked fine and the app ran up. I have a small LCD panel on my Raspberry PI and I can see the app running up here;
and all was looking very good indeed until I hit an unhandled exception that, frankly, I should have seen coming and that exception was as below;
I had to laugh really – the RobotKit.dll library that has survived so well so far in that it was built for Windows 8.1 and yet I’m running it on Windows 10 on PCs, Phones and now IoT devices finally gives up the ghost because it is trying to send a toast notification on a platform that does not support them and, ironically, I never wanted it to send those toast notifications in the first place!
At this point, I must admit that I went and disassembled the RobotKit.dll to see if there was some way of telling it not to send out toast notifications when I didn’t want it to and I managed to disassemble the RobotProvider.ConnectRobot method to;
and so it didn’t seem so ‘dangerous’ to me to attempt to handle the exception that is going to be thrown if this method calls its own toastConnect or toastFailConnect methods as that’s the last thing that the method does anyway.
However, that’s before I noticed that the developer here had written an async public method that returned void which left me scratching my head a little as to what I could possibly do to try and mitigate that.
I don’t know of any ‘easy’ way around this other than to try and mess with the SynchronizationContext that’s being used.
This article is a good read on this scenario and I borrowed the code for the class AsyncSynchronizationContext from that article with just a minor tweak to attempt to add a Replace() method that I can use for my scenario as I really just want to wrap this sync context around this one function call;
namespace BigWhiteBall.Utility
{
using System;
using System.Threading;
/// <summary>
/// Mostly taken from this article
/// http://www.markermetro.com/2013/01/technical/handling-unhandled-exceptions-with-asyncawait-on-windows-8-and-windows-phone-8/
/// </summary>
public class AsyncExceptionSwallowingContext : SynchronizationContext
{
private SynchronizationContext _syncContext;
public AsyncExceptionSwallowingContext()
{
}
public void Replace()
{
var syncContext = Current;
if (syncContext == null)
throw new InvalidOperationException(
"Ensure a synchronization context exists before calling this method.");
var customSynchronizationContext = syncContext as AsyncExceptionSwallowingContext;
if (customSynchronizationContext == null)
{
customSynchronizationContext = new AsyncExceptionSwallowingContext(syncContext);
SetSynchronizationContext(customSynchronizationContext);
}
else
{
SetSynchronizationContext(_syncContext);
}
}
public AsyncExceptionSwallowingContext(SynchronizationContext syncContext)
{
_syncContext = syncContext;
}
public override SynchronizationContext CreateCopy()
{
return new AsyncExceptionSwallowingContext(_syncContext.CreateCopy());
}
public override void OperationCompleted()
{
_syncContext.OperationCompleted();
}
public override void OperationStarted()
{
_syncContext.OperationStarted();
}
public override void Post(SendOrPostCallback d, object state)
{
_syncContext.Post(WrapCallback(d), state);
}
public override void Send(SendOrPostCallback d, object state)
{
_syncContext.Send(d, state);
}
private static SendOrPostCallback WrapCallback(SendOrPostCallback sendOrPostCallback)
{
return state =>
{
try
{
sendOrPostCallback(state);
}
catch (Exception ex)
{
}
};
}
}
}
With that class available to me, I can now wrap up my call to the mis-behaving RobotKitProvider.ConnectRobot as below;
AsyncExceptionSwallowingContext ctx = new AsyncExceptionSwallowingContext();
ctx.Replace();
this.Provider.ConnectRobot(this.sphero);
ctx.Replace();
and that seems to get me out of jail on this one although, naturally, I’d prefer not to have to do it.
I can then run the same code on my Raspberry PI as I’m running on both my PC and my Phone and it works fine, albeit, I’m manipulating the UI with a mouse here;
In Summary
This has been a long post – what has it covered so far?
- Taking a Windows 8.1 library and using it in a Windows 10 project.
- Making that Windows 10 project work across PC, Phone and Raspberry PI 2.
- Dealing with a few hiccups along the way!
I’ll follow up the post with another where I’ll see if I can control the Sphero from the PI without having to use the UI and a mouse – I’ll add some buttons onto a breadboard and see how that works out.
In the meantime, all the code for this post is here for download if you want to do something with it.
Mike Taulty 