Experiments with Shared Holographic Experiences and AllJoyn (Spoiler Alert: this one does not end well!)

Posted on June 29, 2017 by Mike Taulty

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

This post is an interesting one in that it represents two sides of a day or so of technical failure! I tried to get something up and running and it didn’t quite work out but I thought it was worth sharing regardless but apply a large pinch of salt as the result isn’t anything that works too well

Background – Catching up with AllJoyn

It’s been a little while since I looked at AllJoyn and the AllSeen Alliance.

In fact, long enough that the landscape has changed with the merger between the OCF (IoTivity) and the AllSeen Alliance (AllJoyn) with the result that (from the website);

Both projects will collaborate to support future versions of the OCF specification in a single IoTivity implementation that combines the best of both technologies into a unified solution. Current devices running on either AllJoyn or IoTivity solutions will be interoperable and backward-compatible. Companies already developing IoT solutions based on either technology can proceed with the confidence that their products will be compatible with the unified IoT standard that the industry has been asking for.

and so I guess that any use of AllJoyn at this point has to be seen against that backdrop.

Scenario – Using AllJoyn to Share Holograms Across Devices

With that said, I still have AllJoyn APIs in Windows 10 and I wanted to see whether I could use those as a basis for the sharing of Holograms across devices.

The idea would be that a HoloLens app becomes both an AllJoyn consumer and producer and so each device on a network could find all the other devices and then share data such that a common co-ordinate system could be established via world anchors and hologram creation/removal and positioning could also be shared.

If you’re not so familiar with this idea of shared holographic experiences then the official documentation lives here;

https://developer.microsoft.com/en-us/windows/mixed-reality/development

and I’ve written a number of posts around this area of which this one would be the most recent;

Hitchhiking the HoloToolkit-Unity, Leg 13–Continuing with Shared Experiences

My idea for AllJoyn was to avoid any central server and let it do the work of having devices discover each other and connect so that they could form a network where they consume each others’ services as below;

Now, I don’t think that I’d tried this previously but there was something nagging at the back of my mind about the AllJoyn APIs not being something that I could use in the way that I wanted to on HoloLens and I should really have taken heed.

Instead, I ploughed ahead…

Making a “Hello World” AllJoyn Interface

I figured that the “Hello World” here might involve a HoloLens app offering an interface whereby a remote caller could create some object (e.g. a cube) at a particular location in world space.

That seemed like a simple enough thing to figure out and so I sketched out a quick interface for AllJoyn to do something like that;

<interface name=”com.taulty.RemoteHolograms”>
   <method name=”CreateCube”>
     <arg name=”x” type=”d” direction=”in” />
     <arg name=”y” type=”d” direction=”in” />
     <arg name=”z” type=”d” direction=”in” />
   </method>
</interface>

I then struggled a little with respect to knowing where to go next in that the tool alljoyncodegen.exe which I used to use to generate UWP code from an AllJoyn interface seemed to have disappeared from the SDKs.

I found this doc page which suggested that the tool was deprecated and for a while I landed on this page which sounded similar but turned out have only been tested on Linux and not to generate UWP code so was actually very different

I gave up on the command line tool and went off to try and find AllJoyn Studio which does seem to still exist but only for Visual Studio 2015 which was ‘kind of ok’ because I still have VS2015 on my system alongside VS2017.

Whether all these changes are because of the AllJoyn/IoTivity merging, I’ve no idea but it certainly foxed me for a little while.

Regardless, I fed AllJoyn Studio in Visual Studio 2015 my little XML interface definition;

and it spat out some C++/CX code providing me a bunch of boiler plate AllJoyn implementation code which I retargeted to platform version 14393 as the tool seemed to favour 10586;

Writing Some Code for Unity to Consume

At this point, I was a little over-zealous in that I thought that the next step would be to try and write a library which would make it easy for Unity to make use of the code that I’d just had generated.

So, I went off and made a C# library project that referenced the newly generated code and I wrote this code to sit on top of it although I never really got around to figuring out whether this code worked or not as we’ll see in a moment;

namespace AllJoynHoloServer
{
  using com.taulty.RemoteHolograms;
  using System;
  using System.Threading.Tasks;
  using Windows.Devices.AllJoyn;
  using Windows.Foundation;

  public class AllJoynCreateCubeEventArgs : EventArgs
  {
    internal AllJoynCreateCubeEventArgs()
    {
        
    }
    public double X { get; internal set; }
    public double Y { get; internal set; }
    public double Z { get; internal set; }
  }

  public class ServiceDispatcher : IRemoteHologramsService
  {
    public ServiceDispatcher(Action<double, double, double> handler)
    {
      this.handler = handler;
    }
    public IAsyncOperation<RemoteHologramsCreateCubeResult> CreateCubeAsync(
      AllJoynMessageInfo info,
      double x,
      double y,
      double z)
    {
      return (this.CreateCubeAsyncInternal(x, y, z).AsAsyncOperation());
    }
    async Task<RemoteHologramsCreateCubeResult> CreateCubeAsyncInternal(
      double x, double y, double z)
    {
      // Likelihood that the thread we're calling from here is going to
      // break Unity's threading model.
      this.handler?.Invoke(x, y, z);

      return (RemoteHologramsCreateCubeResult.CreateSuccessResult());
    }
    Action<double, double, double> handler;
  }
  public static class AllJoynEventAdvertiser
  {
    public static event EventHandler<AllJoynCreateCubeEventArgs> CubeCreated;

    public static void Start()
    {
      if (busAttachment == null)
      {
        busAttachment = new AllJoynBusAttachment();
        busAttachment.AboutData.DateOfManufacture = DateTime.Now;
        busAttachment.AboutData.DefaultAppName = "Remote Holograms";
        busAttachment.AboutData.DefaultDescription = "Creation and Manipulation of Holograms";
        busAttachment.AboutData.DefaultManufacturer = "Mike Taulty";
        busAttachment.AboutData.ModelNumber = "Number One";
        busAttachment.AboutData.SoftwareVersion = "1.0";
        busAttachment.AboutData.SupportUrl = new Uri("http://www.mtaulty.com");

        producer = new RemoteHologramsProducer(busAttachment);
        producer.Service = new ServiceDispatcher(OnCreateCube);
        producer.Start();
      }
    }
    public static void Stop()
    {
      producer?.Stop();
      producer?.Dispose();
      busAttachment?.Disconnect();
      producer = null;
      busAttachment = null;
    }
    static void OnCreateCube(double x, double y, double z)
    {
      CubeCreated?.Invoke(
        EventArgs.Empty,
        new AllJoynCreateCubeEventArgs()
        {
          X = x,
          Y = y,
          Z = z
        });
    }
    static AllJoynBusAttachment busAttachment;
    static RemoteHologramsProducer producer;
  }
}

There’s nothing particularly exciting or new here, it’s just a static class which hides the underlying AllJoyn code from anything above it and it offers the IRemoteHologramsService over the network and fires a static event whenever some caller remotely invokes the one method on that interface.

I thought that this would be pretty easy for Unity to consume and so I dragged the DLLs into Unity (as per this post) and then added the script below to a blank GameObject to run when the app started up;

#if UNITY_UWP && !UNITY_EDITOR
using AllJoynHoloServer;
#endif
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Startup : MonoBehaviour
{
  // Use this for initialization
  void Start()
  {
#if UNITY_UWP && !UNITY_EDITOR
    AllJoynEventAdvertiser.CubeCreated += this.OnCubeCreated;
    AllJoynEventAdvertiser.Start();
#endif
  }
  #if UNITY_UWP && !UNITY_EDITOR
  void OnCubeCreated(object sender, AllJoynCreateCubeEventArgs args)
  {

  }
  
#endif

}

Clearly, this isn’t fully formed or entirely thought through but I wanted to just see if I could get something up and running and so I tried to debug this code having, first, made sure that my Unity app was asking for the AllJoyn security capability.

Debugging the AllJoyn Experience

I used the IoT AllJoyn Explorer to try and see if my Unity app from the HoloLens was advertising itself correctly on the local network.

That app still comes from the Windows Store and I’ve used it before and it’s always been pretty good for me.

It took me a little while to remember that I need to think about loopback exemption when it comes to this app so that’s worth flagging here.

I found that when I ran my Unity code on the HoloLens, I didn’t seem to see the service being advertised on the AllJoyn network as displayed by the IoT Explorer. I only ended up seeing a blank screen;

in order to sanity check this, I ended up lifting the code that I had built into Unity out of that environment and into a 2D XAML experience to run on my local PC where things lit up as I’d expect – i.e. IoT Explorer then shows;

and so the code seemed to be ok and, at this point, I realised that I could have tested out the whole concept in 2D XAML and never dropped into Unity at all – there’s a lesson in there somewhere! Smile

Having proved the point on my PC, I also ran the same code on my phone and saw a similar result – the service showed up on the network.

However, no matter how I went about it I couldn’t get HoloLens to advertise this AllJoyn service and so I have to think that perhaps that part of AllJoyn isn’t present on HoloLens today.

That doesn’t surprise me too much and I’ve tried to confirm it and will update this post if I get a confirmation either way.

If this is the case though, what might be done to achieve my original aim which was to use AllJoyn as the basis of a shared holographic experience across devices.

I decided that there was more than one way to achieve this…

HoloLens as AllJoyn Consumer, not Producer

It wasn’t what I’d originally had in mind but I figured I could change my scenario such that the HoloLens did not offer an AllJoyn service to be consumed but, instead, consumed an AllJoyn service offered by a device (like a PC or Phone) which could offer a service onto the network. The diagram then becomes…

and so there’s a need for an extra player here (the PC) and also a need for using events or signals in AllJoyn to inform devices when something has happened on ‘the server’ that they need to know about such as a world anchor being created and uploaded or a new hologram being created.

A New Interface with Signals

I figured that I was onto a winner and so set about implementing this. Firstly, I conjured up a basic interface to try and model the code scenarios of;

Devices join/leave the network and other devices might want to know how many devices they are in a shared experience with.
World anchors get uploaded to the network and other devices might want to import them in order to add a common co-ordinate system.
Holograms get added by a user (I’m assuming that they would be added as a child of a world anchor and so there would be an association there).
Holograms get removed by a user.

In addition, I’d also want to add the ability for the transforms (scale, rotate, translate) of a hologram to be changed but I left that to one side while trying to see if I could get these bits to work.

With this in mind, I created a new AllJoyn interface as below;

AllJoyn Interface on Github

and once again I should perhaps have realised that things weren’t going too well here.

Everything about that interface is probably self-explanatory except a couple of small things;

I used GUIDs to identify world anchors and holograms.
I’m assuming that types of holograms can be represented by simple strings – e.g. device 1 may create a hologram of an Xmas tree and I’m assuming that device 2 will get a string “Xmas tree” and know what to do with it.
I wanted to have a single method which returns the [Id/Name/Type/Position] of a hologram but I found that this broke the code generation tool hence my methods GetHologramIdsAndNames and GetHologramTransforms – these should really be one method.

and a larger thing;

My original method for AddWorldAnchor simply took a byte array but I later discovered that AllJoyn seems to have a maximum message size of 128K whereas world anchors can be megabytes and so I added a capability here to “chunk” the world anchor into pieces and that affected this method and also the GetWorldAnchor method.

I should have stopped at this point as this limitation of 128K was flashing a warning light but I ignored it and pressed ahead.

A ‘Server’ App to Implement the Interface

Having generated the code from that new interface (highlighted blue below) I went ahead and generated a UWP app which could act as the ‘server’ (or producer) on the PC (highlighted orange below);

That involved implementing the generated IAJHoloServerService interface which I did as below;

AJHoloService implementation on Github

and then I can build this up (with the UI XAML which isn’t really worth listing) and have it run on my desktop;

waiting for connections to come in.

A Client Library to make it ‘Easier’ from Unity

I also wanted to have a small client library my life easier in the Unity environment in terms of consuming this service and so I added a 3rd UWP project to my solution;

and added a static class which gathered up the various bits of code needed to get the AllJoyn pieces working;

AJHoloServerConnection Implementation on Github

and that relied on this simple callback interface in order to call back into any user of the class which would be my Unity code;

Callback interface on Github

Consuming from Unity

At this point, I should have really tested what it was like to consume this code from a 2D app as I’d have learned a lot while expending little effort but I didn’t do that. Instead, I went and built a largely empty scene in Unity;

Where the Parent object is an empty GameObject and the UITextPrefab is straight from the HoloToolkit-Unity with a couple of voice keywords attached to it via the toolkit’s Keyword Manager;

I made sure my 2 DLLs (the AllJoyn generated DLL plus my client library) were present in a Plugins folder;

with appropriate settings on them;

and made sure that my project was requesting the AllJoyn capability. At this point, I realised that I needed to have some capability where I could relatively easily import/export world anchors in Unity without getting into a lot of code with Update() loops and state machines.

Towards that end, I wrote a couple of simple classes which I think function ok outside of the Unity editor in the UWP world but which wouldn’t work in the editor. I wrote a class to help with exporting world anchors;

WorldAnchorExporter implementation on Github

and a class to help with importing world anchors;

WorldAnchorImporter implementation on Github

and they rely on this a class which tries to do a simplistic bridge between the Update() oriented loop world of Unity and the async world of UWP as I wanted to be able to write async/await code which ‘waited’ for the isLocated flag on a WorldAnchor to be set true;

GameObjectUpdateWatcher implementation on Github

With that all done, I could attach a script to my empty GameObject to form the basis of my logic stringing together the handling of the voice keywords with the import/export of the world anchors, creation of a single, simple type of hologram (a cube) and calls to the AllJoyn service on the PC;

Startup script on Github

That took quite a bit of effort, but was it worth it? Possibly not…

Testing – AllJoyn and Large Buffers

In testing, things initially looked good. I’d run the consumer app on the PC, the client app on the HoloLens and I’d see it connect;

and disconnect when I shut it down and my voice command of “lock” seemed to go through the motions of creating a world anchor and it was quickly exported from the device and then it started to transfer over the network.

And it transferred…

And it took a long time…minutes…

I spent quite some time debugging this. My first investigation was to look at the network performance graph from the HoloLens and it looked like this while transferring a 2-3MB world anchor across the network in chunks (64KB) using the AllJoyn interface that I’d written;

It’s the classic sawtooth and I moved my same client code onto both a Phone and a PC to see if it showed the same behaviour there and, to some extent, it did although it wasn’t as pronounced.

I played quite a bit with the size of the chunks that were being sent over the network and I could see that the gaps between the bursts of traffic seemed to be directly related to the size of the buffer and some native debugging combined with a quick experiment with the Visual Studio profiler pointed to this function in the generated code as being the cause of all my troubles;

and in so far as I can tell this runs some relatively expensive conversion function on every element in the array which burns up a bunch of CPU cycles on the HoloLens prior to the transmission of the data over the network. This seemed to slow down the world anchor transfers dramatically – I would find that it might take minutes for a world anchor to transfer which, clearly, isn’t very practical.

Putting the terrible performance that I’ve created to one side, I hit another problem…

Testing – Unexpected Method Signatures

Once I’d managed to be patient enough to upload a world anchor to my server, I found a particular problem testing out the method which downloads it back to another device. It’s AllJoyn signature in the interface looks like this;

<method name=”GetWorldAnchor”>

    <arg name=”anchorId” type=”s” direction=”in”/>

    <arg name=”byteIndex” type=”u” direction=”in”/>

    <arg name=”byteLength” type=”u” directon=”in”/>

    <arg name=”anchorData” type=”ay” direction=”out”/>

</method>

and I found that when I called this at runtime from the client then my method’s implementation code on the server side wasn’t getting hit. All I was seeing was a client-side AllJoyn error;

ER_BUS_UNEXPECTED_SIGNATURE = 0x9015

now my method has 3 incoming parameters and a return value but it was curious to me that if I used the IoT Explorer on that interface it was showing up differently;

and so IoT Explorer was showing my method as having 2 inward parameters and 2 outgoing parameters or return values which isn’t what the interface specification actually says Confused smile

I wondered whether this was a bug in IoT Explorer or a bug in the runtime pieces and, through debugging the native code, I could use the debugger to cause the code to send 2 parameters rather than 3 and I could see that if I did this then the method call would cross the network and fail on the server side so it seemed that the IoT Explorer was right and my function expected 2 inward parameters and 2 outward parameters.

What wasn’t clear was…why and how to fix?

I spent about an hour before I realised that this problem came down to a type in the XML where the word “direction” had been turned into “directon” and that proved to be causing my problem – there’s a lesson in there somewhere too as the code generation tool didn’t seem to tell me anything about it and just defaulted the parameter to be outgoing.

With that simple typo fixed, I could get my code up and running properly for the first time.

Wrapping Up

Once I’d got past these stumbling blocks, the code as I have it actually seems to work in that I can run through the steps of;

Run the server.
Run the app on my HoloLens.
See the server display that the app has connected.
Use the voice command “lock” to create a world anchor, export it and upload it to the server taking a very long time.
Use the voice command “cube” in a few places to create cubes.
Shutdown the app and repeat 2, 3 above to see the device connect, download the anchor (taking a very long time), import it and recreate the cubes from step 5 where they were previously.

and I’ve placed the source for what I built up here;

https://github.com/mtaulty/AllJoynHoloSharing

but, because of the bad performance on copying the world anchor buffers around, I don’t think this would be a great starting point for implementing a shared holographic server and I’d go back to using the regular pieces from the HoloToolkit rather than trying to take this forward.

That said, I learned some things in playing with it and there’s probably some code in here that I’ll re-use in the future so it was worth while.

Windows 10 IoT Core–More Baby Steps with AllJoyn

Posted on June 15, 2015 by Mike Taulty

Following up on my previous post, I thought it would be ‘good’ to experiment with how I could differentiate between multiple running instances of my amazing ‘lightbulb’ service because it doesn’t seem unreasonable that I might have more than one instance of this service on my network or, even, different implementations of the service.

Initially, I wondered whether I could do this by publishing some kind of ‘location name’ via AllJoyn’s metadata and, at the time of writing, I’m not sure. I had a look at the AboutData that I can use to advertise my service;

https://allseenalliance.org/developers/learn/core/about-announcement/interface#about-data-interface-fields

but none of those fields felt like the “right” place to put an instance specific identifier for my service and so I changed my interface such that it could return a “location” property.

I’m not likely to win any awards for this interface but here’s the new version. I should have called the method GetLocation but I didn’t;

<node name="/com/taulty/lightbulb">
  <interface name="com.taulty.lightbulb">
    <method name="switch">
	<arg name="on" type="b" direction="in"/>
    </method>
    <method name="location">
	<arg name="returnvalue" type="s" direction="out"/>
    </method>
  </interface>
</node>

I rebuilt the generated code using the alljoyncodegen.exe tool and then I altered the universal app that I’d written to provide an implementation of this lightbulb interface on Windows IoT Core on Raspberry PI 2.

My intention was to make it such that this app could be run both on the Raspberry PI 2 where it would switch on a real LED but also on my laptop where the implementation would just toggle something visual on the screen in response to switching the lightbulb on/off.

That’s easy enough to do. I altered the XAML for this universal app to just have 2 simple grids;

<UserControl
    x:Class="LightbulbService.MainControl"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:LightbulbService"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="300"
    d:DesignWidth="400">

  <Grid Padding="40">
    <Grid x:Name="iotGrid" Visibility="Collapsed">
      <Viewbox>
        <StackPanel>
          <TextBlock Text="started advertising as lounge location" TextAlignment="Center"/>
          <TextBlock Text="LED will light up when request is received" TextAlignment="Center"/>
        </StackPanel>
      </Viewbox>
    </Grid>
    <Grid x:Name="noniotGrid" Visibility="Collapsed">
      <Viewbox>
        <StackPanel>
          <TextBlock Text="started advertising as kitchen location" TextAlignment="Center"/>
          <TextBlock Text="background will change colour when request is received" TextAlignment="Center"/>
        </StackPanel>
      </Viewbox>
    </Grid>
  </Grid>
</UserControl>

And then I tweaked the code behind to toggle behaviour between “running in IoT mode” and “running in desktop mode” as;

namespace LightbulbService
{
  using Windows.Devices.AllJoyn;
  using Windows.UI.Xaml;
  using Windows.UI.Xaml.Controls;
  using com.taulty;
  using System;
  using Windows.Foundation;
  using System.Threading.Tasks;
  using Windows.UI.Core;
  using Windows.UI.Xaml.Media;
  using Windows.UI;

  public sealed partial class MainControl : UserControl, IlightbulbService
  {
    public MainControl()
    {
      this.InitializeComponent();
      this.Loaded += OnLoaded;
    }
    private void OnLoaded(object sender, RoutedEventArgs e)
    {
      // NB: I was hoping that the first part of this IF statement would return true on
      // Windows IoT Core but it seems not to and so I've added the OR clause which I
      // didn't really want to.
      Grid grid = this.noniotGrid;
      IlightbulbService service = this;

      if (Windows.Foundation.Metadata.ApiInformation.IsApiContractPresent(
        "Windows.Devices.DevicesLowLevelContract", 1) ||
        Windows.Foundation.Metadata.ApiInformation.IsTypePresent(
        "Windows.Devices.Gpio.GpioController"))
      {
        grid = this.iotGrid;
        service = new GpioLightbulbService(GPIO_LED_PIN, PI_LOCATION);
      }
      grid.Visibility = Visibility.Visible;

      AllJoynBusAttachment busAttachment = new AllJoynBusAttachment();
      busAttachment.AuthenticationMechanisms.Add(AllJoynAuthenticationMechanism.SrpAnonymous);
      lightbulbProducer producer = new lightbulbProducer(busAttachment);
      producer.Service = service;
      producer.Start();
    }
    public IAsyncOperation<lightbulbLocationResult> LocationAsync(AllJoynMessageInfo info)
    {
      return (LocationAsync().AsAsyncOperation());
    }
    static async Task<lightbulbLocationResult> LocationAsync()
    {
      return (lightbulbLocationResult.CreateSuccessResult(THIS_LOCATION));
    }
    public IAsyncOperation<lightbulbSwitchResult> SwitchAsync(AllJoynMessageInfo info, bool interface_on)
    {
      return (SwitchAsync(interface_on).AsAsyncOperation());
    }
    async Task<lightbulbSwitchResult> SwitchAsync(bool on)
    {
      await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
        () =>
        {
          noniotGrid.Background = new SolidColorBrush(on ? Colors.Green : Colors.Red);
        }
      );
      return (lightbulbSwitchResult.CreateSuccessResult());
    }
    static readonly int GPIO_LED_PIN = 5;
    static readonly string THIS_LOCATION = "kitchen";
    static readonly string PI_LOCATION = "lounge";

  }
}

So, this code either advertises the MainPage class itself as an implementation of a lightbulb service where on/off values simply toggle the background colour between red/green or if it’s running on Windows IoT Core it’ll advertise this GpioLightbulbService as the implementation which toggles values on GPIO pin 5;

namespace LightbulbService
{
  using com.taulty;
  using System;
  using System.Threading.Tasks;
  using Windows.Devices.AllJoyn;
  using Windows.Devices.Gpio;
  using Windows.Foundation;

  class GpioLightbulbService : IlightbulbService
  {
    public GpioLightbulbService(int gpioPin, string locationName)
    {
      this.gpioPin = gpioPin;
      this.locationName = locationName;

      gpioController = new Lazy<GpioController>(() =>
      {
        return (GpioController.GetDefault());
      });
      ledPin = new Lazy<GpioPin>(() =>
      {
        var pin = gpioController.Value.OpenPin(this.gpioPin);
        pin.SetDriveMode(GpioPinDriveMode.Output);
        return (pin);
      });
    }
    public IAsyncOperation<lightbulbSwitchResult> SwitchAsync(AllJoynMessageInfo info, 
      bool switchOn)
    {
      return (this.SwitchAsync(switchOn).AsAsyncOperation());
    }
    async Task<lightbulbSwitchResult> SwitchAsync(bool switchOn)
    {
      ledPin.Value.Write(switchOn ? GpioPinValue.Low : GpioPinValue.High);
      return (lightbulbSwitchResult.CreateSuccessResult());
    }
    public IAsyncOperation<lightbulbLocationResult> LocationAsync(AllJoynMessageInfo info)
    {
      return (this.LocationAsync().AsAsyncOperation());
    }
    async Task<lightbulbLocationResult> LocationAsync()
    {
      return (lightbulbLocationResult.CreateSuccessResult(this.locationName));
    }
    int gpioPin;
    string locationName;
    Lazy<GpioController> gpioController;
    Lazy<GpioPin> ledPin;
  }
}

Which is just a minor re-working of the code from the previous blog post.

With that in play, I can run up both of these implementations at the same time with one running on my local laptop and the other on my Raspberry PI 2 respectively and I can then use the AllJoyn Explorer to see them.

NB: I used the CheckNetIsolation tool to ensure that (even outside of VS) I had enabled loopback networking for the relevant apps here as I mentioned in the previous blog post;

Here’s the AllJoyn Explorer seeing my 2 implementations;

Where the rid circle is the implementation coming from the Raspberry PI 2 and the yellow highlight is the local PC and both are running the same app from the code above.

I can then use the explorer to ask for the location of the local service;

And that returns;

And then I can invoke the light on method and the background goes green on my ‘fake’ lightbulb;

And if I use the Raspberry PI implementation, my LED goes on and off as it did previously.

With that in place, I can try and alter my client such that it picks up this new location value and displays it in the UI which is such a minor change to the previous client code that I won’t paste it here.

With that done though, I find that my client app only displays a single implementation of the ‘lightbulb’ service as in the screenshot;

In digging in to this in the debugger, I find that my code which is running in response to the Added handler on my lightbulbWatcher instance is firing twice;

and the first time that code runs, it makes a call to lightbulbConsumer.JoinSessionAsync and that returns a 0 success value.

However, when the second implementation of the service comes along and announces itself, I find that the call to JoinSessionAsync returns a value of 36928 which doesn’t seem to correspond to any of the error values that I could find.

At the time of writing then I can’t use this mechanism to discover 2 instances of my lightbulb service on the network and yet the AllJoyn Explorer is doing just that which makes me wonder whether there’s some other way that I can discover my service which might work?

Firstly, I discovered at least one way to get to the AboutData that my service is publishing – by making a call to AllJoynAboutDataView.GetDataBySessionPortAsync at the time when my service gets Added to my watcher I can get the about data detail;

I also experimented with watching my AllJoynBusAttachment fire StateChanged events to watch it go through its states of Connecting, Connected and so on.

Beyond that, my initial explore around the API gave me the (possibly wrong) impression that it’s specifically set up to work at a higher level while delegating down the harder work to the underlying generated code which I think then sits on top of the C API. That is – it doesn’t feel like I could maybe write the AllJoyn Explorer using this API?

One advantage though is that at least those calls into the C API are in front of me – they’re in the generated code and I could see via native debugging that a call to alljoyn_busattachment_registerbusobject was failing when my code was trying to join a session with the second implementation of the service that it found.

What that didn’t tell me was why it was failing or what the error code might mean but it did lead me to status.h from AllJoyn which told me that 36928 (or 0x9040) meant;

ER_BUS_OBJ_ALREADY_EXISTS = 0x9040

/**< Attempt to add a RemoteObject child that already exists

which was helpful because, previously, I’d been staring at the WinRT projections of these errors and trying to figure out why this one wasn’t listed anywhere.

This made me wonder – maybe I’m only supposed to have a session with one of these objects at a time in this model?

I re-worked the client side code along those lines such that I only attempt to join a session with one lightbulb at a time when I’m actually asking it to do something on my behalf such as tell me its location or switch on/off and that seemed to work quite a bit better.

Consequently, I changed my XAML a little for the light switching app – I won’t paste the whole thing here, I just changed the data template for the list;

<DataTemplate>
          <StackPanel Orientation="Horizontal">
            <TextBlock Text="{Binding ServiceInfo.UniqueName}"/>
            <TextBlock Margin="2,0,2,0" Text="in"></TextBlock>
            <TextBlock Text="{Binding Location}"/>
            <TextBlock Margin="2,0,2,0" Text="made by"></TextBlock>
            <TextBlock Margin="2,0,2,0" Text="{Binding AboutData.Manufacturer}" />
          </StackPanel>
        </DataTemplate>

And I then changed the code behind a little;

namespace lightswitch
{
  using com.taulty;
  using System;
  using System.Collections.ObjectModel;
  using System.ComponentModel;
  using System.Linq;
  using Windows.Devices.AllJoyn;
  using Windows.UI.Core;
  using Windows.UI.Xaml;
  using Windows.UI.Xaml.Controls;

  public sealed partial class MainPage : Page, INotifyPropertyChanged
  {
    public event PropertyChangedEventHandler PropertyChanged;

    public class LightbulbEntry
    {
      public AllJoynAboutDataView AboutData { get; set; }
      public AllJoynServiceInfo ServiceInfo { get; set; }
      public string Location { get; set; }
    }
    public MainPage()
    {
      this.InitializeComponent();
      this.Loaded += OnLoaded;
      this.Lightbulbs = new ObservableCollection<LightbulbEntry>();
    }
    void OnLoaded(object sender, Windows.UI.Xaml.RoutedEventArgs e)
    {
      this.DataContext = this;

      this.StartDiscoveringLightbulbs();
    }
    void StartDiscoveringLightbulbs()
    {
      this.busAttachment = new AllJoynBusAttachment();

      this.busAttachment.AuthenticationMechanisms.Add(AllJoynAuthenticationMechanism.SrpAnonymous);
     
      this.lightbulbWatcher = new lightbulbWatcher(this.busAttachment);

      this.lightbulbWatcher.Added += OnAdded;

      this.lightbulbWatcher.Start();
    }
    async void OnAdded(lightbulbWatcher sender, AllJoynServiceInfo args)
    {
      await this.Dispatcher.RunAsync(
        CoreDispatcherPriority.Normal,
        async () =>
        {
          var aboutData = await AllJoynAboutDataView.GetDataBySessionPortAsync(
            args.UniqueName, this.busAttachment, args.SessionPort);

          string location = string.Empty;

          var session = await lightbulbConsumer.JoinSessionAsync(args, sender);

          if (session.Status == AllJoynStatus.Ok)
          {
            var locationResult = await session.Consumer.LocationAsync();

            if (locationResult.Status == AllJoynStatus.Ok)
            {
              location = locationResult.Returnvalue;
            }
            session.Consumer.Dispose();
          }

          this.Lightbulbs.Add(
            new LightbulbEntry()
            {
              Location = location,
              AboutData = aboutData,
              ServiceInfo = args
            }
          );
        }
      );
    }
    public LightbulbEntry SelectedLightbulb
    {
      get
      {
        return (this.selectedLightbulb);
      }
      set
      {
        if (this.selectedLightbulb != value)
        {
          this.selectedLightbulb = value;
          this.RaisePropertyChanged("SelectedLightbulb");
        }
      }
    }
    void RaisePropertyChanged(string property)
    {
      var changeWatchers = this.PropertyChanged;
      if (changeWatchers != null)
      {
        changeWatchers(this, new PropertyChangedEventArgs(property));
      }
    }
    async void OnLightswitchToggled(object sender, RoutedEventArgs e)
    {
      if (this.SelectedLightbulb != null)
      {
        var sessionResult = await lightbulbConsumer.JoinSessionAsync(
          this.selectedLightbulb.ServiceInfo, this.lightbulbWatcher);

        if (sessionResult.Status == AllJoynStatus.Ok)
        {
          bool value = ((ToggleSwitch)sender).IsOn;
          await sessionResult.Consumer.SwitchAsync(value);

          sessionResult.Consumer.Dispose();
        }
      }
    }
    public ObservableCollection<LightbulbEntry> Lightbulbs { get; private set; }
    AllJoynBusAttachment busAttachment;
    lightbulbWatcher lightbulbWatcher;
    LightbulbEntry selectedLightbulb;
  }
}

And, sure enough, whether I have things correct or not I can get the UI to show multiple ‘lightbulbs’ and I can switch them on and off in this way. Here’s a static screenshot of the ‘light switch’ app running alongside the local ‘light bulb’ app and switching it on;

As I’m sure you can tell from the blog post, this is all very ‘rough and ready’ work on my part but I thought I may as well share experiments in case they help anyone else out.

Mike Taulty

I do some developer stuff on Microsoft technologies in the UK

Category Archives: AllJoyn