Mike Taulty's Blog
Bits and Bytes from Microsoft UK
I Just Want to Be Close To You– Initial Experiments with Proximity

Blogs

Mike Taulty's Blog

Elsewhere

Archives

Back in September, 2011 I went out to the BUILD conference where Microsoft first talked about Windows 8. At that conference (as I’m sure you know) the attendees got a Samsung Slate prototype device which is close to the Samsung ATIV Smart PC Pro.

Being an employee, my slate turned up in a box to the UK around 2 months later (hey, not that I’m complaining) and, for a number of reasons, it’s been my main PC ever since.

One of the things that ‘excited’ me about this device when I first got it was that it had an NFC chip in it. NFC feels to me like one of those Star Trek like technologies that wander into the real world from time to time and I was keen to try it out and especially from the point of view of being able to write code against it using the new ‘Proximity APIs’ that live in WinRT.

Now, at the time the only problem was that I didn’t have any other device which had NFC in it to test against. I mailed the folks on the product team to ask for advice about what devices I might try out with but, frankly, like a lot of Microsoft product teams they didn’t get back to me as I’m just a lowly employee which makes it easy to ignore my pleas for help Smile

So, my NFC dreams withered for a while. It’s a sad story but I’ve been able to move on with my life and think about other things until just the other week…

Arrival of the Nokia Lumia 920

The other week I managed to replace my longstanding HTC Mozart phone (yes, I really was still using this historical artefact) with a nice new, shiny Nokia Lumia 920. It’s a thing of beauty and I particularly like that I can now take photos on the phone that are recognisable rather than something that looks like an impressionist print Winking smile

The 920 has an NFC chip built into it so, finally, I have another device on my desk where I can try out doing ‘some NFC’ from one device to another and I even have heterogeneous devices to try it out with.

The Built In Experience

The first thing I did was to try out the built-in experience of sharing from the Phone to the Windows 8 machine.

I grabbed a picture on my phone from the photos album;

wp_ss_20130208_0002

and then I can choose to share that;

wp_ss_20130208_0003

and one of those options is via ‘Tap N Send’;

wp_ss_20130208_0004

so I complete that;

wp_ss_20130208_0005

and then wiggle the phone around near the back of the Windows 8 machine;

image

and if I tap to receive that content;

image

 

then what happens depends on the ‘Tap N Send’ settings;

image

and by default I get a choice;

image

and can import into the photos app in this particular case.

image

That’s all nice – how would my own app on the phone enable this kind of functionality?

Proximity APIs on the Phone

There’s a number of different capabilities that the proximity APIs on the phone enable for you with the key ones being;

  1. Set up connectivity between two devices in the sense of brokering some kind of comms socket from an app on device 1 to an app on device 2. You’d have to assume that the two apps know something about each other for meaningful data to flow in that scenario.
  2. Read content such as from an NFC tag.
  3. Swap data from the phone to another device which is exactly what I was doing above.

I thought I’d give (3) a try in the first instance and return perhaps to (1) or (2).

Swapping Simple Messages

How do I go about using the proximity APIs to swap simple messages from an app?

The first thing that I can do, is I can grab hold of a ProximityDevice (my phone only has one) and I can start to interact with it. For instance with just this code;

      ProximityDevice device = ProximityDevice.GetDefault();
      
      device.DeviceArrived += d =>
        {
          Debug.WriteLine("Device {0} arrived", d.DeviceId);
        };
      device.DeviceDeparted += d =>
        {
          Debug.WriteLine("Device {0} departed", d.DeviceId);
        };

Once I have that running, just holding up the phone to the back of the Windows 8 machine causes those event handlers to fire as the Windows 8 device ‘arrives’ and as it ‘departs’. Fairly simple.

To share some data, the simplest thing I can do is to ‘publish a message’ but this is limited in that it’s about a fixed size message and I think the message size is, by design, pretty small. For example, on my phone this code;

      ProximityDevice device = ProximityDevice.GetDefault();

      Debug.WriteLine("Max message size {0}", device.MaxMessageBytes);

Prints out the value 10240 or 10K on my Lumia so I’m not going to be able to cram a whole image in there. However, I could publish a URI;

      long messageId = 0;
      ProximityDevice device = ProximityDevice.GetDefault();

      device.DeviceArrived += d =>
        {
          Debug.WriteLine("Device {0} arrived", d.DeviceId);
        };
      device.DeviceDeparted += d =>
        {
          Debug.WriteLine("Device {0} departed", d.DeviceId);
          device.StopPublishingMessage(messageId);
        };

      messageId = device.PublishUriMessage(new Uri("http://www.microsoft.com", UriKind.Absolute));

and that works beautifully from my phone to my Windows 8 slate. Publishing a URI like that is a special case of publishing a message in general. For example, I could have written;

      long messageId = 0;
      ProximityDevice device = ProximityDevice.GetDefault();

      device.DeviceArrived += d =>
        {
          Debug.WriteLine("Device {0} arrived", d.DeviceId);
        };
      device.DeviceDeparted += d =>
        {
          Debug.WriteLine("Device {0} departed", d.DeviceId);
          device.StopPublishingMessage(messageId);
        };

      DataWriter writer = new DataWriter();

      byte[] bits = System.Text.UnicodeEncoding.Unicode.GetBytes("http://www.microsoft.com");

      writer.WriteBytes(bits);

      messageId = device.PublishBinaryMessage("WindowsUri", writer.DetachBuffer());

and that works fine and if you’re a bit worried about that “WindowsUri” magic string on line 20 then the docs page says that this will be ‘ok’ and translated for other devices which don’t know what a WindowsUri might be.

From that same page, there’s an idea of a “WindowsMime” type that I can use for sharing data here;

      long messageId = 0;
      ProximityDevice device = ProximityDevice.GetDefault();

      device.DeviceArrived += d =>
        {
          Debug.WriteLine("Device {0} arrived", d.DeviceId);
        };
      device.DeviceDeparted += d =>
        {
          Debug.WriteLine("Device {0} departed", d.DeviceId);
          device.StopPublishingMessage(messageId);
        };

      DataWriter writer = new DataWriter();

      writer.WriteString("The quick, brown fox jumps over the lazy dog");

      messageId = device.PublishBinaryMessage("WindowsMime.text/plain", writer.DetachBuffer());

which I find quite funny because, by default, this seems to cause Notepad.exe to open up on my Windows 8 machine Smile

So, it’s possible to share a simple, pre-defined message from my phone to my PC – what about the other way around?

Swapping Simple Messages (in reverse)

I know that on Windows 8, if I’m in Internet Explorer and I use the Devices charm to share the page via ‘Tap N Send’ then I should be able to pick that up from an app on my phone. Here’s my trial code;

      long subId = 0;
      ProximityDevice device = ProximityDevice.GetDefault();

      subId = device.SubscribeForMessage("WindowsUri", (s,e) =>
        {
          var type = e.MessageType;

          var bits = e.Data.ToArray();
          var text = System.Text.UnicodeEncoding.Unicode.GetString(bits, 0, bits.Length);

          Debug.WriteLine("Got a message of type [{0}] containing [{1}]",
            type, text);

          device.StopSubscribingForMessage(subId);
        }
      );

and that seems to work ok with one caveat which is that this isn’t perhaps a sensible way to listen for URI messages and the docs say that you don’t need to make a subscription for a URI scheme because the system will invoke the default handler for that scheme anyway which is what happens if I use the code above because even though my code does get the data, the system first puts up a piece of UI saying (essentially) “Do you want to open this link in IE?”.

But, the principle seems to work if I moved it away from URIs toward some other kind of data that I wanted to share or some other scheme than HTTP!

Swapping a Photo

How would I have an app which replicated the built-in experience where the photos app on the Phone can send an image to the photos app on Windows 8?

As far as I know, I can’t do it with this simplistic messaging because the data size is going to be too large. I need another mechanism.

In a first look at this, I got a bit puzzled. It seems that the photos app on the phone can’t be using the “publish binary message” mechanism because the message is way too big and the experience definitely makes it feels like the photo is being streamed across to the Windows 8 machine.

However, all the mechanisms for streaming seem to involve setting up a comms channel (socket) between an app on the phone and an app on another device (my Windows 8 machine) but it seemed to me (from task manager) that data was being transferred from my phone to my Windows 8 machine before any app had run up to receive that data (the photos app ultimately runs up but it seems to be later in the process).

So, how does this work? What is the photos app on the phone doing to share this data to the Windows 8 machine?

I got stumped so, meanwhile, I looked at this idea of 2 communicating “peers”…

Finding Peers

Publishing a simple message (possibly of binary format) is nice but there are time when I might want more complex data to flow between my phone application and some application on another device like a Windows 8 device.

If I want to do that, it’s likely that the applications would either;

  1. Comply with some standard around the data to be exchanged.
  2. Have prior knowledge of each other such that when they ‘meet’ each other they know how to talk.

It feels like scenario (2) is what’s addressed by the PeerFinder class within the SDK. The idea is that the phone app can try and find a peer app on (say) a Windows 8 device and send data to it. This means I now have to have 2 apps – one on the Windows 8 machine and the other on my phone both using the same WinRT APIs to communicate.

It also means that one app knows something about the other in order to start talking to it – some kind of identifier plus, also, whatever protocol that app understands for data sent to/from it.

To get this up and working is actually pretty easy but I’ll confess that it took me about a day to get it sorted out due to a problem with the identifiers that the Windows Phone and Windows 8 apps needed to share.

I had to resort to scouring the web for help and I came across the post that Alex has written up here which covers similar ground to this post but provided some clues about how these identifiers are formed but I still had to actually mail Alex and ask him before I properly figured it out.

The basic flow goes something like this;

  • Both devices use PeerFinder to say that they are available to do comms initiated by a tap gesture (if they can support that)
  • Both devices tell PeerFinder what kind of comms they support (Bluetooth, WiFi Direct, shared network)
  • Both devices tell PeerFinder the identifiers of other apps that they are “compatible” with – i.e. another app that is going to understand whatever traffic they send.
  • Both devices then Start() their PeerFinder and wait for events that go through a sequence of;
    • PeerFound
    • Listening/Connecting
    • Completed
  • At the “Completed” stage, a socket is handed over for data to flow from one device to the other.

The code looks something like this;

      // This is Windows 8 code.
      PeerFinder.AllowBluetooth = true;
      PeerFinder.AllowInfrastructure = true;
      PeerFinder.AllowWiFiDirect = false;
      PeerFinder.AlternateIdentities.Add(
        "WindowsPhone", "{de5db601-dfbd-4d06-98c8-e328088fdd82}");

      Debug.Assert(
        (PeerFinder.SupportedDiscoveryTypes & PeerDiscoveryTypes.Triggered) != 0);

      PeerFinder.TriggeredConnectionStateChanged += (s, e) =>
        {
          if (e.State == TriggeredConnectState.Completed)
          {
            // We have a socket!
            StreamSocket socket = e.Socket;
          }
        };

      PeerFinder.Start();

and on the Windows Phone side (using the same APIs of course – joy!!);

      PeerFinder.AllowBluetooth = true;
      PeerFinder.AllowInfrastructure = true;
      PeerFinder.AllowWiFiDirect = false;
      PeerFinder.AlternateIdentities.Add(
        "Windows", "a6a8b9dc-5e2c-4825-886b-6c677700ceda_54fjpjf3c8m40!App");

      Debug.Assert(
        (PeerFinder.SupportedDiscoveryTypes & PeerDiscoveryTypes.Triggered) != 0);

      PeerFinder.TriggeredConnectionStateChanged += (s, e) =>
        {
          StreamSocket socket = e.Socket;
        };

      PeerFinder.Start();

and the sharp eyed among you will notice it’s the same code apart from the AlternateIdentities that I’m using and I daresay that this code could even be built into a portable class library and then used from both Windows Phone and Windows 8 applications which is “as it should be” in my view and the more we see of that in the future the better.

A word on those identities though because I spent a long time trying to figure them out.

The Alternate Identity Used by the Phone App is the Identity of the Windows 8 App

This is kind of obvious. The phone app is saying “Hey, I have an identity but over on Windows 8 I’m also known with this other identity”. The less obvious thing about it to me is exactly how to form that identity.

The way you build it up is using the package family name from the manifest;

image

and then you follow it with an exclamation mark “!” and then what I found to be the tricky bit. The “id” of the app. Now, from the picture above I expected mine to be “App15” but it’s not. If you close down the graphical editor in Visual Studio and you open the manifest file with a plain XML editor the value you want is;

image

and I puzzled over this for quite a while before managing to get it sorted out.

The Alternate Identity Used by the Windows 8 App is the Identity of the Windows Phone App

The identity of the phone app is much easier and simpler to extract, you simply take it from the manifest shown here in Visual Studio;

image

Tapping Connects the Socket

With that previous code running on both devices, I find that tapping the devices and then pulling them apart establishes the socket between the two. I’m not 100% sure what kind of comms I’m talking over but I think it’s most likely a bluetooth socket in my setup ( I don’t believe my phone does WiFi direct ).

From there, transferring data is “just” a matter of reading and writing stuff up and down a socket which is a fairly common thing to do.

For instance, if my “protocol” between Phone and Windows was to agree that I would always write 1MB of data (easy protocol!) then I might run this code on the phone (no error handling here);

    PeerFinder.TriggeredConnectionStateChanged += async (s, e) =>
        {
          if (e.State == TriggeredConnectState.Completed)
          {
            // We have a socket.
            StreamSocket socket = e.Socket;

            // Make a buffer of 1K.
            byte[] buffer = new byte[1024];

            // Fill it.
            for (int i = 0; i < 1024; i++)
            {
              buffer[i] = (byte)(i % 255);
            }

            // Write it down the wire 1K times.
            for (int i = 0; i < 1024; i++)
            {
              await socket.OutputStream.WriteAsync(buffer.AsBuffer());
            }
            socket.Dispose();
          }
        };

and I might run this code on Windows 8;

   PeerFinder.TriggeredConnectionStateChanged += async (s, e) =>
        {
          if (e.State == TriggeredConnectState.Completed)
          {
            // We have a socket.
            StreamSocket socket = e.Socket;

            // Make a buffer of 1K.
            byte[] buffer = new byte[1024];
            IBuffer readBytes;
            uint totalBytes = 0;

            while (totalBytes < (1024 * 1024))
            {
              readBytes = await socket.InputStream.ReadAsync(buffer.AsBuffer(), 
                (uint)buffer.Length,
                InputStreamOptions.None);

              totalBytes += readBytes.Length;

              Debug.WriteLine("Read {0}K so far", totalBytes / 1024);
            }
            socket.Dispose();
          }
        };

and that seems to transfer the 1MB nicely enough from the phone to the Windows box.

What if the Windows 8 App Isn’t Active?

I find that even if the Windows 8 app is running under the debugger (i.e. where it won’t suspend/resume/terminate) but isn’t actually on the screen then when the phone app tries to use the tap gesture to connect I immediately see a little toast from the OS;

image

and if I actually tap on that Toast then the system will launch my Windows 8 app ( which can, of course, happen even if it’s already running ) and passes it a parameter as the debugger screenshot shows below so my app knows it’s been activated to accept a connection;

 image

How do I go from here to a connected socket? As the doc says and my code/debugging proved to me – the same code of handling the PeerFinder.TriggeredConnectionStateChanged event still works in delivering the socket between the phone and the Windows 8 machine – i.e. I can run the same code to get that socket whether I’m already running and in the foreground or whether I need to be launched and then pick up the socket.

Back to that Photos App

Having managed to get some basic comms going here between two clients over NFC I still don’t know how the photos app on Windows Phone 8 is talking to the Photos app on Windows 8 and sharing that photo with it.

I figured that I might be able to work out what’s going on using some kind of network sniffer but I’ve yet to find one that shows me bluetooth traffic ( I had a look at NetMon and at WireShark but neither seemed to offer blutetooth ).

I’ll continue to poke around and see if I can work it out but if anyone’s got more details on it – leave me a comment or drop me a line. I’d love to know Smile


Posted Fri, Feb 15 2013 12:45 PM by mtaulty
Filed under: , , ,