NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

Just a short post. I’ve been playing with a situation recently where I needed to get an IP address into a holographic app. There’s lots of different ways that you might do this depending on how frequently the IP address might change;

• Hard-code it into the app.
• Put it into a config file (perhaps loaded from some other server).
• Type it into the app’s UI although I think it’s fair to say that typing IP addresses on HoloLens isn’t so much fun.
• Magically download it via some sort of bluetooth service
• Speak it to the application through the microphone and UWP speech recognition
• Put the IP address into a QR code and have the app scan it through the camera

and I’m sure there’s many more but today I thought I’d experiment with what seemed like a fairly natural idea – when I want to give another person an IP address, I usually write it onto a piece of paper and hand it to them or pin it up on the wall.

So, why not with the device?

Because I already have some code which runs on HoloLens and scans for QR codes (see this blog post and accompanying github) it felt like it would be very little effort to change the QR code scanning that happens in that blog post into some “IP Address” OCR recognition and see how well or badly that works out.

Here’s an example of how well/badly it worked out – I think it works quite well but it’s fair to say that I haven’t managed to get it to work with handwriting although that doesn’t surprise me as I’ve never found the OCR engine in UWP to process handwriting and maybe it’s not meant to so I’m not being critical of it when I say that.

Note – it’s “a bit tricky” to record an app on the HoloLens that is actively taking over the webcam so I’m having to show this with static images;

but it feels like a scan of an IP address like that can be done in about 1-2s once the camera has a view of it although, naturally, it’s possible for the camera to get a partial view such as 2.168.0.1 for the above address so a re-scan might be necessary in some circumstances.

In terms of the code here, I simply did some minimal changes to what I already had around QR code processing in that I removed the ZXing library from my project and then changed the public interface so that it looks as it does in the code below and clearly I should do something about renaming the namespace here but this call says “Hey, find me an IP address from the first camera on the system or timeout after 30s”;

MediaFrameQrProcessing.Wrappers.IPAddressScanner.ScanFirstCameraForIPAddress(
        result =>
        {
          UnityEngine.WSA.Application.InvokeOnAppThread(() =>
          {
            // result here is a System.Net.IPAddress...
            this.textMesh.text = result?.ToString() ?? "not found";
          }, 
          false);
        },
        TimeSpan.FromSeconds(30));

and it gives me back an IPAddress instance. In my underlying library, I replaced the QR code frame scanner with a (basic!) IP address frame scanner;

namespace MediaFrameQrProcessing.Processors
{
  using MediaFrameQrProcessing.VideoDeviceFinders;
  using System.Runtime.InteropServices.WindowsRuntime;
  using System.Threading.Tasks;
  using Windows.Devices.Enumeration;
  using Windows.Media.Capture;
  using Windows.Media.Capture.Frames;
  using Windows.Media.Ocr;
  using System;
  using System.Text.RegularExpressions;
  using System.Net;

  public class IPAddressFrameProcessor : MediaCaptureFrameProcessor
  {
    public IPAddress Result { get; private set; }

    public IPAddressFrameProcessor(
      MediaFrameSourceFinder mediaFrameSourceFinder, 
      DeviceInformation videoDeviceInformation, 
      string mediaEncodingSubtype, 
      MediaCaptureMemoryPreference memoryPreference = MediaCaptureMemoryPreference.Cpu) 

      : base(
          mediaFrameSourceFinder, 
          videoDeviceInformation, 
          mediaEncodingSubtype, 
          memoryPreference)
    {
    }
    protected override async Task<bool> ProcessFrameAsync(MediaFrameReference frameReference)
    {
      bool done = false;

      // doc here https://msdn.microsoft.com/en-us/library/windows/apps/xaml/windows.media.capture.frames.videomediaframe.aspx
      // says to dispose this softwarebitmap if you access it.
      using (var bitmap = frameReference.VideoMediaFrame.SoftwareBitmap)
      {
        try
        {
          if (this.ocrEngine == null)
          {
            this.ocrEngine = OcrEngine.TryCreateFromUserProfileLanguages();
            this.regex = new Regex(IP_ADDRESS_PATTERN);
          }
          var results = await this.ocrEngine.RecognizeAsync(bitmap);

          if (results != null)
          {
            var matchingResults = this.regex.Matches(results.Text);

            for (int i = 0; !done && (i < matchingResults.Count); i++)
            {
              IPAddress parsedAddress;

              done = IPAddress.TryParse(matchingResults[i].Value, out parsedAddress);

              if (done)
              {
                this.Result = parsedAddress;
              }
            }
          }
        }
        catch
        {
        }
      }
      return (done);
    }
    Regex regex;
    OcrEngine ocrEngine;

    // Taken from S.O. http://stackoverflow.com/questions/106179/regular-expression-to-match-dns-hostname-or-ip-address
    const string IP_ADDRESS_PATTERN =
      @"(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}";
  }
}

and that all seems to work reasonably well.

The code for this (the library and the 2D XAML test app) is here if anyone wants to play with it.