Following on from this growing series of posts;
Windows 10 UWP–Migrating a Windows 8.1 App (Part 1 of N where N tends to infinity)
Windows 10 UWP–Migrating a Windows 8.1 App (Part 2 of N)
I wanted to begin the process of migrating my camera page and in the process of doing that try and pick up two areas of improvement;
- See if I can benefit from the various improvements that look to have been made around the camera as talked about in the //Build 2015 Session Developing Powerful Camera Apps
- Try and move away from using a version of the QR code recognition library ZXing that I’ve built myself given that there is a WinRT port of ZXing available
In order to do that, I need to dig into my own code and try and remember how my camera capture works because I remember that while coding it back on Windows 8.0 I did not;
- Get things to work the way I wanted them to.
- Get away without some troubles – I remember some ‘bugs’ that I had to code around.
Mostly for my own benefit, the way in which my current camera capture code works is as follows (in Windows 8.1);
- There’s a CapturePage.xaml XAML Page which hosts an AppBar which hosts a few buttons and a UserControl which actually does the capture.
- The UserControl essentially hosts three views (switched in/out by visual state);
- A ‘no camera’ view
- An ‘initialising the camera’ view
- A view which displays the current preview video from the camera along with some data-bound metrics that display how many images per second are being analysed. The video from the camera is displayed by a CaptureElement
When the CapturePage spins up it tries to figure out which cameras there are on the machine (via DeviceInformation.FindAllAsync()) and it looks for DeviceClass.VideoCapture devices and gives priority to cameras that say that they are located on the back of the device.
If one of those is found, then camera preview is started from that device with fairly default settings and then the whole of the image capture and decoding cycle is represented by this single line of code;
await this.cameraCaptureControl.StartCameraPreview(this._currentCamera.Id);
but what happens underneath there isn’t perhaps what you’d expect and it wasn’t what I originally wanted to build in that it’s essentially a separate Task which;
- Creates a stopwatch
- Creates an ImageEncodingProperties specifying width of 1280×960.
- Loops until a QR code is recognised or a cancellation token is signaled (to cancel the task)
- starts the stopwatch
- waits (with a cancellation token) for MediaCapture.CapturePhotoToStreamAsync()
- takes the populated stream (a memory stream) and hands it over (via some messing around) to the ZXing library to see if there’s an identifiable QR code in there
- stops the stopwatch
- updates some metrics about how long that operation took
There’s more code in there than I’m implying but that’s about the basics of it and I think the thing that is surprising is that the app does not in any way try to hook into the video frames but, instead, is periodically capturing PHOTOS and trying to decode those.
I’d really rather that I was hooking into the video feed and taking as many frames from that as it can handle rather than snapping photos. I daresay that I can also do a better job of dealing with the camera in general.
One part of that is that I have code in the app to rotate my view to compensate for scenarios where the device is itself rotated – I believe that I can get rid of this in the light of Windows 10 UWP and I’m looking forward to deleting that code if I get there.
But, first, I thought I’d take a look at seeing if I could remove my dependency on my local build of ZXing and move to the WinRT port of that library.
Moving from ZXing to ZXing 
In the Windows 8.x version of my app, I’d taken the source for ZXing and built it into the app having (as I remember) made a few changes here and there in order to be able to build it for WinRT.
I took away my reference to my local version of ZXing and went off to NuGet to see what I could find and this is what I came up with;
and, in all honesty, so much debug spew comes out of NuGet doing these kinds of operations;
that it’s sometimes hard to know whether the installation of the package actually worked or not but it looks (from the last line) like it did.
Fortunately, all my calls into ZXing were contained in a single class and so I didn’t have to visit much of my code to start trying to replace what I’d done in Windows 8.0.
In doing that though I was keen to keep the existing camera code working while replacing ZXing even if that means that I’ll ultimately do more work than necessary when I rework the camera capture part of that app to (hopefully) work better and with better performance on Windows 10.
Here’s the code that I had on Windows 8.x and, in all honesty, I’ve lost track of why it was attempting to do some kind of pixel conversion but, regardless, that’s what it was doing;
namespace kwiQR.Decoding
{
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Windows.Graphics.Imaging;
using Windows.Storage.Streams;
using ZXing;
using ZXing.Common;
internal static class BarcodeDecoder
{
static Dictionary<DecodeHintType, object> _zxingDecoderHints;
static BarcodeDecoder()
{
_zxingDecoderHints = new Dictionary<DecodeHintType, object>();
_zxingDecoderHints.Add(DecodeHintType.POSSIBLE_FORMATS, new List<BarcodeFormat>() { BarcodeFormat.QR_CODE });
// TODO: do we really need this?
_zxingDecoderHints.Add(DecodeHintType.TRY_HARDER, true);
}
// TODO: work out whether caching some of these objects helps - unsure of their creation
// cost.
// Also, figure out what (if anything) needs disposing here.
public async static Task<string> DecodeStreamToBarcode(IRandomAccessStream photoStream)
{
// WinRT work to try and get to the raw bytes that make up the image.
BitmapDecoder bitmapDecoder = await BitmapDecoder.CreateAsync(photoStream);
BitmapTransform emptyBitmapTransform = new BitmapTransform();
PixelDataProvider pixelDataProvider = await bitmapDecoder.GetPixelDataAsync(
BitmapPixelFormat.Rgba8,
BitmapAlphaMode.Premultiplied,
emptyBitmapTransform,
ExifOrientationMode.RespectExifOrientation,
ColorManagementMode.DoNotColorManage);
byte[] rawRGBPictureBytes = RgbaToRgbArray(pixelDataProvider.DetachPixelData(),
bitmapDecoder.PixelWidth, bitmapDecoder.PixelHeight);
// Now the ZXing bits and pieces.
RGBLuminanceSource zxingLuminanceSource = new RGBLuminanceSource(
rawRGBPictureBytes, (int)bitmapDecoder.PixelWidth, (int)bitmapDecoder.PixelHeight);
HybridBinarizer zxingHybridBinarizer = new HybridBinarizer(zxingLuminanceSource);
BinaryBitmap zxingBinaryBitmap = new BinaryBitmap(zxingHybridBinarizer);
MultiFormatReader zxingMultiformatReader = new MultiFormatReader();
Result zxingResult = zxingMultiformatReader.decode(zxingBinaryBitmap, _zxingDecoderHints);
return (zxingResult?.Text);
}
static byte[] RgbaToRgbArray(byte[] rawRGBAPictureBytes, uint width, uint height)
{
// Now we have the raw bytes, the 'problem' is that this is RGBA whereas the ZXing
// stuff (as I've got it built right now in a mode similar to 'UNITY' wants an
// array of RGB. What to do? Something very cheap and cheerful.
byte[] rawRGBPictureBytes = new byte[width * height * 3];
int i = 0;
int j = 0;
while (i < rawRGBPictureBytes.Length)
{
rawRGBPictureBytes[i] = rawRGBAPictureBytes[j];
i++;
j++;
if (j % 4 == 1)
{
j++;
}
}
return rawRGBPictureBytes;
}
}
}
and the modified code (preserving the interface into the function);
namespace kwiQR.Decoding
{
using System;
using System.Threading.Tasks;
using Windows.Graphics.Imaging;
using Windows.Storage.Streams;
using ZXing;
internal static class BarcodeDecoder
{
static BarcodeReader barcodeReader;
static BarcodeDecoder()
{
barcodeReader = new ZXing.BarcodeReader();
barcodeReader.Options.PureBarcode = false;
barcodeReader.Options.Hints.Add(DecodeHintType.TRY_HARDER, true);
barcodeReader.Options.PossibleFormats = new BarcodeFormat[] { BarcodeFormat.QR_CODE };
barcodeReader.Options.TryHarder = true;
}
public async static Task<string> DecodeStreamToBarcode(IRandomAccessStream photoStream)
{
// I'm hoping that when I get to update the work I'm doing with the camera for Windows 10 I won't
// have to accept the image as a stream here and then decode that stream back into a bunch
// of pixels in order to pass it on to ZXing.
// For now, I'm trying to not break the calling code, so leaving that as it is.
BitmapDecoder bitmapDecoder = await BitmapDecoder.CreateAsync(photoStream);
BitmapTransform emptyBitmapTransform = new BitmapTransform();
PixelDataProvider pixelDataProvider = await bitmapDecoder.GetPixelDataAsync(
BitmapPixelFormat.Rgba8,
BitmapAlphaMode.Premultiplied,
emptyBitmapTransform,
ExifOrientationMode.RespectExifOrientation,
ColorManagementMode.DoNotColorManage);
var zxingResult = barcodeReader.Decode(
pixelDataProvider.DetachPixelData(),
(int)bitmapDecoder.PixelWidth, (int)bitmapDecoder.PixelHeight,
BitmapFormat.RGBA32);
return (zxingResult?.Text);
}
}
}
and so I’ve less code in here and it’s doing less work and my whole solution now drops down from 2 projects (app + class library) to a single project.
And, so far as I can see, the app is still recognising QR codes much as it did previously.
So, that’s a nice jump forward for the app to make and, for me, it reflects the fact that WinRT libraries are much more prevalent than they were back in 2012 when I first wrote this app.
All told, this took me about 2 hours to make the transition.
Mike Taulty 