Grabbing a Photo & Calling Azure Vision API from ‘Pure UWP’ Code

Posted on by

Just using a blog post as a pastie – I had cause to write a function today to take a photo from a camera on a UWP device and send it to the Azure Cognitive Service for Vision to ask for ‘tags’ from the image.

The intention was to call the function from a UWP-specific Unity app running on HoloLens (the code does work on HoloLens). It would need camera, microphone and internet client capabilities to work.

It’s very specific to one task but, clearly, could be wrapped up into some class that made it a lot more general-purpose and exercised more pieces of that API but I just wanted somewhere to put the code in case I want it again in the future. This is what I had…

static async Task<Dictionary<string,double>> TakePhotoAnalyzeAzureForTagsAsync(
            string azureVisionKey,
            string azureVisionBaseEndpoint = "https://westeurope.api.cognitive.microsoft.com")
        {
            var azureVisionApi = "/vision/v1.0/analyze?visualFeatures=Tags";
            Dictionary<String,double> resultDictionary = null;

            // Capture an image from the camera.
            var capture = new MediaCapture();

            await capture.InitializeAsync();

            var stream = new InMemoryRandomAccessStream();

            await capture.CapturePhotoToStreamAsync(
                ImageEncodingProperties.CreateJpeg(), stream);

            stream.Seek(0);

            // Now send that off to Azure for processing
            var httpClient = new HttpClient();
            httpClient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", azureVisionKey);

            var streamContent = new HttpStreamContent(stream);
            streamContent.Headers["Content-Type"] = "application/octet-stream";

            var response = await httpClient.PostAsync(
                new Uri(azureVisionBaseEndpoint + azureVisionApi),
                streamContent);

            if (response.IsSuccessStatusCode)
            {
                var responseString = await response.Content.ReadAsStringAsync();
                JsonObject jsonObject;

                if (JsonObject.TryParse(responseString, out jsonObject))
                {
                    resultDictionary = 
                        jsonObject.GetNamedArray("tags").ToDictionary(
                            tag => tag.GetObject().GetNamedString("name"),
                            tag => tag.GetObject().GetNamedNumber("confidence"));
                }
            }
            return (resultDictionary);
        }

As a related aside, I also revisited this blog post recently as part of looking back at Cognitive Services and its facial APIs and I realised that code needed changing a bit to make it work and so I did that work and dropped it onto github over here;

Example of using both local UWP face detection and Cognitive Service face detection in a UWP app