Back in 2012 when I first got involved in talking to developers and architects about Windows 8 in the UK, one of the things that I heard consistently was feedback of the form;

“Yes, I like the look of that new UI, I can see where you’re going with touch but what I’d really like to do is have one of those new apps interact with desktop software that we’ve already got running on Windows 7”

That word “interact” often meant a number of different things – some examples;

• The modern app needed to read/write data or files that would be shared between it and a desktop app.
• The modern app needed to send/receive “events” from a desktop app such that the two were in sync with one another.
• The modern app needed to call APIs beyond the WinRT APIs and the subset of .NET and “Windows” APIs that are available to a Windows Store app.
  • e.g. this often surfaced around a modern app talking to a piece of hardware via some interface code that already existed.

but there are perhaps many others.

Windows Store apps had been designed to do things like;

1. Be sold/served from the Windows Store without any dependency on an additional piece of software.
2. Have a lifecycle model which was incompatible with existing Windows ‘desktop’ software.
3. Have a security model which was much more restrictive than existing Windows ‘desktop’ software.
4. Have an installation/removal model which was much more lightweight than existing Windows ‘desktop’ software,
5. Run on hardware (specifically, ARM based systems) where Windows had never run before.

which meant that they had largely been designed not to interact with existing desktop applications and so in those early Windows 8 discussions I would often find myself saying something along the lines of;

“No, you can’t have a Store app that does X,Y,Z – it’s not something that Store apps are designed to do”

to which most developers (being curious, inquisitive types) would respond with something like;

“Fine, but I don’t want to put my app in the Store anyway so surely I can break some of these tough restrictions and get my code working if I am just using side-loaded enterprise apps?”

and the unfortunate reality is that the answer was generally;

“No, you’re still not going to get it to work”

and that was pretty much that until the arrival of Windows 8.1 Update with its new feature around “brokered WinRT” components.

//Build 2014 had a session on that;

and there’s also a full document up on MSDN that gives much more of a blow-by-blow account of some of the mechanics of what’s going on here;

Brokered Windows Runtime Components for Side-Loaded Windows Store Apps

In my time, I’ve written quite a lot of COM components, some in C++, some in VB, some in .NET and so I feel like I’ve got some experience in this kind of area but my COM is increasingly rusty and I have not spent nearly so much time on the “modern COM” that came as part of Windows 8 along with IInspectable and so on and so I must admit that while that document is a really good one, I still find myself staring very hard at it and, from time to time, scratching my head.

I read it more than once.

Reading for the First Time

On my first reading of it, what I picked up as I went along was;

1. It is possible to use existing code from a Windows 8.1 Update modern app in an Enterprise side-loaded scenario.
2. There’s a sample to explore and a Visual Studio template to be used as a basic for your own work in this area.
3. The architecture involves running a modern app in one process and a “brokered component” in another process with IPC between the two.
1. As always, this is going to mean “marshalling” of data between the two environments which always has an impact.
2. As always, this involves agreeing on a type system to marshal to/from and that’s the WinRT type system here.
4. This has primarily been thought about from the point-of-view of a .NET developer on .NET 4.5.
5. Unlike building regular WinRT components (in .NET) the process for building one of these brokered components is going to involve defining a separate .winmd file which purely defines the interface and another which implements it.
6. At the moment, this is primarily targeting 32-bit scenarios although the “modern app” side of the equation can be 64-bit.
7. The build scripts for the various pieces would take some analysing.
8. The references for the desktop project would need a bit of analysing – particularly the various “facade” assemblies that are referenced.
9. There’s some complexity in implementing the component which will run on the ‘desktop’ around areas like;
1. Threading.
2. Asynchronous calls.
10. There’s a need to build/register a proxy/stub from within the application’s package.

That document is really good and thorough but I must admit that at the end of my first pass through it I felt like I could do with a diagram or two showing which componentry ran in which process and where various pieces were deployed to.

I’ve Never Been Good at Reading the Instructions…

To help a little with understanding, I figured I’d try and get “Hello World” working via the  “Brokered WinRT Component Project Templates” and then see if I could work backwards from there. The steps I followed are as below.

Firstly, I made a new “Brokered Windows Runtime Component” project – I went with a calculator theme;

and then I went ahead and attempted to create an interface and an implementation of that idea – I only went as far as having an Add function but I made it a bit obscure by making it Async;

namespace MyCalculator
{
  using System;
  using System.Runtime.InteropServices;
  using System.Threading.Tasks;
  using Windows.Foundation;

  [ComVisible(true)]
  public interface ICalculate
  {
    // IAsync... are the set of interfaces that WinRT uses to put a common
    // denominator onto async work. This is true for all WinRT, not special
    // for brokered components.
    IAsyncOperation<int> AddAsync(int x, int y);
  }
  [ComVisible(true)]
  public sealed class Calculator : ICalculate
  {
    public IAsyncOperation<int> AddAsync(int x, int y)
    {
      // just to make this a lot harder than it needs to be, why not run this
      // on the threadpool via Task, capturing x and y above and then 
      Task<int> task = Task.Run<int>(() => x + y);

      // now make that task appear friendly to WinRT
      return (task.AsAsyncOperation());
    }
  }
}

In doing this, I wasn’t 100% sure on whether using the “ComVisible” attributes here was obligatory or not. Next step though was to add a project to generate proxies/stubs for this stuff so I added this project to my solution;

This project appears to contribute nothing other than a .def file with the usual COM related suspects being exported from the DLL but there’s more going on than meets the eye inside this project as we’ll see;

EXPORTS
        DllCanUnloadNow         PRIVATE
        DllGetClassObject       PRIVATE
        DllRegisterServer       PRIVATE
        DllUnregisterServer     PRIVATE

I added a reference to my MyCalculator project;

and did a quick build and that drops me out a proxy/stub DLL;

Next, I made myself a Windows Store app in C#;

and added a reference to the ProxyStub project that I’d just made. I’ll admit, at this point I started to question what it was that I was adding a reference to as the previous project seemed to build a plain-old-DLL rather than anything that I could reference but I went with it;

Then I tried to write some code using my Calculator class inside of that Store app;

      MyCalculator.Calculator calc = new MyCalculator.Calculator();

      var result = await calc.AddAsync(10, 20);

Before running this, I took a few more steps.

Firstly, I copied the .winmd and .pdb files from the MyCalculator project to a folder called d:\temp\binaries

Secondly, I ran as admin and did a regsvr32 on the MyCalculatorProxyStub.dll in that folder;

Thirdly, I made sure that the “ALL APPLICATION PACKAGES” security ‘group’ could read/execute stuff from that folder;

Then I edited the manifest file to add in the Extensions section which listed my type. Note that windows.activatableClass.inProcessServer is somewhat doc’d on MSDN.

and then I tried debugging that code and it worked fine. If I went hunting for a DLLHost.exe to attach to like this one;

then I could attach and see my brokered component code running inside of DllHost.exe;

And, in as far as “Hello World” goes – I was pretty happy. My next steps would be to try and add some references to the component being used and bring in some code (e.g. ADO.NET, File System, WCF) that can’t be usually run in a Windows Store app to see what it’s like to run that code in this context.

But…I must admit that I was a little bit sketchy about what I’d just actually done. I kind of “get” that it worked out but what’s going on at the point where my code news up a MyCalculator.Calculator instance?

I think the main thing that was foxing me was – “what the heck is going on when I build that C++ proxy/stub project?”

TL;DR – What the heck is going on when I build that C++ proxy/stub project? Or…Reading the Instructions

There’s a bunch of custom build steps in that “Brokered Windows Runtime Proxy Stub” project template which tie back closely to a deeper reading of that MSDN document. What they do is something along the lines of;

1. Attempt to kill any dllhost that’s using my component.
2. Use winmdidl and midl to tear apart from C# .winmd component and use the results to build both a proxy/stub and another .winmd that only has “interface” in it, not implementation.
3. Use icacls to attempt to make sure that the output folder is accessible to the “ALL APPLICATON PACKAGES” identifier as I did manually above.

I dug into one or two of those…

1. Run the output of the MyCalculator C# project through the winmdidl compiler to generate IDL from the binary.

Because the binary is implemented in C#, it contains both interface or “shape” along with implementation.

This is done with something like the following command line (simplified here);

winmdidl /metadata_dir:c:\Windows\System32\WinMetadata MyCalculator.winmd

what that does is to spit out an IDL file. By definition, there’s no implementation in an IDL file. The IDL file looks something like;

//
// File generated by WinMDIDL version 8.00.0011
//

import "inspectable.idl";
import "AsyncInfo.idl";
import "EventToken.idl";
import "Windows.Foundation.idl";
// Additional imports to ensure that all required headers are included
cpp_quote("#if defined(__cplusplus)")
cpp_quote("}")
cpp_quote("#endif // defined(__cplusplus)")
cpp_quote("#include <Windows.Foundation.h>")
cpp_quote("#if defined(__cplusplus)")
cpp_quote("extern \"C\" {")
cpp_quote("#endif // defined(__cplusplus)")

// Forward Declare
namespace Windows
{
    namespace Foundation
    {
        interface IStringable;
    }
}
namespace MyCalculator
{
    interface ICalculate;

    runtimeclass Calculator;
}

// Generic instantiations

// Type definition
namespace MyCalculator
{
    [uuid(046A59DD-4907-558E-524D-2215A4F6F6F9)]
    [version(0x01000000)]
    interface ICalculate : IInspectable
    {
        HRESULT AddAsync([in] INT32 x, [in] INT32 y, [out] [retval] Windows.Foundation.IAsyncOperation<INT32>** value);
    }

    [activatable(0x01000000)]
    [marshaling_behavior(agile)]
    [threading(both)]
    [version(0x01000000)]
    runtimeclass Calculator
    {
        [default] interface MyCalculator.ICalculate;
        interface Windows.Foundation.IStringable;
    }
}

2. Run the IDL file produced back through the MIDL compiler

The IDL that was just generated is now run back through the MIDL with a command line something like;

midl.exe /winrt /metadata_dir "c:\Program Files (x86)\Windows Kits\8.1\References\CommonConfiguration\Neutral" /winmd interfaceOnly.winmd

So, now there’s new .winmd file generated only from the IDL which in my case is called interfaceOnly.winmd and bringing up the 2 .winmd files in ILDASM shows the difference;

There’s now an “interface only” version of the .winmd file – the code’s been sucked out of there.

As an aside, this is how a WinRT component written in C++ is always packaged. That is – you can’t embed native code into a .winmd file so a C++ WinRT component is always going to have an “interface only” .winmd file.

This is the .winmd file which the Store app I made above ultimately references and it’s basically a re-packaging of the .winmd file that the MyCalculator project originally produced except the implementation’s all gone and it’s a pure “interface” .winmd file at this point. So…no way that Store app is going to be running any code from inside of this file.

The other thing that drops out of running midl.exe is code for a plain, old, native DLL – the midl compiler with the switches applied produces;

MyCalculator.h

dlldata.c

MyCalculator_i.c

MyCalculator_p.c

and the project template compiles these up into a DLL with the .DEF file that I referenced earlier in the post to export the standard COM exports for registration, class objects and checking whether the DLL is still in use or not.

Registering that DLL via regsvr32.exe adds the registry key (on my system);

which links back via the CLSID to;

and so the proxy/stub is in place for marshalling this interface.

3. When the Store App news up a MyCalculator.Calculator instance – what happens?

I get a bit vague at this point but the StoreApp is .NET code so I can disassemble StoreApp.exe with ILDASM and have a look at the manifest. In there, there’s a piece that says that there’s a reference to the MyCalculator component and this reference was formed by making a reference to the “interface only” .winmd file.

I think that the windowsruntime keyword there means (via “CLR and the Windows Runtime”) that the CLR doesn’t just try and create MyCalculator instances but, instead, delegates that down to the Windows Runtime.

If I debug my Store App using mixed debugging and set a breakpoint just at the point where it is about to instantiate the Calculator;

Then if I load up some symbols for combase.dll I can set a breakpoint in RoGetActivationFactory and I can see that this is getting called and from mixing the parameters with the memory window I can see that it’s MyCalculator.Calculator that we’re dealing with;

So we know that the creation of this “type” is prepared to be delegated to something else. How does that work? In all honesty, I’ve long since forgotten a number of details about how classic COM activation worked and, worse, the details of how modern COM activation works have always been a little obscure to me.

However, it’s COM so the likelihood is that the registry will help – perhaps looking at what happens to the registry during package installation might help in seeing which of the bits that I specified in my app’s manifest show up in the register and how they are used from there during activation of that Calculator type?

From Process Monitor, I spotted a couple of keys being created during package registration that looked a little “useful”;

So, this entry seems to be specifying that we’re expecting to run up the CLR to host this component and there’s the custom attributes;

which seem to say where the binaries have been placed.

If I watch registry access from my StoreApp.exe at the point where it’s trying to instantiate the MyCalculator.Calculator class then I see the sketch of a pattern.

Firstly, some componentry goes and hunts out MyCalculator.Calculator from the registry, find its CustomAttributes, spots the DesktopApplicationPath and decides “this is a job for the winRTDesktopBrokerCLSID”;

A hunt through the registry for that component then goes on and at some point the decision is made to spin up a DllHost to host a CLR-implemented COM component.

and, at some point, that DllHost gets on with the business of hosting the MyCalculator.Calculator component which it seems to do via an IWinRTClassActivator using an IWinRTInprocActivator which is the component (green arrows below) that’s being interrogated in the registry access here just above the point where something reaches out and grabs hold of the .winmd implementation file from the DesktopApplicationPath (red arrows below);

So…in short, I’m not entirely sure of the circuitry between the components here other than to say that;

1. It works.
2. It involves the usual amount of registry access you’d associate with COM.
3. I don’t want to think about it to any greater level of detail than I’ve had to so far.

And, no doubt, there’s some thought to be given here on topics like the lifetime of this “desktop” component which is going to be a reference-counted thing and how that lines up with the lifetime of the Windows Store app that’s using it which can go through its suspend/resume/terminated cycle.

I’d hope to maybe have another look at this and perhaps see what it’s like to use code from that desktop component beyond adding 2 integers together and see what it’s like to do something that’s not possible inside of a Store app – like talk to a WCF service beyond the basic HTTP stack.