Silverlight, HTML and the WebBrowser Control for Offline Apps

“Silverlight is a browser plug-in” Smile

In the early days of Silverlight, it was fairly easy to understand that Silverlight content would always be rendered inside of a browser window (whether that be IE, Chrome, Firefox or Safari) where it was surrounded by the hosting HTML content and JavaScript. The HTML was hosting Silverlight rather than the other way around.

Silverlight has a pretty strong interoperability layer between the CLR code running inside the plug-in and the HTML DOM and JavaScript of the surrounding page so you could use Silverlight to exercise some “control” over the hosting HTML/JS and vice versa but HTML was the starting point.

image

and so if you had a bunch of HTML content to display and wanted to mix in a bit of Silverlight then it was fairly clear what your options were until…

When Silverlight 3 came along, it was possible to take a Silverlight application out of the browser and you ended up with a situation like this;

image

and it’s possible for these applications to run online/offline and to automatically update themselves.

If you then had a bunch of HTML content to display then life became a little difficult for this class of apps because there was no browser around any more and so HTML display wasn’t possible for out of browser apps until…

When Silverlight 4 arrived and introduced the WebBrowser control which only worked out of browser which kind of made sense given the scenario it was trying to solve;

image

and so out-of-browser applications could now display HTML although I think it’s fair to say that the interop layer here is fairly weak compared to the strong interop layer that I talked about between the Silverlight plug-in and its hosting page (in the case where there is one).

I’ll come back to more on the WebBrowser control in a moment but, first, let’s just add in Silverlight 5…

Out-of-browser apps don’t work for all scenarios. Not every user wants to “install” something to the local machine and the developer doesn’t want to present the user with that “installation” dialog that you get with an out-of-browser app. They just want to provide an “in browser” app and so Silverlight 5 adds another tweak to this idea of mixing HTML/Silverlight.

In Silverlight 5, the WebBrowser control can now also be used for Silverlight apps that run in the browser in specific circumstances. As you might imagine, this gets just a little bit complex because you have a situation that looks like this;

image

and the specific circumstances that make this work are;

  1. The application has to be running in a version of Internet Explorer (i.e. by definition, on Windows).
  2. The application has to be running in elevated trust which is something that in browser applications can do in Silverlight 5.

which, for me, makes this feature targeted at a particular class of application – those on the intranet where you have enough control over the desktop to set up the certificates required for a trusted in-browser app and to mandate Internet Explorer as the browser that your user will run.

Now, back to the WebBrowser control – there’s a couple of places where I’ve chatted to people both recently and in the past where this control causes them hassle and that’s what sparked this post. Those areas are;

  1. displaying HTML content when offline
  2. dealing with the “airspace issue”

I’ll come back to these two items in a moment although I don’t think that I’m going to really solve any issues here but first a little bit about the WebBrowser control generally…

The WebBrowser Control

In a regular, out-of-browser application running in the usual security sandbox, the WebBrowser control is immediately useful and can load HTML for you in 3 different ways.

The first is by setting the Source property as in;

image

and it’s interesting to note that there’s no need to set up any cross-domain security policy here – the control will go out to the web and grab the content without looking for clientaccesspolicy.xml on the microsoft.com site.

You can also do a similar thing by using the Navigate() method;

image

and that works just fine too and you can finally load HTML by dropping it straight into a string and loading it that way as in;

image

and that works fine for those situations where you perhaps can’t provide a URL to the HTML that you want to load – I’ll return to this later.

As I said a little earlier, there’s a little bit of an interop bridge between the JavaScript in the HTML hosted in the control and the Silverlight code that’s doing the hosting. So, for example we might have something simple like;

image

and then we might run that JavaScript from C# code;

image

and in the other direction we can do something like;

image

and that’s really the extent of our interoperability with the control – we have no option to reach directly into the HTML DOM other than to pass it a string of HTML to load or to invoke some existing script function to manipulate the DOM.

In these particular cases, I was loading the HTML from a string but if I was actually navigating to HTML files then whether I can use ScriptNotify and InvokeScript depends on where the HTML is coming from. The documentation states;

“The HTML file must be hosted in the same domain as the Silverlight application”

and so I can refactor what I have about to put the HTML into a file and I find that if I load that file from the same domain as my Silverlight app then I can use ScriptNotify as in;

image

but if I load the HTML file from somewhere else, i.e. somewhere other than where the Silverlight XAP came from then I see;

image

and so that’s a limitation around ScriptNotify/InvokeScript.

There’s little else that you can do with the WebBrowser control. You can handle its LoadCompleted event to know when it’s loaded content and you can use its SaveToString() method to save content to a string.

Coming back to those 2 points that I wanted to talk about…

Displaying HTML Content when Offline

One thing that’s not so obvious about the Navigate() method or the Source property on the WebBrowser control is what exactly you can Navigate() to.

It’s fairly obvious that I can do;

image

but what does it mean to do something like;

image

what’s the meaning of a relative URL like this foo.html when the application is running out of browser? What it translates into is going back to the same URL that the XAP file came from in the first place and trying to find foo.html in that location.

In my instance, my XAP file came from;

http://mtaulty/TestingSite/OobApp.xap

and I see this relative URL being resolved back to;

http://mtaulty/TestingSite/foo.html

It’s also quite “interesting” that what this does is establish a base location for the resolution of subsequent resources such as images. If my foo.html contains an image that loads from images/img1.jpg then that will get resolved as;

http://mtaulty/TestingSite/images/img1.jpg

quite naturally as you’d expect.

That’s ok as long as you’re online and can reach the servers that your HTML comes from but one of the advantages of a Silverlight out-of-browser application is that you can take it offline.

If your application is offline or, perhaps for performance reasons, you might want to load your HTML from the filesystem and this is definitely one of the places where things get a little interesting with the WebBrowser control.

Depending on how much HTML/JS content you have, it’s not too hard to see a solution where you either;

  1. package up some HTML/JS into your XAP file
  2. package up some HTML/JS into something like a separate ZIP file

and then download that content from Silverlight code and unpack it to the local filesystem.

Where to put it in the local filesystem? You have 2 choices;

  1. Isolated storage. This is good because you don’t need elevated trust but if you’re then to have the WebBrowser navigate to the content you’ll need the file path to your isolated storage folder which I don’t think you can realistically try to get to unless you are running in elevated trust. Catch 22.
  2. The real filesystem. This is good because it makes for easy, obvious file paths but it means that the application needs elevated trust.

Either way, we seem to end up needing elevated trust.

Regardless, having unpacked that HTML content to somewhere such as c:\temp – how can you navigate to it? What about;

image

As far as I can work out, it’s not possible to get the WebBrowser control to navigate to a file based location like this and this is true for both a sandboxed application and an elevated application.

This is a little bit of a blocker to say the least.

In my case, Silverlight definitely has access to the file c:\temp\foo.html but I can’t ask the browser to load that file for me. I suppose that I could read the file myself and then call NavigateToString() ?

What if I do that?

image

that works a treat but if my HTML has any resources in it such as this image;


<html>
<head>
  <title>my document</title>  
</head>
<body>
  <img alt="my pic" style="width: 480px; height: 320px" src="images/img1.jpg" />
</body>
</html>

then the browser isn’t going to know how to load up that image img1.jpg. One way to try and get around this would be to try and put the image data inline in the HTML by using a “data URL” but that seems to open up a whole world of having to either;

  1. rewrite the HTML up front
  2. rewrite the HTML on the fly as it’s loaded

and neither seem like much fun and I don’t know that they’d help with respect to loading other references items like scripts or stylesheets.

I looked for another way in which I might be able to get the HTML to be loaded as a string but still be able to reference images back in the filesystem and the best that I came up with so far would be to set the <base> on the HTML document directly either through script or through mark-up as in;

<html>  
  <head>    
    <base href="file:///c:/temp/" />
    <title>my document</title>    
  </head>
  <body>
    <img src="images/img1.jpg" />
  </body>
</html>

and that seemed to work ok except when I experimented with a few images I found that some wouldn’t load from the filesystem.

I have yet to figure out why some images loaded and some didn’t but I think this comes down to the way that the sllauncher.exe process has been marked in the registry with respect to what IE features it does/doesn’t allow when hosting HTML.

There’s an explanation of feature controls in IE on MSDN and this specific one appears to be the FEATURE_BLOCK_LMZ_IMG setting which can be used to block images being loaded from the filesystem for processes that host IE.

This value is set to a value of 1 in my registry for the sllauncher.exe process and I noticed that setting it to a value of 0 fixed the image that wouldn’t load for me but I’d need to investigate this more.

image

The other immediate problem with this approach is that if you have a large set of html files then you perhaps don’t want to visit them all and add a new <base> element to them manually.

What if my HTML contained CSS references such as;

<html>  
  <head>    
    <base href="file:///c:/temp/" />
    <title>my document</title>    
    <link type="text/css" rel="stylesheet" href="styles/test.css" />
  </head>
  <body>
    <img src="images/img1.jpg" />
    <img src="images/img2.jpg" />
    <img src="images/img3.jpg" />
  </body>
</html>

well that seems to work reasonably well as well in that my stylesheet definitely got loaded so at least in the simple case this seemed to work.

But what about script? What if my HTML includes script such as;

<html>  
  <head>    
    <base href="file:///c:/temp/" />
    <title>my document</title>        
    <link type="text/css" rel="stylesheet" href="styles/test.css" />
    <script src="scripts/somefunctions.js">
    </script>
  </head>
  <body>
    <div id="x">text one</div>
    <img src="images/img1.jpg" onclick="changeText()"/>
    <img src="images/img2.jpg" />
    <img src="images/img3.jpg" />
  </body>
</html>

where the changeText function comes from the included script.

This one I found a lot harder to figure out. I kept hitting the dreaded “invalid character” error and I suspect that it’s related to this discussion which talks about the IE feature FEATURE_BLOCK_LMZ_SCRIPT which is explained here and certainly on my system I found the registry key;

image

and the description from the docs is that this registry key will stop script from the user’s filesystem being executed inside the process sllauncher.exe which is what was happening to me and I found that if I set this value to zero then my script would load and run just fine so that registry key would need altering if I was to pursue this path further.

I’m curious as to whether PInvoke might light you make use of CoInternetSetFeatureEnabled in the future in order to tweak this for your app rather than trying to poke around in the registry and whether that API can actually tweak this particular feature at runtime.

With that sorted out, I seem to be able to load and run script but what about the big one? What about hyperlinks? If I’m using NavigateToString() to load up HTML then what happens when the user clicks on a hyperlink? It’s not going to work… Confused smile

It’s hard to come up with a particularly great solution here but one idea that I had was around using a bit of jQuery to ensure that use of hyperlinks would be routed through my own code and then passed on to the Silverlight hosting code. That is I added a little bit of global JavaScript to my HTML;

$(document).ready(function (event) {
  $("a").click(function (event) {    
    event.preventDefault();
    notifySilverlightHostOnNavigation(event.target.href);
  });
});

function notifySilverlightHostOnNavigation(url) {
  window.external.notify(url);
}

I’m not very good at JavaScript/jQuery so apologies for whatever I did wrong there Smile but that seems to now “work” in the sense that my Silverlight code now gets notified as you click on hyperlinks.

This chunk of HTML loads that piece of jQuery code from somefunctions.js;

<html>  
  <head>    
    <base href="file:///c:/temp/" />
    <title>my document</title>        
    <link type="text/css" rel="stylesheet" href="styles/test.css" />
    <script src="scripts/jquery.js"></script>
    <script src="scripts/somefunctions.js"></script>
  </head>
  <body>
    <a href="bar.html">click me</a>
    <div id='x'>blank</div>
    <img src="images/img1.jpg" onclick="changeText()"/>
    <img src="images/img2.jpg"/>  
    <img src="images/img3.jpg" />
  </body>
</html>

and so when someone clicks on a hyperlink, this gets passed through to Silverlight code;

    void OnBrowserScriptNotify(object sender, NotifyEventArgs e)
    {
      NavigateToHtmlFile(e.Value);
    }
    void NavigateToHtmlFile(string htmlFile)
    {
      Uri uri = new Uri(htmlFile);      
      string html = File.ReadAllText(uri.LocalPath);
      browser.NavigateToString(html);
    }   

and that makes an attempt to read the HTML file that is being navigated to and push that into the WebBrowser control via NavigateToString() once again.

That seemed to “more or less” work except the next obvious problem is that the Back/Forward navigation menu doesn’t work properly because the browser is being told to NavigateToString() and doesn’t have the notion of a navigation stack around that.

It probably wouldn’t be too hard to add Back/Forward navigation buttons from Silverlight and maintain your own navigation stack as the Silverlight code is taking control of all the navigation here.

Where I’ve ended up then is that to display offline HTML content you’d need to take a bunch of steps;

  1. Packaging HTML/JS/CSS/IMG content for download.
  2. Building an elevated trust application that downloaded the packaged HTML and dropped it into the filesystem.
  3. Altering the registry to at least temporarily change the features that the sllauncher.exe process is allowed to use while hosting HTML such that it can load images and script from the filesystem.
  4. Modifying the HTML such that it sets a <base> location to point to the filesystem and such that it runs that little piece of JavaScript on document load.
    1. this might be done by statically altering the HTML content up front
    2. this might be done by dynamically altering the HTML content as it’s loaded
  5. Possibly building your own navigation stack handling.

That’s not an insignificant piece of work and I’m not so sure how I’d go about 4.2 in a robust way as you’d ideally have a way of parsing the HTML DOM to do this.

But that’s the best I’ve got to around this idea of using the WebBrowser control to display offline content from the filesystem – I’d be keen to know whether others have managed to take this further and whether there’s a different approach that you might take here and whether there’s something I’m missing.

Update (22/06/2011) – What About the Mac?

After I published this post I quite rightly got some questions along the lines of “ok, so even if I went with that scheme, would any of it work on the Mac?”.

I went off and did a bit of basic testing on my iMac and as far as I can tell the scheme that I outlined above works as much on OSX as it does on Windows. That is, I tested;

  1. Loading HTML content from the filesystem and dropping it into the WebBrowser via NavigateToString() seems to work fine.
  2. Having that HTML content pick up CSS, JavaScript and resolve images via the <base> element seems to work fine.
  3. Using that bit of jQuery that I used in order to cause hyperlinks to cause a ScriptNotify inside of the Silverlight code so that it can then load up the next HTML file seemed to work fine.

I’ll try and put together a simple sample that demonstrates some of this and works on both Windows and the Mac.

The Airspace Issue

The other aspect of the WebBrowser control that seems to bite people is the “airspace issue”.

If you look at a Silverlight out-of-browser application using something like Spy++ then you’ll see that it looks something like this;

image

and this is an application that doesn’t use the WebBrowser control at this point but unlike,say, calculator;

image

you’ll notice that Silverlight content is all displayed in a single window where Silverlight takes over the drawing and event handling of everything itself rather than using the regular Windows common controls.

That is – in the Silverlight content, there are no separate Windows windows for the TextBox or the Button whereas in the Calculator those buttons really are Windows windows.

That’s true until you bring in the WebBrowser control which isn’t a real Silverlight control but is, instead, a wrapper around the Windows HTML rendering that lives in mshtml.dll and can either be re-used “raw” or by hosting another web browser control (ShDocVw).

This isn’t exactly surprising – you’d not expect the Silverlight team to write a whole HTML renderer and JavaScript stack when they already exist and if you’ve been around the C/VB sides of the platform you’ll have met these controls before.

Once you bring in the WebBrowser control you’ll notice that it brings in its own Windows window;

image

and the “problem” is that whatever goes on inside of that window is not really integrated with what goes on in the rest of your Silverlight content.

  • For example – if you try and set the Opacity of a WebBrowser control then you’ll notice that it is ignored (NB: you might be able to use WebBrowserBrush to achieve what you want in certain circumstances).
  • For example – if you handle MouseMove events on the UserControl that hosts your WebBrowser then you’ll find that the WebBrowser control swallows those events and does not route them to your handler.
  • For example – if you right mouse on the WebBrowser control then you get the control’s right mouse menu and not the Silverlight right mouse menu (which you might have customised).
  • For example – if you create a WriteableBitmap and put Silverlight content into it that contains the WebBrowser control then you’ll find that the control does not render into that bitmap as other controls do.
  • For example – if you wrap a ScrollViewer around a WebBrowser control then it won’t have the effect that you were hoping it would have and the control will continue to render its own Scrollbars and the Silverlight ScrollViewer will be pretty useless.

and so the WebBrowser within Silverlight represents a different kind of content that doesn’t play very well with the surrounding content and if it doesn’t go far enough to do what you want, you’re likely to be a bit stuck as it’s an all/nothing kind of solution.

I don’t think that the Silverlight airspace limitations are radically different from the WPF ones and so this guide might help here and I’m not aware of ways in which you can modify this behaviour.

Wrapping Up

The WebBrowser control needs a bit of care. If you’re using it to display HTML from the web and you can live with the airspace issues (or mitigate them via the WebBrowserBrush) then you’re in business.

If you need to do something with offline content then it gets complicated quite quickly because the Silverlight control lacks the Base property of the WP7 version of the control and the best means I’ve presented here involves quite a lot of steps to reach what is probably only a partial solution via the NavigateToString() method.

Thoughts?

If you’ve done work around the WebBrowser control with offline HTML then no doubt you’ve solved some of these problems yourself so feel free to comment below or drop me a line and I’ll either update this post or write a new one in the future that incorporates your suggestions.