“Create a Whimsical Animated Silverlight Background”

This falls into the category of “just for fun” Smile

For a little while now I’ve been dropping small Silverlight tutorials onto the ActiveTuts site which is, primarily, geared towards Flash development.

image

The ActiveTuts guys have been a real pleasure to work with, have a very large following and are really good at making your work look professional Smile At this point I have 12 tutorials up on the site and there are some more in the publishing pipeline.

image

If you’re a regular reader of this blog, these videos might be too “introductory” for you but while I was browsing the ActiveTuts+ site the other day I came across a really nice little tutorial for Flash;

image

which I really liked – it’s originally from 2009 and it’s simple enough that a non-Flash person like me could follow along with it and it was inspirational in the sense that it made me wonder whether I could follow similar steps to create a Silverlight version of the same effect and how well/badly that would work out.

And so that’s what I did…

Silverlight Version of the Tutorial

I made a new project in Visual Studio;

image

and set up my main page to host a Canvas at 640×480 and set the background to the same gradient used in the Flash example;

image

from a XAML perspective this is;

<UserControl
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:local="clr-namespace:SilverlightApplication4"
  xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
  xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
  mc:Ignorable="d"
  x:Class="SilverlightApplication4.MainPage">
  <Canvas
    x:Name="LayoutRoot"
    Width="640"
    Height="480">
    <Canvas.Background>
      <RadialGradientBrush>
        <GradientStop
          Color="#FF1F63B4"
          Offset="1" />
        <GradientStop
          Color="#FF02C7FB" />
      </RadialGradientBrush>
    </Canvas.Background>
  </Canvas>
</UserControl>

and then figured that I would try and encapsulate as much of my drawing code into a UserControl so I added a new user control into the project and then drew an Ellipse onto my control before editing its XAML to add a few bits and pieces that Visual Studio can’t easily add;

image

and so the definition of my control ends up being;

<UserControl
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
  xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
  mc:Ignorable="d"
  x:Class="SilverlightApplication4.UserControl1"
  d:DesignWidth="96"
  d:DesignHeight="96">
  <UserControl.Resources>
    <Storyboard
      x:Name="movement"
      Completed="OnMovementCompleted">
      <DoubleAnimation
        x:Name="animationX"
        From="0"
        To="0"
        Storyboard.TargetProperty="(UIElement.RenderTransform).(CompositeTransform.TranslateX)"
        Storyboard.TargetName="ellipse"
        d:IsOptimized="True" />
      <DoubleAnimation
        x:Name="animationY"
        From="0"
        To="0"
        Storyboard.TargetProperty="(UIElement.RenderTransform).(CompositeTransform.TranslateY)"
        Storyboard.TargetName="ellipse"
        d:IsOptimized="True" />
    </Storyboard>
  </UserControl.Resources>
  <Ellipse
    x:Name="ellipse"
    Fill="#FFF4F4F5"
    Width="96"
    Height="96"
    RenderTransformOrigin="0.5,0.5">
    <Ellipse.RenderTransform>
      <CompositeTransform
        x:Name="transform"
        ScaleX="1"
        ScaleY="1"/>
    </Ellipse.RenderTransform>
    <Ellipse.Effect>
      <BlurEffect
        x:Name="effect" />
    </Ellipse.Effect>
  </Ellipse>
</UserControl>

This is more declarative than the Flash version in that I’m creating a control that contains an Ellipse and the Ellipse is already pretty much set up to accept blur, scale values and it already has a Storyboard set up to animate the Ellipse from point A to point B.

Note that the Ellipse, CompositeTransform, Effect and my 2 DoubleAnimations are all named so that I can easily code against them and that lets me write code which will set up various values to random numbers when the control is first used. Here’s the code that forms the rest of the control;

namespace SilverlightApplication4
{
  using System;
  using System.Windows;
  using System.Windows.Controls;

	public partial class UserControl1 : UserControl
	{
    double parentWidth;
    double parentHeight;

		public UserControl1(double parentWidth, double parentHeight)
		{
			// Required to initialize variables
			InitializeComponent();

      this.parentWidth = parentWidth;
      this.parentHeight = parentHeight;
			
			this.Loaded += (s,e) =>
			{
        RandomiseAndStart();
			};
		}
    void RandomiseAndStart()
    {
      // Randomise our opacity
      this.ellipse.Opacity = Randomise(0.2, 0.8);

      // Randomise our blur effect radius
      this.effect.Radius = Randomise(20.0, 30.0);

      // Randomise our size
      double scale = Randomise(0.2, 1.0);
      this.transform.ScaleX = this.transform.ScaleY = scale;

      // Randomise the duration of our animations
      this.animationX.Duration = this.animationY.Duration =
        new Duration(new TimeSpan(0, 0, (int)Randomise(3, 10)));

      // Randomise our start and end positions on the X axis
      this.animationX.From = Randomise(0,
        parentWidth - (scale * this.ellipse.Width));

      this.animationX.To = Randomise(0, 
        parentWidth - (scale * this.ellipse.Width));

      // And for the Y axis
      this.animationY.To = 
        0 -this.ellipse.Height - ((scale * this.ellipse.Height) / 2);

      this.animationY.From =
        this.parentHeight -
          ((this.ellipse.Height - (scale * this.ellipse.Height)) / 2);

      // And start the animation
      this.movement.Begin();
    }
    void OnMovementCompleted(object sender, EventArgs e)
    {
      this.movement.Stop();

      // Go again...
      RandomiseAndStart();
    }
		static UserControl1()
		{
			_random = new Random((int)DateTime.Now.Ticks);
		}
		static double Randomise(double lower, double higher)
		{
      return (lower + (_random.NextDouble() * (higher - lower)));
		}
    static Random _random;
	}
}

and so what’s this doing?

  1. We capture the width and height of our parent control when we are created.
  2. When we are loaded we;
    1. Randomise the opacity value of our ellipse between 0.2 and 0.8
    2. Randomise the blur radius on the blur effect placed onto the ellipse between 20 and 30
    3. Randomise the scale of our ellipse between 0.2 and 1.0.
    4. Randomise the duration of the animation that moves the ellipse so that it lasts between 3 and 10 seconds.
    5. Set up the start points (X,Y) and end points (X,Y) for the animation to move the ellipse
    6. Start the animation.
  3. When the animation completes, we randomise our values again and re-run the animation.

With that all encapsulated into a control with a mixture of declarative behaviour and some code to set up the initial random values, I just need the main page of my application to contain some code to create a set of these controls when we first run up and add them onto the parent Canvas;

	public partial class MainPage : UserControl
	{
		public MainPage()
		{
			// Required to initialize variables
			InitializeComponent();

      this.Loaded += (s, e) =>
        {
          for (int i = 0; i < 30; i++)
          {
            UserControl1 uc = new UserControl1(
              this.LayoutRoot.Width, this.LayoutRoot.Height);

            this.LayoutRoot.Children.Add(uc);
          }
        };
		}
	}

and that’s pretty much it. I get my nice effect and it all looks very pretty;

image

but there’s a little bit of “a problem” – here’s my CPU usage while running this version;

image

Ouch – that’s 2 very busy CPUs – not really likely to be acceptable. What to do?

Performance Optimisation 1 – Remove the Blur Effect

I suspected that the blur effect was hurting me. In Silverlight, effects like this are rendered in software rather than hardware ( if I remember correctly this is mainly for security reasons to avoid runaway effects from the internet going crazy on the GPU ) and so I imagined that the effect was expensive.

I took the effect away to see what pain it was causing me and saw;

image

That’s a significant “win” but the problem is that my background now looks like;

image

and I’ve lost a lot of the subtlety of the original. Not to worry – I figure that I can achieve the same thing that the Blur effect was giving me simply by using a multi-stepped gradient fill and randomising that fill in order to provide the randomised “blur” effect that I had previously. I replaced the solid white fill on the Ellipse with this gradient;

<RadialGradientBrush>
      <GradientStop
        Color="White"
        Offset="0" />
        <GradientStop
          x:Name="gradientStop"
          Color="White"
          Offset="0" />
        <GradientStop
        Color="#00FFFFFF"
        Offset="1" />
    </RadialGradientBrush>

and then made sure that my RandomiseAndStart function had some code to set this up to a random value;

      // Randomise our gradient offset
      this.gradientStop.Offset = Randomise(75.0, 95.0);

and that seems to give me a decent enough look-and-feel;

image

and runs at about 50% of my 2 CPUs. I moved from Task Manager at this point as I wanted a slightly more accurate breakdown focused directly on the iexplore.exe process itself;

image

which is still quite a lot of CPU but is an improvement over the original. Can it be improved?

Performance Optimisation 2 – GPU Acceleration

I figured I’d try my hand at switching on hardware acceleration for my background and so went to the hosting HTML page and switched the settings on;

image

the 2nd setting highlighted there is just a debug flag that lets me see what is not being GPU accelerated and when I run my UI I can see;

image

which suggests that the GPU isn’t involved here ( not that I’d expect it to be at this point ) and so I switched on GPU acceleration for my UserControl;

image

and I wasn’t at all sure that this would help me in this scenario but it does seem to give me a slightly better graph;

image

dropping to an average of about 40% rather than 53% on the previous graph.

But This is Still Way Too High Sad smile

I’m still at 40% whereas the original Flash version looks like this;

image

and so was running at around 7% and I’m still ticking away at 40%. This is not good Sad smile

Performance Optimisation 3 – Frame Rates

The next question has to be whether I’m running to stand still – i.e. am I working harder than I need to for this particular effect? I figured I’d switch on my frame rate counter and see how many frames the plug-in is rendering;

image

and what I see is;

image

Wow, 63 frames per second is probably a little more than I need for this particular animated background so I figure I’ll gate it down a little by setting a maxFrameRate on the plug-in;

image

and my perfmon trace of CPU utilisation is looking a lot more healthy;

image

but, still, this felt a little bit like cheating. I wonder if it’s that UserControl that’s hurting me?

Performance Optimisation 4 – Taking Away the UserControl

I wondered what overhead I was paying for making use of a UserControl that contained an Ellipse rather than just using an Ellipse and so I went down that route and removed my UserControl and just manually created Ellipses from code.

I wrote a second version of my main loop which currently created 30 UserControls and made it create 30 Ellipses directly;

	public MainPage()
		{
			// Required to initialize variables
			InitializeComponent();

      this.Loaded += (s, e) =>
        {
          for (int i = 0; i < 30; i++)
          {
#if UC
            UserControl1 uc = new UserControl1(
              this.LayoutRoot.Width, this.LayoutRoot.Height);

            this.LayoutRoot.Children.Add(uc);
#else
            Ellipse ellipse = CreateEllipse();            
            this.LayoutRoot.Children.Add(ellipse);
            RandomiseAndBegin(ellipse);
#endif
          }
        };
		}

with the routine CreateEllipse doing the work that I had previously left to the definition of the UserControl – i.e. setting up the Ellipse with a Fill, RenderTransform, Storyboard etc.

    Ellipse CreateEllipse()
    {
      Ellipse ellipse = new Ellipse();
      ellipse.CacheMode = new BitmapCache();

      ellipse.Width = 96;
      ellipse.Height = 96;

      CompositeTransform transform = new CompositeTransform();
      ellipse.RenderTransformOrigin = new Point(0.5, 0.5);
      ellipse.RenderTransform = transform;            

      RadialGradientBrush brush = new RadialGradientBrush();
      brush.GradientStops.Add(new GradientStop() { Color = Colors.White, Offset = 0 });
      brush.GradientStops.Add(new GradientStop() { Color = Colors.White, Offset = 0 });
      brush.GradientStops.Add(new GradientStop() { Color = Color.FromArgb(0, 0xFF, 0xFF, 0xFF), Offset = 1 });
      ellipse.Fill = brush;

      DoubleAnimation xAnim = new DoubleAnimation();
      Storyboard.SetTarget(xAnim, ellipse);
      Storyboard.SetTargetProperty(xAnim,
        new PropertyPath("(UIElement.RenderTransform).(CompositeTransform.TranslateX)"));

      DoubleAnimation yAnim = new DoubleAnimation();
      Storyboard.SetTarget(yAnim, ellipse);
      Storyboard.SetTargetProperty(yAnim,
        new PropertyPath("(UIElement.RenderTransform).(CompositeTransform.TranslateY)"));

      Storyboard sb = new Storyboard();
      sb.Children.Add(xAnim);
      sb.Children.Add(yAnim);
      ellipse.Resources.Add("sb", sb);

      sb.Completed += (s, e) =>
        {
          sb.Stop();
          RandomiseAndBegin(ellipse);
        };

      return (ellipse);
    }

and the routine RandomiseAndBegin doing the randomisation work and starting the animations for the first time;

    void RandomiseAndBegin(Ellipse e)
    {
      double scale = Utility.Randomise(0.2, 1.0);
      CompositeTransform transform = e.RenderTransform as CompositeTransform;
      transform.ScaleX = transform.ScaleY = scale;

      RadialGradientBrush brush = (RadialGradientBrush)e.Fill;
      brush.GradientStops[1].Offset = Utility.Randomise(0.5, 0.90);

      e.Opacity = Utility.Randomise(0.2, 0.8);

      Duration duration = new Duration(new TimeSpan(0, 0, (int)Utility.Randomise(3, 10)));
      Storyboard sb = e.Resources["sb"] as Storyboard;
      sb.Children[0].Duration = duration;
      sb.Children[1].Duration = duration;

      DoubleAnimation xAnim = (DoubleAnimation)sb.Children[0];
      xAnim.From =
        Utility.Randomise(0, this.LayoutRoot.Width - (scale * e.Width));
      xAnim.To =
        Utility.Randomise(0, this.LayoutRoot.Width - (scale * e.Width));

      DoubleAnimation yAnim = (DoubleAnimation)sb.Children[1];
      yAnim.From = 
        this.LayoutRoot.Height - ((e.Height - (scale * e.Height)) / 2);
      yAnim.To = 
        0 - e.Height - ((scale * e.Height) / 2);

      sb.Begin();
    }

and I took another look at this instance running

( I noticed while doing this that I’d previously been randomising my gradient offsets to be between 75 and 95 rather than 0.75 and 0.95 but I hadn’t noticed before Smile and it actually looked more like the original Flash version when configured that way );

image

Wow, that’s quite some difference Smile 

Now I’m averaging 0.9% CPU utilisation and I find that if I take away my artificial limit on the framerate then that does little to affect the performance I’m seeing

( by the way – turning off enableGPUAcceleration has a huge difference so I left that one switched on ).

I ramped up from 30 ellipses on the screen to 350 ellipses on the screen and my performance is still better than it was previously even with an order of magnitude more for the plug-in to draw;

image

although the effect gets a little messy;

image

Taking the number of Ellipse instances back down to 30, I wondered if I could now put my BlurEffect back and I found that I could do that without seeming to significantly alter the performance characteristics.

The Example

Here’s the example running in the page;

The Source

Here’s the source for download containing both versions ( via conditional compilation )

Browser <3 Plugin

I should stop reading blogs and Twitter Winking smile

I’ve seen a lot of commentary saying something along the lines of;

“if only ‘HTML5 ‘will come along and then we won’t need all these plugins”

and I find it quite an odd idea.

I suspect what it really means is something like;

“if only ‘HTML5’ will come along and then I won’t need this plugin to play this simple video or a bit of audio”

but who knows?

By the way, I use “HTML5” in quotes like that because I mean “HTML5 and associated standards” but it’s long-winded to write that out every time.

I find it fascinating that folks that seem to like one piece of software (the browser) running on another piece of software (the operating system) then don’t like another piece of software (the plugin) for doing a similar thing. I understand why though.

It does feel to me like this has analogies to the OS (albeit imperfect ones);

Operating Systems

We run operating systems. Myself – I’m running Windows 7 (on a number of machines), Windows Phone 7, Mac OS X Snow Leopard and iOS.

Operating systems provide platforms for applications. New versions come along relatively infrequently. In most cases, the platform is controlled by a vendor like Microsoft or Apple.

No operating system can foresee every client-side need and so we add plugins. We might call them drivers or extensions or even applications.

It’s worth remembering that in the not-so-distant past these plugins included things like a network stack (which if I remember correctly came in as a standard part of the OS in Windows 3.11) and in more recent history have included things like Bluetooth support and multi-touch support.

Over time, functionality that at one time resided in an operating system plug-in becomes so generally accepted as useful that the operating system broadens out to include that functionality and we’ve seen that happen with all of the 3 examples I gave in italics.

Browsers

We run web browsers. Myself – I’m running IE9, FireFox 4.0b6, Chrome 5.0.x and a Safari version or two that I can’t remember as I’m writing this post on my Windows 7 laptop.

Browsers provide a platform for markup display and code execution and surface a subset of the underlying operating system’s functionality.

New versions come along relatively infrequently.

In most cases, the browser is controlled by a vendor like Microsoft, Apple, Mozilla, Google and, in all cases, the browser is implementing a whole tonne of web standards.

The markup/code that the browser runs falls into one of 2 categories;

Standards compliant.

Proprietary which I split out into 2 different categories;

  1. Using proprietary APIs or markup elements that are not standardised but are, instead, limited to one vendor’s particular browser.
  2. Using APIs or markup elements that are not implemented correctly by the browser and so behave differently in one or more vendor’s browsers.

No browser standard addresses every client-side need and so we add plugins. We tend to call them plugins. Smile

Interestingly, the content authored for the browser spans across to other browsers, other platforms and even other devices.

Plugins

We run plugins. Myself – I’m running Silverlight and Flash in the browsers that I run and that’s on both OS X and Windows but not, of course, on iOS.

The OS hosts the browser and the browser hosts the plugin. Whilst the plugin has access to the browser it, crucially, also has access to the underlying operating system.

That is, the plugin is not limited by the browser that is hosting it.

New versions tend to come along relatively frequently compared to browsers or operating systems and people adopt them quickly (e.g. 90%+ of all Silverlight users are already on Silverlight 4).

Thinking specifically about plugins like Silverlight/Flash – these plugins provide a proprietary platform to developers that is owned by a vendor (Microsoft and Adobe in this case).

Interestingly again, the content authored for these plugins spans browsers, platforms and onto other devices.

Applying this (Loosely) to Video

An existing feature like video provides a telling story. Here’s how it went or, at least, how I think it went – there’s a bit of artistic license being applied here;

  1. In the beginning, there was the operating system and the operating system could play videos.
  2. Along came the browser.
  3. People put video content onto the web.
  4. The browser could not play video so when a user clicked on a video, the browser handed the file over to the local media player on the operating system.
  5. Users saw this and demanded “video in the browser”.
  6. Developers writing markup and JavaScript cannot meet this need. They cannot reach beyond the browser to the underlying OS and its obvious ability to play videos. They are sandboxed and this is a good thing.
  7. Plugins sprung up to reach through the browser to the underlying OS and play video. The plugin is not sandboxed, it can reach out to the OS.
  8. Video became successful.
  9. Video playing becomes standardised in future browser versions by adding it to the capabilities inside the browser’s sandbox.

Imagine There’s No Plugins

What if we all had an “HTML5” browser and a new client-side requirement comes along. What can be done go get it to show up in the browser?

  1. It’s not possible. There are no requirements that “HTML5” does not address. Go rethink. You’re an idiot.
  2. Ignore it. Maybe it will go away.
  3. Implement the new requirement using JavaScript and existing HTML markup elements.
  4. Have the browser vendor implement it and provide a proprietary API to their implementation.
  5. Have the standards bodies convene, standardise the requirement and have browser vendors implement it.

I suspect only (3), (4) and (5) are really options.

(3) would be the ideal as no-one has to develop or install a new browser version.

However, because the browser doesn’t (quite rightly!) surface all of the OS functionality to someone writing HTML/JavaScript there’s a lot of things you can’t do without someone else doing some work in the browser layer itself to make sure your code is supported.

Multi-touch might be a good example. In order to get multi-touch events into your HTML/JS I’d say that you need a browser that catches those events from the OS and passes them through to HTML/JS. You need to go beyond the sandbox.

Applying this (Loosely) to Multi-Touch

If I want to pick up multi-touch events and do something reasonable with them inside the browser window today then what are my realistic choices?

4) Have the browser vendor implement it and provide a proprietary API to their implementation.

There are some touch event implementations out there – the window.ontouchstarted, ontouchmove etc events that some browsers surface today.

I struggled quite a lot to figure out which browsers do/don’t support these and on which operating systems. The best I could find was the Modernizer test page. From what I read;

  1. Safari supports these on iOS but I’m not sure whether it does on OS X.
  2. As far as I know (from here) FireFox 4 on Windows 7 supports similar events.
  3. This suggests that Chrome should but that they don’t currently fire.
  4. IE doesn’t to the best of my knowledge.

So, you might be able to get away with this if you could mandate a (non IE) browser choice for your users. That might work out ok on iOS devices of course.

But it’s clear that all these browser vendors implementing a feature in a slightly different way is not really great.

    5) Have the standards bodies convene, standardise the requirement and have the browser vendors implement it.

    I think the W3C is working on this under the “Web Events” banner and is looking to make a recommendation around August 2012 in this area.

    This would be the right way to go in the long term but it’s 2010 and developers are already doing a lot of multi-touch work on all kinds of different devices and operating systems.

    So, what could I do today?

    1. Constrain my users to particular browsers on particular devices (not necessarily unrealistic depending on the devices they use) and go with the proprietary APIs waiting for standards to catch up.
    2. Use a plugin.
      1. Silverlight has support for multi-touch today and it has that support in IE6, 7, 8, 9 and FireFox, Safari and Chrome. What it doesn’t have is that support on iOS and Android. As far as I know, Flash also supports multi-touch today.

    Is This Just About Video and Multi-Touch?

    No, that was just an example.

    I think this is more general than video or multi-touch.  Another example might be 3D.

    It’d be a surprise if 3D didn’t show up more on the web in the future and there’s efforts out there to get X3D into HTML at some point but I’m not too clear where that’s up to right now although there are some implementations of WebGL out there.

    I’d imagine that this is going to show up broadly in plugins first. Adobe announced their “Molehill” APIs for Flash just the other week and are working away at that.

    But, again, that’s just another specific example.

    The Browser has a Plugin Shaped Hole

    My general point is that I think the browser needs an extensibility model to enable people to innovate around browser-based content and, right now, the only model that’s there which lets you take advantage of capabilities not already present in the browser is the plugin model.

    Or did I miss the point?