A Second Experiment with the Visual Layer

Posted on by

Following on from this post, I want to do some more experiments with the new Visual Layer offered by Windows.UI.Composition and I thought that in this post I’d play with rotating some rectangles around on the screen to see how that compares between what I’d do in XAML and what I might do with the Visual Layer.

To that end, I made a simple, blank UWP project in Visual Studio and made a UI with a Canvas on it;

<Page
  x:Class="App287.MainPage"
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:local="using:App287"
  xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
  xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
  mc:Ignorable="d">

  <Grid
    Background="red">
    <Viewbox>
      <TextBlock
        x:Name="txtCount" />
    </Viewbox>
    <Canvas
      Background="Transparent" 
      PointerReleased="OnPointerReleased"
      x:Name="drawCanvas"/>
  </Grid>
</Page>

and I came up with a simple interface to represent a component that can draw a spinning rectangle onto the screen at a given co-ordinate and with a given size. The interface doesn’t quite describe all of what I’ve just written but it’s enough to let me implement the functionality in XAML and with the Visual Layer;

  using Windows.UI;
  using Windows.UI.Xaml.Controls;

  interface IDrawCanvasRectangles
  {
    Color Color { get; set; }
    Canvas DrawCanvas { get; set; }
    void DrawRectangle(int x, int y, int width, int height);
    void ClearRectangles();
  }

and I implemented pieces of this in a base class;

  using System;
  using Windows.UI;
  using Windows.UI.Xaml.Controls;

  abstract class RectangleDrawer : IDrawCanvasRectangles
  {
    public Color Color { get; set; }
    public Canvas DrawCanvas
    {
      get {
        return (this.drawCanvas);
      }
      set
      {
        if (this.drawCanvas != value)
        {
          this.OnDrawCanvasChanging();
          this.drawCanvas = value;
          this.OnDrawCanvasChanged();
        }
      }
    }
    virtual protected void OnDrawCanvasChanging()
    {
    }
    virtual protected void OnDrawCanvasChanged()
    {
    }
    public abstract void ClearRectangles();
    public abstract void DrawRectangle(int x, int y, int width, int height);

    Canvas drawCanvas;
  }

and then wrote some simple code in my code-behind file which handles the PointerReleased button on the Canvas and draws an increasing number of spinning rectangles starting at 1 and ramping up to 4, 16, 256, etc.. It also uses a DispatcherTimer to update a counter 100 times per second to give me some measure of what my XAML UI thread is doing;

 using System;
  using Windows.UI;
  using Windows.UI.Xaml;
  using Windows.UI.Xaml.Controls;
  using Windows.UI.Xaml.Input;

  public sealed partial class MainPage : Page
  {
    public MainPage()
    {
      this.InitializeComponent();

      this.drawer = new XamlRectangleDrawer();
      this.drawer.Color = Colors.White;
      this.drawer.DrawCanvas = this.drawCanvas;

      this.timer = new DispatcherTimer();
      this.timer.Interval = TimeSpan.FromMilliseconds(10);
      this.timer.Tick += OnTimerTick;
      this.timer.Start();
    }
    void OnTimerTick(object sender, object e)
    {
      this.txtCount.Text = (++this.ticks).ToString();
    }
    void OnPointerReleased(object sender, PointerRoutedEventArgs e)
    {
      this.drawer.ClearRectangles();

      var sidelength = Math.Sqrt(this.rectangleCount);

      var fullWidth = this.drawCanvas.ActualWidth /
        sidelength;

      var fullHeight = this.drawCanvas.ActualHeight /
        sidelength;

      var horizontalSpacing = 0.25 * fullWidth;
      var verticalSpacing = 0.25 * fullWidth;

      for (int i = 0; i < sidelength; i++)
      {
        for (int j = 0; j < sidelength; j++)
        {
          this.drawer.DrawRectangle(
            (int)((i * fullWidth) + (horizontalSpacing / 2)),
            (int)((j * fullHeight) + (verticalSpacing / 2)),
            (int)(fullWidth * 0.75),
            (int)(fullHeight * 0.75));
        }
      }
      this.rectangleCount *= 4;
    }
    DispatcherTimer timer;
    long ticks;
    IDrawCanvasRectangles drawer;
    int rectangleCount = 1;
  }

you can see that the code above is using a XamlRectangleDrawer which I derived from my base class to do the XAML drawing;

using System;
  using Windows.Foundation;
  using Windows.UI;
  using Windows.UI.Xaml.Controls;
  using Windows.UI.Xaml.Media;
  using Windows.UI.Xaml.Media.Animation;
  using Windows.UI.Xaml.Shapes;

  class XamlRectangleDrawer : RectangleDrawer
  {
    public override void ClearRectangles()
    {
      this.DrawCanvas.Children.Clear();
    }
    public override void DrawRectangle(int x, int y, int width, int height)
    {
      if (this.FillBrush == null)
      {
        this.FillBrush = new SolidColorBrush(base.Color);
      }
      Rectangle rectangle = new Rectangle()
      {
        Width = width,
        Height = height,
        Fill = FillBrush
      };
      Canvas.SetLeft(rectangle, x);
      Canvas.SetTop(rectangle, y);

      AddSpinAnimationXaml(rectangle);

      this.DrawCanvas.Children.Add(rectangle);
    }
    static void AddSpinAnimationXaml(Rectangle rectangle)
    {
      RotateTransform transform = new RotateTransform();
      rectangle.RenderTransform = transform;
      rectangle.RenderTransformOrigin = new Point(0.5d, 0.5d);

      DoubleAnimation angleAnimation =
        new DoubleAnimation()
        {
          From = 0.0d,
          To = 360
        };

      angleAnimation.EasingFunction = new SineEase();

      Storyboard.SetTarget(angleAnimation, transform);
      Storyboard.SetTargetProperty(angleAnimation, "Angle");

      DoubleAnimation opacityAnimation =
        new DoubleAnimation()
        {
          From = 0.0d,
          To = 1.0d
        };

      opacityAnimation.EasingFunction = new SineEase();

      Storyboard.SetTarget(opacityAnimation, rectangle);
      Storyboard.SetTargetProperty(opacityAnimation, "Opacity");

#if COLOUR_ANIMATIONS
      ColorAnimation colorAnimation =
        new ColorAnimation()
        {
          From = Color.FromArgb(0x55, 0x00, 0x00, 0x00),
          To = Color.FromArgb(0x55, 0xFF, 0xFF, 0xFF)
        };
      Storyboard.SetTarget(colorAnimation, rectangle);
      Storyboard.SetTargetProperty(colorAnimation, "(Shape.Fill).(SolidColorBrush.Color)");
#endif // COLOUR_ANIMATIONS

      Storyboard storyBoard = new Storyboard();
      storyBoard.Duration = TimeSpan.FromSeconds(1);
      storyBoard.RepeatBehavior = RepeatBehavior.Forever;

      storyBoard.Children.Add(angleAnimation);
      storyBoard.Children.Add(opacityAnimation);

#if COLOUR_ANIMATIONS
    storyBoard.Children.Add(colorAnimation);
#endif // COLOUR_ANIMATIONS

      storyBoard.Begin();

    }
    SolidColorBrush FillBrush;
  }

and I also derived a second class from my base class to draw rectangles using the Visual Layer (i.e. Windows.UI.Composition) and to hook that into the Canvas that comes from the XAML world;

 using System;
  using System.Numerics;
  using Windows.UI.Composition;
  using Windows.UI.Xaml.Hosting;

  class VisualRectangleDrawer : RectangleDrawer
  {
    protected override void OnDrawCanvasChanging()
    {
      base.OnDrawCanvasChanging();

      if (base.DrawCanvas != null)
      {
        ElementCompositionPreview.SetElementChildVisual(base.DrawCanvas, null);
      }
      this.containerVisual?.Dispose();
      this.containerVisual = null;

      this.rotationAnimation?.Dispose();
      this.rotationAnimation = null;

      this.brush?.Dispose();
      this.brush = null;

      this.compositor = null;

    }
    protected override void OnDrawCanvasChanged()
    {
      if (base.DrawCanvas != null)
      {
        var canvasVisual = ElementCompositionPreview.GetElementVisual(
          base.DrawCanvas);

        this.compositor = canvasVisual.Compositor;

        this.containerVisual = this.compositor.CreateContainerVisual();

        this.brush = this.compositor.CreateColorBrush(base.Color);

        // NB: I wanted to animate the colour here but, to date, I do not seem to be
        // able to get colour animations to work in the composition bits and I can't
        // find anything on the web that uses them
#if COLOUR_ANIMATIONS
        this.colorAnimation = this.compositor.CreateColorKeyFrameAnimation();
        this.colorAnimation.InsertKeyFrame(0.0f, Color.FromArgb(0x55, 0x00, 0x00, 0x00));
        this.colorAnimation.InsertKeyFrame(1.0f, Color.FromArgb(0x55, 0xFF, 0xFF, 0xFF));
        this.colorAnimation.Duration = TimeSpan.FromSeconds(1);
        this.colorAnimation.IterationBehavior = AnimationIterationBehavior.Forever;

        this.brush.StartAnimation("Color", this.colorAnimation);

#endif // COLOUR_ANIMATIONS

        this.rotationAnimation = this.compositor.CreateScalarKeyFrameAnimation();
        this.rotationAnimation.InsertKeyFrame(0.0f, 0.0f);
        this.rotationAnimation.InsertKeyFrame(1.0f, 360.0f);
        this.rotationAnimation.Duration = TimeSpan.FromSeconds(1);
        this.rotationAnimation.IterationBehavior = AnimationIterationBehavior.Forever;

        this.opacityAnimation = this.compositor.CreateScalarKeyFrameAnimation();
        this.opacityAnimation.InsertKeyFrame(0.0f, 0.0f);
        this.opacityAnimation.InsertKeyFrame(1.0f, 1.0f);
        this.opacityAnimation.Duration = TimeSpan.FromSeconds(1);
        this.opacityAnimation.IterationBehavior = AnimationIterationBehavior.Forever;

        ElementCompositionPreview.SetElementChildVisual(this.DrawCanvas,
          this.containerVisual);
      }
    }
    public override void ClearRectangles()
    {
      this.containerVisual?.Children.RemoveAll();
    }
    public override void DrawRectangle(int x, int y, int width, int height)
    {
      var rectangleVisual = this.compositor.CreateSpriteVisual();
      rectangleVisual.Offset = new Vector3(x, y, 0);
      rectangleVisual.Size = new Vector2(width, height);
      rectangleVisual.CenterPoint = new Vector3((width / 2), (height / 2), 0);
      rectangleVisual.Brush = this.brush;
      rectangleVisual.StartAnimation("RotationAngleInDegrees", this.rotationAnimation);
      rectangleVisual.StartAnimation("Opacity", this.opacityAnimation);
      containerVisual.Children.InsertAtBottom(rectangleVisual);
    }
#if COLOUR_ANIMATIONS
    ColorKeyFrameAnimation colorAnimation;
#endif // COLOUR_ANIMATIONS
    ScalarKeyFrameAnimation opacityAnimation;
    ScalarKeyFrameAnimation rotationAnimation;
    Compositor compositor;
    ContainerVisual containerVisual;
    CompositionColorBrush brush;
  }

What’s with the Conditional Code?

Those 2 implementations are meant to be ‘more or less’ equivalent in that they both draw rectangles that animate their rotation from 0 to 360 degrees over a 1 second period and their opacity from 0 to 1 over the same period.

I also wanted to animate the colour of the rectangles but I couldn’t get my animations to work in the Visual Layer and so I need to return to that and fix it up at a later point but, for now, the sketched code that I was trying to use is conditionally compiled out.

Comparing the Results

I need to flag that I’m being pretty unscientific about comparing the XAML rendering with the Visual Layer rendering – I’m not stating with a clean PC, clean cache, minimal set of services running and I’m not doing detailed recordings of performance monitor or similar.

I just did the most basic comparison and I started off by making sure that my code was using my XamlRectangleDrawer and I ran it up on my PC (Surface Pro 3, Core i7 @ 1.70GHz, 8GB of RAM) and let it draw 1 rectangle;

image

and that was all going very nicely so I ramped it up a little to maybe 256 rectangles (these are animating their rotation and opacity which, naturally, you’re not going to get from a static image here);

image

Now, as far as I understand those performance counters in the top left of the app window what that’s showing me is that I’m getting around 30 frames per second and that drawing a frame is taking my UI thread around 3-4 milliseconds.

I thought I’d ramp up to 1024 rectangles;

image

and then on to 4096;

image

and, at this point, I’m getting around 15 frames per second and they seem to be taking around 25ms to render but it’s all kind of ‘still working’ Smile

When I tapped again to take this up to 16384 I have to confess that the PC exploded Winking smile Ok, it didn’t. But it became mostly unusable and to the point where it was actually quite hard to kill the application and get my PC back. 16384 seemed to be the point where the XAML rendering here ‘topped out’ for this PC.

What about the Visual Layer renderer? I changed the code to create one of those and got it drawing 4096 rectangles;

image

Now, there’s a smoothness to what’s being displayed here that’s hard to describe but I can definitely feel a difference between what’s being drawn here and the XAML drawing although it might be an easing animation that’s fooling me Smile

I ramped that up to 16384;

image

and the numbers seem to suggest that this is still doing 30fps with a 1-2ms render time but, naturally, these counters reflect what the UI thread is doing and I’m only asking that to update a textblock on the screen on a 10ms frequency.

The work is being done by the composition engine and it displays its counters in the top right of the screen so if we add them together;

image

I’m not sure how to screenshot them! but my app says it’s doing 30fps with 1-2ms to draw a frame and the composition engine says it’s doing 60fps with about 15ms to draw a frame.

Ramping up to 65536 rectangles…things get a little nuts;

image

but my app is still kicking out 30fps and the composition engine says that each frame is taking around 60ms to render.

Time to Try a Phone

Having tried the PC, I thought I’d try a phone and see what happens there and so I ran the same example up on a Lumia 920 running Windows 10 Mobile Preview to see how the 2 variants worked there.

Here’s the XAML variant running on my phone – note that I’m not at all sure about the impact of using the ‘project my screen’ app so I made sure that I wasn’t running it other than for this screenshot;

image

What I saw here was that the XAML variant ran something like this;

64 rectangles: 18fps, 20ms per frame

256 rectangles: 6fps, 60-100ms per frame

1024 rectangles: 1fps, 200+ms per frame

the render times started to become quite variable on the phone for this example beyond the 64 rectanlge mark.

Switching to the Visual Layer variant gave me better results, running something like this;

64 rectangles: 30fps, 8ms per frame (System: 60fps, 6ms)

256 rectangles: 26fps, 10ms per frame (System:60fps, 13ms)

1024 rectangles: 30fps, 10ms per frame (System: 60fps, 45ms)

4096 rectangles: 15fps, 10ms per frame (System: 60fps, 170ms)

and it’s perhaps worth saying that at 4096 rectangles the UI thread on the phone was clearly struggling to keep that ticking counter updated – there were definite pauses in there.

Anything to be drawn from this quick, hacky experiment?

Ok, so this was a slightly crazy experiment and I didn’t take any steps to do proper analysis or to try and optimise my XAML rendering here but it reinforced my expectation that using the visual layer like this gave me ‘better’ results in the sense that I can push more through it and, essentially I feel like I’m switching between;

  1. The XAML implementation where I’m asking the UI thread to do a lot of work before we hand things off to the compositor.
  2. The Visual Layer implementation where I do a lot less work on the UI thread and hand a bunch of the animation and drawing over to the compositor directly.

I’m still experimenting with the Visual Layer at this point but I feel I’m getting more of a ‘grip’ on how I do things with it and what it might do for me in a real-world app.

Want the code? Here for download.