Intro, History, Blurb
With the 1511 release of Windows, I noticed that one of the refreshed areas from a developer’s perspective was that of the ‘composition’ APIs that live in;
which I think are known by the nicer name of the ‘Visual Layer’ and, for me, these first showed up at //Build 2015 and there’s a session about them under the banner of the ‘New Visual Layer’;
(that’s quite some title!) and I watched it at the time but, in all honesty, I didn’t really ‘get it’ and the APIs were in a preview form so it felt like I didn’t need to ‘get it’ completely just at that point.
With the recent updates that come through in Windows 10 update 1511 I’m starting to feel that I really should ‘get it’ and so I thought that I’d start to dig in.
This post is just some early steps – whether I’ll have fully ‘got it’ by the end of the post, I’m unsure 
For Universal Windows Apps, there seem to be a few pieces of technology that are fundamental in terms of presenting something onto the screen;
- I can use ‘XAML’ in the sense that I can use the classes within Windows.UI.Xaml that derive from something like UIElement and I can put them into panels and containers and get them onto the screen either through use of declarative XAML or imperative code.
- In this world, I’m using retained graphics in that if I add an Ellipse to a UI then it stays there until I change/remove it, I don’t have to manually redraw it on each frame or similar.
- I can use Win2D to draw in the 2D world, that is – I can go and grab the win2d.uwp package and then it gives me access to elements like CanvasControl (and much more)
- This moves me into the realm of immediate mode graphics where I enter into a draw/clear loop via the Draw event on that control and I take entire responsibility for what goes on within that control.
- I can use DirectX APIs to draw in both the 2D and the 3D worlds via either native means (i.e. write some C++) or via interop means (i.e. maybe use SharpDX).
- If I’m using DirectX for 2D drawing – i.e. where X is [Direct2D, DirectWrite,DirectImage] then I can probably use Win2D as that brings those technologies into the UWP world already.
- I can use something that’s layered on top of one or more of these (e.g. Unity, MonoGame) to raise the abstraction level and/or maybe get some x-platform capability as well.
It’s interesting to think about where these technologies come from – i.e.
- XAML is part of the Universal Windows Platform and it’s a technology that’s natively implemented such that it can show up in the same way in both .NET applications and C++ applications without those C++ applications taking a dependency on .NET. Being part of the UWP means that this is available on all Windows 10 devices.
- Win2D is being built out in the open on GitHub and is delivered as a NuGet package.
- DirectX APIs are part of the UWP as well with some of the commonly used abstractions/wrappers for them (e.g. SharpDX, Unity, etc) coming from non-Microsoft sources that may or may not be open source.
I can also combine some or all of 1-4 above in order to allow the separate technologies to play to their strengths – e.g. I might have some Direct3D game sitting underneath a set of game controls provided by a XAML layer.
In doing those kinds of combinations, I don’t want to have separate ‘islands’ of rectangular content in an application (the old ‘airspace issue’) but, instead, I want to be able to layer content from one technology with content from another technology and do that in a performant way and for XAML that brings in classes like SwapChainPanel class (or SurfaceImageSource) which can integrate DirectX content into the XAML world.
In the XAML world, it’s possible to layer lots of UI elements on top of each other with transparency and then it’s also possible to have more layers coming from these other, non-XAML technologies and so it’s clearly necessary to have some component take responsibility for ‘compositing’ these layers together such that, ultimately, a decision is made about how to light up each pixel according to the composed layers that contribute to that pixel.
While that needs to happen at the application level, it also needs to happen at the desktop level when content from N applications might overlap at a single place on the desktop and this is done (since Windows Vista in 2006) by the Desktop Window Manager (DWM) which radically changed the way in which Windows applications draw in that from Vista onwards they draw off-screen and the DWM composites the results back onto the desktop.
There’s a developer API to get involved in that composition, it’s the DirectComposition (COM) API and, as the documentation link says;
“DirectComposition is a Windows component that enables high-performance bitmap composition with transforms, effects, and animations. Application developers can use the DirectComposition API to create visually engaging user interfaces that feature rich and fluid animated transitions from one visual to another. “
So, not only does this layer compose but it also can also apply transformations, effects and animations while doing that composition.
Digging In – Re-Reading Kenny Kerr’s Article from MSDN Magazine
That’s all (possibly incorrect!) history. Back here at the end of 2015, what does Windows.UI.Composition bring to the party?
The first thing to say is that these are part of the UWP – they’re not out there on GitHub or similar and you don’t bring them into your application via a NuGet package, they are just ‘there’ in the UWP.
I think a great place to start is a deep read of Kenny Kerr’s article;
I’ll admit that I flicked through this article quite a few weeks ago but it didn’t sink in for me although even on first reading you get that Windows.UI.Composition is a new, WinRT API which brings these capabilities of composition into the UWP but isn’t just a wrapper of the DirectComposition COM APIs – it provides its own, unified, retained graphics approach.
Recently, I’ve gone back and re-read Kenny’s article and delved off into the supporting articles and I’ve worked through it in Visual Studio – i.e. I followed Kenny’s path and I created a new, blank, UWP application with no XAML rendering in it and I used the Compositor to put Visuals on the screen and I started to get those core ideas of;
Compositor, Visual (plus ContainerVisual and SpriteVisual).
This was my ‘Hello World’ (from Kenny’s code) after making a blank project, removing App.xaml/.cs and MainPage.xaml/.cs and setting the entry point (via the manifest) to be App283.View in order that I could have a ‘zero XAML’ application and purely use the composition APIs to draw something;
namespace App283
{
using System.Numerics;
using Windows.ApplicationModel.Core;
using Windows.Foundation;
using Windows.UI;
using Windows.UI.Composition;
using Windows.UI.Core;
class View : IFrameworkView, IFrameworkViewSource
{
static void Main()
{
CoreApplication.Run(new View());
}
public IFrameworkView CreateView()
{
return this;
}
public void SetWindow(CoreWindow window)
{
this.compositor = new Compositor();
this.compositionTarget = this.compositor.CreateTargetForCurrentView();
// Make a visual which is a container to put other visuals into.
ContainerVisual container = this.compositor.CreateContainerVisual();
this.compositionTarget.Root = container;
// Make a visual which paints itself with a brush.
SpriteVisual visual = this.compositor.CreateSpriteVisual();
// Tell it where it is.
visual.Size = new Vector2(100, 100);
visual.Offset = new Vector3(10, 10, 0);
// Tell it how to paint itself
visual.Brush = this.compositor.CreateColorBrush(Colors.Red);
// Put it into the container.
container.Children.InsertAtTop(visual);
}
public void Run()
{
CoreWindow window = CoreWindow.GetForCurrentThread();
window.Activate();
window.Dispatcher.ProcessEvents(CoreProcessEventsOption.ProcessUntilQuit);
}
public void Initialize(CoreApplicationView applicationView)
{
}
public void Load(string entryPoint)
{
}
public void Uninitialize()
{
}
CompositionTarget compositionTarget;
Compositor compositor;
}
}
and, sure enough, I get my nice red rectangle on the UI;
and so I thought that I’d go crazy with it 
and tried to add in another rectangle by adding code after the creation of the first one;
visual = this.compositor.CreateSpriteVisual();
// Tell it where it is.
visual.Size = new Vector2(100, 100);
visual.Offset = new Vector3(20, 20, 1);
// Tell it how to paint itself
visual.Brush = this.compositor.CreateColorBrush(
Color.FromArgb(128, 0, 255, 0));
// Put it into the container.
container.Children.InsertAtTop(visual);
and it feels like I’m doing some ‘composition’ here by adding my red and green together;
Brushes? Surfaces? Devices?
For a while I got bogged down here in the description that says that we have;
- Visual – base object
- ContainerVisual – like Visual but with children
- SpriteVisual – like ContainerVisual but with a brush which renders pixels
So, SpriteVisual feels like its mostly where the action happens but what are these brushes? There are;
- CompositionColorBrush – a solid colour
- CompositionEffectBrush – a brush that applies an effect.
- CompositionSurfaceBrush – a brush that paints a visual with a composition surface ( ICompositionSurface ).
I’ve already tried a colour brush and I’m not ready to try an effect brush and so it feels like CompositionSurfaceBrush is where drawing of anything really starts but the docs left me puzzling about where I might get an ICompositionSurface from although they do tell me that it’s “only exposed to native code” so that was perhaps a hint.
After a little bit of poking around, I see that Win2D offers some experimental support here in Microsoft.Graphics.Canvas.UI.Composition.
I wondered whether that might be something that I could get working and so, tentatively, I tried it out (with help from the sample here) and after maybe 30 minutes or so I managed to draw a couple of rectangles to a Visual using a CompositionSurfaceBrush;
using Microsoft.Graphics.Canvas;
using Microsoft.Graphics.Canvas.UI.Composition;
using System.Numerics;
using Windows.ApplicationModel.Core;
using Windows.Foundation;
using Windows.Graphics.DirectX;
using Windows.Graphics.Display;
using Windows.UI;
using Windows.UI.Composition;
using Windows.UI.Core;
class View : IFrameworkView, IFrameworkViewSource
{
static void Main()
{
CoreApplication.Run(new View());
}
public IFrameworkView CreateView()
{
return this;
}
void GetCanvasAndGraphicsDevices()
{
// NB: I think I'm meant to handle the device lost event here.
// Get a canvas device.
this.canvasDevice = CanvasDevice.GetSharedDevice();
this.canvasDevice.DeviceLost += OnDeviceLost;
if (this.graphicsDevice == null)
{
this.graphicsDevice = CanvasComposition.CreateCompositionGraphicsDevice(
this.compositor, this.canvasDevice);
}
else
{
CanvasComposition.SetCanvasDevice(this.graphicsDevice, this.canvasDevice);
}
this.graphicsDevice.RenderingDeviceReplaced += OnRenderingDeviceReplaced;
}
void OnDeviceLost(CanvasDevice sender, object args)
{
sender.DeviceLost -= this.OnDeviceLost;
this.GetCanvasAndGraphicsDevices();
}
void OnRenderingDeviceReplaced(
CompositionGraphicsDevice sender, RenderingDeviceReplacedEventArgs args)
{
this.Redraw();
}
void Redraw()
{
using (var drawingSession = CanvasComposition.CreateDrawingSession(
this.drawingSurface))
{
drawingSession.FillRectangle(
new Rect(0, 0, 200, 200),
Colors.Red);
drawingSession.FillRectangle(
new Rect(200, 200, 200, 200),
Colors.Green);
}
}
public void SetWindow(CoreWindow window)
{
this.compositor = new Compositor();
this.GetCanvasAndGraphicsDevices();
// Make a visual.
SpriteVisual drawingVisual = this.compositor.CreateSpriteVisual();
drawingVisual.Size = new Vector2(400, 400);
// Create a drawing surface.
this.drawingSurface = graphicsDevice.CreateDrawingSurface(
new Size(400, 400),
DirectXPixelFormat.B8G8R8A8UIntNormalized,
DirectXAlphaMode.Premultiplied);
// Create a drawing surface brush.
CompositionSurfaceBrush brush = this.compositor.CreateSurfaceBrush(
this.drawingSurface);
drawingVisual.Brush = brush;
this.Redraw();
// Make a visual which is a container to put other visuals into.
ContainerVisual container = this.compositor.CreateContainerVisual();
container.Children.InsertAtTop(drawingVisual);
this.compositionTarget = this.compositor.CreateTargetForCurrentView();
this.compositionTarget.Root = container;
DisplayInformation.DisplayContentsInvalidated += OnDisplayContentsInvalidated;
}
void OnDisplayContentsInvalidated(DisplayInformation sender, object args)
{
// Apparently, this is needed...as it will raise the device lost event if
// necessary.
CanvasDevice.GetSharedDevice();
}
public void Run()
{
CoreWindow window = CoreWindow.GetForCurrentThread();
window.Activate();
window.Dispatcher.ProcessEvents(CoreProcessEventsOption.ProcessUntilQuit);
}
public void Initialize(CoreApplicationView applicationView)
{
}
public void Load(string entryPoint)
{
}
public void Uninitialize()
{
}
CompositionTarget compositionTarget;
Compositor compositor;
CanvasDevice canvasDevice;
CompositionGraphicsDevice graphicsDevice;
CompositionDrawingSurface drawingSurface;
}
as below;
Animations – Hello World
I moved on to follow Kenny’s lead in terms of adding some animation by changing my SetWindow method to be;
public void SetWindow(CoreWindow window)
{
this.compositor = new Compositor();
this.compositionTarget = this.compositor.CreateTargetForCurrentView();
// Make a visual which is a container to put other visuals into.
ContainerVisual container = this.compositor.CreateContainerVisual();
this.compositionTarget.Root = container;
// Make a visual which paints itself with a brush.
SpriteVisual visual = this.compositor.CreateSpriteVisual();
// Tell it where it is.
visual.Size = new Vector2(100, 100);
visual.Offset = new Vector3(10, 10, 0);
// Tell it how to paint itself
visual.Brush = this.compositor.CreateColorBrush(Colors.Red);
// Put it into the container.
container.Children.InsertAtTop(visual);
ScalarKeyFrameAnimation anim = this.compositor.CreateScalarKeyFrameAnimation();
anim.InsertKeyFrame(0.0f, 0.0f);
anim.InsertKeyFrame(1.0f, 360.0f);
anim.Duration = TimeSpan.FromSeconds(1);
anim.IterationBehavior = AnimationIterationBehavior.Forever;
visual.StartAnimation("RotationAngleInDegrees", anim);
}
and, sure enough, that gave me an animation around the origin of the element itself although I didn’t quite manage to combine the right set of properties to rotate it around a relative offset of 0.5,0.5 just yet, but I will do;
I read through Kenny’s explanation of ‘Expressions’ but I need to dig into that more in order to really get a grip as to what’s going on there.
XAML Integration Hello World
Kenny’s article finishes up by talking about XAML integration and so I made myself a second, blank, UWP app from the template, added a Button to the UI (called ‘myButton’) and then added a bit of code to rotate that button from the XAML world;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs e)
{
var xamlVisual = ElementCompositionPreview.GetElementVisual(this.myButton);
var compositor = xamlVisual.Compositor;
var animation = compositor.CreateScalarKeyFrameAnimation();
animation.Duration = TimeSpan.FromSeconds(1.0d);
animation.IterationBehavior = AnimationIterationBehavior.Forever;
animation.InsertKeyFrame(0.0f, 0.0f);
animation.InsertKeyFrame(1.0f, 360.0f);
xamlVisual.StartAnimation("RotationAngleInDegrees", animation);
}
}
This all ‘worked’ exactly as you’d expect it to including (as far as I can tell) the hit-testing of the button as it rotated.
It’s worth flagging that this type of animation is working ‘below’ the XAML level and so animating this button is much like applying a render transform and not like applying a WPF style layout transform so if I make this into 3 buttons on my screen;
<StackPanel VerticalAlignment="Center" HorizontalAlignment="Center">
<Button
FontSize="24"
Width="192"
Height="192"
Content="One" />
<Button
x:Name="myButton"
FontSize="24"
Width="192"
Height="192"
Content="Two" />
<Button
FontSize="24"
Width="192"
Height="192"
Content="Three" />
</StackPanel>
then the animation of the middle button has no effect on the positioning of the other two;
I found it quite ‘odd’ to be plugging in at this level below XAML and playing around with an animation like this although, clearly, in this case I could build these animations entirely in the XAML world.
What’s not so easy to do in the UWP XAML world is to use effects and the composition layer can do those too.
Time Out – A New Video Arrives
While this post was in draft form, a new video arrived onto Channel 9 with more info on the new composition layer. I’ve embedded the link below;
I watched this from start to finish and the guys do a great job of explaining things and I really liked their slide-show demo but I still didn’t feel that I’d got a complete grasp of the bits here and so I ploughed on with my own tinkering and continued to effects…
Hello World of Effects
Having applied a simple animation, I thought that I’d try my luck in applying an effect.
There’s an interesting set of dependencies when it comes to effects because the description of an effect is done by the IGraphicsEffect interface which lives in the UWP but then implementation pieces lie in Win2D and so to use effects you need to bring in the Win2D.uwp NuGet package.
There are lots of effects in Microsoft.Graphics.Canvas.Effects listed here but not all of those are applicable for composition and the [NoComposition] attribute on that page flags which ones they are and is used to make the shorter list of effects listed here. I can only guess that the difference between the lists is to do with this being either ongoing work or perhaps certain of these effects are not suitable for being implemented in the composition layer for one reason or another. While digging here, I noticed that Win2D has support for are custom shaders which don’t make it across into the set of effects supported for composition.
So, armed with a list of effects and a Visual to apply one or more of them to, how do I do that?
I went back to the code that I had for drawing 2 rectangles onto the screen and I added code to that SetWindow method as below;
public void SetWindow(CoreWindow window)
{
this.compositor = new Compositor();
this.GetCanvasAndGraphicsDevices();
// Make a visual.
SpriteVisual drawingVisual = this.compositor.CreateSpriteVisual();
drawingVisual.Size = new Vector2(400, 400);
// Create a drawing surface.
this.drawingSurface = graphicsDevice.CreateDrawingSurface(
new Size(400, 400),
DirectXPixelFormat.B8G8R8A8UIntNormalized,
DirectXAlphaMode.Premultiplied);
// Create a drawing surface brush.
CompositionSurfaceBrush drawingBrush = this.compositor.CreateSurfaceBrush(
this.drawingSurface);
// Create a parameter for the source of the effect - this will be our
// surface brush that we draw two rectangles into
CompositionEffectSourceParameter sourceParameter =
new CompositionEffectSourceParameter("input");
// Create a descripton of an effect.
SaturationEffect saturationEffect = new SaturationEffect()
{
Saturation = 0.1f,
Source = sourceParameter
};
// Create an effects factory.
CompositionEffectFactory effectFactory = this.compositor.CreateEffectFactory(saturationEffect);
// Create a brush that represents the effect.
CompositionEffectBrush effectBrush = effectFactory.CreateBrush();
// Tell the brush that its source is the original brush.
effectBrush.SetSourceParameter("input", drawingBrush);
// Tell the visual
drawingVisual.Brush = effectBrush;
// Make a visual which is a container to put other visuals into.
ContainerVisual container = this.compositor.CreateContainerVisual();
container.Children.InsertAtTop(drawingVisual);
this.compositionTarget = this.compositor.CreateTargetForCurrentView();
this.compositionTarget.Root = container;
this.Redraw();
DisplayInformation.DisplayContentsInvalidated += OnDisplayContentsInvalidated;
}
the essential change here is that on line 44 above I set the drawingVisual.Brush to draw with an effectBrush rather than with the drawingBrush which is what the original code did. Then between lines 18 and 41 I create that effectBrush telling it to use a saturation effect where the saturation is set to 0.1f and drawingBrush is used as the source of the effect.
This gives me a nice, faded output as you’d expect;
and I guess that I can then combine my “Hello World” of animations with my “Hello World” of effects and animate this value from 0.0 to 1.0 over time as in;
public void SetWindow(CoreWindow window)
{
this.compositor = new Compositor();
this.GetCanvasAndGraphicsDevices();
// Make a visual.
SpriteVisual drawingVisual = this.compositor.CreateSpriteVisual();
drawingVisual.Size = new Vector2(400, 400);
// Create a drawing surface.
this.drawingSurface = graphicsDevice.CreateDrawingSurface(
new Size(400, 400),
DirectXPixelFormat.B8G8R8A8UIntNormalized,
DirectXAlphaMode.Premultiplied);
// Create a drawing surface brush.
CompositionSurfaceBrush drawingBrush = this.compositor.CreateSurfaceBrush(
this.drawingSurface);
// Create a parameter for the source of the effect - this will be our
// surface brush that we draw two rectangles into
CompositionEffectSourceParameter sourceParameter =
new CompositionEffectSourceParameter("input");
// Create a descripton of an effect.
SaturationEffect saturationEffect = new SaturationEffect()
{
Name = "myEffect",
Saturation = 0.1f,
Source = sourceParameter
};
// Create an effects factory, specifying that I want to animate the
// Saturation property.
CompositionEffectFactory effectFactory =
this.compositor.CreateEffectFactory(
saturationEffect,
new[] { "myEffect.Saturation" });
// Create a brush that represents the effect.
CompositionEffectBrush effectBrush = effectFactory.CreateBrush();
// Tell the brush that its source is the original brush.
effectBrush.SetSourceParameter("input", drawingBrush);
// Tell the brush that we want to animate its Saturation property.
// NB: Not at all sure that I'm doing this right.
effectBrush.Properties.InsertScalar("myEffect.Saturation", 0.0f);
// Tell the visual
drawingVisual.Brush = effectBrush;
// Make an animation that animates the Saturation property of the
// effect brush.
ScalarKeyFrameAnimation anim = this.compositor.CreateScalarKeyFrameAnimation();
anim.InsertKeyFrame(0.0f, 0.0f);
anim.InsertKeyFrame(0.5f, 1.0f);
anim.InsertKeyFrame(1.0f, 0.0f);
anim.Duration = TimeSpan.FromSeconds(1);
anim.IterationBehavior = AnimationIterationBehavior.Forever;
effectBrush.StartAnimation("myEffect.Saturation", anim);
// Make a visual which is a container to put other visuals into.
ContainerVisual container = this.compositor.CreateContainerVisual();
container.Children.InsertAtTop(drawingVisual);
this.compositionTarget = this.compositor.CreateTargetForCurrentView();
this.compositionTarget.Root = container;
this.Redraw();
DisplayInformation.DisplayContentsInvalidated += OnDisplayContentsInvalidated;
}
where I fumbled quite badly over this code was in trying to figure out the strings that get passed into the APIs at lines 40, 50, 63 – I’m not at all sure how the property/naming system works but by flicking through samples I came up with something that seemed to do what I wanted it to do as per the screen capture below;
and so I kind of got something working but I wasn’t too sure about what I was doing
The last thing that I wanted to do was reconnect this back to the XAML world…
Adding Effects to XAML
How do I add effects into XAML? In the previous code, we gave an effect a ‘source parameter’ with a line of code like this one;
effectBrush.SetSourceParameter("input", drawingBrush);
and that second parameter is a CompositionBrush and so the question I guess becomes one of how I get a CompositionBrush from a XAML element because the brush I used in the previous example came from somewhere down in Win2D.
It’s easy enough to get a Visual from the XAML world using the methods on the ElementCompositionPreview class but the methods on that class look to be there to enable some specific scenarios;
and it feels like the scenario of painting a Visual with a brush that comes from the XAML world might not be one of them – I must admit that was a bit of a surprise as I would have thought that being able to easily add effects such as blur, shadow, etc. to XAML elements would be one of the first ‘easy wins’ that the composition framework might enable but it looks like that’s not yet the case.
What can be done? The only thought that I’ve had to date is that while it’s far from ideal, it’s possible to ask a portion of a XAML tree to render itself to a bitmap and it’s also possible to ask Win2D to render bitmaps to the screen.
With that in mind, I made a simple UI;
<Page
x:Class="App285.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App285"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:w2d="using:Microsoft.Graphics.Canvas.UI.Xaml"
mc:Ignorable="d">
<Grid
Background="White">
<w2d:CanvasControl
x:Name="canvasControl"
HorizontalAlignment="Left"
VerticalAlignment="Top"
Draw="OnDraw">
</w2d:CanvasControl>
<Grid
x:Name="myXamlGrid"
HorizontalAlignment="Center"
VerticalAlignment="Center">
<TextBlock
Text="Hello World"
FontSize="72"
FontFamily="Segoe UI Light" />
</Grid>
</Grid>
</Page>
and so I just have a CanvasControl from Win2D and I have a Grid with a TextBlock in it and the Grid is called myXamlGrid.
I then tried to write some code which would;
- Run a DispatcherTimer such that (approx) 60 times per second we would ask the myXamlGrid control to render itself as a bitmap using a RenderTargetBitmap.
- Run an OnDraw loop for the CanvasControl such that it would loop, picking up the latest published bitmap from (1) above and render it into the top left of the UI.
I ended up with this code below and, no doubt, it could do with being a bit more elegant but I’m just trying things out;
namespace App285
{
using Microsoft.Graphics.Canvas;
using Microsoft.Graphics.Canvas.UI.Xaml;
using System;
using System.Threading;
using Windows.Graphics.Imaging;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Media.Imaging;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.renderBitmap = new Lazy<RenderTargetBitmap>(
() =>
new RenderTargetBitmap()
);
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs e)
{
this.timer = new DispatcherTimer()
{
Interval = TimeSpan.FromMilliseconds(16)
};
this.timer.Tick += OnTimerTick;
this.timer.Start();
this.canvasControl.Width = this.myXamlGrid.ActualWidth;
this.canvasControl.Height = this.myXamlGrid.ActualHeight;
}
async void OnTimerTick(object sender, object e)
{
if (Interlocked.CompareExchange(ref this.bufferStatusFlag, WRITING, READY_TO_WRITE) == READY_TO_WRITE)
{
await this.renderBitmap.Value.RenderAsync(this.myXamlGrid);
var pixels = await renderBitmap.Value.GetPixelsAsync();
this.drawBitmap = SoftwareBitmap.CreateCopyFromBuffer(pixels,
BitmapPixelFormat.Bgra8,
this.renderBitmap.Value.PixelWidth,
this.renderBitmap.Value.PixelHeight,
BitmapAlphaMode.Premultiplied);
Interlocked.Exchange(ref this.bufferStatusFlag, READY_TO_READ);
}
}
void OnDraw(CanvasControl sender, CanvasDrawEventArgs args)
{
args.DrawingSession.Units = CanvasUnits.Pixels;
if (Interlocked.CompareExchange(
ref this.bufferStatusFlag, READING, READY_TO_READ) == READY_TO_READ)
{
this.canvasBitmap?.Dispose();
this.canvasBitmap = CanvasBitmap.CreateFromSoftwareBitmap(sender.Device,
this.drawBitmap);
args.DrawingSession.DrawImage(canvasBitmap);
this.drawBitmap.Dispose();
Interlocked.Exchange(ref this.bufferStatusFlag, READY_TO_WRITE);
}
else if (this.canvasBitmap != null)
{
args.DrawingSession.DrawImage(canvasBitmap);
}
sender.Invalidate();
}
CanvasBitmap canvasBitmap;
int bufferStatusFlag;
SoftwareBitmap drawBitmap;
Lazy<RenderTargetBitmap> renderBitmap;
DispatcherTimer timer;
static readonly int READY_TO_WRITE = 0;
static readonly int WRITING = 1;
static readonly int READY_TO_READ = 2;
static readonly int READING = 3;
}
}
and, sure, enough that gives me a replica of the myXamlGrid Grid in the top left corner of the screen that is being drawn with Win2D although I’m not sure that the sizing is working the way in which I expect;
but then I fumbled quite a lot in trying to figure out how to bring the composition APIs into the code above. Logically, what I thought I might do was;
- Create some kind of composition drawing surface or graphics device or similar.
- Plug that into the CanvasControl such that when it draws via its OnDraw methods it’s actually drawing to the composition layer.
- Plug an effect into the composition layer.
but I wasn’t sure about how to achieve (2).
The CanvasControl has a property called CustomDevice but it wasn’t clear to me whether there was any relationship between this property and a CompositionGraphicsDevice and I didn’t find one and I couldn’t find any use of the CanvasControl in the composition samples that I came across and I’d have to say that it felt like I was wandering into a valley somewhere between the composition APIs and the Win2D APIs and there didn’t seem to be a map to guide me.
So, I went back to the drawing board and took away the CanvasControl while preserving the code which rendered the XAML UI to a bitmap on a timer and trying to combine that with the code that I’d written previously in the non-XAML sample.
My UI just became this;
<Page
x:Class="App285.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App285"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:w2d="using:Microsoft.Graphics.Canvas.UI.Xaml"
mc:Ignorable="d">
<Grid
Background="White">
<Grid
x:Name="myHostGrid"
HorizontalAlignment="Left"
VerticalAlignment="Top" />
<Grid
x:Name="myXamlGrid"
HorizontalAlignment="Center"
VerticalAlignment="Center">
<StackPanel
Orientation="Horizontal">
<Ellipse
Fill="Red"
Width="100"
Height="100" />
<Rectangle
Fill="Green"
Width="100"
Height="100" />
</StackPanel>
</Grid>
</Grid>
</Page>
and my code became simpler in some ways;
using Microsoft.Graphics.Canvas;
using Microsoft.Graphics.Canvas.Effects;
using Microsoft.Graphics.Canvas.UI.Composition;
using System;
using System.Numerics;
using Windows.Foundation;
using Windows.Graphics.DirectX;
using Windows.Graphics.Display;
using Windows.Graphics.Imaging;
using Windows.UI.Composition;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Hosting;
using Windows.UI.Xaml.Media.Imaging;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.renderBitmap = new RenderTargetBitmap();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs e)
{
this.InitialiseCompositionElements();
this.timer = new DispatcherTimer()
{
Interval = TimeSpan.FromMilliseconds(16)
};
this.timer.Tick += OnTimerTick;
this.timer.Start();
}
async void OnTimerTick(object sender, object e)
{
// Not sure whether we will ever actually *be* busy, but just thinking we might
if (!this.busy)
{
this.busy = true;
// Render our XAML to a bitmap. This bitmap will be sized in real pixels rather
// than in effective pixels.
await this.renderBitmap.RenderAsync(this.myXamlGrid);
var pixels = await renderBitmap.GetPixelsAsync();
using (var drawBitmap = SoftwareBitmap.CreateCopyFromBuffer(pixels,
BitmapPixelFormat.Bgra8,
this.renderBitmap.PixelWidth,
this.renderBitmap.PixelHeight,
BitmapAlphaMode.Premultiplied))
{
using (var session = CanvasComposition.CreateDrawingSession(this.drawingSurface))
{
try
{
using (var canvasBitmap =
CanvasBitmap.CreateFromSoftwareBitmap(session.Device, drawBitmap))
{
// We want to draw here in effective pixels, not real pixel sizes.
// (I think!)
DisplayInformation info = DisplayInformation.GetForCurrentView();
session.DrawImage(
canvasBitmap,
new Rect(
0,
0,
this.renderBitmap.PixelWidth / info.RawPixelsPerViewPixel,
this.renderBitmap.PixelHeight / info.RawPixelsPerViewPixel));
}
}
catch (Exception ex)
{
// TBD: I sometimes get a 'catastrophic failure' from the call above
// to create a software bitmap and I don't yet know why or what
// I'm doing wrong.
}
}
}
this.busy = false;
}
}
void InitialiseCompositionElements()
{
// Get the compositor from the XAML element that I have.
var compositor =
ElementCompositionPreview.GetElementVisual(this.myXamlGrid).Compositor;
// Actual Width/Height are going to be returned in effective pixels which
// is going to differ from the size of the bitmap that we'll render from
// the XAML.
var width = (float)this.myXamlGrid.ActualWidth;
var height = (float)this.myXamlGrid.ActualHeight;
// Make our visual.
var spriteVisual = compositor.CreateSpriteVisual();
spriteVisual.Size = new Vector2(width, height);
// Get a device
var sharedDevice = CanvasDevice.GetSharedDevice();
// So that we can get a composition graphics device.
var compositionGraphicsDevice =
CanvasComposition.CreateCompositionGraphicsDevice(compositor, sharedDevice);
// So that we can make a drawing surface
this.drawingSurface = compositionGraphicsDevice.CreateDrawingSurface(
new Size(width, height),
DirectXPixelFormat.B8G8R8A8UIntNormalized,
DirectXAlphaMode.Premultiplied);
// So that we can make a brush
var surfaceBrush = compositor.CreateSurfaceBrush(this.drawingSurface);
// Create our effect description
var saturationEffect = new SaturationEffect()
{
Source = new CompositionEffectSourceParameter("source"),
Saturation = 0.5f
};
// So that we can make a factory
var effectFactory = compositor.CreateEffectFactory(saturationEffect);
// SO that we can make a brush
var effectBrush = effectFactory.CreateBrush();
// which uses our drawing brush as its source
effectBrush.SetSourceParameter("source", surfaceBrush);
// We want to paint our visual with our effect brush
spriteVisual.Brush = effectBrush;
// And drop it into the grid we've left empty for it in the XAML
ElementCompositionPreview.SetElementChildVisual(this.myHostGrid, spriteVisual);
}
bool busy;
CompositionDrawingSurface drawingSurface;
RenderTargetBitmap renderBitmap;
DispatcherTimer timer;
}
and, yes, I did end up with a copy of my simple UI drawn by Win2D and composited by Windows.UI.Composition with a SaturationEffect applied;
but it took a little bit of head scratching to get there and I’d need to think some more now about how I would take this and use it as the basis to build something such that I could use the new composition layer to simply add a set of effects to existing XAML elements.
I learned a lot while writing this post, I’m fairly sure that the code has problems in a number of places but I feel a little more familiar with Windows.UI.Composition than I did when I started writing and I know I’ve got quite a bit more to figure out on it so I’ll perhaps have some more posts as I play around with it in coming weeks.
Enjoy!
Mike Taulty 