One of the Windows 10/UWP areas that I’m very interested in watching progress are the Composition APIs (the Visual Layer) that first appeared in the 1511 build but are continuing to grow in the upcoming Anniversary Edition.
I have watched a few of the sessions (including breakouts and pre-recorded) from //Build 2016 around composition and I was going to make a list of them here but then I realised that the composition team had already done this 
They have their github page;
and within there is the ‘newsletter’ that the guys are publishing with their updates which is something I found really valuable;
https://github.com/Microsoft/WindowsUIDevLabs/wiki/Windows-UI-Newsletter—May-Edition
and within that there is the list of all the sessions from //Build 2016 relating to the broader topic of Windows UI with around 8-9 relating directly to the visual layer topics;
https://github.com/Microsoft/WindowsUIDevLabs/wiki/Windows-UI-Newsletter—May-Edition#build-sessions
At the time of writing, I have watched some of these but not all of these but I will get there
I also spotted that some of my earlier posts are referenced from the team’s github so I’d better try and keep up with what’s happening with these APIs!
Revisiting the Previous Visual Layer Post
The last post that I wrote in my earlier experiments with the APIs here was around how easy/difficult it might be to walk up to some arbitrary XAML element and add an effect to it like blur/saturation etc. to it.
At the time of that post, on the 10586 SDK I found that it wasn’t really so easy because it involved me as a developer having to render the XAML content to a bitmap before then rendering that bitmap back to the screen with the composition layer and that might be ok if the XAML in question was relatively static but it’s a bit of a challenge if the bitmap has to be constantly updated where the XAML content is dynamically changing.
I did it by using some kind of timer but that was a long way from ideal.
Moving on to the Current Preview SDK – New Brushes!
With the current preview of the 14332 SDK, I wanted to check whether this scenario had moved on and it looks like it has with the arrival of the CompositionBackdropBrush.
At the time of writing, I can’t find a proper documentation link around that class but I found a decent discussion around it on StackOverflow which pointed back to the sample code and so on and it intrigued me enough to want to try it out for myself and that’s what this post is about.
To recap, the Visual Layer deals with Visuals and there’s the derived SpriteVisual which is a Visual that is painted with a CompositionBrush and in the 10586 SDK the brushes are CompositionColorBrush, CompositionEffectBrush and CompositionSurfaceBrush.
In the 14332 SDK, the number of available brushes seems to double to include CompositionBackdropBrush, CompositionMaskBrush, CompositionNineGridBrush.
For this post, it’s the CompositionBackdropBrush that interests me and I found it to be a curious thing
The CompositionBackdropBrush
Let’s say that I have this simple UI with a ViewBox and a TextBlock;
<Page
x:Class="App3.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App3"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"
x:Name="myGrid">
<Viewbox>
<TextBlock
x:Name="txtCount" />
</Viewbox>
</Grid>
</Page>
and some code-behind which updates on a timer;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs args)
{
int value = 0;
this.timer = new DispatcherTimer();
this.timer.Interval = TimeSpan.FromSeconds(1);
this.timer.Tick += (s, e) =>
{
// not a great idea to ++ this variable in a format string here.
this.txtCount.Text = $"{++value}";
};
this.timer.Start();
}
DispatcherTimer timer;
}
then perhaps I want to take this whole ‘chunk’ of UI and use it as some kind of brush as input to an effect or maybe just to mirror the UI somewhere else on the screen as I can in WPF with a VisualBrush.
How I might do that wasn’t immediately obvious to me using CompositionBackgroundBrush because I could only find one way to create one.
Specifically;
var gridVisual = ElementCompositionPreview.GetElementChildVisual(this.myGrid);
var compositor = gridVisual.Compositor;
var brush = compositor.CreateBackdropBrush();
and I found this one a bit puzzling in terms of that call to CreateBackdropBrush – i.e. which backdrop am I getting here because I’m making this call on the compositor itself and I don’t seem to tell the compositor about the element that I would like to get the backdrop from.
My expectation would be something like;
- Get some visual for a UI element
- Create a brush from that visual
- Use that brush somewhere
but I don’t think that’s quite how things work here. I think the sequence is something more like;
- Create a visual
- Create a backdrop brush
- Apply the brush to the visual
- Position the visual ‘over’ the content which will become the brush
Applying an Effect
I thought I’d see how that worked out. I moved my existing grid into a row of a sub-grid and added a button to my UI;
<Page
x:Class="App3.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App3"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Grid.RowDefinitions>
<RowDefinition />
<RowDefinition
Height="Auto" />
</Grid.RowDefinitions>
<Grid x:Name="myGrid">
<Viewbox>
<TextBlock
x:Name="txtCount" />
</Viewbox>
</Grid>
<Button
Grid.Row="1"
Content="Blur"
HorizontalAlignment="Center"
Click="OnEffectButton" />
</Grid>
</Page>
and then added some code behind the button;
namespace App3
{
using Microsoft.Graphics.Canvas.Effects;
using System;
using System.Numerics;
using Windows.UI.Composition;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Hosting;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs args)
{
int value = 0;
this.timer = new DispatcherTimer();
this.timer.Interval = TimeSpan.FromSeconds(1);
this.timer.Tick += (s, e) =>
{
// not a great idea to ++ this variable in a format string here.
this.txtCount.Text = $"{++value}";
};
this.timer.Start();
}
DispatcherTimer timer;
void OnEffectButton(object sender, RoutedEventArgs e)
{
if (this.effectVisual != null)
{
this.effectVisual.Dispose();
this.effectVisual = null;
}
else
{
var gridVisual = ElementCompositionPreview.GetElementVisual(this.myGrid);
var compositor = gridVisual.Compositor;
this.effectVisual = compositor.CreateSpriteVisual();
// We'd need to resize this as/when the grid resized.
this.effectVisual.Size = new Vector2(
(float)this.myGrid.ActualWidth,
(float)this.myGrid.ActualHeight);
GaussianBlurEffect blurEffect = new GaussianBlurEffect()
{
BorderMode = EffectBorderMode.Hard, // NB: default mode here isn't supported yet.
Source = new CompositionEffectSourceParameter("source")
};
var effectFactory = compositor.CreateEffectFactory(blurEffect);
var effectBrush = effectFactory.CreateBrush();
effectBrush.SetSourceParameter("source", compositor.CreateBackdropBrush());
this.effectVisual.Brush = effectBrush;
ElementCompositionPreview.SetElementChildVisual(this.myGrid, this.effectVisual);
}
}
SpriteVisual effectVisual;
}
}
and now I’ve got a blurred counter updating in real time;
I then wondered whether that blur might be something I could drag around with my finger (or mouse/pen) and so I changed my UI;
<Page
x:Class="App3.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App3"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Grid.RowDefinitions>
<RowDefinition />
</Grid.RowDefinitions>
<Grid
x:Name="myGrid"
PointerPressed="OnPointerPressed"
PointerMoved="OnPointerMoved"
PointerReleased="OnPointerReleased">
<Viewbox>
<TextBlock
x:Name="txtCount" />
</Viewbox>
</Grid>
</Grid>
</Page>
and the code behind it;
namespace App3
{
using Microsoft.Graphics.Canvas.Effects;
using System;
using System.Numerics;
using Windows.UI;
using Windows.UI.Composition;
using Windows.UI.Input;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Hosting;
using Windows.UI.Xaml.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
this.effectSize = new Vector2(200, 200);
}
void OnLoaded(object sender, RoutedEventArgs args)
{
int value = 0;
this.timer = new DispatcherTimer();
this.timer.Interval = TimeSpan.FromSeconds(1);
this.timer.Tick += (s, e) =>
{
// not a great idea to ++ this variable in a format string here.
this.txtCount.Text = $"{++value}";
};
this.timer.Start();
}
void AddEffect()
{
var gridVisual = ElementCompositionPreview.GetElementVisual(this.myGrid);
var compositor = gridVisual.Compositor;
this.effectVisual = compositor.CreateSpriteVisual();
// We'd need to resize this as/when the grid resized.
this.effectVisual.Size = this.effectSize;
GaussianBlurEffect blurEffect = new GaussianBlurEffect()
{
BorderMode = EffectBorderMode.Hard, // NB: default mode here isn't supported yet.
BlurAmount = 5.0f,
Source = new CompositionEffectSourceParameter("source")
};
var effectFactory = compositor.CreateEffectFactory(blurEffect);
var effectBrush = effectFactory.CreateBrush();
effectBrush.SetSourceParameter("source", compositor.CreateBackdropBrush());
this.effectVisual.Brush = effectBrush;
ElementCompositionPreview.SetElementChildVisual(this.myGrid, this.effectVisual);
}
void OnPointerPressed(object sender, PointerRoutedEventArgs e)
{
this.AddEffect();
this.PositionEffect(e.GetCurrentPoint(this.myGrid));
}
void PositionEffect(PointerPoint point)
{
var halfWidth = this.effectSize.X / 2;
var halfHeight = this.effectSize.Y / 2;
var idealPositionX = (float)point.Position.X - halfWidth;
var idealPositionY = (float)point.Position.Y - halfHeight;
idealPositionX = Math.Max(0, idealPositionX);
idealPositionX = Math.Min(idealPositionX, (float)this.myGrid.ActualWidth - this.effectSize.X);
idealPositionY = Math.Max(0, idealPositionY);
idealPositionY = Math.Min(idealPositionY, (float)this.myGrid.ActualHeight - this.effectSize.Y);
this.effectVisual.Offset = new Vector3(
idealPositionX, idealPositionY, 0);
}
void OnPointerMoved(object sender, PointerRoutedEventArgs e)
{
if (this.effectVisual != null)
{
this.PositionEffect(e.GetCurrentPoint(this.myGrid));
}
}
void OnPointerReleased(object sender, PointerRoutedEventArgs e)
{
this.effectVisual?.Dispose();
this.effectVisual = null;
}
SpriteVisual effectVisual;
Vector2 effectSize;
DispatcherTimer timer;
}
}
and, sure enough, I had a 200×200 pixel square of ‘blur effect’ that I could drag around my UI.
It would perhaps make sense if I had richer content (like an image) at this point but hopefully the 0 in the screenshot below looks suitably blurred at the top left corner.
As an aside, I noticed that my PointerMoved events didn’t fire when I was using this via touch (they did for mouse) and I’m guessing that’s because they were being intercepted by the visual layer in a way that I haven’t quite understood just yet so I need to return to that – I think this might relate to interactions at the visual layer.
A Non-Rectangular Effect
With this kind of working, I wondered whether I could make that effect brush fill something other than a rectangle – e.g. can I make it circular or do visuals always have to be rectangular?
I had a few attempts at this.
- Initially, I thought I might try some XAML trick with a Path similar but I fairly quickly ground to a halt when I realised I ultimately needed to be able to clip the visual itself.
- I then tried to look at the Clip property of a CompositionBrush but that seemed to limited to a rectangle.
- I then started to look at the CompositionMaskBrush which seemed like the right thing to be using but I couldn’t find too many uses of it out there other than a sample here which seemed to be using a transparent PNG as the source which tells a mask how to clip.
I tried to follow that CompositionMaskBrush route based on what I saw in the sample but the sample was more about adding a shadow to an image. When I tried to bend this to suit my purposes, it seemed that the CompositionMaskBrush did not support taking its input from either a CompositionEffectBrush or a CompositionBackdropBrush.
I got a bit blocked at that point until…
A Non-Rectangular Visual
Being a bit stuck, I was about to ask the @windowsui Twitter account for help when I saw that they had retweeted Ratish’s tweet about;
with links to his github and that sounded like exactly what I was trying to figure out – Visuals that aren’t square.
I had a quick poke into the code and it looks like the ‘trick’ that I was missing was to the one of rendering a geometry of some sort to a surface so that the composition APIs could use it as a mask.
Ratish’s code has that trick and so I thought that I’d give it a try and I did that for around 10 minutes before realising that I was heading back to the same place as my previous attempts – i.e. I was still going to get stuck by CompositionMaskBrush not allowing me to pass a Source brush that came from a CompositionBackdropBrush.
So…at the time of writing, I’m not sure whether I can/can’t apply an effect in a non-rectangular way here but I’m still pleased with how easy it becomes to add an effect to a ‘live’ XAML UI.
Finishing Up
I took my UI back to something really simple – just an image from the web;
<Page
x:Class="App3.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App3"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"
x:Name="myGrid"
SizeChanged="OnGridSizeChanged">
<Image
Source="http://www.mrwallpaper.com/wallpapers/mauritius-blue-bay-beach.jpg"
Stretch="Uniform" />
</Grid>
</Page>
and I changed my code to animate the blur over that UI (noting that the UI could be something more complex than just a static image);
namespace App3
{
using Microsoft.Graphics.Canvas.Effects;
using System;
using System.Numerics;
using Windows.UI.Composition;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Hosting;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void SizeVisual()
{
if (this.effectVisual != null)
{
this.effectVisual.Size = new Vector2(
(float)this.myGrid.ActualWidth, (float)this.myGrid.ActualHeight);
}
}
void OnLoaded(object sender, RoutedEventArgs args)
{
var gridVisual = ElementCompositionPreview.GetElementVisual(this.myGrid);
var compositor = gridVisual.Compositor;
this.effectVisual = compositor.CreateSpriteVisual();
this.SizeVisual();
GaussianBlurEffect blurEffect = new GaussianBlurEffect()
{
Name = "Blur",
BorderMode = EffectBorderMode.Hard, // NB: default mode here isn't supported yet.
BlurAmount = 0.0f,
Source = new CompositionEffectSourceParameter("source")
};
var effectFactory = compositor.CreateEffectFactory(
blurEffect,
new string[] { "Blur.BlurAmount" });
var effectBrush = effectFactory.CreateBrush();
var blurAnimation = compositor.CreateScalarKeyFrameAnimation();
var easingFunction = compositor.CreateLinearEasingFunction();
blurAnimation.InsertKeyFrame(0.0f, 0.0f);
blurAnimation.InsertKeyFrame(0.5f, 10.0f, easingFunction);
blurAnimation.InsertKeyFrame(1.0f, 0.0f, easingFunction);
blurAnimation.IterationBehavior = AnimationIterationBehavior.Forever;
blurAnimation.Duration = TimeSpan.FromSeconds(5);
effectBrush.StartAnimation("Blur.BlurAmount", blurAnimation);
effectBrush.SetSourceParameter("source", compositor.CreateBackdropBrush());
this.effectVisual.Brush = effectBrush;
ElementCompositionPreview.SetElementChildVisual(this.myGrid, this.effectVisual);
}
void OnGridSizeChanged(object sender, SizeChangedEventArgs e)
{
this.SizeVisual();
}
SpriteVisual effectVisual;
}
}
and I’m back to a nicely animated blur over whatever UI is resident in the Grid named ‘myGrid’.
What I’ve yet to figure out though is whether I can take one of these brushes and use it to paint some other ‘area’ of the screen – I’ll return to that in a later post.
Mike Taulty 
Pingback: Windows 10 Anniversary Update Preview–Composition and the CompositionBackdropBrush | Tech News
Pingback: The Morning Brew #2108 | Tech News
Pingback: The week in .NET – 06/07/2016 | Tech News
Pingback: Windows 10 Anniversary Update Preview–Composition and Expressions – Mike Taulty
Hi MIke,
Nice article. It is possible to create custom shaped Backdrop brushes using CompositionProToolkit. Check this out https://github.com/ratishphilip/CompositionProToolkit