Following up on these 3 posts;
Windows, UI and Composition (the Visual Layer)
A Second Experiment with the Visual Layer
A Third Experiment with the Visual Layer–Images
I wanted to see if I could combine at a basic level some of what I did in the first post with some of what I did in the third post in order to see if I could take a XAML element such as a Grid and do something like;
myGrid.Saturate(30);
with the composition APIs adding the saturation effect for me.
How easy/hard is that to do?
I guess that I have to start by adding some kind of Saturate extension method to a XAML type and I could choose something like UIElement or FrameworkElement but I think it’s probably sensible to try and use a Panel as I’ve a suspicion that I might need a panel for what I need to do.
I made a small piece of XAML with an image, some textblocks and a couple of buttons in it;
<Page
x:Class="App291.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App291"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Grid.RowDefinitions>
<RowDefinition />
<RowDefinition
Height="Auto" />
</Grid.RowDefinitions>
<Grid
x:Name="myGrid">
<Grid.RowDefinitions>
<RowDefinition />
<RowDefinition />
<RowDefinition />
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition />
<ColumnDefinition Width="Auto"/>
</Grid.ColumnDefinitions>
<Image
Source="Images/kermit.jpg"
Stretch="Fill"
Grid.RowSpan="3" />
<Viewbox
Grid.Row="0"
Grid.Column="1"
Stretch="Fill">
<Grid
Background="Red">
<TextBlock
Text="Red"
Foreground="White" />
</Grid>
</Viewbox>
<Viewbox
Grid.Row="1"
Grid.Column="1"
Stretch="Fill">
<Grid
Background="Green">
<TextBlock
Text="Green"
Foreground="White" />
</Grid>
</Viewbox>
<Viewbox
Grid.Row="2"
Grid.Column="1"
Stretch="Fill">
<Grid
Background="Blue">
<TextBlock
Text="Blue"
Foreground="White" />
</Grid>
</Viewbox>
</Grid>
<StackPanel
Grid.Row="2"
Orientation="Horizontal">
<Button
Margin="12"
Content="Change Saturation"
Click="OnApply">
</Button>
<Button
Margin="12"
Content="Restore Saturation"
Click="OnRemove">
</Button>
</StackPanel>
</Grid>
</Page>
which looks like this;
and then it’s not too tricky to write some code behind that attempts to ‘Saturate’ that Grid named myGrid;
private void Button_Click(object sender, RoutedEventArgs e)
{
this.myGrid.Saturate(30);
}
that doesn’t do anything at this point.
What should it do? Ideally, I’d like to be able to just walk up to a portion of the live XAML tree and apply an effect to it like I can in WPF or like I could in Silverlight and I’m hoping that’s where the composition APIs mixed with Win2D will ultimately go but I don’t know if that’s so simple at this point or whether it’s a core scenario for those APIs.
In my original post on the composition APIs under the section “Adding Effects to XAML” I got blocked on trying to get a CompositionBrush from a ‘live’ XAML element and the only way that I could see of doing it was to capture a static image of a XAML element and to apply an effect to that.
To date, that’s as close as I’ve got so my approach to implement that Saturate method might be;
- Render the element to a bitmap.
- Create a composition brush out of that bitmap using Win2D.
- Insert a new visual as the last, top-most child of the Panel.
- Paint that visual with the brush.
- Apply the effect to that visual.
I wondered how this might work out in the real world and so I tried it and, the first thing that I realised was that it’s going to have to be Async rather than a synchronous process and the second thing that I realised was that there might be quite a lot of stuff to dispose of in order to remove the effect and so my code-behind file with handlers for those 2 buttons quickly became;
using System;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
}
async void OnApply(object sender, RoutedEventArgs e)
{
this.effectContext = await this.myGrid.SaturateAsync(30);
}
void OnRemove(object sender, RoutedEventArgs e)
{
this.effectContext?.Dispose();
this.effectContext = null;
}
IDisposable effectContext;
}
I wrote SaturateAsync as an extension method in a static class and it turned into quite a bit of code;
using Microsoft.Graphics.Canvas.Effects;
using System;
using System.Numerics;
using System.Threading.Tasks;
using Windows.Foundation;
using Windows.Graphics.Imaging;
using Windows.UI.Composition;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Hosting;
using Windows.UI.Xaml.Media.Imaging;
public static class PanelExtensions
{
class EffectContext : IDisposable
{
// no finalizer as I have no unmanaged resources that I'm
// aware.
public void Dispose()
{
// tagging this onto disposal is a bit naughty but there was a
// need to dipose the other bits so this removes the SpriteVisual
// from the ContainerVisual it's living in.
this.spriteVisual.Parent.Children.RemoveAll();
// these bits all need disposal...
this.effectBrush.Dispose();
this.imageBrush.Dispose();
this.spriteVisual.Dispose();
}
public SpriteVisual spriteVisual;
public CompositionImageBrush imageBrush;
public CompositionEffectBrush effectBrush;
}
static ContainerVisual EnsureContainerVisualInXamlPanel(Panel panel)
{
var containerVisual =
ElementCompositionPreview.GetElementChildVisual(panel) as ContainerVisual;
if (containerVisual == null)
{
Compositor compositor = ElementCompositionPreview.GetElementVisual(panel).Compositor;
containerVisual = compositor.CreateContainerVisual();
ElementCompositionPreview.SetElementChildVisual(panel, containerVisual);
}
return (containerVisual);
}
static async Task<SoftwareBitmap> RenderXamlElementToBGRASoftwareBitmapAsync(UIElement element)
{
var renderedBitmap = new RenderTargetBitmap();
// Unsure of the behaviour here because if I specify a width, height they seem to
// be ignored here so leaving those parameters out and then resizing it myself
// later on.
await renderedBitmap.RenderAsync(element);
// Get the pixels out of that bitmap as BGRA8
var pixels = await renderedBitmap.GetPixelsAsync();
// Copy into a software bitmap.
var softwareBitmap = SoftwareBitmap.CreateCopyFromBuffer(
pixels,
BitmapPixelFormat.Bgra8,
renderedBitmap.PixelWidth,
renderedBitmap.PixelHeight,
BitmapAlphaMode.Premultiplied);
return (softwareBitmap);
}
public static async Task<IDisposable> SaturateAsync(
this Panel panel, int saturationPercentage)
{
var compositor =
ElementCompositionPreview.GetElementVisual(panel).Compositor;
// We put a container visual into the Xaml panel if we haven't
// done that already. The container will stay there forever
// because I don't know how to remove it. But...we'll add/remove
// a child sprite visual to it.
var containerVisual = EnsureContainerVisualInXamlPanel(panel);
// the return value which provides a way for our caller to
// dispose of what we give them when they are done.
var returnContext = new EffectContext();
// this is not necessarily the same width/height as the
// rendered bitmap we're about to make.
var panelSize = new Size(panel.ActualWidth, panel.ActualHeight);
// Render the XAML Panel to a software bitmap via a
// RenderTargetBitmap.
using (var renderedSoftwareBitmap =
await RenderXamlElementToBGRASoftwareBitmapAsync(panel))
{
// Create an instance of my 'composition image brush' from that.
returnContext.imageBrush = CompositionImageBrush.FromBGRASoftwareBitmap(
compositor,
renderedSoftwareBitmap,
panelSize);
}
// Create a saturation effect brush taking our image brush as
// the source.
returnContext.effectBrush = CreateSaturationEffectBrushWithSource(
compositor, saturationPercentage / 100.0f, returnContext.imageBrush.Brush);
// Create a new Sprite Visual to draw our image.
returnContext.spriteVisual = compositor.CreateSpriteVisual();
returnContext.spriteVisual.Size =
new Vector2((float)panelSize.Width, (float)panelSize.Height);
// We want to paint our visual with our effect brush
returnContext.spriteVisual.Brush = returnContext.effectBrush;
// and add it to the container visual that's now in the panel.
containerVisual.Children.InsertAtTop(returnContext.spriteVisual);
// Quite a lot of stuff now needs disposing!
return (returnContext);
}
static CompositionEffectBrush CreateSaturationEffectBrushWithSource(
Compositor compositor,
float saturationAmount,
CompositionBrush sourceBrush)
{
CompositionEffectBrush brush = null;
using (
var saturationEffect = new SaturationEffect()
{
Source = new CompositionEffectSourceParameter("source"),
Saturation = saturationAmount
})
{
// So that we can make a factory
using (var effectFactory = compositor.CreateEffectFactory(saturationEffect))
{
// SO that we can make a brush
brush = effectFactory.CreateBrush();
// which uses our drawing brush as its source
brush.SetSourceParameter("source", sourceBrush);
}
}
return (brush);
}
}
and you should apply a pinch of salt as I’m still very much trying to figure out how this works but what that SaturateAsync method does is to;
- Use the ElementCompositionPreview support to grab a Visual from the XAML Panel and, from there, to get a Compositor
- Place a ContainerVisual as the child of the XAML Panel – this provides a visual to parent a SpriteVisual off because I can’t find a way to remove a visual from being the child of a XAML element and so adding an intermediate Visual helps me get rid of the SpriteVisual later on.
- Renders the XAML Panel into a SoftwareBitmap via way of a RenderTargetBitmap.
- Turns the SoftwareBitmap into a CompositionBrush using similar code to that posted in the previous post.
- Creates a CompositionEffectBrush for a saturation effect using the brush created in the previous step as the source for the effect.
- Adds a SpriteVisual painted with the effect brush into the ContainerVisual that’s been parented by the Panel
That’s quite a few steps and (I think) it leaves a few pieces to later be disposed of and so I return a class I called EffectContext which wraps up those pieces and (hopefully) disposes of them when the user is done with the effect.
I had some fun and games with that in trying to;
- Add/Remove a visual as the child of a XAML panel. There is the ElementCompositionPreview.SetElementChildVisual method but there doesn’t seem to be a way of removing that child visual at a later point and hence me adding an additional ContainerVisual in between the XAML Panel and the SpriteVisual that draws the effect over the top.
- Rendering a XAML Panel into a bitmap using RenderTargetBitmap – I’m unsure whether this renders based on effective or actual pixels and it seems to ignore requests to render at smaller sizes. Also, if the content in the Panel that’s being rendered doesn’t take the entire space then the bitmap ends up being the wrong size. I found that making the background of the Panel ‘Transparent’ seemed to help for that case.
That code uses a modified version of the CompositionImageBrush that I knocked together in the previous blog post which is as below for this sample;
using Microsoft.Graphics.Canvas;
using Microsoft.Graphics.Canvas.UI.Composition;
using System;
using Windows.Foundation;
using Windows.Graphics.DirectX;
using Windows.Graphics.Imaging;
using Windows.UI.Composition;
public class CompositionImageBrush : IDisposable
{
private CompositionImageBrush()
{
}
public CompositionBrush Brush
{
get
{
return (this.drawingBrush);
}
}
void CreateDevice(Compositor compositor)
{
this.graphicsDevice = CanvasComposition.CreateCompositionGraphicsDevice(
compositor, CanvasDevice.GetSharedDevice());
}
void CreateDrawingSurface(Size drawSize)
{
this.drawingSurface = this.graphicsDevice.CreateDrawingSurface(
drawSize,
DirectXPixelFormat.B8G8R8A8UIntNormalized,
DirectXAlphaMode.Premultiplied);
}
void CreateSurfaceBrush(Compositor compositor)
{
this.drawingBrush = compositor.CreateSurfaceBrush(
this.drawingSurface);
}
public static CompositionImageBrush FromBGRASoftwareBitmap(
Compositor compositor,
SoftwareBitmap bitmap,
Size outputSize)
{
CompositionImageBrush brush = new CompositionImageBrush();
brush.CreateDevice(compositor);
brush.CreateDrawingSurface(outputSize);
brush.DrawSoftwareBitmap(bitmap, outputSize);
brush.CreateSurfaceBrush(compositor);
return (brush);
}
void DrawSoftwareBitmap(SoftwareBitmap softwareBitmap, Size renderSize)
{
using (var drawingSession = CanvasComposition.CreateDrawingSession(
this.drawingSurface))
{
using (var bitmap = CanvasBitmap.CreateFromSoftwareBitmap(drawingSession.Device,
softwareBitmap))
{
drawingSession.DrawImage(bitmap,
new Rect(0, 0, renderSize.Width, renderSize.Height));
}
}
}
public void Dispose()
{
// TODO: I'm unsure about the lifetime of these objects - is it ok for
// me to dispose of them here when I've done with them and, especially,
// the graphics device?
this.drawingBrush.Dispose();
this.drawingSurface.Dispose();
this.graphicsDevice.Dispose();
}
CompositionGraphicsDevice graphicsDevice;
CompositionDrawingSurface drawingSurface;
CompositionSurfaceBrush drawingBrush;
}
and here’s the earth shattering effect that this ultimately produces 
Again, this post isn’t meant to be suggesting that you go out and do something like what I’m doing here – I’m just trying to wander around the Windows.UI.Composition APIs and get a bit of a feeling for them and, more than likely, there’s better ways of doing this kind of thing.
Mike Taulty 