The other day, I came across a slight snag in trying to render the contents of a UI containing an Ink Canvas to a bitmap on Windows 10 and the UWP.
Let’s imagine that I’ve got a UI that looks something like this;
Which I’ve made in XAML like this;
<Page
x:Class="App311.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App311"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<RelativePanel>
<Grid
RelativePanel.AlignVerticalCenterWithPanel="True"
RelativePanel.AlignHorizontalCenterWithPanel="True"
x:Name="topLeftGrid">
<StackPanel>
<Grid
x:Name="parentGrid"
Background="SkyBlue"
Width="300">
<Grid.RowDefinitions>
<RowDefinition
Height="Auto" />
<RowDefinition
Height="Auto" />
<RowDefinition
Height="150" />
</Grid.RowDefinitions>
<TextBox
Margin="8"
Grid.Row="0"
Header="First Name"
IsReadOnly="True"
Text="Mickey" />
<TextBox
Margin="8"
Grid.Row="1"
Header="Last Name"
IsReadOnly="True"
Text="Mouse" />
<Grid
Grid.Row="2">
<Rectangle
Margin="8,24,8,24"
Stroke="Black" />
<TextBlock
Text="Signature"
Margin="8,0,0,0"
HorizontalAlignment="Left"
VerticalAlignment="Top" />
</Grid>
<InkCanvas
x:Name="myCanvas" Grid.RowSpan="3"/>
</Grid>
<Button
Content="Render to 2nd Image"
Click="OnRenderToSecondImage"
HorizontalAlignment="Stretch" />
</StackPanel>
</Grid>
<Grid
RelativePanel.AlignTopWith="topLeftGrid"
RelativePanel.RightOf="topLeftGrid">
<Image
x:Name="secondImage"
Stretch="None" />
</Grid>
</RelativePanel>
</Page>
and you can see that there’s an InkCanvas covering the whole of the Grid that has the 2 TextBox’s in it.
The idea of the Button labelled ‘Render to 2nd Image’ is that it will use a RenderTargetBitmap to make a ‘copy’ of the whole UI which sits above it on the screen and render that into the Image named secondImage in the XAML.
I have this code behind the Button;
async void OnRenderToSecondImage(object sender, RoutedEventArgs e)
{
var renderBitmap = new RenderTargetBitmap();
await renderBitmap.RenderAsync(this.parentGrid);
this.secondImage.Source = renderBitmap;
}
However, if I try this out then I see that the ink from the InkCanvas disappears;
and, a bit surprisingly, it disappears both from the original UI on the left and it doesn’t show up rendered to the bitmap copy on the right either.
Fascinatingly, if I resize the Window just a touch;
then the ink reappears on the left. That’ll need returning to I think but where I got stuck most was in how to solve what seemed to be one of the limitations of the RenderTargetBitmap – i.e. it seems not to render the ink from an InkCanvas.
What to do? I was stumped for quite a long time until I learned that Win2D is able to Draw Ink and so I wondered if I could use Win2D to re-draw the bitmap that comes out of RenderTargetBitmap with the ink back on top of it.
Here’s my first attempt at that;
async void OnRenderToSecondImage(object sender, RoutedEventArgs e)
{
// The original bitmap from the screen, missing the ink.
var renderBitmap = new RenderTargetBitmap();
await renderBitmap.RenderAsync(this.parentGrid);
var renderBitmapPixels = await renderBitmap.GetPixelsAsync();
var win2DDevice = CanvasDevice.GetSharedDevice();
var win2DTarget = new CanvasRenderTarget(
win2DDevice, renderBitmap.PixelWidth, renderBitmap.PixelHeight, 96.0f);
using (var win2dRenderedBitmap =
CanvasBitmap.CreateFromBytes(
win2DDevice,
renderBitmapPixels,
renderBitmap.PixelWidth,
renderBitmap.PixelHeight,
Windows.Graphics.DirectX.DirectXPixelFormat.B8G8R8A8UIntNormalized))
{
using (var win2dSession = win2DTarget.CreateDrawingSession())
{
// Draw the rendered bitmap
win2dSession.DrawImage(win2dRenderedBitmap);
// Overlay the ink on top.
win2dSession.DrawInk(this.myCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
}
// Extract the bitmap from the CanvasTarget back into a format that
// we can hand over to a XAML Image.
var outputBitmap = new SoftwareBitmap(
BitmapPixelFormat.Bgra8,
renderBitmap.PixelWidth,
renderBitmap.PixelHeight,
BitmapAlphaMode.Premultiplied);
outputBitmap.CopyFromBuffer(
win2DTarget.GetPixelBytes().AsBuffer());
// Now feed that to the XAML image.
var source = new SoftwareBitmapSource();
await source.SetBitmapAsync(outputBitmap);
this.secondImage.Source = source;
}
and here’s how it came out;
What’s happened there? Well, the ink seems to have been rendered ok but the original blue rectangle seems to have magnified its role a little.
What I think’s quite interesting about this example is that if I set this breakpoint;
and then take a look at this output;
then I see the rendered bitmap is 450×453 but here’s the same breakpoint on the same code run for a second time;
and the only difference here is that for the first example I had the app’s window on my Surface Pro 3’s internal monitor whereas for the second example I had the window on an external monitor.
So…the size of the bitmap that you get from RenderTargetBitmap changes depending on the monitor that your window is currently on and I guess that makes sense in that the API is presumably trying to give you a pixel-based representation that accurately represents the UI that is on a screen that might be scaling (in my case) either at 1.0x or at 1.5x.
How to fix that? Firstly, I went off and tried to read a little more around how Win2D deals with pixels and there’s some interesting behaviours in the APIs around how they deal with sizes passed as floats rather than those passed as integers.
I played around with a whole bunch of scenarios but it seemed to me that I was stuck with either;
- Asking the RenderTargetBitmap to render itself and then (on a high DPI screen) scaling the output down.
- Asking the RenderTargetBitmap to render itself at a scaled down size.
and both of these result in a less-than-perfect representation of the rendered UI for a monitor that’s (ironically) higher DPI.
One variant of the code that seemed to be about as good as I could get it was;
// The original bitmap from the screen, missing the ink.
var renderBitmap = new RenderTargetBitmap();
await renderBitmap.RenderAsync(this.parentGrid);
var bitmapSizeAt96Dpi = new Size(
renderBitmap.PixelWidth,
renderBitmap.PixelHeight);
var renderBitmapPixels = await renderBitmap.GetPixelsAsync();
var win2DDevice = CanvasDevice.GetSharedDevice();
var displayInfo = DisplayInformation.GetForCurrentView();
using (var win2DTarget = new CanvasRenderTarget(
win2DDevice,
(float)this.parentGrid.ActualWidth,
(float)this.parentGrid.ActualHeight,
96.0f))
{
using (var win2dSession = win2DTarget.CreateDrawingSession())
{
using (var win2dRenderedBitmap =
CanvasBitmap.CreateFromBytes(
win2DDevice,
renderBitmapPixels,
(int)bitmapSizeAt96Dpi.Width,
(int)bitmapSizeAt96Dpi.Height,
Windows.Graphics.DirectX.DirectXPixelFormat.B8G8R8A8UIntNormalized,
96.0f))
{
win2dSession.DrawImage(win2dRenderedBitmap,
new Rect(0, 0, win2DTarget.SizeInPixels.Width, win2DTarget.SizeInPixels.Height),
new Rect(0, 0, bitmapSizeAt96Dpi.Width, bitmapSizeAt96Dpi.Height));
}
win2dSession.Units = CanvasUnits.Pixels;
win2dSession.DrawInk(this.myCanvas.InkPresenter.StrokeContainer.GetStrokes());
}
// Get the output into a software bitmap.
var outputBitmap = new SoftwareBitmap(
BitmapPixelFormat.Bgra8,
(int)win2DTarget.SizeInPixels.Width,
(int)win2DTarget.SizeInPixels.Height,
BitmapAlphaMode.Premultiplied);
outputBitmap.CopyFromBuffer(
win2DTarget.GetPixelBytes().AsBuffer());
// Now feed that to the XAML image.
var source = new SoftwareBitmapSource();
await source.SetBitmapAsync(outputBitmap);
this.secondImage.Source = source;
}
}
and that ‘kind of works’ – here’s what I see on my lower DPI monitor;
but the output on my higher DPI monitor isn’t quite as good;
There’s a blur to that bitmap on the right hand side but I have yet to come up with a way to skirt around that.
In both cases, the ink on the left has still disappeared which isn’t exactly ‘desirable’. For the moment, I solve that in my case by moving where the InkCanvas sits in the visual tree. That is, I change my UI definition to be;
<Page
x:Class="App311.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App311"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<RelativePanel>
<Grid
RelativePanel.AlignVerticalCenterWithPanel="True"
RelativePanel.AlignHorizontalCenterWithPanel="True"
x:Name="topLeftGrid">
<StackPanel>
<Grid> <!-- sneaky new grid to parent ink canvas for me -->
<Grid
x:Name="parentGrid"
Background="SkyBlue"
Width="300">
<Grid.RowDefinitions>
<RowDefinition
Height="Auto" />
<RowDefinition
Height="Auto" />
<RowDefinition
Height="150" />
</Grid.RowDefinitions>
<TextBox
Margin="8"
Grid.Row="0"
Header="First Name"
IsReadOnly="True"
Text="Mickey" />
<TextBox
Margin="8"
Grid.Row="1"
Header="Last Name"
IsReadOnly="True"
Text="Mouse" />
<Grid
Grid.Row="2">
<Rectangle
Margin="8,24,8,24"
Stroke="Black" />
<TextBlock
Text="Signature"
Margin="8,0,0,0"
HorizontalAlignment="Left"
VerticalAlignment="Top" />
</Grid>
</Grid>
<InkCanvas
x:Name="myCanvas" />
</Grid>
<Button
Content="Render to 2nd Image"
Click="OnRenderToSecondImage"
HorizontalAlignment="Stretch" />
</StackPanel>
</Grid>
<Grid
RelativePanel.AlignTopWith="topLeftGrid"
RelativePanel.RightOf="topLeftGrid">
<Image
x:Name="secondImage"
Stretch="None" />
</Grid>
</RelativePanel>
</Page>
and you’ll notice that I have re-parented the InkCanvas into a Grid (labelled ‘sneaky’ in the XAML) which is sitting above the original ‘parentGrid’ Grid and so calling RenderAsync on that ‘parentGrid’ is no longer having any impact on the InkCanvas and I see;
(on my higher DPI monitor).
For the moment, that’s all I know – I thought I’d share in case anyone hits something similar and feel free to let me know that I’m doing it wrong 
Mike Taulty 