I’ve done some pieces of work with the Windows inking capabilities since they came into the platform in Windows 8.0 (and before with what was present in WPF) and I’ve also worked with the advancements that came in DirectInk in Windows 10 and I think that was a big leap forward in terms of (at least);
- the improvements in performance and latency in terms of making the ink experience feel fluid and natural for the user
- the improvements in the developer experience in terms of moving more of the common experience into the platform without damaging the user’s experience
One thing I hadn’t really done before today though was to investigate custom drying of ink properly.
I’d seen people talk about it and I’d read about it but I hadn’t written any code with it and, usually, that’s where I learn most.
I saw it talked about in these places which are great references;
- Harness the Full Power of Digital Inking in Your Universal Windows App (coverage starts at 27m)
- Pen and Stylus Interactions
That first session was recorded back in 2015 and back then it talked about custom drying needing some DirectX implementation and it seemed complex enough to put on the ‘come back to later’ pile as I didn’t have a pressing need to experiment with it.
However, since then Win2D came along and added capabilities to render ink (which I used in this post) but I hadn’t thought to revisit this topic of custom drying with Win2D.
Until today.
Now, it’s worth saying that there’s an official Win2D sample called ‘InkExample’ on GitHub so if you want an official view then I’d advise looking at the sample. What follows below is just my experimentation in trying to get my head around this notion of custom drying.
As an aside, I did borrow a couple of pieces from that sample and they helped me a lot in understanding how this works.
I started off by making a simple UI in XAML;
<Grid
Background="PowderBlue"
x:Name="grid">
<w2d:CanvasControl
x:Name="canvasControl"
xmlns:w2d="using:Microsoft.Graphics.Canvas.UI.Xaml"
Draw="OnDraw" />
<InkCanvas
x:Name="inkCanvas" />
</Grid>
where the CanvasControl comes from the win2d.uwp package.
My aim was to write code to start doing ‘custom drying’ and so I began with;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
// Have to switch on custom drying early in the process and before
// the InkCanvas has Loaded.
this.inkSync = this.inkCanvas.InkPresenter.ActivateCustomDrying();
this.inkCanvas.InkPresenter.StrokesCollected += OnStrokesCollected;
}
void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
var strokes = this.inkSync.BeginDry();
// TODO: render?
this.inkSync.EndDry();
}
void OnDraw(
CanvasControl sender,
CanvasDrawEventArgs args)
{
}
InkSynchronizer inkSync;
}
and, sure enough, every stroke that I inked with my pen would appear on the InkCanvas in ‘wet’ form as I built it up but as soon as I released my pen, it would ‘dry’ and disappear because I hadn’t written any code to dry it.
I really like these ‘wet’ and ‘dry’ terms – they’re really descriptive. The OS/platform does the work to capture the ink in its ‘wet’ form on its own thread to avoid latency etc. and then when the ink is captured, it hands it off to the UI thread to ‘dry’ it which, to me, really means to draw it in a way that’s more ‘permanent’ than the way it’s been drawn so far. The ‘wet’/’dry’ terminology captures that well.
I need to transport the ink strokes from the OnStrokesCollected event handler over into the OnDraw handler so that they can be drawn.
The two modified functions are as below;
void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
this.wetInkStrokes = this.inkSync.BeginDry();
this.canvasControl.Invalidate();
}
void OnDraw(
CanvasControl sender,
CanvasDrawEventArgs args)
{
// If we only need to add some newly 'wet' ink then we do that.
if (this.wetInkStrokes != null)
{
args.DrawingSession.DrawInk(this.wetInkStrokes);
this.wetInkStrokes = null;
this.inkSync.EndDry();
}
}
IReadOnlyList<InkStroke> wetInkStrokes;
InkSynchronizer inkSync;
}
This is better than what I had before in that now when I draw a stroke and then release the pen, the stroke does not disappear 
However, when I draw and release the next stroke, the first stroke disappears.
Naturally, that’s because Win2D isn’t a retained graphics system and so I need to draw everything every time, I can’t just forget the ink that has gone before.
That said, I can do something a little like retained graphics in that I can use a CanvasRenderTarget and keep it around as long as I can. Most of the code got changed to make that happen and so I’m including all of it again;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
// Have to switch on custom drying early in the process and before
// the InkCanvas has Loaded.
this.inkSync = this.inkCanvas.InkPresenter.ActivateCustomDrying();
this.inkCanvas.InkPresenter.StrokesCollected += OnStrokesCollected;
this.canvasControl.SizeChanged += OnCanvasControlSizeChanged;
}
void OnCanvasControlSizeChanged(object sender, SizeChangedEventArgs e)
{
this.renderTarget?.Dispose();
this.renderTarget = null;
}
void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
this.wetInkStrokes = this.inkSync.BeginDry();
this.canvasControl.Invalidate();
}
void OnDraw(
CanvasControl sender,
CanvasDrawEventArgs args)
{
bool createRenderTarget = (this.renderTarget == null);
if (createRenderTarget)
{
this.renderTarget = new CanvasRenderTarget(
this.canvasControl,
this.canvasControl.Size);
}
using (var renderSession = this.renderTarget.CreateDrawingSession())
{
if (createRenderTarget)
{
// have to clear out the render target on first use.
renderSession.Clear(Colors.Transparent);
}
// If we only need to add some newly 'wet' ink then we do that.
if (this.wetInkStrokes != null)
{
// Draw the ink to the render target.
renderSession.DrawInk(this.wetInkStrokes);
this.wetInkStrokes = null;
this.inkSync.EndDry();
}
}
// Draw the render target to the screen.
args.DrawingSession.DrawImage(this.renderTarget);
}
CanvasRenderTarget renderTarget;
IReadOnlyList<InkStroke> wetInkStrokes;
InkSynchronizer inkSync;
}
Now, this works a little better still in that I can draw N ink strokes and they all survive around on the screen but only up until the point where I (e.g.) resize the window and then I need to redraw all the ink which, so far, I haven’t captured so maybe it’s time to do that now. Most of the code changed again so listing it all out here;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
// Have to switch on custom drying early in the process and before
// the InkCanvas has Loaded.
this.inkSync = this.inkCanvas.InkPresenter.ActivateCustomDrying();
this.inkCanvas.InkPresenter.StrokesCollected += OnStrokesCollected;
this.canvasControl.SizeChanged += OnCanvasControlSizeChanged;
this.strokeContainer = new InkStrokeContainer();
}
void OnCanvasControlSizeChanged(object sender, SizeChangedEventArgs e)
{
this.renderTarget?.Dispose();
this.renderTarget = null;
}
void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
this.wetInkStrokes = this.inkSync.BeginDry();
this.strokeContainer.AddStrokes(args.Strokes);
this.canvasControl.Invalidate();
}
void OnDraw(
CanvasControl sender,
CanvasDrawEventArgs args)
{
bool createRenderTarget = (this.renderTarget == null);
if (createRenderTarget)
{
this.renderTarget = new CanvasRenderTarget(
this.canvasControl,
this.canvasControl.Size);
}
using (var renderSession = this.renderTarget.CreateDrawingSession())
{
if (createRenderTarget)
{
// have to clear out the render target on first use.
renderSession.Clear(Colors.Transparent);
}
var strokesToDraw =
this.wetInkStrokes ?? this.strokeContainer.GetStrokes();
// Draw the ink to the render target.
renderSession.DrawInk(strokesToDraw);
if (this.wetInkStrokes != null)
{
this.wetInkStrokes = null;
this.inkSync.EndDry();
}
}
// Draw the render target to the screen.
args.DrawingSession.DrawImage(this.renderTarget);
}
InkStrokeContainer strokeContainer;
CanvasRenderTarget renderTarget;
IReadOnlyList<InkStroke> wetInkStrokes;
InkSynchronizer inkSync;
}
and that seemed to work fairly well although I’m unsure whether the call to EndDry here is at the right point or whether it should really be done after the last DrawImage call is made.
It’s also worth saying that the official Win2d sample that I referenced does some more complex work around calling EndDry after the CanvasControl has actually rendered so it’s worth checking that out and this discussion thread around it.
One of the reasons for using custom drying in the first place is to allow ink content to be layered with other content – e.g. using InkCanvas in its default mode wouldn’t allow a scenario like this one below;
and so here in the Z-Order we have an InkCanvas on top of a Win2D CanvasControl just like we did before. However, the Win2D CanvasControl is then drawing a Circle, drawing the ink on top of it and then drawing a square on top of that and, to be clear, it’s a slightly weird effect I’m cooking up because the ‘wet’ ink was drawn “over” the square as the screenshot below shows;
but it ‘dries’ under the rectangle and so it ‘moves’ when you release the pen which feels a bit odd but this technique opens up the potential to layer ink like this with other content.
It also opens up the possibility of drawing the ink in any number of custom ways and I borrowed an idea from that official Win2D inking sample to draw the ink using an outline geometry instead of just asking Win2D to draw it directly.
I added a little ToggleSwitch to my UI;
<Grid
Background="PowderBlue"
x:Name="grid">
<w2d:CanvasControl
x:Name="canvasControl"
xmlns:w2d="using:Microsoft.Graphics.Canvas.UI.Xaml"
Draw="OnDraw" />
<InkCanvas
x:Name="inkCanvas" />
<ToggleSwitch
OffContent="Regular"
OnContent="Geometry"
IsOn="False"
HorizontalAlignment="Center"
VerticalAlignment="Bottom"
Toggled="OnDrawingModeChanged" />
</Grid>
and I also updated the default size of my pen to be 4×4 and the colour to be white. This then gives me this UI;
and it changed a bit of the code so here’s the entire code-behind file for completeness;
namespace CustomDrying
{
using Microsoft.Graphics.Canvas;
using Microsoft.Graphics.Canvas.Geometry;
using Microsoft.Graphics.Canvas.UI.Xaml;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;
using Windows.Foundation;
using Windows.UI;
using Windows.UI.Input.Inking;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
// Have to switch on custom drying early in the process and before
// the InkCanvas has Loaded.
this.inkSync = this.inkCanvas.InkPresenter.ActivateCustomDrying();
this.inkCanvas.InkPresenter.StrokesCollected += OnStrokesCollected;
var defaultAttr = this.inkCanvas.InkPresenter.CopyDefaultDrawingAttributes();
defaultAttr.Size = new Size(4, 4);
defaultAttr.Color = Colors.White;
this.inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(defaultAttr);
this.canvasControl.SizeChanged += OnCanvasControlSizeChanged;
this.strokeContainer = new InkStrokeContainer();
this.drawWithGeometry = false;
}
void OnCanvasControlSizeChanged(object sender, SizeChangedEventArgs e)
{
this.renderTarget?.Dispose();
this.renderTarget = null;
}
void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
this.wetInkStrokes = this.inkSync.BeginDry();
this.strokeContainer.AddStrokes(args.Strokes);
this.canvasControl.Invalidate();
}
void OnDraw(
CanvasControl sender,
CanvasDrawEventArgs args)
{
bool createRenderTarget = (this.renderTarget == null);
if (createRenderTarget)
{
this.renderTarget = new CanvasRenderTarget(
this.canvasControl,
this.canvasControl.Size);
}
var centre = new Vector2(
(float)sender.ActualWidth / 2.0f,
(float)sender.ActualHeight / 2.0f);
// Draw a circle at the 'back' in the z-order.
args.DrawingSession.FillCircle(
centre,
Math.Min((float)sender.ActualWidth / 2.5f, (float)sender.ActualHeight / 2.5f),
Colors.Red);
using (var renderSession = this.renderTarget.CreateDrawingSession())
{
if (createRenderTarget)
{
// have to clear out the render target on first use.
renderSession.Clear(Colors.Transparent);
}
var strokesToDraw =
this.wetInkStrokes ?? this.strokeContainer.GetStrokes();
// Draw the ink to the render target.
this.DrawInk(renderSession, strokesToDraw);
if (this.wetInkStrokes != null)
{
this.wetInkStrokes = null;
this.inkSync.EndDry();
}
}
// Draw the render target to the screen.
args.DrawingSession.DrawImage(this.renderTarget);
// Draw a rectangle 'over' that.
args.DrawingSession.FillRectangle(
new Windows.Foundation.Rect(
centre.X - 100,
centre.Y - 100,
200,
200),
Colors.Blue);
}
void OnDrawingModeChanged(object sender, RoutedEventArgs e)
{
this.drawWithGeometry = !this.drawWithGeometry;
}
void DrawInk(CanvasDrawingSession session,
IReadOnlyList<InkStroke> strokes)
{
if (!this.drawWithGeometry)
{
// Win2D already knows how to draw ink so let it do it the
// regular way.
session.DrawInk(strokes);
}
else
{
this.DrawInkWithGeometry(session, strokes);
}
}
void DrawInkWithGeometry(
CanvasDrawingSession ds,
IReadOnlyList<InkStroke> strokes)
{
var strokeStyle = new CanvasStrokeStyle
{
DashStyle = CanvasDashStyle.Dash
};
var geometry = CanvasGeometry.CreateInk(ds, strokes).Outline();
// We assume all strokes are the same colour. Not necessarily
// right.
var colour = strokes.First().DrawingAttributes.Color;
ds.DrawGeometry(geometry, colour, 1, strokeStyle);
}
InkStrokeContainer strokeContainer;
CanvasRenderTarget renderTarget;
IReadOnlyList<InkStroke> wetInkStrokes;
InkSynchronizer inkSync;
bool drawWithGeometry;
}
}
I daresay that I’ve got a few things wrong in there but it seems to work quite nicely and I learned quite a lot while putting it together incrementally and the sample and the article referenced earlier helped a lot. Of course, the sample should be your definitive place to go but I thought I’d write this down as it helped me get a better understanding than I had before I started on it.
Mike Taulty 