There’s a demo in this video (at the 7min mark, sorry for not being able to directly link to it here);
where Brian seems to tap on a map with his pen to mark one spot, tap again to mark another and then draw an ink line between the two points and the map responds by giving a measurement of the line that’s been drawn and can use that as the basis for getting directions.
That may or may not be a feature of the Maps app on the Windows 10 Anniversary Edition, we’ll have to wait and see what happens when the OS ships but it’s a cool feature if it’s there 
What interested me more though was how this might be achieved? That is – how do you combine the platform pieces which are primarily;
such that you can ink on a map in this way without losing the regular capabilities of the MapControl?
I thought about this for a while and tried a few approaches and it wasn’t as simple as I might have hoped it would be.
The Ideal Approach
In an ideal world, I would place an InkCanvas on top of a MapControl in the z-order and I’d then somehow tell the InkCanvas to pass through all non-pen related events to the MapControl underneath it while waiting for some kind of sequence of;
- Tap to mark position 1
- Tap to mark position 2
- Line drawn between position 1 and 2
which it could then (perhaps) pass down to the MapControl to work out routes or similar.
The challenge with this is that the InkCanvas is very definitely not a transparent control in the sense that if you put it over some other content then pointer events will not reach that underlying content. That’s even the case if you tell the InkCanvas to set its inking mode to process only pen events (the default). This isn’t surprising in the XAML world, it’s just “how it is”, events don’t travel through controls.
It might be possible to capture all the pointer events on the InkCanvas and then somehow manually ‘pass them on’ to the underlying MapControl but I suspect that will always be a little bit ‘leaky’ in that I doubt that the MapControl surfaces APIs which can perfectly mimic the way in which it responds to gestures like pinch, rotate, etc.
I didn’t think that approach would be viable and so I set about trying a few other approaches which all centre around the assumption that mouse/touch events should drive the map interaction and pen events should (depending on context) be picked up by an InkCanvas that needs to be activated on demand.
That activation means being able to pick up pointer events at some level ‘over’ or ‘within’ the MapControl so as to be able to tell when a pen is being used versus when a mouse/touch are being used and I tried a few ways of doing this as below…
Approach One – Ask the Map.
The MapControl has events like MapTapped which feels like it might be useful but I got blocked in the early stages because the event abstracts away the details of the Pointer event which caused it to fire.
That is, if I handle MapTapped then I can’t tell whether the event was fired by a pen or by mouse/touch and so I don’t know whether to make some InkCanvas visible or not.
Initially, then, these sorts of events don’t really seem to help and so I went on to another approach.
Approach Two – Handle Pointer Events ‘Around’ the Map
My next attempt was to try and use lower-level pointer events around the MapControl in the sense that I’m trying to get hold of the pointer events either before the MapControl gets them or maybe after it gets them.
This turns out to be ‘a bit more difficult’ than I’d hoped for and it seems that this is because the MapControl is quite ‘hungry’ when it comes to pointer events.
Here’s a quick example. If I have this UI;
<Page
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:MapApp"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:Maps="using:Windows.UI.Xaml.Controls.Maps"
x:Class="MapApp.MainPage"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"
PointerEntered="OnGridPointerEntered"
PointerExited="OnGridPointerExited"
PointerPressed="OnGridPointerPressed">
<Button
HorizontalAlignment="Stretch"
VerticalAlignment="Stretch"
Content="Button" />
</Grid>
</Page>
with this code-behind;
namespace MapApp
{
using System.Diagnostics;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
}
private void OnGridPointerEntered(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerEntered));
}
private void OnGridPointerExited(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerExited));
}
private void OnGridPointerPressed(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerPressed));
}
}
}
Then I find that as I move my mouse into and out of this Window and click on the button then I see these traces in my output window;
Ok, so I get PointerEntered and PointerExited and I’m not entirely surprised that I don’t get PointerPressed because I have a Button taking all of the available space and Button turns raw pointer events into some higher order event “Click” so if I wanted to see those events as well I’d have to be a bit more sneaky;
namespace MapApp
{
using System.Diagnostics;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs args)
{
this.AddHandler(
UIElement.PointerPressedEvent,
new PointerEventHandler(
(s,e) =>
{
Debug.WriteLine("The real pressed handler");
}
),
true);
}
void OnGridPointerEntered(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerEntered));
}
void OnGridPointerExited(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerExited));
}
void OnGridPointerPressed(object sender, PointerRoutedEventArgs e)
{
Debug.WriteLine(nameof(OnGridPointerPressed));
}
}
}
and then I get the output that I want;
and that’s all well understood and the event handler added by AddHandler fires before the Button’s Click event but let’s say that I now replace this Button with a MapControl;
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"
PointerEntered="OnGridPointerEntered"
PointerExited="OnGridPointerExited"
PointerPressed="OnGridPointerPressed">
<Maps:MapControl
HorizontalAlignment="Stretch"
VerticalAlignment="Stretch"/>
</Grid>
and I leave my code-behind alone then the results that I see are;
No, not a single event. No pointer entered, no pointer exited and no pointer pressed. It seems that the MapControl swallows all of those events and doesn’t give me much of a chance to get involved. Changing the UI to;
<Page
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:MapApp"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:Maps="using:Windows.UI.Xaml.Controls.Maps"
x:Class="MapApp.MainPage"
mc:Ignorable="d">
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Maps:MapControl
HorizontalAlignment="Stretch"
VerticalAlignment="Stretch"/>
</Grid>
</Page>
and the code behind to;
namespace MapApp
{
using System.Diagnostics;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs args)
{
this.AddHandler(
UIElement.PointerEnteredEvent,
new PointerEventHandler(
(s, e) =>
{
Debug.WriteLine("The real entered handler");
}
),
true);
this.AddHandler(
UIElement.PointerEnteredEvent,
new PointerEventHandler(
(s, e) =>
{
Debug.WriteLine("The real exited handler");
}
),
true);
this.AddHandler(
UIElement.PointerPressedEvent,
new PointerEventHandler(
(s,e) =>
{
Debug.WriteLine("The real pressed handler");
}
),
true);
}
}
}
makes no difference, I still don’t see any events firing here.
I wondered if I could do something at the “Window level” here, that is;
void OnLoaded(object sender, RoutedEventArgs args)
{
var window = CoreWindow.GetForCurrentThread();
window.PointerEntered += (s, e) =>
{
};
window.PointerExited += (s, e) =>
{
};
}
but those events don’t fire either once I’ve got a MapControl taking up the entirety of my UI’s space.
Approach Three – An InkCanvas In Front of the Map
I went back to explore the ‘ideal approach’ again, wondering whether I might be able to have the InkCanvas initially visible but in “passive” mode using some of the control’s capabilities around “unprocessed input”.
That is, I made my UI;
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Maps:MapControl/>
<InkCanvas
x:Name="inkCanvas" />
</Grid>
with some code;
namespace MapApp
{
using Windows.Devices.Input;
using Windows.UI.Core;
using Windows.UI.Input.Inking;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Maps;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.Loaded += OnLoaded;
}
void OnLoaded(object sender, RoutedEventArgs args)
{
inkCanvas.InkPresenter.InputDeviceTypes =
CoreInputDeviceTypes.Pen |
CoreInputDeviceTypes.Mouse |
CoreInputDeviceTypes.Touch;
inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.None;
inkCanvas.InkPresenter.UnprocessedInput.PointerEntered += OnEntered;
}
void OnEntered(Windows.UI.Input.Inking.InkUnprocessedInput sender, PointerEventArgs args)
{
if (args.CurrentPoint.PointerDevice.PointerDeviceType != PointerDeviceType.Pen)
{
this.inkCanvas.Visibility = Visibility.Collapsed;
}
else
{
this.inkCanvas.InkPresenter.InputProcessingConfiguration.Mode =
InkInputProcessingMode.Inking;
}
}
}
}
This “kind of works” in the sense that if I approach the UI with a pen then I can ink whereas if I approach the UI with a mouse or with touch I can manipulate the map.
I say “kind of” because it’s fair to say that if I approach the UI with touch then the very first touch event is used up in dismissing the InkCanvas and a user would notice that they always have to tap twice which isn’t perfect.
The bigger problem though is that this trick is essentially;
- Have an InkCanvas dormant over the whole UI
- Look at the first pointer event. If it is a pen then activate the InkCanvas. Otherwise, hide the InkCanvas so that further pointer events go to the map.
- When the pointer that we’re dealing with exits, put the InkCanvas back, dormant, over the whole UI so that we can repeat the process we’ve just gone through.
The challenge is that I can’t do that last step without some kind of “pointer exiting” event from the MapControl and I don’t get one.
Approach Four – Put the InkCanvas In the Map
This approach is a little like the first approach but I remembered that the MapControl has an ability to host XAML elements via its Children property. Now, the documentation talks about;
“Display XAML user interface elements such as a Button, a HyperlinkButton, or a TextBlock by adding them as Children of the MapControl. You can also add them to the MapItemsControl, or bind the MapItemsControl to an item or a collection of items.”
but it doesn’t say that I can’t add an InkCanvas in there I made sure that I was handling the MapTapped event on the MapControl;
<Grid
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Maps:MapControl
x:Name="map"
MapTapped="OnMapTapped"/>
</Grid>
and put some code behind that;
namespace MapApp
{
using Windows.Devices.Input;
using Windows.Foundation;
using Windows.UI;
using Windows.UI.Core;
using Windows.UI.Input.Inking;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Maps;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
}
void OnMapTapped(MapControl sender, MapInputEventArgs args)
{
if (this.map.MapElements.Count < 2)
{
var mapIcon = new MapIcon()
{
Location = args.Location,
NormalizedAnchorPoint = new Point(0.5, 0.5)
};
this.map.MapElements.Add(mapIcon);
// When you add a 2nd point of interest, we put an InkCanvas over the entire map
// waiting to see what you do next. If your next move is to use a pen then
// we'll wait for an ink stroke. If your next move is to use something
// other than a pen then we'll get rid of the InkCanvas and get out of your
// way.
if (this.map.MapElements.Count == 2)
{
this.InsertInkCanvasOverlay();
}
}
}
void InsertInkCanvasOverlay()
{
this.inkCanvas = new InkCanvas();
// Cover the map. Would need to handle resize as well really.
this.inkCanvas.Width = this.map.ActualWidth;
this.inkCanvas.Height = this.map.ActualHeight;
// We don't want it to ink just yet, we want to wait to see what happens next in
// terms of input mechansim.
this.inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.None;
// Make sure we watch all the events that interest us.
this.inkCanvas.InkPresenter.StrokeInput.StrokeEnded += OnStrokeEnded;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerEntered += OnInkCanvasPointerEntered;
// NB: with the pen, we will 'lose' the pointer if it moves much distance from the
// screen after tapping the 2nd point of interest. We could do a more detailed
// check using the "IsInContact" property but I haven't done yet.
this.inkCanvas.InkPresenter.UnprocessedInput.PointerExited += OnPointerExited;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerLost += OnPointerLost;
// Format it
var drawingAttr = this.inkCanvas.InkPresenter.CopyDefaultDrawingAttributes();
drawingAttr.PenTip = PenTipShape.Rectangle;
drawingAttr.Size = new Size(4, 4);
drawingAttr.IgnorePressure = true;
drawingAttr.Color = Colors.Orange;
this.inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(drawingAttr);
// put it into the map control.
this.map.Children.Add(this.inkCanvas);
}
void OnStrokeEnded(InkStrokeInput sender, PointerEventArgs args)
{
// TODO: Now there's just the small matter of taking that ink stroke
// and somehow turning it into a route.
}
void OnPointerLost(InkUnprocessedInput sender, PointerEventArgs args)
{
this.RemoveInkCanvas();
}
void OnPointerExited(InkUnprocessedInput sender, PointerEventArgs args)
{
this.RemoveInkCanvas();
}
void OnInkCanvasPointerEntered(InkUnprocessedInput sender, PointerEventArgs args)
{
if (args.CurrentPoint.PointerDevice.PointerDeviceType == PointerDeviceType.Pen)
{
this.inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.Inking;
}
else
{
this.RemoveInkCanvas();
}
}
void RemoveInkCanvas()
{
if (this.inkCanvas != null)
{
this.map.Children.Remove(this.inkCanvas);
this.inkCanvas.InkPresenter.StrokeInput.StrokeEnded -= this.OnStrokeEnded;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerEntered -= this.OnInkCanvasPointerEntered;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerExited -= this.OnPointerExited;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerLost -= this.OnPointerLost;
this.inkCanvas = null;
// for now, I also remove any points of interest on the map but it's a TBD
// whether you'd really want to do this.
this.map.MapElements.Clear();
}
}
InkCanvas inkCanvas;
}
}
So, essentially this leaves the map open to regular use but it uses the MapTapped event to add up to 2 points of interest onto the map (with no real way of clearing them yet). At the point where the 2nd POI gets added, an InkCanvas is inserted over the Map in “stealth” mode in that it isn’t inking or erasing so all input should be unprocessed.
Event handlers are added to the “UnprocessedInput” events such that if the next event comes from a Pen, we switch the mode into Inking and allow the ink to happen whereas if the next event comes from mouse or touch then we get rid of the InkCanvas.
That allows me to draw between two points as below;
which is ‘a start’ and leaves the next step which would be to figure out whether this can be turned into route directions.
From Ink Line to Route Directions
I’m not sure of what the best way is to change the collection of strokes that I’ve now captured into a driving/walking route so I went for the simple option and decided that I’d make some assumptions;
- It’s ok to take just a single ink stroke
- It’s safe to assume that a single ink stroke is not going to be made up of too many rendering segments
but these assumptions may not hold and so the code below might well need changing. It’s worth saying that it’s possible to get all the ink points within an ink stroke so an alternate (or better) solution might be to take some sampling of the ink points but I haven’t tried that to date.
Here’s the code with additional handlers for the StrokesCollected event of the InkPresenter.
namespace MapApp
{
using System;
using System.Collections.Generic;
using System.Linq;
using Windows.Devices.Geolocation;
using Windows.Devices.Input;
using Windows.Foundation;
using Windows.Services.Maps;
using Windows.UI;
using Windows.UI.Core;
using Windows.UI.Input.Inking;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Maps;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.tapPositions = new MapInputEventArgs[POI_COUNT];
this.map.SizeChanged += OnMapSizeChanged;
}
void OnMapTapped(MapControl sender, MapInputEventArgs args)
{
var mapElementCount = this.map.MapElements.Count;
if (mapElementCount < POI_COUNT)
{
this.tapPositions[mapElementCount] = args;
var mapIcon = new MapIcon()
{
Location = args.Location,
NormalizedAnchorPoint = new Point(0.5, 0.5)
};
this.map.MapElements.Add(mapIcon);
// When you add a 2nd point of interest, we put an InkCanvas over the entire map
// waiting to see what you do next. If your next move is to use a pen then
// we'll wait for an ink stroke. If your next move is to use something
// other than a pen then we'll get rid of the InkCanvas and get out of your
// way.
mapElementCount = this.map.MapElements.Count;
if (mapElementCount == POI_COUNT)
{
this.InsertInkCanvasOverlay();
}
}
}
void InsertInkCanvasOverlay()
{
this.inkCanvas = new InkCanvas();
// Cover the map.
this.inkCanvas.Width = this.map.ActualWidth;
this.inkCanvas.Height = this.map.ActualHeight;
// We don't want it to ink just yet, we want to wait to see what happens next in
// terms of input mechansim.
this.inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.None;
// Make sure we watch all the events that interest us.
this.inkCanvas.InkPresenter.StrokesCollected += OnStrokesCollected;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerEntered += OnInkCanvasPointerEntered;
// NB: with the pen, we will 'lose' the pointer if it moves much distance from the
// screen after tapping the 2nd point of interest. We could do a more detailed
// check using the "IsInContact" property but I haven't done yet.
this.inkCanvas.InkPresenter.UnprocessedInput.PointerExited += OnPointerExited;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerLost += OnPointerLost;
// Format it
var drawingAttr = this.inkCanvas.InkPresenter.CopyDefaultDrawingAttributes();
drawingAttr.PenTip = PenTipShape.Rectangle;
drawingAttr.Size = new Size(4, 4);
drawingAttr.IgnorePressure = true;
drawingAttr.Color = Colors.Orange;
this.inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(drawingAttr);
// put it into the map control.
this.map.Children.Add(this.inkCanvas);
}
async void OnStrokesCollected(InkPresenter sender, InkStrokesCollectedEventArgs args)
{
// We're going to take the first stroke, we're not going to try and
// somehow stitch together many strokes :-S
var firstStroke = args.Strokes.FirstOrDefault();
if (firstStroke != null)
{
// How to split this down into some waypoints?
var geoPoints = new List<Geopoint>();
geoPoints.Add(this.tapPositions[0].Location);
// TBD: Is using the rendering segments a good idea here or would
// it make more sense to take some sampling of the ink points
// along the way? Not sure.
foreach (var segment in firstStroke.GetRenderingSegments())
{
Geopoint geoPoint = null;
this.map.GetLocationFromOffset(segment.Position, out geoPoint);
geoPoints.Add(geoPoint);
}
geoPoints.Add(this.tapPositions[1].Location);
var routeResult = await MapRouteFinder.GetDrivingRouteFromWaypointsAsync(geoPoints);
// We should do something about failures too? 🙂
if (routeResult.Status == MapRouteFinderStatus.Success)
{
var routeView = new MapRouteView(routeResult.Route);
routeView.OutlineColor = Colors.Black;
routeView.RouteColor = Colors.Orange;
this.map.Routes.Add(routeView);
}
this.RemoveInkCanvas();
}
}
void OnMapSizeChanged(object sender, SizeChangedEventArgs e)
{
if (this.inkCanvas != null)
{
this.inkCanvas.Width = e.NewSize.Width;
this.inkCanvas.Height = e.NewSize.Height;
}
}
void OnPointerLost(InkUnprocessedInput sender, PointerEventArgs args)
{
this.RemoveInkCanvas();
this.RemovePOI();
}
void OnPointerExited(InkUnprocessedInput sender, PointerEventArgs args)
{
this.RemoveInkCanvas();
this.RemovePOI();
}
void OnInkCanvasPointerEntered(InkUnprocessedInput sender, PointerEventArgs args)
{
if (args.CurrentPoint.PointerDevice.PointerDeviceType == PointerDeviceType.Pen)
{
this.inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.Inking;
}
else
{
this.RemoveInkCanvas();
}
}
void RemoveInkCanvas()
{
if (this.inkCanvas != null)
{
this.map.Children.Remove(this.inkCanvas);
this.inkCanvas.InkPresenter.StrokesCollected -= this.OnStrokesCollected;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerEntered -= this.OnInkCanvasPointerEntered;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerExited -= this.OnPointerExited;
this.inkCanvas.InkPresenter.UnprocessedInput.PointerLost -= this.OnPointerLost;
this.inkCanvas = null;
}
}
void RemovePOI()
{
this.map.MapElements.Clear();
for (int i = 0; i < POI_COUNT; i++)
{
this.tapPositions[i] = null;
}
}
void OnClear(object sender, RoutedEventArgs e)
{
this.RemovePOI();
this.RemoveInkCanvas();
this.map.Routes.Clear();
}
static readonly int POI_COUNT = 2;
MapInputEventArgs[] tapPositions;
InkCanvas inkCanvas;
}
}
and that produces a “prototype” experience that you can see in the screen capture below;
That’s as far as I’ve gone with this to date, feel free to feedback in the comments here and let me know of better directions that I could have taken.
Mike Taulty 
Pingback: Windows 10, UWP and Experimenting with Inking onto a Map Control | Tech News