One of the things that Windows 8 did when it came along with its new app development model was to do some unification around the way in which “a device that can point” is represented.
In the early days, I found myself often looking for a “Mouse Down” style event only to have to keep reminding myself that it might not be a mouse that the user is using and so I needed to think in terms of “Pointers” rather than mice.
If I extend this out to code then I can quickly knock up a blank app in .NET which has a Canvas to draw on;
<Page x:Class="App209.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App209"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Canvas Background="Red"
PointerPressed="OnDown"
PointerReleased="OnUp"
PointerMoved="OnMoved"
x:Name="myCanvas"/>
</Grid>
</Page>
and then I could create a quick “scribble” like experience in that I can track the pointer events and draw lines between points;
namespace App209
{
using System.Collections.Generic;
using Windows.UI;
using Windows.UI.Input;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Shapes;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.pointers = new Dictionary<uint, PointerPoint>();
}
void OnDown(object sender, PointerRoutedEventArgs e)
{
this.pointers[e.Pointer.PointerId] = e.GetCurrentPoint(this.myCanvas);
}
void OnUp(object sender, PointerRoutedEventArgs e)
{
this.pointers.Remove(e.Pointer.PointerId);
}
void OnMoved(object sender, PointerRoutedEventArgs e)
{
if (this.pointers.ContainsKey(e.Pointer.PointerId))
{
PointerPoint origin = this.pointers[e.Pointer.PointerId];
PointerPoint current = e.GetCurrentPoint(this.myCanvas);
Line line = new Line();
line.Stroke = new SolidColorBrush(Colors.Black);
line.StrokeThickness = this.CalculateLineThickness(origin);
line.X1 = origin.Position.X;
line.Y1 = origin.Position.Y;
line.X2 = current.Position.X;
line.Y2 = current.Position.Y;
this.myCanvas.Children.Add(line);
this.pointers[e.Pointer.PointerId] = current;
}
}
int CalculateLineThickness(PointerPoint point)
{
int lineThickness = 1;
switch (point.PointerDevice.PointerDeviceType)
{
case Windows.Devices.Input.PointerDeviceType.Mouse:
break;
case Windows.Devices.Input.PointerDeviceType.Pen:
lineThickness = 2;
break;
case Windows.Devices.Input.PointerDeviceType.Touch:
lineThickness = 5;
break;
default:
break;
}
return (lineThickness);
}
Dictionary<uint, PointerPoint> pointers;
}
}
and then I can draw on this using both mouse and touch points to produce art of great future value;
and I can use multiple fingers (up to 10 I think on my Dell XPS 12) at the same time and draw with both the mouse and touch at the same time too. I can also use pen and that works just fine too.
When I look at the details of what’s being reported here in my OnDown method up above, I can see stuff like;
and so the OS knows that this is a touch point rather than a mouse point and has a tonne of info around what’s going on including;
and if I do the same thing with a mouse rather than a touch point then I’m going to see slightly different results and the same is true if I’m using a pen.
This low-level commonality flows through into higher level constructs in the sense that if I have a couple of controls on a screen like this;
<Page x:Class="App211.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App211"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<StackPanel Orientation="Horizontal" HorizontalAlignment="Center"
VerticalAlignment="Center">
<ListView MaxHeight="192">
<ListView.Items>
<x:Int32>1</x:Int32>
<x:Int32>2</x:Int32>
<x:Int32>3</x:Int32>
<x:Int32>4</x:Int32>
<x:Int32>5</x:Int32>
<x:Int32>6</x:Int32>
<x:Int32>7</x:Int32>
<x:Int32>8</x:Int32>
<x:Int32>9</x:Int32>
<x:Int32>10</x:Int32>
</ListView.Items>
</ListView>
<Button Content="Click Me" />
</StackPanel>
</Grid>
</Page>
then I can scroll that list and click the button using my finger on the touch screen;
or using the mouse or the pen which will surface additional UI like the scroll bar here;
and that’s all part of dealing with mice, touch and pen in any Windows 8.1 application although, naturally, not all user interfaces are going to work equally well across the 3 and, additionally, some user interfaces are going to specialise their handling of particular input devices.
Bringing in the Kinect
One possible role for a Kinect is to use it as a fancy kind of “pointer” device.
You could use the low level skeletal tracking bits to treat some part( s ) of a body as a pointer and somehow turn the Kinect into a form of pointer device. You’d have quite a lot of work to do though in deciding which bits of the body to track and you’d have to choose something against which to track them to (e.g. track the right hand relative to the hip or the head or somesuch) and so on.
While that’s do-able, it might not be the best plan unless you’ve got specific reasons for doing it because;
- your new mechanism might not be overly discoverable for the user unless you convince other app developers to follow the same scheme
- you’d have to write all that code yourself
Both of those can be avoided by using the bits that are talked about in the Channel 9 video below;
which reveals that there’s already framework support that’s been done around a specific set of gestures for using the hands of 1 or 2 users as an input mechanism. That work includes the steps needed around the user’s initial engagement with the device, defining a “PHIZ” interaction region for the user to move their hands within, rendering the hand position on screen as a set of cursors and for interaction models involving pressing, manipulating and linking that up with existing controls and so on.
The key to this seems to be the KinectRegion (I can’t find this documented here anywhere at the time of writing). This is a control that (as you’d see in the video) lives in the Microsoft.Kinect.Xaml.Controls assembly and inherits from ContentControl so that you can shove whatever kind of UI you’d like within one of those controls;
It’s worth pointing out that adding the reference above also brings in a reference to Microsoft.Kinect.Toolkit.Input which I’ll return to later;
If I take that original “scribble” example and change the UI such that it contains a KinectRegion as per below;
<Page x:Class="App209.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App209"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:k="using:Microsoft.Kinect.Xaml.Controls"
mc:Ignorable="d">
<k:KinectRegion>
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<Canvas Background="Red"
PointerPressed="OnDown"
PointerReleased="OnUp"
PointerMoved="OnMoved"
x:Name="myCanvas" />
</Grid>
</k:KinectRegion>
</Page>
then I can get in front of the sensor and do the “engagement gesture” (see picture above) and, instantly, I see that my hand is represented;
and that the open/grabbed nature of my hand is also represented;
I can also easily change between right and left hands and that works pretty seamlessly too.
It’s worth saying that I think I can take control of this “engage with the UI” experience by plugging in my own implementation of the interface IKinectEngagementManager (although I can’t find the docs on this yet) and introducing that implementation to the KinectRegion via its SetKinectOne/TwoPersonManualEngagement method rather than allowing it to default to the system engagment method.
Probably also worth saying that I think the cursors shown here can be turned off and replaced and so on – you’re not stuck with the hands that are shown in the pictures here.
That’s more advanced than where I’ve got to yet though – to continue my simple example, I can divide up the screen such that some of the available area is within a KinectRegion and some isn’t – here below I’m making the top left 2/3rds of the screen part of the KinectRegion;
<Page x:Class="App209.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App209"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:k="using:Microsoft.Kinect.Xaml.Controls"
mc:Ignorable="d">
<Grid Background="Azure">
<Grid.RowDefinitions>
<RowDefinition Height="2*" />
<RowDefinition Height="*" />
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="2*" />
<ColumnDefinition Width="*" />
</Grid.ColumnDefinitions>
<k:KinectRegion>
<Canvas Background="Red"
PointerPressed="OnDown"
PointerReleased="OnUp"
PointerMoved="OnMoved"
x:Name="myCanvas" />
</k:KinectRegion>
</Grid>
</Page>
that produces an “interesting” result as at the boundaries you can see that my “hand” is locked to that region no matter where I move it;
One of the things that I find interesting about the way in which KinectRegion behaves is that my first assumption would have been that I would be getting PointerMoved events as my “engaged” hand wandered over this region on the screen. That’s not the case though. I don’t get any pointer moved style events from the mouse/pen/touch code above when I’m using the Kinect here.
I tried to press down on the Canvas in question but that didn’t respond (it’s perhaps unusual to press on a Canvas rather than, say, a Button) until I tweaked a little attached property;
and then pressing on the Canvas gave me a visual response signifying that the press had been recognised;
but…none of this raises any of the pointer events that I might have been hoping for so I don’t see the same effect as switching between mouse/pen/touch although I clearly see this hand cursor moving around as though it was a pointer.
What gives? While the KinectRegion has integrated and given me this functionality, I don’t think that it goes as far as pretending to be a pointer like a mouse/pen/touch but it does come pretty close if I approach things a slightly different way.
Firstly, I’ll refactor my code a little;
namespace App209
{
using System.Collections.Generic;
using Windows.Foundation;
using Windows.UI;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Shapes;
using WindowsPreview.Kinect.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.canvasPositions = new Dictionary<uint, Point>();
}
void TrackDown(uint pointerId, Point point)
{
this.canvasPositions[pointerId] = point;
}
void TrackUp(uint pointerId)
{
this.canvasPositions.Remove(pointerId);
}
void OnPointerDown(object sender, PointerRoutedEventArgs e)
{
var pt = e.GetCurrentPoint(this.myCanvas);
this.TrackDown(e.Pointer.PointerId, pt.Position);
}
void OnPointerUp(object sender, PointerRoutedEventArgs e)
{
this.TrackUp(e.Pointer.PointerId);
}
void OnPointerMoved(object sender, PointerRoutedEventArgs e)
{
if (this.canvasPositions.ContainsKey(e.Pointer.PointerId))
{
var pt = e.GetCurrentPoint(this.myCanvas);
JoinCanvasLine(
e.Pointer.PointerId,
(PointerDeviceType)e.Pointer.PointerDeviceType,
pt.Position);
}
}
private void JoinCanvasLine(uint pointerId, PointerDeviceType deviceType, Point currentCanvasPosition)
{
if (this.canvasPositions.ContainsKey(pointerId))
{
var previousCanvasPosition = this.canvasPositions[pointerId];
Line line = new Line();
line.Stroke = new SolidColorBrush(Colors.Black);
line.StrokeThickness = this.CalculateLineThickness(deviceType);
line.X1 = previousCanvasPosition.X;
line.Y1 = previousCanvasPosition.Y;
line.X2 = currentCanvasPosition.X;
line.Y2 = currentCanvasPosition.Y;
this.myCanvas.Children.Add(line);
}
this.canvasPositions[pointerId] = currentCanvasPosition;
}
int CalculateLineThickness(PointerDeviceType type)
{
int lineThickness = 1;
switch (type)
{
case PointerDeviceType.Mouse:
break;
case PointerDeviceType.Pen:
lineThickness = 2;
break;
case PointerDeviceType.Touch:
lineThickness = 5;
break;
case PointerDeviceType.Kinect:
lineThickness = 8;
break;
default:
break;
}
return (lineThickness);
}
Dictionary<uint, Point> canvasPositions;
}
}
with the attempt being to abstract some of the code away from being mouse/pointer/pen specific towards being more open to movement events from another kind of device like Kinect.
With that in place, I can pick up events from the Kinect that look a lot like the existing pointer events but aren’t related to them in the inheritance sense. These events are generated from the KinectCoreWindow class. This is analogous to the events that I’ve been handling so far which have their origins in the CoreWindow class. I can handle events like PointerEntered/PointerExited/PointerMoved which are passed an event args of type KinectPointerEventArgs which then contains a KinectPointerPoint value.
That KinectPointerPoint value delivers a bunch of information.
and there’s both a Position a RawPosition and then the Properties value here is of type KinectPointerPointProperties which has more info;
I can get information around whether the user is “engaged” or not, “in range” or not, whether it’s a left/right hand, the rotation of the hand and there’s info here to link this back to skeletal tracking data if needed. There are also 3 (count them!) position values across Position, RawPosition, UnclampedPosition. I think having the first 2 is in line with how mouse/pen/touch operate and the 3rd one is an addition specific to Kinect.
How to make use of this? I could add a little more code to try and handle those events;
namespace App209
{
using System.Collections.Generic;
using Windows.Foundation;
using Windows.UI;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Shapes;
using WindowsPreview.Kinect.Input;
public sealed partial class MainPage : Page
{
public MainPage()
{
this.InitializeComponent();
this.canvasPositions = new Dictionary<uint, Point>();
this.Loaded += this.OnLoaded;
}
void OnLoaded(object sender, Windows.UI.Xaml.RoutedEventArgs e)
{
var window = KinectCoreWindow.GetForCurrentThread();
window.PointerEntered += OnKinectPointerEntered;
window.PointerExited += OnKinectPointerExited;
window.PointerMoved += OnKinectPointerMoved;
}
Point ScaleKinectPointToCanvasPoint(Point kinectPoint)
{
return (
new Point()
{
X = kinectPoint.X * this.myCanvas.ActualWidth,
Y = kinectPoint.Y * this.myCanvas.ActualHeight
}
);
}
bool IsApplicableKinectPoint(KinectPointerPoint kinectPoint)
{
return (kinectPoint.Properties.IsEngaged &&
kinectPoint.Properties.IsInRange);
}
void OnKinectPointerEntered(KinectCoreWindow sender, KinectPointerEventArgs args)
{
if (IsApplicableKinectPoint(args.CurrentPoint))
{
var pt = ScaleKinectPointToCanvasPoint(args.CurrentPoint.Position);
this.TrackDown(args.CurrentPoint.PointerId, pt);
}
}
void OnKinectPointerMoved(KinectCoreWindow sender, KinectPointerEventArgs args)
{
if (IsApplicableKinectPoint(args.CurrentPoint))
{
var pt = ScaleKinectPointToCanvasPoint(args.CurrentPoint.Position);
this.JoinCanvasLine(
args.CurrentPoint.PointerId,
PointerDeviceType.Kinect,
ScaleKinectPointToCanvasPoint(args.CurrentPoint.Position));
}
}
void OnKinectPointerExited(KinectCoreWindow sender, KinectPointerEventArgs args)
{
if (IsApplicableKinectPoint(args.CurrentPoint))
{
this.TrackUp(args.CurrentPoint.PointerId);
}
}
void TrackDown(uint pointerId, Point point)
{
this.canvasPositions[pointerId] = point;
}
void TrackUp(uint pointerId)
{
this.canvasPositions.Remove(pointerId);
}
void OnPointerDown(object sender, PointerRoutedEventArgs e)
{
var pt = e.GetCurrentPoint(this.myCanvas);
this.TrackDown(e.Pointer.PointerId, pt.Position);
}
void OnPointerUp(object sender, PointerRoutedEventArgs e)
{
this.TrackUp(e.Pointer.PointerId);
}
void OnPointerMoved(object sender, PointerRoutedEventArgs e)
{
if (this.canvasPositions.ContainsKey(e.Pointer.PointerId))
{
var pt = e.GetCurrentPoint(this.myCanvas);
JoinCanvasLine(
e.Pointer.PointerId,
(PointerDeviceType)e.Pointer.PointerDeviceType,
pt.Position);
}
}
private void JoinCanvasLine(uint pointerId, PointerDeviceType deviceType, Point currentCanvasPosition)
{
if (this.canvasPositions.ContainsKey(pointerId))
{
var previousCanvasPosition = this.canvasPositions[pointerId];
Line line = new Line();
line.Stroke = new SolidColorBrush(Colors.Black);
line.StrokeThickness = this.CalculateLineThickness(deviceType);
line.X1 = previousCanvasPosition.X;
line.Y1 = previousCanvasPosition.Y;
line.X2 = currentCanvasPosition.X;
line.Y2 = currentCanvasPosition.Y;
this.myCanvas.Children.Add(line);
}
this.canvasPositions[pointerId] = currentCanvasPosition;
}
int CalculateLineThickness(PointerDeviceType type)
{
int lineThickness = 1;
switch (type)
{
case PointerDeviceType.Mouse:
break;
case PointerDeviceType.Pen:
lineThickness = 2;
break;
case PointerDeviceType.Touch:
lineThickness = 5;
break;
case PointerDeviceType.Kinect:
lineThickness = 8;
break;
default:
break;
}
return (lineThickness);
}
Dictionary<uint, Point> canvasPositions;
}
}
and so I’m handling 6 different types of events now and sending them largely all to the same drawing code and it “kind of works”;
but, the problem is that I have no real equivalent of “pointer pressed/released” events here and so this code pretty much just draws from the moment that the user is engaged until the moment that they try to disengage. The code clearly needs something a little more like pointer down/up.
It seemed obvious to use the grip/release gesture that the KinectRegion clearly already knows about to implement it but, to be honest, I had to do quite a bit of head-scratching to figure out at what level that gesture was being understood by the SDK bits. As far as I can figure out, there’s a model in the implementation that considers items to be “manipulatable” or “pressable” and that seems to be the way to plug in.
Now, this is where I wander out onto even thinner ice as I’m not 100% sure that I know how this works but I took a little look at the sample that I got from this thread;
and figured that perhaps what I needed to try and do was to make a “manipulatable Canvas”. Hence – my XAML shifted to;
<Page x:Class="App209.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App209"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:k="using:Microsoft.Kinect.Xaml.Controls"
mc:Ignorable="d">
<Grid Background="Azure">
<Grid.RowDefinitions>
<RowDefinition Height="2*" />
<RowDefinition Height="*" />
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="2*" />
<ColumnDefinition Width="*" />
</Grid.ColumnDefinitions>
<k:KinectRegion x:Name="region" >
<local:PointerDrawCanvas Background="Red" />
</k:KinectRegion>
</Grid>
</Page>
where this new PointerDrawCanvas is trying to be “all things to all men” in the sense that it’s trying to handle mouse/pen/touch and also the Kinect manipulation gesture represented by the “gripped hand drag”. I sketched out this attempt based on the code I’d got so far;
public class PointerDrawCanvas : Canvas, IKinectControl
{
public PointerDrawCanvas()
{
this.canvasPositions = new Dictionary<uint, Point>();
this.PointerPressed += this.OnPointerDown;
this.PointerMoved += this.OnPointerMoved;
this.PointerReleased += this.OnPointerUp;
}
// IKinectControl...
public IKinectController CreateController(IInputModel inputModel, KinectRegion kinectRegion)
{
return (new PointerDrawableCanvasController(inputModel, kinectRegion));
}
public bool IsManipulatable
{
get { return (true); }
}
public bool IsPressable
{
get { return (false); }
}
internal Point ScaleKinectPointToCanvasPoint(Point kinectPoint)
{
return (
new Point()
{
X = kinectPoint.X * this.ActualWidth,
Y = kinectPoint.Y * this.ActualHeight
}
);
}
internal void TrackDown(uint pointerId, Point point)
{
this.canvasPositions[pointerId] = point;
}
internal void TrackUp(uint pointerId)
{
this.canvasPositions.Remove(pointerId);
}
internal void JoinCanvasLine(uint pointerId, PointerDeviceType deviceType, Point currentCanvasPosition)
{
if (this.canvasPositions.ContainsKey(pointerId))
{
var previousCanvasPosition = this.canvasPositions[pointerId];
Line line = new Line();
line.Stroke = new SolidColorBrush(Colors.Black);
line.StrokeThickness = this.CalculateLineThickness(deviceType);
line.X1 = previousCanvasPosition.X;
line.Y1 = previousCanvasPosition.Y;
line.X2 = currentCanvasPosition.X;
line.Y2 = currentCanvasPosition.Y;
this.Children.Add(line);
}
this.canvasPositions[pointerId] = currentCanvasPosition;
}
void OnPointerDown(object sender, PointerRoutedEventArgs e)
{
var pt = e.GetCurrentPoint(this);
this.TrackDown(e.Pointer.PointerId, pt.Position);
}
void OnPointerUp(object sender, PointerRoutedEventArgs e)
{
this.TrackUp(e.Pointer.PointerId);
}
void OnPointerMoved(object sender, PointerRoutedEventArgs e)
{
if (this.canvasPositions.ContainsKey(e.Pointer.PointerId))
{
var pt = e.GetCurrentPoint(this);
JoinCanvasLine(
e.Pointer.PointerId,
(PointerDeviceType)e.Pointer.PointerDeviceType,
pt.Position);
}
}
int CalculateLineThickness(PointerDeviceType type)
{
int lineThickness = 1;
switch (type)
{
case PointerDeviceType.Mouse:
break;
case PointerDeviceType.Pen:
lineThickness = 2;
break;
case PointerDeviceType.Touch:
lineThickness = 5;
break;
case PointerDeviceType.Kinect:
lineThickness = 8;
break;
default:
break;
}
return (lineThickness);
}
Dictionary<uint, Point> canvasPositions;
}
and then I have this “controller” which this class creates in response to IKinectControl.CreateController;
public class PointerDrawableCanvasController : IKinectManipulatableController
{
IManipulatableModel inputModel;
KinectRegion kinectRegion;
public PointerDrawableCanvasController(IInputModel inputModel, KinectRegion kinectRegion)
{
this.inputModel = (IManipulatableModel)inputModel;
this.kinectRegion = kinectRegion;
// TODO: need to remove these event handlers at some point perhaps?
this.inputModel.ManipulationStarted += OnManipulationStarted;
this.inputModel.ManipulationUpdated += OnManipulationUpdated;
this.inputModel.ManipulationCompleted += OnManipulationCompleted;
}
void OnManipulationStarted(IManipulatableModel sender, KinectManipulationStartedEventArgs args)
{
var pt = this.Canvas.ScaleKinectPointToCanvasPoint(args.Position);
this.Canvas.TrackDown(sender.CapturedPointerId, pt);
}
void OnManipulationUpdated(IManipulatableModel sender, KinectManipulationUpdatedEventArgs args)
{
var pt = this.Canvas.ScaleKinectPointToCanvasPoint(args.Position);
this.Canvas.JoinCanvasLine(sender.CapturedPointerId, PointerDeviceType.Kinect, pt);
}
void OnManipulationCompleted(IManipulatableModel sender, KinectManipulationCompletedEventArgs args)
{
var pt = this.Canvas.ScaleKinectPointToCanvasPoint(args.Position);
this.Canvas.TrackUp(sender.CapturedPointerId);
}
PointerDrawCanvas Canvas
{
get
{
return ((PointerDrawCanvas)this.inputModel.Element);
}
}
public FrameworkElement Element
{
get { return (this.inputModel.Element as FrameworkElement); }
}
public ManipulatableModel ManipulatableInputModel
{
get { return (this.inputModel as ManipulatableModel); }
}
}
and that’s pretty much it – I’m not sure about using the IManipulatableModel.CapturedPointerId in the controller code – that might be dubious. More generally, none of this code does the right thing around capturing/releasing pointers but this seemed to give me what I was basically looking for in that I can now draw with the hand gesture as in the video below;
so I more or less got to what I wanted and I learned quite a bit along the way and the code for this is here if you want it.
Going back to the example that I was using where I had a couple of controls – I can wrap a KinectRegion into that example too;
<Page x:Class="App211.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:App211"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
xmlns:k="using:Microsoft.Kinect.Xaml.Controls">
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<k:KinectRegion>
<StackPanel Orientation="Horizontal"
HorizontalAlignment="Center"
VerticalAlignment="Center">
<ListView MaxHeight="192"
FontSize="24"
Padding="48">
<ListView.Items>
<x:Int32>1</x:Int32>
<x:Int32>2</x:Int32>
<x:Int32>3</x:Int32>
<x:Int32>4</x:Int32>
<x:Int32>5</x:Int32>
<x:Int32>6</x:Int32>
<x:Int32>7</x:Int32>
<x:Int32>8</x:Int32>
<x:Int32>9</x:Int32>
<x:Int32>10</x:Int32>
</ListView.Items>
</ListView>
<Button Content="Click Me"
Padding="48" />
</StackPanel>
</k:KinectRegion>
</Grid>
</Page>
and that “just works” albeit it’s probably not presenting a UI that a user would actually want to use but the mechanics are there as the quick video capture below shows;
and you should refer back to the video above about recommendations around such UIs and minimum sizes and so on and also around using other controls like the KinectUserViewer in order to help the user figure out how the device is viewing them. But, for now, I learned a few things about pointer events and about how regular controls can be plugged into a “Kinectable” UI.
Mike Taulty 