Windows 10, UWP and Composition–Light and Shade

One of the functional areas that the composition APIs offer are the abilities to add both lighting and shadowing effects to UI and I was interested in trying them out after some of my efforts with general effects like blur.

It’s worth saying that the official samples have a DropShadow sample and some lighting samples within them but I wanted to experiment for myself and especially around how I might create this kind of effect that the UI team shared on one of their newsletters – this picture;

image

had me thinking “how do you do that?” and it got my interest.

Shadows and Fog

I’ll admit that there’s another version of this blog post where I fumbled around a bit trying to understand what the Shadow property was for on a SpriteVisual and trying to make use of it in a way that really didn’t work out for me and caused me to go looking for the manual.

Fortunately, at the time that I was writing the post, I did a search and I found this brand new article on MSDN;

Using the Visual Layer with XAML

which has a recipe for drop shadows and that helped a lot and caused me to more or less start over and rework the post entirely as I had spent a little time barking up the wrong tree.

I also realised that I should have watched this video from //Build as well around effects, lighting and shadows before trying to make it up on my own Smile 

image

Armed with some of what I saw in these materials, my first experiment with shadows was to make a blank UI.

  <Grid
    x:Name="grid"
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">

  </Grid>

and to write a tiny bit of code which set up a SpriteVisual which did not paint itself at all but which did have a Shadow set for it as below;

      var compositor = ElementCompositionPreview.GetElementVisual(this.grid).Compositor;
      var spriteVisual = compositor.CreateSpriteVisual();
      spriteVisual.Size = new Vector2(100, 100);
      var dropShadow = compositor.CreateDropShadow();
      dropShadow.Offset = new Vector3(10, 10, 0);
      dropShadow.Color = Colors.Orange;
      spriteVisual.Shadow = dropShadow;
      ElementCompositionPreview.SetElementChildVisual(this.grid, spriteVisual);

and the interesting thing to me was that this Visual does indeed paint a shadow even though it doesn’t paint any content.

My earlier attempts had assumed something quite different and that’s why I’d fumbled around quite a lot before getting to what now seems like a simple realisation – a SpriteVisual can paint both content and a shadow and, by default, the shadow is rectangular;

Capture

The other key part of what I learned in that recipe is that the DropShadow has a Mask property that you can use to shape the shadow.

I’m unsure how to get hold of such a mask for arbitrary content but, from the recipes post, a new method has been added to Image, TextBlock and ShapeGetAlphaMask() which makes this very easy for those specific element types.

So, if my challenge was to add a shadow to a TextBlock then my UI can become;

  <Grid
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid HorizontalAlignment="Center" VerticalAlignment="Center">
      <Grid
        x:Name="grid" />
      <TextBlock
        x:Name="txtBlock"
        Text="Drop Shadow"
        FontSize="48"
        HorizontalAlignment="Center"
        VerticalAlignment="Center" />
    </Grid>
  </Grid>

and my code can become;

      var compositor = ElementCompositionPreview.GetElementVisual(this.grid).Compositor;
      var spriteVisual = compositor.CreateSpriteVisual();
      spriteVisual.Size = this.grid.RenderSize.ToVector2();

      var dropShadow = compositor.CreateDropShadow();
      dropShadow.Mask = this.txtBlock.GetAlphaMask();
      dropShadow.Offset = new Vector3(10, 10, 0);
      spriteVisual.Shadow = dropShadow;

      ElementCompositionPreview.SetElementChildVisual(this.grid, spriteVisual);

and that works out well;

Capture

but I think the heavy lifting here is really being done by TextBlock.GetAlphaMask() and I wondered how I might do this for arbitrary content in the absence of a helper method?

For instance, TextBox doesn’t have a GetAlphaMask() property and neither does Slider and so on.

It’s worth noting that in the official samples, there’s a use of a circular image to mask a shadow in the “Shadow Playground” such that you can have either a rectangular shadow;

Capture

or a circular one;

Capture

and that makes it fairly clear that if you know the shape of the mask up-front then you can represent it by an image and use that to mask the shadow.

But, let’s say that I do have arbitrary content in XAML, how do I apply a shadow to it? Something like a Slider?

I searched around a little and found a similar question here;

Are shadows only for images?

and that was useful to me for 2 reasons.

One was that my local samples clone had got out of step with the remote (and I hadn’t noticed) and so I’d been staring at out of date samples thinking that my git pull had been updating them when it hadn’t been.

The other was that it seemed to answer my question with the supplied CompositionShadow user control until I took a look into it and realised that it’s hard-wired to assume that the content of the control is one of the three that have had a GetAlphaMask() method added to them – i.e. either a TextBlock, Image or Shape.

This then led me back to the same question that I started with which would be how I could apply a shadow to an arbitrary piece of UI which isn’t;

  • Image, TextBlock, Shape
  • Rectangular
  • Defined by a pre-defined shape that can be masked by an image of that shape

Now, I must admit that maybe this question isn’t particularly important given that this list probably covers 95%+ of the useful cases anyway but it still left me curious and I’m not sure how I’d do it at the time of writing other than to possibly try and render the XAML element as an image and then use that as a mask of some sort.

I gave it a quick try even though it’s likely to be prohibitively expensive to do it and I use the CompositionImageBrush class that I wrote in this post and with this UI;


  <Grid
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid
      HorizontalAlignment="Center"
      VerticalAlignment="Center">
      <Grid
        x:Name="grid"
        HorizontalAlignment="Stretch"
        VerticalAlignment="Stretch" />
      <Slider
        x:Name="slider"
        Width="284" />
    </Grid>
  </Grid>

and this chunk of code-behind;

    async void OnLoaded(object sender, RoutedEventArgs e)
    {
      var gridVisual = ElementCompositionPreview.GetElementVisual(this.grid);

      var spriteVisual = gridVisual.Compositor.CreateSpriteVisual();
      spriteVisual.Size = this.slider.RenderSize.ToVector2();

      var bitmap = new RenderTargetBitmap();

      await bitmap.RenderAsync(
        this.slider,
        (int)this.slider.ActualWidth,
        (int)this.slider.ActualHeight);

      var pixels = await bitmap.GetPixelsAsync();

      using (var softwareBitmap = SoftwareBitmap.CreateCopyFromBuffer(
        pixels,
        BitmapPixelFormat.Bgra8,
        bitmap.PixelWidth,
        bitmap.PixelHeight,
        BitmapAlphaMode.Premultiplied))
      {
        var brush = CompositionImageBrush.FromBGRASoftwareBitmap(
          gridVisual.Compositor,
          softwareBitmap,
          new Size(bitmap.PixelWidth, bitmap.PixelHeight));

        var dropShadow = gridVisual.Compositor.CreateDropShadow();
        dropShadow.Mask = brush.Brush;
        dropShadow.Offset = new Vector3(50, 50, 0);
        spriteVisual.Shadow = dropShadow;
      }
      ElementCompositionPreview.SetElementChildVisual(this.grid, spriteVisual);
    }

then it “kind of worked”;

Capture

although, clearly, that work would need re-doing every time the value changed and the slider resized and that kind of thing so it’s probably not a very “practical” solution Smile

No doubt there’s a better way and if you know of one, let me know in the comments below and I’ll update the post but it feels to me like the addition of the 3 GetAlphaMask() methods to Image, TextBlock and Shape signals the idea that there’s not perhaps a simple, one-method-fits-all approach that the UI team could easily add (otherwise, they’d almost certainly have done so Smile).

Lights

In terms of experimenting with lights, I spun up a UI with an image in it that looked like this;

 <Grid
    Background="Black"
    x:Name="grid">
    <Image
      Stretch="UniformToFill"
      Source="ms-appx:///Assets/cat.jpg" />
  </Grid>

and some code that tried to create a PointLight and point it at that UI;

    void OnLoaded(object sender, RoutedEventArgs e)
    {
      var gridVisual = ElementCompositionPreview.GetElementVisual(this.grid);
      var compositor = gridVisual.Compositor;

      var pointLight = compositor.CreatePointLight();
      pointLight.CoordinateSpace = gridVisual;
      pointLight.Color = Colors.White;

      pointLight.Offset = new Vector3(
        (float)this.grid.ActualWidth / 2,
        (float)this.grid.ActualHeight / 2,
        100);

      pointLight.Targets.Add(gridVisual);
    }

and it didn’t seem to have much of an effect so I thought that I’d create the visual to display the image myself using the ImageLoader class that’s part of the Windows UI Labs samples. So, I took the Image element out of the UI altogether and wrote this code instead;

    void OnLoaded(object sender, RoutedEventArgs e)
    {
      var gridVisual = ElementCompositionPreview.GetElementVisual(this.grid);
      var compositor = gridVisual.Compositor;

      var imageLoader = ImageLoaderFactory.CreateImageLoader(compositor);

      var image = imageLoader.LoadImageFromUri(new Uri("ms-appx:///Assets/cat.jpg"));
      var surfaceBrush = compositor.CreateSurfaceBrush(image);

      var spriteVisual = compositor.CreateSpriteVisual();
      spriteVisual.Brush = surfaceBrush;
      spriteVisual.Size = this.grid.RenderSize.ToVector2();

      var pointLight = compositor.CreatePointLight();
      pointLight.CoordinateSpace = gridVisual;
      pointLight.Color = Colors.White;

      pointLight.Offset = new Vector3(
        (float)this.grid.ActualWidth / 2,
        (float)this.grid.ActualHeight / 2,
        100);

      pointLight.Targets.Add(gridVisual);

      ElementCompositionPreview.SetElementChildVisual(this.grid, spriteVisual);
    }

and that gave me more like the desired effect;

Capture

so I’m not sure what went wrong in my trying to apply this to an existing XAML element. Once the light is created, it’s not too difficult to animate it around a little and give it that sort of ‘roaming spotlight’ effect. I added in a little bit of code;

      var centreX = (float)this.grid.ActualWidth / 2.0f;
      var centreY = (float)this.grid.ActualHeight / 2.0f;
      var deltaX = centreX * 0.9f;
      var deltaY = centreY * 0.9f;
      var upperZ = 250;
      var lowerZ = 100;

      var animation = compositor.CreateVector3KeyFrameAnimation();
      animation.Duration = TimeSpan.FromSeconds(5);
      animation.InsertKeyFrame(0.0f, new Vector3(centreX - deltaX, centreY, upperZ));
      animation.InsertKeyFrame(0.25f, new Vector3(centreX, centreY - deltaY, lowerZ));
      animation.InsertKeyFrame(0.5f, new Vector3(centreX + deltaX, centreY, upperZ));
      animation.InsertKeyFrame(0.75f, new Vector3(centreX, centreY + deltaY, lowerZ));
      animation.InsertKeyFrame(1.0f, new Vector3(centreX - deltaX, centreY, upperZ));
      animation.IterationBehavior = AnimationIterationBehavior.Forever;

      pointLight.StartAnimation("Offset", animation);

and that creates a reasonable animated effect with very little code;

and it’s pretty easy to get that spun up and working without writing tonnes of code around it. There’s 3 other types of light to experiment with here, AmbientLight, DistantLight and SpotLight and I’d intend to try replacing my PointLight with the latter two to see how that works out.

But that’s for another post…

Windows 10 Anniversary Update Preview, Visual Layer–Mocking Up the Lock Screen

I wanted something relatively simple to experiment with using some of the things that I’d picked up about the Visual Layer when writing these posts;

Visual Layer Posts

and from Rob’s posts;

Rob’s Posts

and, specifically, I wanted to try and do a little bit more with interactions that I’d started playing with;

Windows 10, UWP and Composition– Experimenting with Interactions in the Visual Layer

and so I thought I’d make a stab at a cheap reproduction of what I see with the Windows 10 lock-screen’s behaviour which (purely from staring at it) seems to;

  1. Slide up with the user’s finger.
  2. Fade out the text it displays as it slides
  3. On completion of “enough” of a slide, hides the text and appears to both zoom and darken the lock screen image before displaying the logon box.

Here’s a screen capture of my attempt to date;

and you’ll probably notice that it’s far from perfect but (I hope) it captures a little of what the lock-screen does.

In experimenting with this, I used a Blank UWP app on SDK preview 14388 with Win2d.uwp referenced and I had a simple piece of XAML as my UI;

  &lt;Grid
    Background=&quot;Red&quot;
    PointerPressed=&quot;OnPointerPressed&quot;
    x:Name=&quot;xamlRootGrid&quot;&gt;
    &lt;Image
      x:Name=&quot;xamlImage&quot;
      Source=&quot;ms-appx:///Assets/lockImage.jpg&quot;
      HorizontalAlignment=&quot;Left&quot;
      VerticalAlignment=&quot;Top&quot;
      Stretch=&quot;UniformToFill&quot; /&gt;
    &lt;!-- this grid is here to provide an easy place to add a blur to the image behind it --&gt;
    &lt;Grid
      x:Name=&quot;xamlBlurPlaceHolder&quot; /&gt;
    &lt;Grid
      x:Name=&quot;xamlContentPanel&quot;
      HorizontalAlignment=&quot;Stretch&quot;
      VerticalAlignment=&quot;Stretch&quot;&gt;
      &lt;StackPanel
        HorizontalAlignment=&quot;Left&quot;
        VerticalAlignment=&quot;Bottom&quot;
        Margin=&quot;48,0,0,48&quot;&gt;
        &lt;TextBlock
          Text=&quot;09:00&quot;
          FontFamily=&quot;Segoe UI Light&quot;
          Foreground=&quot;White&quot;
          FontSize=&quot;124&quot; /&gt;
        &lt;TextBlock
          Margin=&quot;0,-30,0,0&quot;
          Text=&quot;Thursday, 14th July&quot;
          FontFamily=&quot;Segoe UI&quot;
          Foreground=&quot;White&quot;
          FontSize=&quot;48&quot; /&gt;
        &lt;TextBlock
          Text=&quot;Jim's Birthday&quot;
          Margin=&quot;0,48,0,0&quot;
          FontFamily=&quot;Segoe UI Semibold&quot;
          Foreground=&quot;White&quot;
          FontSize=&quot;24&quot; /&gt;
        &lt;TextBlock
          Text=&quot;Friday All Day&quot;
          FontFamily=&quot;Segoe UI Semibold&quot;
          Foreground=&quot;White&quot;
          FontSize=&quot;24&quot; /&gt;
      &lt;/StackPanel&gt;
    &lt;/Grid&gt;
  &lt;/Grid&gt;

and you’ll probably notice that I don’t quite have the fonts or spacing quite right but it’s an approximation and then I wrote some code behind to try and achieve what I wanted;

namespace App12
{
  using Microsoft.Graphics.Canvas.Effects;
  using System;
  using System.Numerics;
  using Windows.UI;
  using Windows.UI.Composition;
  using Windows.UI.Composition.Interactions;
  using Windows.UI.Xaml;
  using Windows.UI.Xaml.Controls;
  using Windows.UI.Xaml.Hosting;
  using Windows.UI.Xaml.Input;

  public static class VisualExtensions
  {
    public static Visual GetVisual(this UIElement element)
    {
      return (ElementCompositionPreview.GetElementVisual(element));
    }
  }
  public sealed partial class MainPage : Page, IInteractionTrackerOwner
  {
    public MainPage()
    {
      this.InitializeComponent();
      this.Loaded += OnLoaded;
    }
    void OnLoaded(object sender, Windows.UI.Xaml.RoutedEventArgs e)
    {
      // The visual for our root grid
      this.rootGridVisual = this.xamlRootGrid.GetVisual();

      // Keep hold of our compositor.
      this.compositor = this.rootGridVisual.Compositor;

      // The visual for the grid which contains our text content.
      this.contentPanelVisual = this.xamlContentPanel.GetVisual();

      // And for the image
      this.imageVisual = this.xamlImage.GetVisual();

      // Set up the centre point for scaling the image 
      // TODO: need to alter this on resize?
      this.imageVisual.CenterPoint = new Vector3(
        (float)this.xamlRootGrid.ActualWidth / 2.0f,
        (float)this.xamlRootGrid.ActualHeight / 2.0f,
        0);

      // Get the visual for the grid which sits in front of the image that I can use to blur the image
      this.blurPlaceholderVisual = this.xamlBlurPlaceHolder.GetVisual();

      // Create the pieces needed to blur the image at a later point.
      this.CreateDarkenedVisualAndAnimation();

      this.CreateInteractionTrackerAndSource();

      // NB: Creating our animations here before the layout pass has gone by would seem
      // to be a bad idea so we defer it. That was the big learning of this blog post.

    }
    void CreateInteractionTrackerAndSource()
    {
      // Create an interaction tracker with an owner (this object) so that we get
      // callbacks when interesting things happen, this was a major learning for
      // me in this piece of code.
      this.interactionTracker = InteractionTracker.CreateWithOwner(this.compositor, this);

      // We're using the root grid as the source of our interactions.
      this.interactionSource = VisualInteractionSource.Create(this.rootGridVisual);

      // We only want to be able to move in the Y direction.
      this.interactionSource.PositionYSourceMode = InteractionSourceMode.EnabledWithoutInertia;

      // From 0 to the height of the root grid (TODO: recreate on resize)
      this.interactionTracker.MaxPosition = new Vector3(0, (float)this.xamlRootGrid.ActualHeight, 0);
      this.interactionTracker.MinPosition = new Vector3(0, 0, 0);

      // How far do you have to drag before you unlock? Let's say half way.
      this.dragThreshold = this.xamlRootGrid.ActualHeight / 2.0d;

      // Connect the source to the tracker.
      this.interactionTracker.InteractionSources.Add(this.interactionSource);
    }

    void CreateDarkenedVisualAndAnimation()
    {
      var darkenedSprite = this.compositor.CreateSpriteVisual();
      var backdropBrush = this.compositor.CreateBackdropBrush();

      // TODO: resize?
      darkenedSprite.Size = new Vector2(
        (float)this.xamlRootGrid.ActualWidth,
        (float)this.xamlRootGrid.ActualHeight);

      // I borrowed this effect definition from a Windows UI sample and
      // then tweaked it.
      using (var graphicsEffect = new ArithmeticCompositeEffect()
      {
        Name = &quot;myEffect&quot;,
        Source1Amount = 0.0f,
        Source2Amount = 1.0f,
        Source1 = new ColorSourceEffect()
        {
          Name = &quot;Base&quot;,
          Color = Color.FromArgb(255, 0, 0, 0),
        },
        Source2 = new CompositionEffectSourceParameter(&quot;backdrop&quot;)
      })
      {
        this.darkenImageAnimation = this.compositor.CreateScalarKeyFrameAnimation();
        this.darkenImageAnimation.InsertKeyFrame(0.0f, 1.0f);
        this.darkenImageAnimation.InsertKeyFrame(0.0f, 0.6f);
        this.darkenImageAnimation.Duration = TimeSpan.FromMilliseconds(250);

        using (var factory = this.compositor.CreateEffectFactory(graphicsEffect,
          new string[] { &quot;myEffect.Source2Amount&quot; }))
        {
          this.mixedDarkeningBrush = factory.CreateBrush();
          this.mixedDarkeningBrush.SetSourceParameter(&quot;backdrop&quot;, backdropBrush);
          darkenedSprite.Brush = this.mixedDarkeningBrush;
        }
      }
      ElementCompositionPreview.SetElementChildVisual(this.xamlBlurPlaceHolder, darkenedSprite);
    }

    void OnPointerPressed(object sender, PointerRoutedEventArgs e)
    {
      // First time around, create our animations.
      if (this.positionAnimation == null)
      {
        LazyCreateDeferredAnimations();
      }
      if (e.Pointer.PointerDeviceType == Windows.Devices.Input.PointerDeviceType.Touch)
      {
        // we send this to the interaction tracker.
        this.interactionSource.TryRedirectForManipulation(
          e.GetCurrentPoint(this.xamlRootGrid));
      }
    }
    void LazyCreateDeferredAnimations()
    {
      // opacity.
      this.opacityAnimation = this.compositor.CreateExpressionAnimation();

      this.opacityAnimation.Expression =
        &quot;1.0 - (tracker.Position.Y / (tracker.MaxPosition.Y - tracker.MinPosition.Y))&quot;;

      this.opacityAnimation.SetReferenceParameter(&quot;tracker&quot;, this.interactionTracker);

      this.contentPanelVisual.StartAnimation(&quot;Opacity&quot;, this.opacityAnimation);

      // position.
      this.positionAnimation = this.compositor.CreateExpressionAnimation();
      this.positionAnimation.Expression = &quot;-tracker.Position&quot;;
      this.positionAnimation.SetReferenceParameter(&quot;tracker&quot;, this.interactionTracker);
      this.contentPanelVisual.StartAnimation(&quot;Offset&quot;, this.positionAnimation);

      // scale for the background image when we &quot;unlock&quot;
      CubicBezierEasingFunction easing = this.compositor.CreateCubicBezierEasingFunction(
        new Vector2(0.5f, 0.0f),
        new Vector2(1.0f, 1.0f));

      // this animation and its easing don't 'feel' right at all, needs some tweaking
      this.scaleAnimation = this.compositor.CreateVector3KeyFrameAnimation();
      this.scaleAnimation.InsertKeyFrame(0.0f, new Vector3(1.0f, 1.0f, 1.0f), easing);
      this.scaleAnimation.InsertKeyFrame(0.2f, new Vector3(1.075f, 1.075f, 1.0f), easing);
      this.scaleAnimation.InsertKeyFrame(1.0f, new Vector3(1.1f, 1.1f, 1.1f), easing);
      this.scaleAnimation.Duration = TimeSpan.FromMilliseconds(500);
    }

    // From hereon in, these methods are the implementation of IInteractionTrackerOwner.
    public void CustomAnimationStateEntered(
      InteractionTracker sender, 
      InteractionTrackerCustomAnimationStateEnteredArgs args)
    {
    }
    public void IdleStateEntered(
      InteractionTracker sender, 
      InteractionTrackerIdleStateEnteredArgs args)
    {
      if (this.unlock)
      {
        // We make sure that the text disappears
        this.contentPanelVisual.Opacity = 0.0f;

        // We try and zoom the image a little.
        this.imageVisual.StartAnimation(&quot;Scale&quot;, this.scaleAnimation);

        // And darken it a little.
        this.mixedDarkeningBrush.StartAnimation(&quot;myEffect.Source2Amount&quot;, this.darkenImageAnimation);
      }
      else
      {
        sender.TryUpdatePosition(Vector3.Zero);
      }
    }
    public void InertiaStateEntered(
      InteractionTracker sender, 
      InteractionTrackerInertiaStateEnteredArgs args)
    {
    }  
    public void InteractingStateEntered(
      InteractionTracker sender, 
      InteractionTrackerInteractingStateEnteredArgs args)
    {
      this.unlock = false;
    }
    public void RequestIgnored(
      InteractionTracker sender, 
      InteractionTrackerRequestIgnoredArgs args)
    {
    }
    public void ValuesChanged(
      InteractionTracker sender, 
      InteractionTrackerValuesChangedArgs args)
    {
      if (!this.unlock &amp;&amp; (args.Position.Y &gt; this.dragThreshold))
      {
        this.unlock = true;
      }
    }
    bool unlock;
    double dragThreshold;
    InteractionTracker interactionTracker;
    VisualInteractionSource interactionSource;
    Visual rootGridVisual;
    Visual contentPanelVisual;
    Visual blurPlaceholderVisual;
    Compositor compositor;
    ExpressionAnimation positionAnimation;
    ExpressionAnimation opacityAnimation;
    ScalarKeyFrameAnimation darkenImageAnimation;
    CompositionEffectBrush mixedDarkeningBrush;
    Vector3KeyFrameAnimation scaleAnimation;
    Visual imageVisual;
  }
}

What’s that code doing?

  1. At start-up
    1. getting hold of a bunch of Visuals for the various XAML UI elements.
    2. creating a Visual (darkenedSprite) which lives in the Grid named xamlBlurPlaceHolder and which will effectively paint itself with a mixed combination of the colour black and the image which sits under it in the Z-order.
    3. creating an animation (darkenImageAnimation) which will change the balance between black/image when necessary.
    4. creating an interaction tracker and an interaction source to track the Y movement of the touch pointer up the screen within some limits.
  2. On pointer-pressed
    1. Creating an animation which will cause the text content to slide up the screen wired to the interaction tracker
    2. Creating an animation which will cause the text content to fade out wired to the interaction tracker
    3. Creating an animation which will later be used to scale the image as the lock-screen is dismissed (this could, perhaps, be done earlier)
    4. Passing the pointer event (if it’s touch) across to the interaction tracker

In building that out, I learned 2 main things. One was that things have changed since build 10586 and I need to read the Wiki site more carefully as talked about in this post.

The other was around how to trigger the ‘dismissal’ of my lock-screen at the point where the user’s touch point has travelled far enough up the screen.

I was puzzled by that for quite a while. I couldn’t figure out how I was meant to know what the interaction tracker was doing and I kept looking for events without finding any.

Equally, I couldn’t figure out how to debug what the interaction tracker was doing when my code didn’t work.

That changed when I came across IInteractionTrackerOwner and the InteractionTracker.CreateWithOwner() method. Whether I have this right or not, it let me plug code (and diagnostics) into the InteractionTracker and I used the ValueChanged method to try and work out when the user’s touch point has gone 50% of the way up the screen so that I can then dismiss the lock-screen.

I don’t dismiss it immediately though. Instead, I wait for the IdleStateEntered callback and in that code I try to take steps to;

  1. Set the opacity of the text panel to 0 so that it disappears.
  2. Begin the animation on the image Visual so as to zoom it a little
  3. Begin the animation on the composite brush that I have so as to darken the image by mixing it with Black.

The other thing that I learned was that I don’t understand the lighting features of the Visual Layer well enough yet and that I need to explore them some more in isolation to try and work that out.

But, the main thing for me here was to learn about IInteractionTrackerOwner and hence sharing that here (even in this rough, experimental form).

Windows 10 Anniversary Update 14388, UWP, Visual Layer–Offsets on Preview SDK 14388

Just a small thing but I’m posting it here in case it helps anyone else. I spent a good few ‘minutes’ trying to figure out why this piece of XAML;

   <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        
        <Rectangle x:Name="xamlRectangle" Width="96" Height="96" Fill="Black" HorizontalAlignment="Left" VerticalAlignment="Bottom">
            
        </Rectangle>
    </Grid>

married up with this piece of code behind;

  public sealed partial class MainPage : Page
  {
    public MainPage()
    {
      this.InitializeComponent();
      this.Loaded += this.OnLoaded;
    }
    void OnLoaded(object sender, RoutedEventArgs e)
    {
      this.rectangleVisual = ElementCompositionPreview.GetElementVisual(
        this.xamlRectangle);

      this.rectangleVisual.Offset = new Vector3(100, -100, 0);
    }
    Visual rectangleVisual;
  }

Wasn’t giving me the result that I expected on Windows 10 Anniversary Update 14388 with SDK preview 14388.

What it gives me is a black rectangle which is not offset by the 100,-100 vector that I’ve specified.

1

However, if I simply retarget my project to target SDK build 10586 and run it on the same machine then I find that I get the results I expect.

2

I initially thought this was a bug but I learned that it was a change between 10586 and the builds post 14332 – the detail is here and I think I need to read it at least another 5 times before I understand it 🙂

Update

I have now read that article more than 5 times and I’m still trying to understand it although I think I have slightly more of a handle on its implications even if I don’t quite understand the mechanics.

In terms of dealing with Offset, I think I understand that if XAML thinks that it has set some value for Visual.Offset then it will override whatever value I set for Visual.Offset when it comes to do a layout pass.

For example, in the code/XAML scenario that I have further up in this post I have a Rectangle which is set to be arranged in the Bottom/Left of its parent Grid.

I also have a Loaded event handler which attempts to set the Offset property to (100,-100,0) and on SDK 14388 nothing happens whereas on 10586 the offset gets applied.

It’s worth saying that the same thing appears to happen if I try and start an animation which targets “Offset” from the Loaded handler – it doesn’t do anything on 14388.

Why? Because after my code changes the Offset value to (100,-100,0) XAML is doing a layout pass and setting it back to whatever value is necessary to position the rectangle at the bottom left of the Grid.

If I change my Loaded handler to something like this;

    async void OnLoaded(object sender, RoutedEventArgs e)
    {
      this.rectangleVisual = ElementCompositionPreview.GetElementVisual(
        this.xamlRectangle);

      await Task.Delay(5000);

      this.rectangleVisual.Offset = new Vector3(100, -100, 0);
    }

then I notice different behavior in that after 5 seconds the rectangle disappears 🙂 Why? Presumably because my asynchronously waiting for 5 seconds gives the thread time to go back and process the layout work before completing this function which then applies an Offset of (100,-100,0) from the origin of the Window which makes the rectangle (at 96 pixels high) disappear.

If I then resize the Window to force another layout pass then the Rectangle re-appears back in the bottom left hand corner of the Window.

So, the XAML layer has had to take an active hand in the positioning of this Rectangle and it attempts to reassert its authority every time it does a layout.

However, if I change my XAML such that the VerticalAlignment of the Rectangle is set to “Top” then the XAML layer doesn’t explicitly set the Visual.Offset value on any of its layout passes and so there’s no need for the artificial delay that I added into the code – the Rectangle positions itself off screen from the start and stays that way regardless of resizing the Window because the XAML pieces have never been involved in explicitly setting the Offset property in the first place.

I must admit that I find this a little tricky to reason about and I guess that it would lead me towards not letting XAML position elements that I then wanted reposition using the Visual layer although that feels a bit restrictive in terms of not being then able to use a lot of ‘goodness’ that XAML’s layout system can give me like proportional sizing and so on. I perhaps need to experiment a little more with this to figure out more around how it works and how to best code with it rather than against it 🙂

Update 2

Rob has written a post here which digs into more detail around what I was seeing in this post and I think the 2 diagrams that he’s posted help massively in explaining what happens in the Anniversary Update versus the November Update.