Windows 10 1607, UWP and Lifecycle on HoloLens (2)

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

In this previous post, I’d taken a bit of a look at the updated UWP app lifecycle as it runs on HoloLens and I’d finished off that post by saying that I was starting to see how the actions of ‘placement’ and ‘launching’ apps worked within the HoloLens shell but I wanted to;

“think on it a little more from the point of view of ‘state management’ and come back to it in a future post”

and that’s the purpose of this post.

Since that last post, I’ve had more of a chance to look at what some of the built-in apps on HoloLens do and, specifically, I spent some time experimenting with the Settings app and the Edge browser.

In the screenshot below, I’ve ‘placed’ the Settings app into 2 different places in the environment – I opened the app on the right first and then the one on the left. On the right hand app, I then navigated to the ‘System’ page before switching back to the left hand app;

image

the left-hand app has been launched (from its unique secondary tile) and has moved itself to the ‘System’ page. If I then move that app to (e.g.) the ‘Brightness’ tab;

image

and then re-launched the app on the right then it also jumps to the ‘Brightness’ tab;

image

and so it feels like there’s one Window here, one set of content within it and these two placements are operating as one.

If I contrast this with the Edge browser then that behaves differently which is not perhaps a huge surprise as on the PC it operates as an app that can have multiple windows whereas the Settings app does not.

I’ll call these 2 ‘placements’ ‘left Edge’ and ‘right Edge’.

image

It’s immediately clear from the screenshot that these are operating independently – left Edge is showing Bing.com and right Edge is showing Microsoft.com. I couldn’t say whether the Edge team had to write a small piece of tailored code here to achieve that or whether it’s a natural side-effect of the way that they implement their existing multi-windowing behaviour.

If I then run a number of other apps so as to cause these to suspend/resume and so on then no matter what I do these 2 placements remember their individual state as a user would expect them to although some of the time when I return back to them it does feel like they are reloading the page which is not an unreasonable optimisation to make.

What if my own UWP app wanted to mimic this Edge behaviour or the Settings behaviour?

I made a blank UWP app with 5 photos of guitars in this Images folder;

image

and wrote a little ‘view model’ to represent an image as below;

namespace MyBlankApp
{
  using System.Threading.Tasks;
  using Windows.Storage;
  using Windows.UI.Xaml.Media;
  using Windows.UI.Xaml.Media.Imaging;

  using System;
  using System.IO;

  class ImageViewModel : ViewModelBase
  {
    internal ImageSource Image
    {
      get
      {
        return (this.image);
      }
      set
      {
        base.SetProperty(ref this.image, value);
      }
    }
    ImageSource image;

    // public to keep ComboBox.DisplayMemberPath happy.
    public string Title
    {
      get
      {
        return (this.title);
      }
      set
      {
        base.SetProperty(ref this.title, value);
      }
    }
    string title;

    internal async Task PopulateFromFileAsync(StorageFile file)
    {
      var source = new BitmapImage();
      var stream = await file.OpenReadAsync();
      source.SetSource(stream);
      this.Image = source;
      this.Title = Path.GetFileNameWithoutExtension(
        file.Name);
    }
  }
}

and then another little ‘view model’ which has a list of those images;

namespace MyBlankApp
{
  using System;
  using System.Collections.ObjectModel;
  using System.Threading.Tasks;
  using Windows.ApplicationModel;

  class ImageListViewModel : ViewModelBase
  {
    public ObservableCollection<ImageViewModel> Images
    {
      get
      {
        return (this.images);
      }
      set
      {
        base.SetProperty(ref this.images, value);
      }
    }
    ObservableCollection<ImageViewModel> images;

    public ImageViewModel SelectedImage
    {
      get
      {
        return (this.Images[this.SelectedIndex]);
      }
    }

    public int SelectedIndex
    {
      get
      {
        return (this.selectedIndex);
      }
      set
      {
        base.SetProperty(ref this.selectedIndex, value);
        base.OnPropertyChanged("SelectedImage");
      }
    }
    int selectedIndex;

    internal async Task PopulateFromFolderAsync()
    {
      var assetsFolder = 
        await Package.Current.InstalledLocation.GetFolderAsync("Assets");
      var imagesFolder =
        await assetsFolder.GetFolderAsync("Images");
     
      this.Images = new ObservableCollection<ImageViewModel>();

      var files = await imagesFolder.GetFilesAsync();

      foreach (var file in files)
      {
        var image = new ImageViewModel();

        await image.PopulateFromFileAsync(file);

        this.Images.Add(image);
      }
      this.SelectedIndex = 0;
    }
  }
}

and I got rid of my MainPage.xaml/.xaml.cs and replaced the with a MainControl.xaml;

<UserControl
  x:Class="MyBlankApp.MainControl"
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:local="using:MyBlankApp"
  xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
  xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
  mc:Ignorable="d"
  d:DesignHeight="300"
  d:DesignWidth="400">

  <Grid>
    <Image
      Source="{x:Bind ViewModel.SelectedImage.Image,Mode=OneWay}"
      Stretch="Uniform" />
    <ComboBox
      ItemsSource="{x:Bind ViewModel.Images,Mode=OneWay}"
      SelectedIndex="{x:Bind ViewModel.SelectedIndex,Mode=TwoWay}"
      DisplayMemberPath="Title"
      VerticalAlignment="Top"
      HorizontalAlignment="Left"      
      Width="132"
      Margin="96">
    </ComboBox>
  </Grid>
</UserControl>

and some minimal code behind;

using System;
using System.ComponentModel;
using System.Runtime.CompilerServices;
using Windows.UI.Xaml.Controls;

namespace MyBlankApp
{
  public sealed partial class MainControl : UserControl,
    INotifyPropertyChanged
  {
    public event PropertyChangedEventHandler PropertyChanged;

    public MainControl()
    {
      this.InitializeComponent();
    }

    internal ImageListViewModel ViewModel
    {
      get
      {
        return (this.viewModel);
      }
      set
      {
        this.SetProperty(ref this.viewModel, value);
      }
    }
    ImageListViewModel viewModel;

    bool SetProperty<T>(ref T storage, T value,
      [CallerMemberName] String propertyName = null)
    {
      if (object.Equals(storage, value)) return false;

      storage = value;
      this.OnPropertyChanged(propertyName);
      return true;
    }
    void OnPropertyChanged([CallerMemberName] string propertyName = null)
    {
      this.PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
    }
  }
}

and I wrote a little class to manage ‘state’ for me. This is overkill because the entire state in this ‘app’ is a single integer value which represents which of the images has been selected and that’s it. Writing a class to manage an integer is probably a bit over the top;

namespace MyBlankApp
{
  using System;
  using System.IO;
  using System.Threading.Tasks;
  using Windows.Storage;

  static class StateManagement
  {
    static internal async Task SaveStateAsync(int state)
    {
      // Our state is simply an integer (the selected image). We could
      // put it into App settings but let's put it into a file to be
      // similar to what a real app would do.
      var stateFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(
        stateFileName, CreationCollisionOption.ReplaceExisting);

      await FileIO.WriteTextAsync(stateFile, state.ToString());
    }
    static internal async Task<int> LoadStateAsync()
    {
      var state = 0;

      try
      {
        var stateFile = await ApplicationData.Current.LocalFolder.GetFileAsync(
          stateFileName);

        var contents = await FileIO.ReadTextAsync(stateFile);

        int.TryParse(contents, out state);
      }
      catch (FileNotFoundException)
      {
      }
      return (state);
    }
    const string stateFileName = "statefile.bin";
  }
}

and, finally, I worked on my App class and made use of OnLaunched to try and make sure that state is restored in the event of a previous termination of the app and the Suspending event to store that state. I also decided to let the App class ‘own’ the view model and hand it to the ViewModel which isn’t perhaps what I’d usually do but this isn’t a complex example;

namespace MyBlankApp
{
  using System.Threading.Tasks;
  using Windows.ApplicationModel;
  using Windows.ApplicationModel.Activation;
  using Windows.UI.Xaml;

  sealed partial class App : Application
  {
    public App()
    {
      this.InitializeComponent();
      this.Suspending += OnSuspending;
    }
    protected async override void OnLaunched(LaunchActivatedEventArgs args)
    {
      int? selectedIndex = null;

      if (args.PreviousExecutionState == ApplicationExecutionState.Terminated)
      {
        selectedIndex = await StateManagement.LoadStateAsync();
      }
      await this.LoadViewModelAsync(selectedIndex);

      this.EnsureUI();
    }
    async Task LoadViewModelAsync(int? selectedIndex)
    {
      if (this.viewModel == null)
      {
        this.viewModel = new ImageListViewModel();
        await this.viewModel.PopulateFromFolderAsync();
      }
      if (selectedIndex.HasValue)
      {
        this.viewModel.SelectedIndex = (int)selectedIndex;
      }
    }
    void EnsureUI()
    {
      if (Window.Current.Content == null)
      {
        var ui = new MainControl();
        ui.ViewModel = this.viewModel;

        Window.Current.Content = ui;
      }
      Window.Current.Activate();
    }
    async void OnSuspending(object sender, SuspendingEventArgs e)
    {
      var deferral = e.SuspendingOperation.GetDeferral();

      await StateManagement.SaveStateAsync(this.viewModel.SelectedIndex);

      deferral.Complete();
    }
    ImageListViewModel viewModel;
  }
}

Now I have an app and I can run it on the PC to exercise 2 different scenarios;

  • Run->Change State->Close->Run
  • Run->Change State->Suspend & Shutdown->Run

For the first scenario, the app runs and it displays the default guitar;

image

and if I switch to a different guitar;

image

and then close the app and re-open it then it does what I expect in that there’s no saved state so it reverts back to the default guitar;

image

if I then switch it to a different guitar;

image

and I use the debugger to go around the suspend & shutdown->launch cycle then the app will preserve state and so starts back up on the same guitar;

image

and that’s all as I would expect.

It’s worth saying that in both cases I see the LeavingBackground and EnteredBackground events firing as I would expect them to.

If I then run this on the HoloLens emulator then I see slightly different behaviour.

For the Run->Change State->Close->Run scenario, I don’t see the EnteredBackground event fire in the debugger as the app closes, I only see the Suspending event fire. Additionally, when I re-launch the app I see it restore state – i.e. the app’s previous execution state is marked as ‘Terminated’ coming in to the re-launch whereas I would expect it to be ‘ClosedByUser’. I don’t think that’s any big deal but it does seem to be a subtle difference if my debugging isn’t misleading me but it had me take out any dependencies I’d made on the Entered/LeavingBackground events.

For the Run->Change State->Suspend & Shutdown->Run scenario, things seem to work exactly as they do on the PC including the EnteredBackground event.

If I launch the app twice in the environment then it behaves like the settings app. That is, with these two apps side-by-side;

image

and then if I change the guitar in the window on the right hand-side;

image

then the left hand window is now showing a momentarily stale screenshot as it isn’t active but when I tap on it to re-launch it the UI updates;

image

and so it’s like the Settings app – two placements that manifest themselves as one Window, one set of content and going around the suspend->terminate->launch cycle works as I’d expect it to.

One thing that I noticed while experimenting here is that if I press F5 to debug my app on the emulator then I see the Tile ID which activates the app to be “App” as below;

image

If, instead, I use the debugger setting to ‘Do not launch, but debug my code when it starts’ then when I launch the app from the start screen I see a GUID as the tile id;

image

What’s more interesting is what happens if I then place multiple copies of the app as below using the regular F5 method;

image

then what I see in the debugger is;

  • App on right runs first, is launched with a tile Id of “App”
  • App on left runs next, is launched with a GUID tile Id
  • App on right is clicked on and launched again and now has a GUID tile Id

I think this is the debugger tricking me because if I go with the ‘Do not launch’ method then I see;

  • App on right runs first, is launched with a GUID tile Id
  • App on left runs first, is launched with a GUID tile Id
  • App on right is clicked and launched again with the same GUID tile Id it had at step 1

So I had to apply a little care in debugging here to (hopefully) not get fooled.

What if my code wanted to behave more like the Edge app and have content that varied per placement? I can have multiple copies of my state indexed by the tile Id that’s currently active and then just switch between them at the point where a particular copy gets launched.

I made a little ‘state’ class to represent this – again, it’s perhaps overkill but it’s basically just a dictionary<string,T>;

namespace MyBlankApp
{
  using System.Collections.Generic;
  using System.Threading.Tasks;
  using Windows.UI.StartScreen;
  using System;
  using System.Linq;

  internal class StateBag<T>
  {
    internal StateBag()
    {
      this.state = new Dictionary<string, T>();
    }
    internal T GetStateForInstance(string instanceId)
    {
      T stateValue = default(T);
      this.state.TryGetValue(instanceId, out stateValue);
      return (stateValue);
    }
    internal void SetStateForInstance(string instanceId, T state)
    {
      this.state[instanceId] = state;
    }
    internal async Task TidyOldInstancesAsync()
    {
      var secondaryTiles = await SecondaryTile.FindAllAsync();

      var orphanedKeys = state.Keys.Except(secondaryTiles.Select(s => s.TileId)).ToArray();

      foreach (var key in orphanedKeys)
      {
        state.Remove(key);
      }
    }
    Dictionary<string, T> state;
  }
}

and then I modified my state management class to persist these for me (adding a little bit of control over the JSON serializer);

namespace MyBlankApp
{
  using Newtonsoft.Json;
  using Newtonsoft.Json.Serialization;
  using System;
  using System.Collections.Generic;
  using System.IO;
  using System.Linq;
  using System.Reflection;
  using System.Threading.Tasks;
  using Windows.Storage;

  static class StateManagement
  {
    class PrivateContractResolver : DefaultContractResolver
    {
      protected override List<MemberInfo> GetSerializableMembers(Type objectType)
      {
        var flags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;
        MemberInfo[] fields = objectType.GetFields(flags);
        return fields
            .Concat(objectType.GetProperties(flags).Where(propInfo => propInfo.CanWrite))
            .ToList();
      }

      protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
      {
        return base.CreateProperties(type, MemberSerialization.Fields);
      }
    }
    static StateManagement()
    {
      jsonSerializerSettings = new JsonSerializerSettings()
      {
        ContractResolver = new PrivateContractResolver()
      };
    }
    static internal async Task SaveStateAsync<T>(StateBag<T> state)
    {
      // Our state is simply an integer (the selected image). We could
      // put it into App settings but let's put it into a file to be
      // similar to what a real app would do.
      var stateFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(
        stateFileName, CreationCollisionOption.ReplaceExisting);

      var json = JsonConvert.SerializeObject(state, jsonSerializerSettings);

      await FileIO.WriteTextAsync(stateFile, json);
    }
    static internal async Task<StateBag<T>> LoadStateAsync<T>()
    {
      StateBag<T> state = null;

      try
      {
        var stateFile = await ApplicationData.Current.LocalFolder.GetFileAsync(
          stateFileName);

        var contents = await FileIO.ReadTextAsync(stateFile);

        state = JsonConvert.DeserializeObject<StateBag<T>>(
          contents, jsonSerializerSettings);
      }
      catch (FileNotFoundException)
      {
      }
      return (state);
    }
    static JsonSerializerSettings jsonSerializerSettings;
    const string stateFileName = "statefile.bin";
  }
}

and then I modified my App class;

namespace MyBlankApp
{
  using System.Threading.Tasks;
  using Windows.ApplicationModel;
  using Windows.ApplicationModel.Activation;
  using Windows.UI.Xaml;

  sealed partial class App : Application
  {
    public App()
    {
      this.InitializeComponent();
      this.state = new StateBag<int>();
      this.Suspending += OnSuspending;
    }
    protected async override void OnLaunched(LaunchActivatedEventArgs args)
    {
      if (args.PreviousExecutionState == ApplicationExecutionState.Terminated)
      {
        this.state = await StateManagement.LoadStateAsync<int>();
      }

      // If we've already been launched, make sure we record the selected index
      // for that tile ID because we can be launched->launched->launched
      // without necessarily suspending. 
      if (!string.IsNullOrEmpty(this.currentTileId))
      {
        this.state.SetStateForInstance(this.currentTileId, this.viewModel.SelectedIndex);
      }
      // Note the last tile that launched.
      this.currentTileId = args.TileId;

      var selectedIndex = this.state.GetStateForInstance(this.currentTileId);

      await this.LoadViewModelAsync(selectedIndex);

      this.EnsureUI();
    }
    async Task LoadViewModelAsync(int selectedIndex)
    {
      if (this.viewModel == null)
      {
        this.viewModel = new ImageListViewModel();
        await this.viewModel.PopulateFromFolderAsync();
      }
      this.viewModel.SelectedIndex = selectedIndex;
    }
    void EnsureUI()
    {
      if (Window.Current.Content == null)
      {
        var ui = new MainControl();
        ui.ViewModel = this.viewModel;

        Window.Current.Content = ui;
      }
      Window.Current.Activate();
    }
    async void OnSuspending(object sender, SuspendingEventArgs e)
    {
      var deferral = e.SuspendingOperation.GetDeferral();

      this.state.SetStateForInstance(this.currentTileId,
        this.viewModel.SelectedIndex);

      await StateManagement.SaveStateAsync<int>(this.state);

      deferral.Complete();
    }
    string currentTileId;
    StateBag<int> state;
    ImageListViewModel viewModel;
  }
}

where, really, the key change is that I now maintain my state (i.e. the integer representing the selected guitar) on a per-tile-ID basis and I switch between the instances at the point where the OnLaunched override is executed.

That lets me then have as many placements as I like of my app;

image

and when I click on the dormant ones to re-launch them they behave like the Edge app in that the content that was in that specific placement is remembered and re-displayed and that should survive across suspend/terminate etc.

Ultimately, this post became long when it’s really trying to describe a simple concept – in this environment, it’s my choice as to whether I want my app to display one set of content across multiple placements or content per placement and the way I seem to be able to do it is to maintain a dictionary <TileID,State> and to save/load/refresh it at the right points (keeping in mind that tile IDs can go away and state will need tidying up.

One thought on “Windows 10 1607, UWP and Lifecycle on HoloLens (2)

Comments are closed.