Mike Taulty's Blog
Bits and Bytes from Microsoft UK
Windows 8.1 Store, Azure Mobile Services & Offline Data

Blogs

Mike Taulty's Blog

Elsewhere

Archives

Following up on that post from yesterday, the other thing that I noticed with Mobile Services yesterday was the new support for working online/offline (via a local SQLite store).

There’s a “getting started” post on the website about this and I don’t think the client-side support stretches beyond .NET clients just yet.

There’s also discussion of this in some of the //Build sessions, specifically;

although that wanders off a little into a Xamarin discussion in the latter half of the session so you may want to watch/skip that depending on what you already know about Xamarin.

In terms of my own “getting” started, I can make myself a blank Windows 8.1 Store project inside of Visual Studio and then quickly add a mobile service;

image

leading to;

image

and then I can create a table inside of that service, let’s call it customers for example;

image

leading to;

image

and then with that in place I could write a little class which represents what data I want within my customers table;

 [DataContract(Name="customers")]
  class Customer
  {
    [DataMember(Name="id")]
    public string Id { get; set; }

    [DataMember(Name="firstname")]
    public string FirstName { get; set; }
    
    [DataMember(Name="lastname")]
    public string LastName { get; set; }
    
    [DataMember(Name="spam")]
    public bool SpamWithEmailInvitations { get; set; }
    
    [DataMember(Name="version")]
    [Version]
    public string Version { get; set; }
  }

Note the inclusion of the member called Version and that it has a Microsoft.WindowsAzure.Version attribute stamped on it. For quite a while now Mobile Services has created 3 additional columns on a table called _version, _createdAt, _updatedAt for the purposes of being able to detect conflicts on the server-side. In order to do this in combination with offline working, that version column needs to show up and be understood on the client side.

I can then add a reference to my project for Azure Mobile Services from NuGet;

image

and I can write a little class to wrap up my service access;

  class DataManager
  {
    void Initialise()
    {
      if (this.proxy == null)
      {
        this.proxy = new MobileServiceClient(
          "https://mttestservice.azure-mobile.net/");
      }
    }
    IMobileServiceTable<T> GetTable<T>()
    {
      this.Initialise();
      return (proxy.GetTable<T>());
    }
    public async Task<IEnumerable<Customer>> GetCustomersAsync()
    {
      var customers = await this.GetTable<Customer>().ReadAsync();
      return (customers);
    }
    public async Task InsertCustomerAsync(Customer customer)
    {
      await this.GetTable<Customer>().InsertAsync(customer);
    }
    MobileServiceClient proxy;
  }

and then use it to insert a record;

      DataManager manager = new DataManager();
      
      Customer c = new Customer()
        {
          FirstName = "Fred",
          LastName = "Astair",
          SpamWithEmailInvitations = false
        };

      await manager.InsertCustomerAsync(c);

and to query existing records;

      DataManager manager = new DataManager();
      
      var customers = await manager.GetCustomersAsync();

Now if I want to be able to do offline working on this data then I need to take a few steps and it’s important to realise that this is not an automatic “SQL Server replication” style feature but, instead, a way in which the application can ask for certain data to be available in a local offline store ( SQLite for Windows 8.1 here ) and then take explicit steps to push data to/from that local store back to the cloud ( with conflict resolution ) when the app determines that the user is online/offline ( probably based on the NetworkInformation class ).

The first step is to add a reference to SQLite. I already have the installed as an extension to Visual Studio and I got it from here: http://sqlite.org/download.html looking for the “Precompiled Binaries for Windows Runtime” side of things. That means that I can add a reference to my project;

image

Adding that reference means that we just added a reference to a native component (i.e. not .NET code) which means that we need to explicitly build packages for different processor architectures rather than just building MSIL code. So, Visual Studio warns;

image

until I say “ok, I’ll just build for x86 for now then”;

image

With SQLite available, the Mobile Services folks have built a store on top of it which plugs into the MobileServiceClient and so I can add support for that from NuGet;

image

and now (by way of example) I can change my data manger class a little bit – adding some explicitly named methods here;

namespace App97
{
  using Microsoft.WindowsAzure.MobileServices;
  using Microsoft.WindowsAzure.MobileServices.SQLiteStore;
  using Microsoft.WindowsAzure.MobileServices.Sync;
  using System.Collections.Generic;
  using System.Threading.Tasks;

  class DataManager
  {
    static readonly string DATABASE_FILE = "local.db";

    async Task InitialiseAsync()
    {
      if (this.proxy == null)
      {
        this.proxy = new MobileServiceClient(
          "https://mttestservice.azure-mobile.net/");
      }
      await this.InitialiseDatabaseAsync();
    }
    async Task InitialiseDatabaseAsync()
    {
      if (!this.proxy.SyncContext.IsInitialized)
      {
        // Create/open DB.
        MobileServiceSQLiteStore store =
          new MobileServiceSQLiteStore("local.db");

        // Make sure that store defines our table (at least). NB: repeating
        // this seems to be "ok".
        store.DefineTable<Customer>();

        // Tell our proxy about our database and specify a default
        // IMobileServicesSyncHandler to do the work.
        await this.proxy.SyncContext.InitializeAsync(
          store, new MobileServiceSyncHandler());
      }
    }
    async Task<IMobileServiceSyncTable<T>> GetTable<T>()
    {
      await this.InitialiseAsync();
      return (proxy.GetSyncTable<T>());
    }
    public async Task<IEnumerable<Customer>> GetCustomersAsync()
    {
      var table = await this.GetTable<Customer>();
      var customers = await table.ReadAsync();

      return (customers);
    }
    public async Task InsertCustomerAsync(Customer customer)
    {
      var table = await this.GetTable<Customer>();

      await table.InsertAsync(customer);
    }
    public async Task SyncWithCloudAsync()
    {
      // NB: would be nice to deal with cancellation here at least.
      // NB: pulling this data from cloud->local database first
      // pushes data from the local store to the cloud first.
      // NB: calling when offline will cause errors that would
      // need to be handled.
      var table = await this.GetTable<Customer>();
      await table.PullAsync();
    }
    MobileServiceClient proxy;
  }
}

and the main change here is the initialisation of the SyncContext on the MobileServiceClient in the InitialiseDatabaseAsync method and also the fact that I am no longer making calls to MobileServiceClient.GetTable<T> but, instead, I’m making calls to MobileServiceClient.GetSyncTable<T> which means that my data access now is against the local SQLite store rather than the cloud store. I’ve then got the simplistic SyncWithCloudAsync method to attempt to rationalise my local table with the cloud table.

With 1 record in the cloud if I now execute this code;

      DataManager manager = new DataManager();
      
      var customers = await manager.GetCustomersAsync();

then I’m not going to see any data because the data is being pulled from the local SQLite database and there’s nothing in that database. If I change that code to include;

      DataManager manager = new DataManager();

      await manager.SyncWithCloudAsync();

      var customers = await manager.GetCustomersAsync();

then I’m going to get my 1 customer record back from the table because it’s been sync’d with the cloud first. Naturally, deciding when and where to call my SyncWithCloudAsync method within my app would be a crucial thing to get right.

Similarly, if I wrote code such as;

      DataManager manager = new DataManager();

      Customer c = new Customer()
      {
        FirstName = "Ginger",
        LastName = "Rogers",
        SpamWithEmailInvitations = true
      };
      await manager.InsertCustomerAsync(c);

then my local SQLite store is going to contain 2 records and my cloud is going to contain 1 record until I call my sync method again to update the cloud from the local table. It’s worth saying that because the id columns in Mobile Services are Guids rather than integers it’s possible to create records offline without worrying about creating collisions on the ids.

As an aside, the framework here is running through the table’s RESTful access methods on the service so if (e.g.) I have some custom script running on the service side to somehow filter/alter the data then the sync framework here is going to run through that code. It’s not using some “back door” type mechanism to get to the data.

Nonetheless, there may well be conflicts if I have 2 users working on the same data or if I have 1 user on multiple devices working on the same data so I guess conflicts will have to be handled.

In some frameworks you can define sets of columns to watch for the purposes of optimistic concurrency checking but, here, I think it’s just done at the level of a whole record and the version column is acting as the watchman on whether a record has got out of synchronisation or not.

I didn’t include an update mechanism on my DataManager class but I could relatively easily add one;

public async Task UpdateCustomerAsync(Customer customer)
    {
      var table = await this.GetTable<Customer>();
      await table.UpdateAsync(customer);
    }

and now I can cause trouble. The normal way to do this would be to run multiple instances of the app but let’s say that I write code such as this;

   DataManager manager = new DataManager();

      // we are in sync...
      await manager.SyncWithCloudAsync();

      // we get the customers from the local table
      var customers = await manager.GetCustomersAsync();
    
      // we update one of them
      var first = customers.First();

      first.LastName = "Smith";

      // we update the local table.
      await manager.UpdateCustomerAsync(first);

      // we update the cloud.
      await manager.SyncWithCloudAsync();

and everything’s fine and our changes make it to the cloud. What if I opened up a direct route to the cloud though? That is, what if I added this method to my data manager which went around the local database!

    public async Task UpdateCustomerDirectWithCloudAsync(Customer customer)
    {
      await this.InitialiseAsync();
      await this.proxy.GetTable<Customer>().UpdateAsync(customer);
    }

Now I can cause some trouble, code like this for instance;

      DataManager manager = new DataManager();

      // we are in sync...
      await manager.SyncWithCloudAsync();

      // we get the customers from the local table
      var customers = await manager.GetCustomersAsync();
    
      // we update one of them
      var first = customers.First();

      first.LastName = "Jones";

      // we update the local database - note that this will leave the VERSION
      // field in the record (object) alone.
      await manager.UpdateCustomerAsync(first);

      // we update the cloud DIRECTLY - note that this will change the VERSION
      // field in the record (object).
      await manager.UpdateCustomerDirectWithCloudAsync(first);

      // we try to sync...we're going to fail...
      try
      {
        await manager.SyncWithCloudAsync();
      }
      catch (MobileServicePushFailedException ex)
      {
        foreach (var error in ex.PushResult.Errors)
        {
          Debug.WriteLine(
            string.Format("Hit an error with table {0}, status is {1}, item was {2}",
            error.TableName,
            error.Status,
            error.Item));
        }
      }

and this dumps out output to the debug monitor;

Hit an error with table customers, status is PreconditionFailed, item was {
  "id": "7D398BA8-1480-4499-A12D-BA9C0DA8BFC7",
  "firstname": "Fred",
  "lastname": "Jones",
  "spam": false,
  "__version": "AAAAAAAAB94="
}

Naturally, I might want to not expose these kinds of exception directly to the caller of my DataManager class so I could take steps there to wrap things up nicely but I won’t do that here.

Instead, I can make a decision about what to do about a conflict like this. There’s generally three things I can do;

  1. Decide that the client database wins.
  2. Decide that the server database wins.
  3. Ask the user whether the client database or the server database wins.

This is handled by the implementation of IMobileServicesSyncHandler which I pass to the call to IMobileServicesSyncContext.Initialize. I can change my DataManager class to accept a handler – changes below are to the constructor and the InitialiseDatabaseAsync methods (not listed whole class again);

  class DataManager
  {
    static readonly string DATABASE_FILE = "local.db";

    public DataManager(IMobileServiceSyncHandler syncHandler = null)
    {
      this.syncHandler = syncHandler != null ?
        syncHandler : new MobileServiceSyncHandler();
    }
    async Task InitialiseAsync()
    {
      if (this.proxy == null)
      {
        this.proxy = new MobileServiceClient(
          "https://mttestservice.azure-mobile.net/");
      }
      await this.InitialiseDatabaseAsync();
    }
    async Task InitialiseDatabaseAsync()
    {
      if (!this.proxy.SyncContext.IsInitialized)
      {
        // Create/open DB.
        MobileServiceSQLiteStore store =
          new MobileServiceSQLiteStore("local.db");

        // Make sure that store defines our table (at least). NB: repeating
        // this seems to be "ok".
        store.DefineTable<Customer>();

        // Tell our proxy about our database and specify a default
        // IMobileServicesSyncHandler to do the work.
        await this.proxy.SyncContext.InitializeAsync(
          store, this.syncHandler);
      }
    }

With reference to the sample here I could then try and make a “server side wins” sync handler;

  class ServerWinsSyncHandler : IMobileServiceSyncHandler
  {
    public async Task<JObject> ExecuteTableOperationAsync(
      IMobileServiceTableOperation operation)
    {
      // NB: assuming 2nd retry will work (might not be a good thing)
      JObject jObject = null;

      try
      {
        jObject = await operation.ExecuteAsync();
      }
      catch (MobileServicePreconditionFailedException ex)
      {
        // pick up the server-side value from the exception.
        jObject = ex.Value;
      }
      return (jObject);
    }

    public Task OnPushCompleteAsync(MobileServicePushCompletionResult result)
    {
      return (Task.FromResult(0));
    }
  }

and try it out;

     DataManager manager = new DataManager(new ServerWinsSyncHandler());

      // we are in sync...
      await manager.SyncWithCloudAsync();

      // we get the customers from the local table
      var customers = await manager.GetCustomersAsync();
    
      // we update one of them
      var first = customers.First();

      first.LastName = "Local";

      // we update the local database - note that this will leave the VERSION
      // field in the record (object) alone.
      await manager.UpdateCustomerAsync(first);

      first.LastName = "Cloud";

      // we update the cloud DIRECTLY - note that this will change the VERSION
      // field in the record (object).
      await manager.UpdateCustomerDirectWithCloudAsync(first);

      // we try to sync - the handler ensures that the conflict leaves
      // the cloud winning
      await manager.SyncWithCloudAsync();

      // This goes back to our local DB...
      customers = await manager.GetCustomersAsync();
      first = customers.First();
      Debug.Assert(first.LastName == "Cloud");

and that final assertion holds true – i.e. the conflict was detected and the cloud version of the record was left in the local database.

If I want to ensure that the client side of things wins then I can write a different sync handler but I haven’t quite managed to get the implementation of that to work for me just yet so I’ll update the post to add that last piece as/when I’ve got it working a little better than I have right now Smile

Update – trying to get a “client” wins handler working.

I talked to the Mobile Services team around why I couldn’t get a handler working that adopted a strategy of letting the client-side “win” when it came to a conflict between the client-side and the cloud-side.

In doing that, I think I hit a bit of a glitch that is resolved in an updated pre-release version of the mobile services SDK and so I installed an updated version of the SDK which was published yesterday;

image

and then wrote a ClientWinsSyncHandler which executes the request, checks for an exception back from the server and in the presence of that exception it attempts to replace the version number that the client-side record has with the version number that the cloud-side record has and then attempts to do the update again. That looks like;

  class ClientWinsSyncHandler : IMobileServiceSyncHandler
  {
    public async Task<JObject> ExecuteTableOperationAsync(
      IMobileServiceTableOperation operation)
    {
      JObject jObject = null;
      bool retry = true;

      try
      {
        // try for the first time.
        jObject = await operation.ExecuteAsync();
        retry = false;
      }
      catch (MobileServicePreconditionFailedException ex)
      {
        // take the version number that the server says it has and
        // stamp it into the client record so that on the next 
        // try there "should" be no conflict.
        operation.Item[MobileServiceSystemColumns.Version] =
          ex.Value[MobileServiceSystemColumns.Version];
      }
      if (retry)
      {
        // try again. this "should" work this time around although,
        // clearly, based on timing in the real world there could
        // be more conflicts so we might really want to loop.
        jObject = await operation.ExecuteAsync();
      }
      return (jObject);
    }
    public Task OnPushCompleteAsync(MobileServicePushCompletionResult result)
    {
      return (Task.FromResult(0));
    }
  }

and then keeping that previous DataManager implementation I can test this out with something like;

      DataManager manager = new DataManager(new ClientWinsSyncHandler());

      // we are in sync...
      await manager.SyncWithCloudAsync();

      // we get the customers from the local table
      var customers = await manager.GetCustomersAsync();
    
      // we update one of them
      var first = customers.First();

      first.LastName = "Local";

      // we update the local database - note that this will leave the VERSION
      // field in the record (object) alone.
      await manager.UpdateCustomerAsync(first);

      first.LastName = "Cloud";

      // we update the cloud DIRECTLY - note that this will change the VERSION
      // field in the record (object).
      await manager.UpdateCustomerDirectWithCloudAsync(first);

      // we try to sync - the handler ensures that the conflict leaves
      // the local version winning - i.e. it goes from our local store
      // to the cloud and then comes back again to the local store.
      await manager.SyncWithCloudAsync();

      // This goes back to our local DB...
      customers = await manager.GetCustomersAsync();
      first = customers.First();
      Debug.Assert(first.LastName == "Local");

and that final assertion now works out fine on that new version of the SDK – i.e. the local data “wins” and the cloud data is replaced when the 2 records are out of sync.


Posted Tue, Apr 8 2014 11:54 AM by mtaulty