Experiments with Shared Holograms and Azure Blob Storage/UDP Multicasting (Part 5)

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

A quick follow-up to my previous post where I’d done a little testing around changing the rotation of a shared hologram.

I modified the script in the solution named Rotate.cs to become a script named MoveScaleRotateTest.cs and changed the behaviour so as to do some scaling and translation alongside the rotation.

I then adopted the same ‘strategy’ of trying to test this out using one HoloLens and my console-based recording application;

  • Create some shared holograms on the HoloLens which sends an ‘object created’ message to the console application and a world anchor blob to the cloud.
  • Tap on a shared hologram which starts it slowly rotating, scaling and translating and sending ‘object transformed’ messages to the console application.
  • Tap again on a shared hologram which sends an ‘object deleted’ message to the console application.
  • Restart the HoloLens application.
  • Replay all the recorded messages from the console application to see if the HoloLens application behaves correctly when it receives those messages by creating the hologram in the right place, transforming it in the right way before ultimately deleting it again.

and things seem to work out reasonably well Smile so I pushed those changes back to github.

With that, I’m reasonably happy that this seems to work although I’d like to test on multiple devices.

However, there’s still a big item missing from the ‘TODO’ list that I had at the end of this post and that’s the notion of having some kind of ‘memory’ of the shared scene that’s being built up.

So far, the code that I’ve written can cope with a scenario where N devices are on a network and holograms are created, deleted and manipulated (rotate, scale, translate) while all the devices are alive and receiving multicast messages.

What it doesn’t handle at all is a scenario where one device comes along, sets up a ‘scene’ with some shared holograms, world anchors, transformations etc. and then another device joins into that scene at a later point in time having missed all the detailed messages that have been multicast while it wasn’t “present” on the network.

How does a joining device get an up to date picture of the shared holograms that other devices have already created? Right now, that’s a “TBD” for this particular set of blog posts but I’ll come back to it if I can find some time to put something in place…

Experiments with Shared Holograms and Azure Blob Storage/UDP Multicasting (Part 4)

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

Following up on my previous post, one of the “to do” items was to allow a hologram to move, rotate, scale and have that change in its transform carried across the network to other devices.

I’m going to assume for this post that the hologram is not going to want to move so much as to cause it to change its parent and move to another world-anchored parent as that’d require more work right now so I’m thinking of small movements relative to the parent object that are reflected by changes in the local position, rotation and scale properties of a transform.

I added a settable property to my SharedHologramsController to flag whether the code should attempt to synchronise transforms on the shared objects that it has created;

image

and an interval at which to attempt synchronisation.

Naturally, a lot of GameObjects being shared over the network with a high interval is going to equate to a lot of network messages so there’s a trade-off to be made there.

I then added a MonoBehaviour derived script to actually poll a GameObject and watch for changes to its Transform.LocalPosition, LocalRotation and LocalScale properties before dispatching them over the network using the TransformMessage message that already existed in the project.

That script is called TransformSynchronizer and it’s fairly simple (and perhaps too simple! Smile).

That’s all the changes to enable that sort of behaviour but I’d really like to test it properly with multiple devices as so far I’ve only been able to use the approach that I mentioned in an earlier post of having the HoloLens multicast messages to a console application which then sends them back again as a way of simulating a second device.

To start to test this out, I modified the test scene in the Unity project to add a behaviour such that the first tap on a cube will start it slowly rotating whereas the second tap on a cube will now delete it. That let me test changes to the cube’s rotation changing and I need to add some more code to test out changes to position and scale but rotation seems to work “reasonably”.

All of that code is contained in the TestScript in the Unity project.

I’ll make subsequent updates if I find that changes to local position and local scale don’t behave in a suitable way.

Experiments with Shared Holograms and Azure Blob Storage/UDP Multicasting (Part 3)

NB: The usual blog disclaimer for this site applies to posts around HoloLens. I am not on the HoloLens team. I have no details on HoloLens other than what is on the public web and so what I post here is just from my own experience experimenting with pieces that are publicly available and you should always check out the official developer site for the product documentation.

Following up on my previous post, one of the “to do” items was to avoid the hard-coding of the mechanism via which ‘shared’ holograms were to be created.

In the example code below (taken from the TestScript in the Unity project) which makes a call to SharedCreator.Create…

        SharedHologramsController.Instance.Creator.Create(
            "Cube",
            position,
            forward,
            new Vector3(0.1f, 0.1f, 0.1f),
            cube =>
            {
                ChangeMaterial(cube, this.GreenMaterial);
                cube.AddComponent<BoxCollider>();
            }
        );

the Create method expects to be passed the name of a Unity primitive like “Cube” or “Sphere”. I updated the repo so as to add a simple interface via which this can hopefully be abstracted;

using System;
using UnityEngine;

public interface ICreateGameObjects
{
    void CreateGameObject(string gameObjectSpecifier, Action<GameObject> callback);
}

and then exposed a property on the SharedCreator via which an implementation can be plugged in. By default, that property value will be an instance of a class named PrimitiveGameObjectCreator which simply encapsulates logic which was previously hard-wired into the SharedCreator class.

While doing this, I was very conscious that I really need to just bite the bullet and plug in an IoC container which would probably make the whole thing a lot nicer. Pete has some notes on that in his post here.