Intel RealSense Camera (F200): Hand Alerts & Gesture Alerts

Posted on by

Adding to this set of posts I wanted to begin to look at the hand tracking capabilities and I figured I’d try and start simple with by looking at the capabilities around ‘alerts’ and ‘gestures’ and it feels very familiar to what I did with facial alerts here and I wrote a comparable console application in the first instance;

namespace ConsoleApplication1
{
  using System;
  using System.Collections.Generic;
  using System.Linq;

  class PXCMStatusException : Exception
  {
    public PXCMStatusException(pxcmStatus status)
    {
      this.Status = status;
    }
    public pxcmStatus Status { get; private set; }
  }
  static class PXCMStatusExtensions
  {
    public static void ThrowOnFail(this pxcmStatus status)
    {
      if (!status.Succeeded())
      {
        throw new PXCMStatusException(status);
      }
    }
    public static bool Succeeded(this pxcmStatus status)
    {
      return (status == pxcmStatus.PXCM_STATUS_NO_ERROR);
    }
  }
  class Program
  {
    static void Main(string[] args)
    {
      Console.WriteLine("Hit a key to end...");

      using (PXCMSenseManager senseManager = PXCMSenseManager.CreateInstance())
      {
        senseManager.EnableHand();

        var handModule = senseManager.QueryHand();

        using (var handConfiguration = handModule.CreateActiveConfiguration())
        {
          handConfiguration.EnableAllAlerts().ThrowOnFail();
          handConfiguration.EnableAllGestures().ThrowOnFail();

          handConfiguration.SubscribeAlert(hand =>
            {
              Console.WriteLine(
                "A hand alert has happened! [{0}] [{1}]",
                hand.frameNumber, hand.label);
            }
          ).ThrowOnFail();

          handConfiguration.SubscribeGesture(gesture =>
            {
              Console.WriteLine("A gesture has happened! [{0}] [{1}] [{2}] [{3}]",
                  gesture.frameNumber, gesture.handId, gesture.name, gesture.state);
            }
          ).ThrowOnFail();

          handConfiguration.ApplyChanges().ThrowOnFail();
        }

        senseManager.Init();

        senseManager.StreamFrames(false);

        Console.ReadKey();

        handModule.Dispose();
        senseManager.Dispose();
      }
    }
  }
}

and then I can sit in front of this, raise a single hand and attempt to do a gesture or two (the SDK lists them all out and I’ll revisit in a follow on post) driving simple console output like;

handgestureetc

and you can see the start of a fist gesture in there and a full_pinch and various hand alerts around whether the hand is inside or outside of borders and so on. The full set of hand alerts are;

 public enum AlertType
  {
    ALERT_HAND_DETECTED = 1,
    ALERT_HAND_NOT_DETECTED = 2,
    ALERT_HAND_TRACKED = 4,
    ALERT_HAND_NOT_TRACKED = 8,
    ALERT_HAND_CALIBRATED = 16,
    ALERT_HAND_NOT_CALIBRATED = 32,
    ALERT_HAND_OUT_OF_BORDERS = 64,
    ALERT_HAND_INSIDE_BORDERS = 128,
    ALERT_HAND_OUT_OF_LEFT_BORDER = 256,
    ALERT_HAND_OUT_OF_RIGHT_BORDER = 512,
    ALERT_HAND_OUT_OF_TOP_BORDER = 1024,
    ALERT_HAND_OUT_OF_BOTTOM_BORDER = 2048,
    ALERT_HAND_TOO_FAR = 4096,
    ALERT_HAND_TOO_CLOSE = 8192,
    ALERT_HAND_LOW_CONFIDENCE = 16384,
  }

and it occurs to me that somewhere in the state machine represented by this state is a state that represents a ‘happy’ state and I figure I’ll be writing that code soon but this was my ‘Hello World’ in using hand alerts, gestures.