Note: these are early notes based on some initial experiments with the Silverlight 5 beta, apply a pinch of salt to what you read.

As you’d expect, Silverlight can already play audio and has been able to since day one via the MediaElement class which doubles up for also playing video.

If I remember rightly, audio support is for mp3, WMA7, 8 and 9 and also for AAC but does not include WAV files. MediaElement is pretty easy to use but if you’ve got requirements like;

1. Multiple media elements playing back on loops
2. Multiple media elements playing back at different volumes
3. Multiple media elements playing back at different pitches

then it gets very hard very quickly.

Further, if you need to have sound that responds with low latency then you run into more complexities and you can find lots of discussion threads out on there on the web (like this one) that talk about how you can try and achieve low latency playback via MediaElement but it’s a struggle.

Silverlight 5 makes that easier by introducing a new way of playing audio via the “XNA” SoundEffect and SoundEffectInstance classes which offer a new, low-latency way of playing audio with options around the volume, pitch and looping.

I thought about how I might experiment with this and I remembered the old “Silverlight Grand Piano” sample back from Silverlight 1 which you can still run live on the web by clicking the image below;

This sample was based around the MediaElement and you might notice some latency when you first hit a key on that piano with the mouse.

I thought I’d take this and quickly re-work it in Silverlight 5 to use the new SoundEffect class. Whilst I was at it, I also added keyboard support and touch support although I’d have to say that the visuals weren’t really designed to;

1. Have more than one piano key pressed at a time.
2. Be stretched (so I left it fixed sized).

and so it’s a little clunky.

Using the New SoundEffect APIs

The first thing I did was to convert the sample’s use of WMA to WAV because the SoundEffect API supports PCM WAV files and so I have a bunch of WAVs in the project;

when the application starts up I create a dictionary of SoundEffect for each note that we can play (note – I didn’t try to be clever and have just the one note and alter the pitch but that might be a smart move ).

I have this simple array of piano notes;

    static string[] notes = 
    {
      "C",
      "CSharp",
      "D",
      "DSharp",
      "E",
      "F",
      "FSharp",
      "G",
      "GSharp",
      "A",
      "ASharp",
      "B",
    };

and when the application starts up I build a dictionary that maps from the name of the note to the SoundEffect for that note. That is;

    Dictionary<string, SoundEffect> allNoteEffects;

    Dictionary<string, SoundEffectInstance> playingNoteInstances;
   
    void InitialiseEffects()
    {
      this.allNoteEffects = new Dictionary<string, SoundEffect>();
      this.playingNoteInstances = new Dictionary<string, SoundEffectInstance>();

      foreach (var note in notes)
      {
        this.allNoteEffects[note] =
          SoundEffect.FromStream(GetResourceStreamForNote(note));
      }
    }

On line 13 I used SoundEffect.FromStream() to load up the relevant SoundEffect from the embedded WAV file in the resources of my assembly.

Then whenever someone hits a key on the piano (via either the mouse, keyboard or touch) I need to play that particular SoundEffect and so I use;

 void PlayNote(string note)
    {
      if (!this.playingNoteInstances.ContainsKey(note))
      {
        if (this.allNoteEffects.ContainsKey(note))
        {
          SetNoteImageVisibility(note, Visibility.Visible);
          SoundEffectInstance instance = this.allNoteEffects[note].CreateInstance();
          this.playingNoteInstances[note] = instance;          
          instance.Play();
        }
      }
    }

what this is doing is;

1. Checking that we aren’t already playing the particular note.
2. Checking that the note is valid (I get into a particular problem with “E#” otherwise )
3. Using (one line 8) SoundEffect.CreateInstance() to make a SoundEffectInstance from my SoundEffect.
4. Storing (on line 9) the SoundEffectInstance so that we could later Dispose() of it.
5. Playing (on line 10) the sound effect instance.

Note that I don’t think I absolutely needed to use SoundEffectInstance in this scenario but I wanted to experiment with it – you can see how these 2 classes compare below and going down the route of SoundEffectInstance seems to offer a lot more control;

At some point, my user will take their finger off the piano key and I simply Dispose() of the SoundEffectInstance (or instances if they held down multiple piano keys) that was either still playing or had stopped. That’s not the only way to go – I could keep the instances around and re-play them and so on but it suited my logic here.

You can run the modified version (on SL5) by clicking on the image below;

and you should be able to drive it either with;

1. The mouse. Left button down starts the play and left button up disposes immediately so if you don’t hold the mouse button down you’ll get a short note.
2. The keyboard. Use the keys C, D, E, F, G, A, B and the SHIFT key for “sharp” (hopefully )
3. Touch – I’ve only got 2 touch-points on my monitor here but that seemed to work reasonably.

Note that if you double click on the label at the bottom and go full screen then you’ll find that you lose touch support and keyboard support in the browser.

Note also that the double-click is handled by Silverlight 5’s new ClickCount property

    void OnFullScreen(object sender, MouseButtonEventArgs e)
    {
      if (e.ClickCount == 2)
      {
        Application.Current.Host.Content.IsFullScreen =
          !Application.Current.Host.Content.IsFullScreen;
      }
    }

The source code is here for download if you want to poke around yourself. Note that it’s all “code behind” as it was put together quickly.