Ok, ok – this will be the last post about webcams and capturing video. I’ve got to stop 🙂

BUT…I thought I’d modify the code from the last post a little in order to try and capture audio along with the video. This opened up some “interesting questions” in that in Silverlight you derive from an AudioSink to capture audio and from a VideoSink to capture video.

Now, those 2 base classes look awfully similar and if you want to capture both audio and video and try to synchronise the two then it feels a bit painful that you can’t implement some capturing interfaces rather than have to derive from base classes.

Also, even though you feed both sinks into a CaptureSource, I noticed that I was getting audio samples a little before I was getting any video samples and so it looks like there a job involved to try and synchronize the two together.

Silverlight does tell you where in the time sequence these various samples come from and it tells you how long they last but it could be expensive to try and analyse that information and act appropriately so I simply;

1. Throw away any audio samples that arrive prior to my video samples arriving.
2. Accept that there’s a race around (1) and that I probably throw away a little too much/too little audio.
3. Assume that once I’ve started capturing audio and video they are then in sync and I don’t have to analyse the start times/durations to get them back into sync.

Those might not be the smartest things to do 🙂

Then there’s threading. I noticed that my AudioSink methods OnStartCapture/OnSamples etc. are called on some worker thread and my VideoSink methods OnCaptureStarted/OnSample etc are called on some other worker thread.

This causes a bit of fun because I need to write a RIFF AVI file which has a header that can’t really be written out until I’ve seen the VideoFormat and AudioFormat that are passed into OnStartCapture from these multiple threads. So I then have to try and write out the header whilst queueing up any video/audio frames that arrive in the meantime and, subsequently, writing those out as chunks to the RIFF file in the order that they arrived which I use async I/O for relying on my queue to keep the IOs in order.

What I have “seems to work” but I’d want to refactor it out quite a bit to feel more comfortable that it survives all these threads running through it and I’d probably rework the AudioSink/VideoSink/AVI File Writing part of this code if I was to look at it again in the future ( which I’m not going to 🙂 ).

And with all that said;

1. Adding uncompressed audio to an uncompressed video file makes an enormous file format even larger 🙂
2. I still can’t get the resultant AVI file to play in anything other than QuickTime 🙂

but it was still fun to experiment with. The new version of the application with audio is running in the page below;

and if you want to download the source for your own experimentation then that’s here too.

Enjoy!

( no more posts from me about webcams )

( promise )

( really )

( ever )

( no, genuinely, that’s it, no more, done, over )

( then again, I would like to just refactor that code a little…. 😉 )