Apply a pinch of salt here, this is a work-in-progress….:-)

I wanted to follow up a little on the blog post I wrote about capturing video/audio from the webcam/microphone with Silverlight 4. In that post I’d written a little about how you could take the audio stream and write it out in some format ( specifically, I chose to write my captured audio out as a WAV file ) but I’d deliberately left out trying to write out the video stream because it felt like it would be “more difficult” and, naturally, I like to avoid any challenge that appears tricky 🙂

Tim’s also got a great video up on Silverlight.NET that also provides a much more “interactive” view of how to get this stuff up and running and builds it up from scratch so I’d recommend that as background as well.

But…what to do about actually writing the video out somewhere and in what format? Capturing the video isn’t too tricky. The question is how to go from the frames captured to a recognisable video format and whether that’s do-able in a cross-platform way?

It’s been asked about on Stack Overflow and there’s a suggestion there around converting the bitmap video frames to JPEG and then writing out an AVI/RIFF file using MJPG which is an uncompressed format so you don’t have to write lots of compression code.

I did quite a bit of reading about the RIFF format.

I initially took this suggested route and looked at the FJCore library for doing the JPEG conversion. This JPEG translation is relatively expensive and so I quickly realised that it’d be a struggle to do it “real-time” ( not a big surprise ) as the video frames come in from the webcam so I structured my code to be a two step process;

1. Capture the raw video from the webcam into isolated storage files
2. Offer an export that takes that format and translates it into an .AVI outside of isolated storage – specifically I’ve chosen to save the file into the MyVideos folder which means that the application needs to be both out-of-browser and Trusted in order to work. I’ll return to this point about file streams in a moment.

However, I’ve only partially gone down this path so far in that I pretty quickly realised that I didn’t have to write JPEGs and so I avoided that for now and wrote out an AVI using the DIB handler device-indepdenent-bitmaps instead ( at 24bpp ). It makes for big files.

This post and code are still very much a work-in-progress though because;

• Having skipped using MJPG and JPEGs, I could possibly skip the whole two-step architecture that I started with and write the .AVI file in real-time.
• There’s probably a way of avoiding the translation that I do from 32bppARGB format to 24bppRGB format and just writing out the raw frames instead thereby making the former bullet point even more likely.

but that’s not where my code is just yet because of the way I originally set about it but I’ll revisit it and see if I can cut it down.

However, I thought it was worth publishing this as a work-in-progress because it’s unlikely that you’d actually want to go with an uncompressed format like I’m doing here because the files end up so big. So, perhaps it is more likely that you’d want to use something like the two-stage process that I’m using here.

Just returning back to point 2 above. Ther'e’s no real need for this application to be trusted because it simply saves videos into isolated storage and then translates those at a later point into some other file which it could prompt the user for. However…I do that file saving on a background thread.

If the application stays sandboxed then it needs to use the SaveFileDialog in order to get to a FileStream and if it does that then, as far as I can tell, that FileStream has thread affinity with the UI thread which means that writing to it from a background thread becomes tricky as I have to use the Dispatcher to get over to the UI thread all the time.

Consequently, for now I went with making the application trusted because then it can open a file directly in MyVideos and not end up with this thread affinity.

I’ll look into this separately as it’s not something I’d been too aware of previously.

Anyway, back to the story – I’ll not paste code here as there’s quite a bit of it and, as I say, this is work-in-progress in that I suspect I can remove a lot of the two-stage code that I’ve described above and cut down a lot of what I wrote.

Instead, I’ll post the project and source but please bear in mind that it’s just a quickly-thrown-together sample and not particularly well structured and has a few gremlins that I need to sort out.

The “app” that I’ve got right now looks like this when you run it in the browser;

and then when it’s running trusted out of the browser it looks like this ( yep, not pretty 🙂 );

Now, as part of the two stage process, this app needs a bunch of isolated storage space and so the slider at the bottom can be used to make that “big” ( note – the code will just fall over if it runs out of isolated storage space whilst capturing video );

Having given myself some more isolated storage space for the app, I can switch on the web cam;

Note that the app won’t show the video until you hit the “Capture Video” button at which point it’ll start writing video into a file ( “Video1” ) in isolated storage.

A note about this. At the moment, I’m finding it’s easier to have a separate piece of UI for switching on the camera as I find that if I put that functionality deep within some function ( even a function which has been directly invoked from a UI action ) then I get exceptions about the functionality not being “UI invoked”. Hence why I have a separate “switch the camera on” button here.

If I then click the “Capture Video” button then there’s a rectangle that will get painted with the webcam video and it’ll be captured into an isolated storage file until the “Stop Capturing” button is clicked.

When the "Stop Capturing” button is clicked, the video file should show up in the ListBox below;

and now that temporary, intermediary file ( written in isolated storage using my own simplistic format ) can be exported using the "Copy to My Videos” button which will copy it to MyVideos\Video1.avi using a background thread but effectively showing a model progress dialog which blocks the rest of the UI until it’s done – note the code that does the file writing is pretty inefficient;

and then it should show up as a .AVI file in MyVideos;

and then it should be playable but I have a confession 🙂

Right now, I can only get it to play in QuickTime and not in Media Player – this is because I have some bug that I’ve failed to nail down yet in the way I’ve written the .AVI file so feel free to jump in and fix that for me but here it is playing inside of QuickTime outside of Silverlight altogether;

So – with all that said it’s a start but there’s a bunch of work I need to do here;

1. Revisit and see if I can move away from this two-pass approach to a single-pass approach.
1. Or, possibly, stick with the two-pass approach because it probably lends itself to file formats that involve compression! 🙂
2. Revisit and see about removing the need for the application to be trusted whilst still writing to a file from a background thread.
3. Generally tidy up the code as it’s a lot messy ( don’t look for some MVVM implementation here – it’s all a bit cobbled together ).
4. Look to use something other than uncompressed bitmaps as the video format as they’re huge 🙂

But it’s a start – here’s the project for download if you want to take a look and move it on yourself in the meantime.

You’ll need Visual Studio 2010 Beta 2 and the Silverlight 4 Beta Development Bits.

Enjoy!