Just an update to this post as I got a little worried after publishing it :-)

In case it wasn't obvious - you wouldn't really want to use a clientaccesspolicy.xml file like the one that I used which says "Allow anybody to do anything" as that'd more than likely be a bad idea.

In my specific case, that would mean that any Silverlight application could try (if it knew the URL) to access my local console application via its SOAP based interface and that's almost not what I'd want.

So...I'd want to tailor client access policy to only allow cross-site access (in this case to the local machine via my SOAP service) to sites that I trusted.