I’ve been wanting to catch up and explore the (relatively) new Silverlight PivotViewer control for a little while but just hadn’t managed to find the spare cycles.
The announcement is about 6 weeks old at this point which shows how far “behind the times” I’m running
A number of public examples of the use of PivotViewer have appeared on the web in the time since the announcement with perhaps my own favourite being the MSDN Magazine example which I’ve embedded in the page here;
Embedding the application into the page here doesn’t perhaps show it in its best light so you might want to pop over to see it on its original page.
If you haven’t seen it already it’s worth experimenting with including/excluding items from the data-set using the various facets displayed on the left hand side;
and then also changing the way the items are displayed ( grid view or graph view ), how they are sorted and how they are sized;
it’s also worth sorting by something like “Topic” because there’s a lot of topics and too many to fit onto one “graph” so they end up being presented in ranges;
which can then be “zoomed” into – e.g. I can zoom into the “Microsoft Office to Remoting” stack to see those actual topics;
I can also zoom in to the individual items and there’s clearly some DeepZoom going on there and in this particular sample when I get zoomed into an item I get some more detail on it and I have a couple of “actions” that I can perform with it;
so I can “Get the Code”, “Read the Article” ( both of which are regular hyperlinks ) or I can also use this little panel over on the right hand side to navigate in a different way within the PivotViewer collection that I’m viewing.
The first thing I did with this application was to “View Source” in my browser ( actually, these days I just hit F12 in IE8 ) and that showed me that the application is hitting a Pivot collection located at;
http://pivot.blob.core.windows.net/msdn-magazine/msdnmagazine.cxml ( this is also displayed when you first run the app up in the top left )
and I had a bit of a poke around in that (big) CXML file that defines the collection that the control is displaying before remembering that this same CXML format is something that I can open in the Pivot application from Live Labs and so I can then be running the 2 views onto the same collection side-by-side – one provided by Silverlight in a browser and the other in a desktop application (as far as I know there’s no “control” for the desktop right now, there’s “just” the application);
In terms of figuring this stuff out – the main centre for learning about Pivot looks to be your friendly, neighbourhood Silverlight.net site which has surprisingly glossy videos walking you through an overview, some scenarios and a high-level “how to” video and links across to more info on the Live Labs site.
I downloaded the bits from here and installed ( took about 20s ).
Then I read the release notes ( pretty decent, mentioned some good points about MIME types, cross domain policy files, performance for more than 3000 items and also some caching issues across the Silverlight PivotViewer and the Pivot client application ).
Beyond that, I figure this is about 2 things – the collections and the control.
Collections
There’s 2 parts to collections. There’s the collection that represents the data that is displayed on the screen and then there’s the collection that represents the DeepZoom images that are presented along with the data.
How to put those collections together?
Simple Collections
I started with a “Hello World”. There’s plenty of these already out there but there’s nothing like making your own so I made a new Silverlight project and added a collection.xml file to the ClientBin folder on the website where my Silverlight XAP would end up. That file looked like;
<?xml version="1.0"?> <Collection Name="Pictures That Ship With Windows" SchemaVersion="1.0" xmlns="http://schemas.microsoft.com/collection/metadata/2009" xmlns:p="http://schemas.microsoft.com/livelabs/pivot/collection/2009" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <FacetCategories> <FacetCategory Name="Type Of Picture" Type="String" IsFilterVisible="True" IsMetaDataVisible="True"/> </FacetCategories> <Items ImgBase="Images/UntitledProject1/Exported Data/mikesimages/dzc_output.xml"> <Item Id="0" Img="#0" Name="Windows"> <Facets> <Facet Name="Type Of Picture"> <String Value="Windows"/> </Facet> </Facets> <Description>This is the Windows picture</Description> </Item> <Item Id="1" Img="#1" Name="Landscape"> <Facets> <Facet Name="Type Of Picture"> <String Value="Lanscapes"/> </Facet> </Facets> <Description>This is the landscape picture</Description> </Item> <Item Id="2" Img="#2" Name="Nature"> <Facets> <Facet Name="Type Of Picture"> <String Value="Nature"/> </Facet> </Facets> <Description>This is the nature picture</Description> </Item> </Items> </Collection>
and so it’s easy to see that I’ve put 3 items into my collection, they are 3 of the pictures that ship with Windows and I’ve categorised them via a facet called “Type of Picture” into [Windows/Landscapes/Nature]. Easy.
I then dropped into the DeepZoom Composer and, pretty much like the instructions here just did;
- Import the source images to be included in the collection.
- Drag the images to the Layout work area. It doesn’t matter how they are laid out.
- Use Custom Export, specifying Images as the Output Type, and exporting as a collection.
Again – pretty easy and I made sure that the exported image collection went into that folder that’s specified in the ImgBase property of my Items collection up there in the collection.xml file so that the control would find them.
With that in place, I added references to the Pivot assemblies from my Silverlight Project, dragged a PivotViewer control onto the design surface and wrote a little line of code to wire it up to the collection.xml file;
this.Loaded += (s, e) => { // hmm, this api seems to not cope with a relative uri this.pivotViewer1.LoadCollection( "http://localhost:32768/SilverlightApplication27.Web/ClientBin/collection.xml", null); };
and all seemed to work reasonably well in that I got an app that did the right thing with respect to this tiny collection;
Now, there’s a small number of images in that set of images that ship with Windows but it’s a large enough set that I wouldn’t want to type all the details in so I’d want to write code to build up that collection rather than type it out myself into an XML file.
That would probably make sense written for PowerShell but, in the short term, I just wrote a quick console application and referenced the DeepZoomTools.dll ( see here ) in order to get my DeepZoom images created by reading through the c:\windows\web\wallpaper\ folder;
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.IO; using System.Xml.Linq; using Microsoft.DeepZoomTools; namespace ConsoleApplication7 { class Program { static XName CollectionName(string name) { return (XName.Get(name, nsCollection)); } static string PathToPictureType(string path) { return ( Path.GetFileName(Path.GetDirectoryName(path))); } static void Main(string[] args) { string imageFolder = Environment.GetFolderPath( Environment.SpecialFolder.Windows); // seems to be the standard place for Windows 7, at least. imageFolder = Path.Combine(imageFolder, "Web", "Wallpaper"); var images = Directory.GetFiles(imageFolder, "*.jpg", SearchOption.AllDirectories). Select((path, index) => Tuple.Create(index, path, PathToPictureType(path))); XElement items = new XElement(CollectionName("Items"), new XAttribute("ImgBase", outputImageFile), images.Select(image => new XElement(CollectionName("Item"), new XAttribute("Id", image.Item1), new XAttribute("Img", string.Format("#{0}", image.Item1)), new XElement(CollectionName("Facets"), new XElement(CollectionName("Facet"), new XAttribute("Name", facetPictureType), new XElement(CollectionName("String"), new XAttribute("Value", image.Item3))))))); XElement xml = new XElement(CollectionName("Collection"), new XAttribute("Name", "Pictures That Ship With Windows"), new XAttribute("SchemaVersion", "1.0"), new XElement(CollectionName("FacetCategories"), new XElement(CollectionName("FacetCategory"), new XAttribute("Name", facetPictureType), new XAttribute("Type", "String"), new XAttribute("IsFilterVisible", "True"), new XAttribute("IsMetaDataVisible", "True"))), items); CollectionCreator creator = new CollectionCreator(); // this is wonderfully optimistic 🙂 creator.Create( images.Select(i => i.Item2).ToList(), outputImageFile); // as is this xml.Save(collectionFile); } const string collectionFile = @"c:\temp\collection.xml"; const string outputImageFile = @"c:\temp\outputImages.xml"; const string facetPictureType = "Type of Picture"; const string nsCollection = "http://schemas.microsoft.com/collection/metadata/2009"; } }
so, that is all a bit hard-coded and accepting of defaults in the object model but it worked fine in that it emitted 4 things;
- collection.xml file that I can quickly hack to update the ImgBase property to point to the proper web URL where I’m going to host the corresponding DeepZoom Image collection
- outputImages.xml – the DeepZoom image collection metadata that I’m going to host on my web server – this is the file that I’ll point ImgBase at
- outputImage_files – folders containing DeepZoom image bits
- outputImage_images – folders containing DeepZoom image bits
and so I can copy those bits to the right place on my web server and update the files so the links are to the right things and I’m now viewing all those windows images in the PivotViewer;
I’m pretty sure that I could have achieved the same thing for this particular collection by using the Excel tool and just bringing in the ~30 images and typing in a few details but I’m naturally lazy.
Now, the docs call this a Simple Collection in the sense that it’s just one huge XML file containing everything and it’s all pre-generated. The docs partition collection types into;
- Simple collections containing up to 3,000 items. In these collections, the user's experience is defined by previously generated (or static) XML.
- Linked collections, generally containing several thousands of items. These collections consist of multiple inter-linked simple collections.
- Dynamic collections, unbounded in size. In these collections, the user's experience is defined by XML generated dynamically in response to user action.
Linked Collections
I could group my simple collections into a linked collection of simple collections which would get me into potentially 10’s of thousands of items but has some ( naturaly ) limitations ( as cribbed from the docs );
- Storage – Large linked collections can result in very large numbers of image files. Due to likely overlaps among different segments of a linked collection, the number of image files can quickly multiply into the many millions.
- Updates – If the collection content rarely changes, a large number of image files may not pose a major problem. The addition or removal of an item, however, often requires large portions of the Deep Zoom image tile pyramid to be regenerated (read more about Deep Zoom tile pyramids here: Collection Image Content). As a result, maintaining a large collection that needs to be updated frequently can be cumbersome.
- User Experience – Great linked collections can be difficult to create when the data set at hand does not lend itself naturally to division into smaller segments. They are also limited in their ability to respond to un-anticipated queries, and do not naturally support search across their segments.
I thought I’d see if I could generate a linked collection even though my set of 31 items doesn’t really need one but, hey, that doesn’t mean the concept is any different.
I thought I’d go with 2 collections, one being “world” ( architecture, landscapes, nature ) and the other being “abstract” ( characters, scenes, Windows ). So, in the first instance I can change my console application to spit out 2 such collections ( sure, it’s still pretty messy code );
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.IO; using System.Xml.Linq; using Microsoft.DeepZoomTools; namespace ConsoleApplication7 { class Program { static XName CollectionName(string name) { return (XName.Get(name, nsCollection)); } static string PathToPictureType(string path) { return ( Path.GetFileName(Path.GetDirectoryName(path).ToLower())); } static void WriteImagesToCollection( IEnumerable<Tuple<int, string, string>> imageInfo, string outputCollectionFile, string outputImageFile) { XElement items = new XElement(CollectionName("Items"), new XAttribute("ImgBase", "INSERT PROPER URL HERE"), imageInfo.Select(image => new XElement(CollectionName("Item"), new XAttribute("Id", image.Item1), new XAttribute("Img", string.Format("#{0}", image.Item1)), new XElement(CollectionName("Facets"), new XElement(CollectionName("Facet"), new XAttribute("Name", facetPictureType), new XElement(CollectionName("String"), new XAttribute("Value", image.Item3))))))); XElement xml = new XElement(CollectionName("Collection"), new XAttribute("Name", "Pictures That Ship With Windows"), new XAttribute("SchemaVersion", "1.0"), new XElement(CollectionName("FacetCategories"), new XElement(CollectionName("FacetCategory"), new XAttribute("Name", facetPictureType), new XAttribute("Type", "String"), new XAttribute("IsFilterVisible", "True"), new XAttribute("IsMetaDataVisible", "True"))), items); CollectionCreator creator = new CollectionCreator(); // this is wonderfully optimistic 🙂 creator.Create( imageInfo.Select(i => i.Item2).ToList(), outputImageFile); // as is this xml.Save(outputCollectionFile); } static IEnumerable<Tuple<int, string, string>> FilterImageList( IEnumerable<string> imageList, Func<string, bool> predicate) { return (imageList. Where(predicate). Select( (path, index) => Tuple.Create(index, path, PathToPictureType(path)))); } static void Main(string[] args) { string imageFolder = Environment.GetFolderPath( Environment.SpecialFolder.Windows); // seems to be the standard place for Windows 7, at least. imageFolder = Path.Combine(imageFolder, "Web", "Wallpaper"); var allImages = Directory.GetFiles(imageFolder, "*.jpg", SearchOption.AllDirectories).ToList(); var worldPictureTypes = new string[] { "architecture", "landscapes", "nature" }; var worldImages = FilterImageList(allImages, image => worldPictureTypes.Contains(PathToPictureType(image))); var abstractImages = FilterImageList(allImages, image => !worldPictureTypes.Contains(PathToPictureType(image))); WriteImagesToCollection(worldImages, @"c:\temp\worldCollection.xml", @"c:\temp\worldImages.xml"); WriteImagesToCollection(abstractImages, @"c:\temp\abstractCollection.xml", @"c:\temp\abstractImages.xml"); } const string facetPictureType = "Type of Picture"; const string nsCollection = "http://schemas.microsoft.com/collection/metadata/2009"; } }
I can then take the 2 collections and 2 sets of DeepZoom images that drop out of that process and drop them onto my website (with the ImgBase URL property correctly fixed) so that the PivotViewer can pick them up.
Then I can easily add some little radio buttons that flick between one collection and the other as in;
and all that’s doing is switching the collection at runtime as the radio button changes as below;
string url = string.Format( "http://localhost:32768/SilverlightApplication27.Web/ClientBin/{0}collection.xml", (bool)radioWorld.IsChecked ? "world" : "abstract"); this.pivotViewer1.LoadCollection(url, null);
but the “problem” with that is that if I do something like;
- Narrow my filter to only include “landscapes” and “scenes” in the world collection
- Switch to the abstract collection
- Narrow my filter to only include “characters” and “scenes”
- Switch back to the world collection
then at point (4) things get messy because the PivotViewer is reset rather than returning to my previous view on the world collection so the experience feels ugly.
I’d like the control to preserve that view so that I can “navigate” back to it.
The control offers this via a property called ViewerState ( explained here ) which lets me get to a state bag that represents the view that’s being used and the facets that are being filtered on so I could write a little code to capture that and put it back in place as the user switches between collections. So, with a member variable on my class (string) called previousState I can change that previous code to;
string url = string.Format( "http://localhost:32768/SilverlightApplication27.Web/ClientBin/{0}collection.xml", (bool)radioWorld.IsChecked ? "world" : "abstract"); string tempState = this.pivotViewer1.ViewerState; this.pivotViewer1.LoadCollection(url, this.previousState); this.previousState = tempState;
and that makes the jumping backwards and forwards experience better than it was. I also experimented with preserving the CurrentItemId of the control across these “jumps” as well in case the user was focused on a specific item when they made the jump into the other collection.
I’m not sure how this integrates with Silverlight’s navigation system. I could see my embedding this into a navigation page and then as Silverlight navigation occurs I could treat the fragment part of the URL handed to a Silverlight page as a URL fragment to be handed to the PivotViewer and so integrate the two. That’s for a later experiment…
But how to link items in one collection to items in another? An item has an Href property which I could use to jump a user to details about that item but it doesn’t feel right to attempt to hijack that Href in order to jump to a related collection.
There’s no need though because the notion of related/linked collections is already present. I can use the extensions schema ( documented here and here ) in order to add a set of related collections to an item. As an example, I manually hacked one of my items to include;
<Item Id="0" Img="#0"> <Extension> <ext:Related xmlns:ext="http://schemas.microsoft.com/livelabs/pivot/collection/2009"> <ext:Link Name="Link A" Href="http://www.microsoft.com"/> <ext:Link Name="Link B" Href="http://www.microsoft.com"/> <ext:Link Name="Link C" Href="http://www.microsoft.com"/> </ext:Related> </Extension>
and the info panel updates to include those links across to related collections as in;
Nice – so how do those hyperlinks fire? They fire via the LinkClicked event on the PivotViewer control and the link is part of the LinkEventArgs that’s passed to me.
That event is also passed for other hyperlink clicks in the Info Panel if those links don’t resolve into the current collection so if I gave an Item a Name and an Href set to something like http://www.microsoft.com then I’d see;
and clicking on that would fire the same event for that external URL.
I can see how I can use Related Links in order to link to other collections but I’m not 100% sure how to structure those links.
For instance – if I want to link all the architecture images in the world collection to the scenes images in the abstract collection then how I construct a hyperlink here which includes both the collection and the facet that I want?
I’d expected to use something like;
http://vroot/abstractCollection.cxml#$facet0$=Type Of Picture=EQ.scenes
but the documentation for PivotViewer.LoadCollection says that you can’t pass URLs like that to LoadCollection but, instead, you need to pass the collection URL and the ViewerState separately.
So, it feels a little like I’m using a single data attribute here on the Related hyperlink to store 2 pieces of information. I’d have liked to be able to store any arbitrary data on one of those related links – for example;
<!-- pseudo xml --> <Item Id="0"> <Related> <Link Name="foo" Href="http://vroot/abstractCollection.cxml"> <State> <Property Key="bar" Value="1"/> </State> </Link> </Related> </Item>
and then have that additional State bag up there delivered to my code when someone clicks the link.
Anyway, that doesn’t look to be something that I can do right now. At this point, I unzipped the sample application that comes with the PivotViewer and I see it using links that include the collection URL and the ViewerState and then using code to pull them apart again when the link is clicked and feed the 2 separate pieces of information into PivotViewer.LoadCollection.
I didn’t want to duplicate the entire URL to my 2 collections in every link to them so I went with my own internal URI formatted like;
Href = “info:world:landscapes”
which has the advantage of using a bit less space in the XML and not embedding the location of the .cxml files into that XML either. I can then pick up the tag world/abstract in my code and link to through to the right .cxml file.
I hacked my code that’s generating my collection files and I do mean hacked because I think my use of Tuple here has just about made this code unmaintainable and pretty inefficient but I was being lazy with regard to creating more classes to throw at the problem;
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.IO; using System.Xml.Linq; using Microsoft.DeepZoomTools; namespace ConsoleApplication7 { class Program { static XName CollectionName(string name) { return (XName.Get(name, nsCollection)); } static string PathToPictureType(string path) { return ( Path.GetFileName(Path.GetDirectoryName(path).ToLower())); } static void WriteImagesToCollection( IEnumerable<Tuple<int, string, string>> imageInfo, Func<string, Tuple<string,string>> relatedCollectionMapper, string outputCollectionFile, string outputImageFile) { XElement items = new XElement(CollectionName("Items"), new XAttribute("ImgBase", "INSERT PROPER URL HERE"), imageInfo.Select(image => new XElement(CollectionName("Item"), new XElement(CollectionName("Extension"), new XElement(XName.Get("Related", nsExtensions), new XElement(XName.Get("Link", nsExtensions), new XAttribute("Name", "Related Collection"), new XAttribute("Href", string.Format("info:{0}:{1}", relatedCollectionMapper(image.Item3).Item1, relatedCollectionMapper(image.Item3).Item2))))), new XAttribute("Id", image.Item1), new XAttribute("Img", string.Format("#{0}", image.Item1)), new XElement(CollectionName("Facets"), new XElement(CollectionName("Facet"), new XAttribute("Name", facetPictureType), new XElement(CollectionName("String"), new XAttribute("Value", image.Item3))))))); XElement xml = new XElement(CollectionName("Collection"), new XAttribute("Name", "Pictures That Ship With Windows"), new XAttribute("SchemaVersion", "1.0"), new XElement(CollectionName("FacetCategories"), new XElement(CollectionName("FacetCategory"), new XAttribute("Name", facetPictureType), new XAttribute("Type", "String"), new XAttribute("IsFilterVisible", "True"), new XAttribute("IsMetaDataVisible", "True"))), items); CollectionCreator creator = new CollectionCreator(); // this is wonderfully optimistic 🙂 creator.Create( imageInfo.Select(i => i.Item2).ToList(), outputImageFile); // as is this xml.Save(outputCollectionFile); } static IEnumerable<Tuple<int, string, string>> FilterImageList( IEnumerable<string> imageList, Func<string, bool> predicate) { return (imageList. Where(predicate). Select( (path, index) => Tuple.Create(index, path, PathToPictureType(path)))); } static void Main(string[] args) { string imageFolder = Environment.GetFolderPath( Environment.SpecialFolder.Windows); // seems to be the standard place for Windows 7, at least. imageFolder = Path.Combine(imageFolder, "Web", "Wallpaper"); var allImages = Directory.GetFiles(imageFolder, "*.jpg", SearchOption.AllDirectories).ToList(); var worldPictureTypes = new string[] { "architecture", "landscapes", "nature" }; var worldImages = FilterImageList(allImages, image => worldPictureTypes.Contains(PathToPictureType(image))); var abstractImages = FilterImageList(allImages, image => !worldPictureTypes.Contains(PathToPictureType(image))); var relatedCollections = new Tuple<string,string>[] { Tuple.Create("architecture", "scenes"), Tuple.Create("landscapes", "characters"), Tuple.Create("nature", "windows") }; WriteImagesToCollection( worldImages, pictureType => Tuple.Create("abstract", relatedCollections.First(rc => rc.Item1 == pictureType).Item2), @"c:\temp\worldCollection.xml", @"c:\temp\worldImages.xml"); WriteImagesToCollection( abstractImages, pictureType => Tuple.Create("world", relatedCollections.First(rc => rc.Item2 == pictureType).Item1), @"c:\temp\abstractCollection.xml", @"c:\temp\abstractImages.xml"); } const string facetPictureType = "Type of Picture"; const string nsCollection = "http://schemas.microsoft.com/collection/metadata/2009"; const string nsExtensions = "http://schemas.microsoft.com/livelabs/pivot/collection/2009"; } }
Anyway! my output collections now contain related links such as this one which says that this image has related content in the abstract collection;
<Item Id="0" Img="#0"> <Extension> <Related xmlns="http://schemas.microsoft.com/livelabs/pivot/collection/2009"> <Link Name="Related Collection" Href="info:abstract:scenes" /> </Related> </Extension>
so I just need a little code to pull about those URLs when someone clicks on one ( my collections are stored in worldCollection.xml and abstractCollection.xml );
void OnLinkClicked(object sender, System.Windows.Pivot.LinkEventArgs e) { if (e.Link.Scheme == "info") { string[] pieces = e.Link.AbsoluteUri.Split(':'); string collection = pieces[1]; string pictureType = pieces[2]; string url = string.Format( "http://localhost:32768/SilverlightApplication27.Web/ClientBin/{0}collection.xml", collection); string viewerState = string.Format("Type of Picture=EQ.{0}", pictureType); this.pivotViewer1.LoadCollection(url, viewerState); } }
and that all seems to work reasonably well to let me jump from one collection to its related collection so in the way that this image from the nature set in the world collection can make a jump for me to the windows set in the abstract collection;
Ok, enough on linked collections.
Dynamic Collections
The ultimate in flexibility comes from generating collections dynamically – clearly going dynamic means that there’s going to be wins in terms of letting you store less additional data and also in terms of how you deal with collection items that are changing which isn’t so easy with a simple or linked collection.
At the time of writing, I haven’t stared too hard at dynamic collections other than reading through the docs up here and thinking that this picture;
is a bit scary but, ultimately, it looks to come down to something to generate the CXML and something to generate the tiles for the DeepZoom so not perhaps quite as scary as that picture suggests.
What I did do though was to run the Wikipedia example and watch it with Fiddler – I’m not sure if there’s a Silverlight PivotViewer version of the Wikipedia collection. It’s on the web here;
but I found that I needed to run up the LiveLabs Pivot desktop application to actually view the collections.
I couldn’t find a Silverlight version so I stole the HTML/JS source from that page and made a quick, hacked together version that I put here – not sure that giving the Silverlight part of that web page 50% in a div was a good idea but there you go;
I think I’d get the same effect by monitoring the desktop version as the Silverlight version but I wanted to be sure and what I see by watching this is;
- Initial request for a collection such as /Wikipeda.cxml?Subject Area=Music
- Initial response of a Collection with 100 items in it with facets of Wikipedia Category, Subject Area, Rank, Years
- An interesting looking ImgBase property for the DeepZoom images for the collection which looks to be generated on the fly
- Lots of requests using the dynamically generated ImgBase for the DeepZoom images
- More requests for those DeepZoom tiles as I zoom into the imagery
I’m sure there’s lots of fancy stuff going on server-side around generation and caching of the CXML collection and the DeepZoom imagery but that’s where I’m leaving dynamic collections for the moment. There is an some sample code around that tile building hanging off the docs.
The other side of this is about what the control itself can do as it views these collections but I’d better move that into a follow on post seeing how long this one has got already…