Following on from the previous post, to get this work started I wanted to build a little library that supported what I wanted to do with the Twitter API.
There’s probably already a .NET library out there but I didn’t search for it and my needs were fairly small and specific.
The first thing that I wanted to do was to build a custom configuration file section to represent my configuration. In my configuration I wanted to be able to set;
- The language to use for searching twitter
- The set of search terms to search for. I wanted each search term to have the configuration options of;
- On/Off – i.e. isIncluded when we do searching
- Filter – i.e. I wanted to be able to filter to only include “positive” tweets, “negative” tweets, tweets that are explicitly either “positive” or “negative” and then “all” tweets.
I don’t know what you do when you want a custom configuration file section but I know what I do. I use the Configuration File Section Designer which I wrote about here. If you don’t use this designer then I guess that either;
- You don’t build custom configuration file sections in .NET. Fair enough.
- You have a lot of spare time on your hands and don’t like progress 🙂
Seriously, it’s brilliant and it’s also a good example of what T4 templates can do for you.
And so…here’s my “design” of my configuration file section or my .csd file;
what you can’t see on the diagram is all the properties that are set on these items such as the code Namespace being SearchTermsConfig or that language is of type string and is required and so on and so forth.
What this ultimately gives me is a set of generated classes and, specifically, I can just write code like this;
static void Main(string[] args)
{
SearchTermsConfiguration config = SearchTermsConfiguration.Instance;
Console.WriteLine("language is {0}", config.language);
foreach (SearchTerm term in config.SearchTerms)
{
Console.WriteLine("Term is {0}", term.Term);
Console.WriteLine("IsIncluded is {0}", term.IsIncluded);
Console.WriteLine("Filter is {0}", term.Filter);
}
}
against a config file like this;
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="searchTermsConfiguration"
type="SearchTermsConfig.SearchTermsConfiguration,ConsoleApplication3"/>
</configSections>
<!-- Set language to "en", "fr", etc. -->
<searchTermsConfiguration language="en">
<searchTerms>
<!-- isIncluded is a bool (true/false) which can be changed in the UI anyway -->
<!-- filter is an enum which can be;
Positive
Negative
PositiveAndNegative (not the same as "All" below)
All
-->
<searchTerm term="Microsoft"
isIncluded="true"
filter="All"/>
<searchTerm term="Windows"
isIncluded="true"
filter="All"/>
</searchTerms>
</searchTermsConfiguration>
</configuration>
and it just works 🙂 which is very cool. Ok, enough about the configuration section designer.
I didn’t want to just use these classes inside the rest of my code so I wrote some companion classes such as this little enum to represent my simplistic filtering;
[Flags]
public enum SearchTermFilter
{
// Bit of a weird bitmask here. 0 means everything. Bits 1 and 2 filter it down.
All = 0x0,
Positive = 0x1,
Negative = 0x2,
PositiveAndNegative = 0x3, // Only here so it can go in the config file.
}
and this SearchTerm class which in many ways mimics what I have in the configuration file but adds some more things that’ll be useful to me at a later point;
It’s basically a class that can return the Term, whether it’s currently valid (IsTermValid), whether it’s currently in-use (IsTermIncluded), what the filter is (Filter, FilterIncludesNegative, FilterIncludesPositive) and the Uri that you need to use to hit Twitter to get an Atom feed for one of these things (AtomFeedUri).
I also wrote a little collection class which keeps hold of these SearchTerms for me and to load them from the configuration file (using the generated classes from the configuration section designer);
public class SearchTermsConfiguration : ObservableCollection<SearchTerm>
{
public static SearchTermsConfiguration LoadFromConfig()
{
SearchTermsConfiguration collection = new SearchTermsConfiguration();
foreach (SearchTermsConfig.SearchTerm configItem in
SearchTermsConfig.SearchTermsConfiguration.Instance.SearchTerms)
{
collection.Add(new SearchTerm()
{
Term = configItem.Term,
IsIncluded = configItem.IsIncluded,
Filter =
(SearchTermFilter)Enum.Parse(
typeof(SearchTermFilter), configItem.Filter)
});
}
return (collection);
}
public static string Language
{
get
{
return (SearchTermsConfig.SearchTermsConfiguration.Instance.language);
}
}
}
and you can see in there that I “hide” the underlying generated classes such as SearchTermsConfig.SearchTerm in order to produce an ObservableCollection of my own SearchTerm class. You might also spot the string from the configuration file being turned into an instance of my SearchTermFilter enumeration.
With that in place, I need something that reads one of these Twitter feeds that correspond to a particular search term and so I wrote a little class called AtomFeedReader (bad name because it’s more specific than the name implies). I won’t include the whole source for it here but it looks like this and brings in another class with a bad name, AtomFeedItem;
The idea of this AtomFeedReader is that I construct it with a SearchTerm and then I can go and call DequeueTopItem. What it then does;
- Checks its list of queuedItems to see if there is an AtomFeedItem at the top of that queue. If so, return it.
- If not, begin an async download to hit Twitter, get the Atom feed for the search term, convert the items found there to AtomFeedItems and enqueue them hoping that someone will later call DequeueTopItem and then this time they’ll get some data.
So…pretty simple stuff and AtomFeedReader is expected to be called on some kind of timer with the caller trying DequeueTopItem to see if anything’s available. I’ll come back to HasImageNeedingDownload at a later point as it’s not obvious at all.
What I need now is a class to tie together multiple of these AtomFeedReader classes based on my configuration and poll them from time to time to see if they’ve got any interesting data. I wrote that class and it’s not so big so I’ll paste it here;
public class AtomFeedManager
{
public event EventHandler<AtomFeedItemAvailableEventArgs> FeedItemAvailable;
public event EventHandler<AtomFeedItemUnavailableEventArgs> FeedItemUnavailable;
public AtomFeedManager(SearchTermsConfiguration searchTerms,
TimeSpan timerInterval)
{
this.searchTerms = searchTerms;
lastTimeOfFeedItem = DateTime.Now;
currentIndex = new ModuloInt(searchTerms.Count);
InitialiseReaders();
OnTimerElapsed(null, null);
timer = new Timer(timerInterval.TotalMilliseconds);
timer.Elapsed += OnTimerElapsed;
timer.Start();
}
void InitialiseReaders()
{
readers = new List<AtomFeedReader>();
foreach (SearchTerm term in searchTerms)
{
readers.Add(new AtomFeedReader(term));
}
}
void OnTimerElapsed(object sender, ElapsedEventArgs e)
{
int startIndex = currentIndex;
AtomFeedReader reader = null;
AtomFeedItem item = null;
while (item == null)
{
reader = readers[currentIndex];
item = reader.DequeueTopItem();
currentIndex++;
if (currentIndex == startIndex)
{
break;
}
}
if (item != null)
{
lastTimeOfFeedItem = DateTime.Now;
}
if ((item != null) && (FeedItemAvailable != null))
{
FeedItemAvailable(this, new AtomFeedItemAvailableEventArgs()
{
SearchTerm = reader.SearchTerm,
FeedItem = item
});
}
else if ((item == null) && (FeedItemUnavailable != null))
{
FeedItemUnavailable(this, new AtomFeedItemUnavailableEventArgs()
{
UnavailableTime = DateTime.Now - lastTimeOfFeedItem
});
}
}
List<AtomFeedReader> readers;
SearchTermsConfiguration searchTerms;
Timer timer;
ModuloInt currentIndex;
DateTime lastTimeOfFeedItem;
}
You can see that it sets up a set of AtomFeedReader based on the SearchTermsConfiguration that’s passed to it and then it ticks on a timer. On each timer tick, it goes to each AtomFeedReader and calls DequeueTopItem. It does this until any reader produces an item or until we’ve exhausted the set.
Then we fire one of two events – FeedItemAvailable or FeedItemUnavailable to notify any interested parties that we either found something interesting or we didn’t. Those events either carry the details of the new AtomFeedItem that’s become available or they carry a TimeSpan which says how long it is since we last saw a new AtomFeedItem from anywhere.
So…putting all these classes together into a library that I’ll call TwitterAtomFeedLibrary I can go and write code against them like;
class Program
{
static void WriteColour(ConsoleColor color,
string formatString, params object[] args)
{
Console.ForegroundColor = color;
Console.WriteLine(formatString, args);
Console.ResetColor();
}
static void Main(string[] args)
{
// Load my configured search terms
SearchTermsConfiguration config = SearchTermsConfiguration.LoadFromConfig();
// Set up my feed manager on a 5 second interval
AtomFeedManager feedManager = new AtomFeedManager(config, new TimeSpan(0, 0, 5));
// Handle events
feedManager.FeedItemAvailable += (s, e) =>
{
WriteColour(ConsoleColor.Yellow, "Feed item available for search term {0}",
e.SearchTerm.Term);
WriteColour(ConsoleColor.Green, "\tAuthor:{0}", e.FeedItem.Author);
WriteColour(ConsoleColor.Green, "\tAuthor Image:{0}", e.FeedItem.AuthorImageUri);
WriteColour(ConsoleColor.Green, "\tTitle: [{0}]", e.FeedItem.Title);
WriteColour(ConsoleColor.Green, "\tPublished Date: {0}", e.FeedItem.PublishedDate.ToShortDateString());
};
feedManager.FeedItemUnavailable += (s, e) =>
{
WriteColour(ConsoleColor.Cyan, "No feed items available for {0} seconds",
e.UnavailableTime.TotalSeconds);
};
// Sit back and relax
Console.ReadLine();
}
}
and it throws out content like this from the Console;
Now that I’ve got this AtomFeedManager class and its friends like SearchTerm I can go and put a WPF UI onto them which is where I’ll head in the next post.
Meanwhile, the sourcecode from this post (with the little test program code above) is all here for download.
Mike Taulty 