Showing posts with label RSS. Show all posts
Showing posts with label RSS. Show all posts

Friday, August 21, 2009

Pimpin' the blog part #3

Today we'll look at how I've build my own RSS feed in ASP.NET 3.5 and using Linq2SQL.

Before we dive into that, I would like to announce that my articles are now fed trough this RSS feed to CodeProject.com. The latest of my articles (at least most of them) are now available there too. I would like to thank Shawn Ewington from the CodeProject for working with me an supporting me, so I could achieve this bigger audience.

Now, back to business. As you may remember from part #1 of this series, the first goal for me was to customize my RSS feed, so I could add categories to my feed without messing up my navigation. I also lined out the fases I would go trough to get to my new and improved blog and after leaving you in part #2 I reached step three, wich means I now have my content up and running in a database on the new platform and it gets updated trough RSS from the Blogger.com platform.

I've now used the code I wrote for part #2 as a starting point for building the new RSS feed. As I had a great experience with the SyndicationFeed class for parsing a feed, I figured I might as well use it to publish a feed as well. The general steps needed to get the feed published are these:
  1. Setup a SyndicationFeed object with some general information like title, author, etc.
  2. Load the articles from the database and convert them into feed items
  3. Publish the feed
The first step actually did take some work as I wanted a lot of things to be configurable trough my web.config and also it takes some extra code to set most of the properties. To help me out with loading these settings I've build a SyndicationSettings class which holds the constants for the config key names and accesses the ConfigurationManager to get the actual values. I've made it a static class, so it is as easy to use as possible.

Setting up the feed is done in the SetupFeed method:
private static SyndicationFeed SetupFeed()
{
SyndicationFeed feed = new SyndicationFeed();
SyndicationPerson person = new SyndicationPerson(SyndicationSettings.SyndicationPersonEmail,
SyndicationSettings.SyndicationPersonName, SyndicationSettings.SyndicationPersonUrl);
feed.Authors.Add(person);
feed.Contributors.Add(person);

feed.Categories.Add(new SyndicationCategory("CodeProject"));

feed.Copyright = new TextSyndicationContent(SyndicationSettings.Copyright,
TextSyndicationContentKind.Plaintext);
feed.Description = new TextSyndicationContent(SyndicationSettings.Description);
feed.Generator = GeneratorName;
feed.Id = SyndicationSettings.FeedId;
feed.Links.Add(new SyndicationLink(new Uri(SyndicationSettings.HomePageLink)));
feed.Title = new TextSyndicationContent(SyndicationSettings.FeedTitle);
return feed;
}

As you can see I've created a SyndicationPerson object to be reused both as the author and a contributor.

The next step is to load the articles from the database and convert them into SyndicationItem instances. First I needed to get them from the database, for which I expanded the functionality of the StorageConnection class I wrote in part #2. The basics are pretty straight forward, but as I was still fairly new to Linq2Sql I struggled a bit with getting the categories to load with the data automatically. Here is the GetArticles method from the StorageConnection:
public static List<Article> GetArticles(bool loadCategories, int maxNumberOfArticles)
{
List<Article> articles = new List<Article>();

DataLoadOptions dataLoadOptions = new DataLoadOptions();
if (loadCategories)
{
dataLoadOptions.LoadWith<Article>(article => article.ArticleCategories);
dataLoadOptions.LoadWith<ArticleCategory>(articleCategory => articleCategory.Category);
}
using (Developers42_DataClassesDataContext context = new Developers42_DataClassesDataContext())
{
context.LoadOptions = dataLoadOptions;
articles = context.Articles.OrderByDescending(article => article.PublicationDate).Take(
maxNumberOfArticles).ToList();
}
return articles;
}

I've used the LoadOptions property to let the Linq2Sql framework know I want to include both the ArticleCategory and the Article entities while loading the Articles. Further more I used the OrderByDescending method to specify that I want the Articles on PublicationDate and I used the Take method to specify that I only want to load up to 25 articles at a time.
Converting the Article objects into SyndicationItem objects is as simple as calling the constructor including most of the properties and then adding the categories.

As the final step all that needs to be done is to publish the result to the client. To do this, we need to first clear the existing response buffer, to make sure we don't send anything already in the template. Next we can simply create and XmlWriter that has the response stream as the underlying stream and call the SaveToRss20 method on the SyndicationFeed with the XmlWriter as a parameter. This is how the code looks:

XmlWriterSettings settings = new XmlWriterSettings();
settings.Encoding = Encoding.UTF8;
XmlWriter writer = XmlWriter.Create(Page.Response.OutputStream, settings);

Page.Response.ClearContent();
feed.SaveAsRss20(writer);
writer.Flush();

As you may have noticed, building your own RSS feed is easy when using the SyndicationFeed class. To help you even further I've included a link to the source here. It also includes the code from part #2 for parsing an RSS feed.

I hope this post was useful to you. Next time we'll look at parts of the new version of the Developers 42 blog.

Thursday, August 13, 2009

Pimpin' the blog part #2

In this episode of Pimpin' the blog we are going to have a look at how I will sync my data between Blogger.com and my new hosting platform. This involves a not well known feature from .NET in relation to RSS and using Linq2SQL. Eventually we'll end up with a tool that reads by Blogger RSS Feed and stores it in a SQL Server 2005 Database (or any other compatible database for that matter).

Thoughts on synchronization
As I laid out in part #1 I'm far from ready to give up my Blogger account as I still have many things to replace before I can do so. However I don't feel much for keeping two stores for the same information synchronized by hand. I have better things to do with my time than that (altough not that much better :-) ).
The first thing that came to mind was actually RSS as it keeps all the blog aggregate sites up to date as well, so why not use that? Besides, the final trigger to do all this was to customize my RSS feed in the first place, so why not use it as a source?
As a good developer I also went to see if I had any alternatives. I could opt for a HTTP/HTML spider, but it would be awkward, messy and complex. I could try and automate the export process for Blogger blogs, but again, awkward, messy and complex.

Loading a feed
So RSS it is then. The entire process is relatively simple:
  1. Get the feeds xml content
  2. Parse the feed into articles, etc.
  3. Store the articles and related data in the database
The first step is easy. Just take an HttpWebRequest and point it at the feed. Here is the code:

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(FeedUrl);
if (UseProxy)
{
request.Proxy = newWebProxy(ProxyUrl);
}

HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();

So as you can see. it first sets up the HttpWebRequest, so it can get to the RSS Feed (using a proxy if needed). Then it just gets the response stream which then contains the XML for the RSS Feed.

The next step got me thinking. The first solution that popped into my mind was to use Linq2Xml. However that would involve a lot of code to get to all the different parts of information I needed. I googled around, read some blogs, until I ran into someone mentioning the SyndicationFeed object that's new in .NET Framework 3.5. I figured I would give that a try to see how it works and I could always go back to parsing the feed myself.

Here is the code to actually load the response stream into a SyndicationFeed instance:

XmlReader reader = XmlReader.Create(responseStream);

Feed = SyndicationFeed.Load(reader);

Wow, that was easy, now wasn't it? Keep in mind however that you do need to add a reference to both System.ServiceModel and System.ServiceModel.Web to make this work.
What I ended up with is a class that would handle loading the feed into a SyndicationFeed object that handled everything I needed in under fifty lines of code!

So that tackled step two of the process.
All that's left is to store it into the database. As I mentioned earlier, I chose to use Linq2Sql to handle this for me. Why? I have had extensive experience with Entity Framework and I do think that for large solutions it can be a good choice, however it does take a lot of effort to make it do what you want, which is not what I needed here.
I read up on Microsofts strategy on data access and why both Entity Framework and Linq2Sql are pushed and found out that Linq2Sql is actually meant to support RAD on smaller projects, or at least for smaller data access layers. As my data model only consists of three tables, I guess my project would qualify as small.



I'm not going to bother you with the details on how I stored my articles trough Linq2Sql and just go ahead and post a link to the code below.
The main program to control this is more interesting:

string feedUrl = ConfigurationManager.AppSettings[FeedUrlConfigKey];
RssFeedReader reader = newRssFeedReader(feedUrl);
reader.UseProxy = UseProxyTrueValue.Equals(ConfigurationManager.AppSettings[UseProxyConfigKey],
StringComparison.OrdinalIgnoreCase);
if (reader.UseProxy)
{
reader.ProxyUrl = ConfigurationManager.AppSettings[ProxyUrlConfigKey];
}
reader.ReadFeed();

foreach (SyndicationItem feedItem in reader.Feed.Items)
{
List<string> categories = newList<string>();
foreach (SyndicationCategory category in feedItem.Categories)
{
categories.Add(category.Name);
}
StorageConnection.AddArticle(feedItem.Title.Text, feedItem.Summary.Text, feedItem.PublishDate.Date,
feedItem.Id, categories.ToArray());
}


First I set up my RssFeedReader instance and call ReadFead on it. This results in a SyndicationFeed on which I iterate trough the Items collection. Then I get the categories and feed them into my StorageConnection class which makes sure everything is properly stored in the database. The StorageConnection class makes sure nothing is duplicated even if the same article is added more then once.

Here is the source:


In part #3 of this series, we'll look into building a new RSS feed with some customizations, based on the data we've retrieved today.

Sunday, August 9, 2009

Pimpin' the blog part #1

In this article I will describe to you why I want to change my blog, what I'm trying to achieve, how I plan to do that and whats in it all for you.

Why?
After writing only a few articles in Blogger and reading up on how to build a usable blog, I knew that at some point I would have to replace Blogger with something custom build. As time passed and I tried to do more and more things with my blog, I ran into more and more things that I couldn't do as I wanted to do them.

Recently, as some of you may have noticed, I got a comment from Sean Ewington, who is the Chief Technical Editor for The Code Project. He invited me to publish my blog on CodeProject.com (thanks, Sean). To do this I would simply add my blog's RSS feed to their system and add a category to my articles, stating that they should be available on CodeProject as well. Unfortunately that last step is not possible for me, as that would mean I would now have a CodeProject category on my blog, which is not what I want.

This latest thing on the list of things you can't do with Blogger, triggered me to finaly get to work on this and plan some new code.

What?
Some things I want to be able to do after the complete transformation has happened:
  • Customize my RSS feed
  • Easier to customize styling
  • Have more control on advertising (individual posts have specific ads, etc.)
  • Use Silverlight for at least some part of my blog
  • Have a better layout with three columns
  • Have an easier to maintain and easier to use navigation
How?
To get this all done requires some planning. It won't happen all in one go, obviously. First thing I should state is that I had a windows hosting account lying around for a while now and I will be leveraging that account for my blog 2.0.
Here are the steps I plan to follow for getting up and running with a new platform.
  1. Get a database up and running to store content
  2. Get the content on the new platform to sync with the content on Blogger
  3. Build a new RSS feed and publish it trough my feedburner account (so if you have a subscription, you won't notice the change).
  4. Design a new layout for my blog
  5. Implement the new layout
  6. Test the two blogs side by side
After that I'm not sure. I might want to redirect all traffic coming to http://jvdveen.blogger.com/ to my new site, but I'm not sure what will happen with search engines and their spiders and I'm not sure if I'm even allowed to do this. However I'm still happy with the Blogger editor and I'm not sure if I want to build a completely new editor so I can then kill my Blogger account. This is something I'll be thinking about later on. If you have any suggestions, please let me know.

Why should you care?
Well, you don't have to care, however... as the title of this post suggests, I will be publishing about the whole process of creating my new blogging platform and then moving in. Not only might you learn something, I will also make available handy code in the process, that you might use yourself. So stick around, learn and benefit.