Why Did I Do This?

I really enjoy a lot of the blog posts and articles at Psychology Today. There must be many thousands of articles in 776 individual blogs organized into 45 topics (as of today, assuming my counting is correct) and several dozen new ones are written each week.

The Psychology Today home page aggregates all the articles across all the topics and blogs and displays the 30 most recent items from across the site in the “The Latest” section.

I wanted to be able to easily stay updated on new content across the site. I could have subscribed to all 45 Topic feeds in Google Reader, but I really didn’t want to add 45 new feeds; I only wanted to add one single feed.

My Preliminary Research

First I searched around on the Psychology Today web site for an RSS feed like I wanted, but I couldn’t find one. Then I turned to Google to see if I could find one mentioned there, and I didn’t find anything. So I then turned to investigating the Psychology Today web site more closely. Here is what I discovered.

Each topic has an RSS feed for it, and each blog has an RSS feed for it as well. For example, you can view the most recent articles by the writers in the Procrastination topic at the Procrastination topic page at
http://www.psychologytoday.com/topics/procrastination/,
and access the Procrastination topic RSS feed at
http://www.psychologytoday.com/topics/procrastination/feed.
One of the blogs in the Procrastination topic is “Don’t Delay” by Timothy A. Pachyl. You can view the most recent posts on this blog at
http://www.psychologytoday.com/blog/dont-delay,
and access the Don’t Delay RSS feed at
http://www.psychologytoday.com/blog/dont-delay/feed.

If you look a those URLs, you’ll notice they use a very simple naming convention for the URL paths for the the topics and blogs. The RSS feed can then be accessed by simply appending /feed onto the end of the topic or blog URL.

Unfortunately, you’ll get a 404 error page if you just append /feed onto the root URL like so:
http://www.psychologytoday.com/feed. There also doesn’t seem to be a single RSS feed that aggregates all the content from the entire site.

Rolling My Own Psychology Today RSS Feed

At this point, I determined that if I wanted an RSS feed of all the items from the Psychology Today home page then I was going to have to do it myself. i thought about writing my own PHP script to periodically parse out their home page, but I decided that before I did this I would see if there was a good service that could do this more easily.

A quick search uncovered a few services that will create an RSS feed from the content on any site. I experimented with a few of these and none of them worked to my satisfaction. Then I discovered Feed43. This site let me essentially write a script that parses through the Psychology Today home page, extracts the content in the “The Latest” section, then I could mark up how that extracted content would get rendered in an RSS feed. In just a few minutes I had a working RSS feed of the Latest Items from the Psychology Today Home Page. Here is the results:
http://www.feed43.com/5658507871028441.xml.

Warnings and Limitations

I really do like the Feed43 service. One thing that you’ll need to keep in mind though is that they only scrape the home page and rebuild the feed once every six hours (they will scrape it once an hour for $29/year) but since this is currently the only site I’m using this for I can’t justify that expense. Another thing you’ll need to remember is that if they redesign their web site and the code on their home page changes then this feed may no longer work. As long as I’m interested in aggregating their content in this way, I’ll try to maintain this feed and keep it working, but I’m not making any promises.

Final Words

So, if you’re a psychology geek like I’ve become in the the past couple years, enjoy your new Psychology Today Home Page RSS Feed! If you find this useful, let me know in the comments. As for me, I’ve got some Sunday afternoon reading to do…

Writing this was much easier than I initially expected. I looked at the code for a couple other plugins that added something before or after the post, and reviewed the WordPress Plugin API. I was up and running in no time.

I had initially hard-coded a button last weekend when I made the site live. That was certainly more challenging than using the APIs and doing it right. If you have a WordPress blog, know a little bit about PHP (or are willing to learn), and want to make your blog do something new and interesting, then I strongly encourage you to spend a little time investigating how to create your own WordPress plugins. You’ll have a lot of fun, learn a lot, and contribute to your neighbors on the web as well.

Find out more about the “Zap Read This Post” plugin for WordPress.