General Hacks Ideas Web Development

RSS Feed of The Latest Items from the Psychology Today Home Page

I just used the Feed43 web site to create an RSS of all the items that appear in the “The Latest” section on the home page of the Psychology Today web site. Theoretically, this should contain all the latest content from across the Psychology Today web site. If you just want to subscribe to it, here it is:
http://www.feed43.com/5658507871028441.xml

Why Did I Do This?

I really enjoy a lot of the blog posts and articles at Psychology Today. There must be many thousands of articles in 776 individual blogs organized into 45 topics (as of today, assuming my counting is correct) and several dozen new ones are written each week.

The Psychology Today home page aggregates all the articles across all the topics and blogs and displays the 30 most recent items from across the site in the “The Latest” section.

I wanted to be able to easily stay updated on new content across the site. I could have subscribed to all 45 Topic feeds in Google Reader, but I really didn’t want to add 45 new feeds; I only wanted to add one single feed.

My Preliminary Research

First I searched around on the Psychology Today web site for an RSS feed like I wanted, but I couldn’t find one. Then I turned to Google to see if I could find one mentioned there, and I didn’t find anything. So I then turned to investigating the Psychology Today web site more closely. Here is what I discovered.

Each topic has an RSS feed for it, and each blog has an RSS feed for it as well. For example, you can view the most recent articles by the writers in the Procrastination topic at the Procrastination topic page at
http://www.psychologytoday.com/topics/procrastination/,
and access the Procrastination topic RSS feed at
http://www.psychologytoday.com/topics/procrastination/feed.
One of the blogs in the Procrastination topic is “Don’t Delay” by Timothy A. Pachyl. You can view the most recent posts on this blog at
http://www.psychologytoday.com/blog/dont-delay,
and access the Don’t Delay RSS feed at
http://www.psychologytoday.com/blog/dont-delay/feed.

If you look a those URLs, you’ll notice they use a very simple naming convention for the URL paths for the the topics and blogs. The RSS feed can then be accessed by simply appending /feed onto the end of the topic or blog URL.

Unfortunately, you’ll get a 404 error page if you just append /feed onto the root URL like so:
http://www.psychologytoday.com/feed. There also doesn’t seem to be a single RSS feed that aggregates all the content from the entire site.

Rolling My Own Psychology Today RSS Feed

At this point, I determined that if I wanted an RSS feed of all the items from the Psychology Today home page then I was going to have to do it myself. i thought about writing my own PHP script to periodically parse out their home page, but I decided that before I did this I would see if there was a good service that could do this more easily.

A quick search uncovered a few services that will create an RSS feed from the content on any site. I experimented with a few of these and none of them worked to my satisfaction. Then I discovered Feed43. This site let me essentially write a script that parses through the Psychology Today home page, extracts the content in the “The Latest” section, then I could mark up how that extracted content would get rendered in an RSS feed. In just a few minutes I had a working RSS feed of the Latest Items from the Psychology Today Home Page. Here is the results:
http://www.feed43.com/5658507871028441.xml.

Warnings and Limitations

I really do like the Feed43 service. One thing that you’ll need to keep in mind though is that they only scrape the home page and rebuild the feed once every six hours (they will scrape it once an hour for $29/year) but since this is currently the only site I’m using this for I can’t justify that expense. Another thing you’ll need to remember is that if they redesign their web site and the code on their home page changes then this feed may no longer work. As long as I’m interested in aggregating their content in this way, I’ll try to maintain this feed and keep it working, but I’m not making any promises.

Final Words

So, if you’re a psychology geek like I’ve become in the the past couple years, enjoy your new Psychology Today Home Page RSS Feed! If you find this useful, let me know in the comments. As for me, I’ve got some Sunday afternoon reading to do…

Related Posts

5 Comments

  • Reply
    John
    4/30/2012 at 5:01 am

    Thanks for the interesting idea. I’ll give it a try later 😀

  • Reply
    rickyspears
    4/30/2012 at 1:27 pm

    John – If you have a web site you want to monitor but they don’t offer an RSS feed, Feed43 is great!

  • Reply
    Joanna
    10/26/2012 at 2:50 pm

    exactly what I was looking for, thanks!

  • Reply
    Ken
    5/27/2013 at 8:36 am

    Ricky, thank you for sharing your RSS feed.

  • Reply
    liz
    9/5/2013 at 9:11 pm

    really useful! thanks a lot!

  • Leave a Reply