When you want to reuse content that you find online, you have a few options:
- Copy the data by hand
- Writing a screen-scraping tool to automate
- Existing software tools
There are many different ways to transform the data once it is downloaded, and we have written a lot of screen-scraping tools over the years. Whether I was downloading content about viruses, or migrating content from an old copy of a website it seems to me like everyone could use screen-scraping in some ways.
Being able to syndicate information from RSS feeds is a much simpler process than building browser automation tools for the same task. It really depends on whether the site that you are looking at allows syndication. Sometimes we need to use a premium wordpress plugin to adjust our syndication schedules.
There is a lot of complexity to this process, so it is necessary to define the requirements up front and then we build a project plan and start the process.