“Sometimes, when developing a web application, you want to
acquire data from another source to put on your site. For example,
portal sites like portaloo place the latest news headlines on their
pages, and the news headlines are constantly updated without human
intervention.”
“I ran into a similar problem designing the web site and
database system of Memphis Scholastic Chess. They were spending 5-6
hours per week manually locating the latest ratings for 900 players
(having to navigate through around 75 individual pages) and then
skim the list for players in our organization on the United States
Chess Federation web site. I used Python to write a server site
program that runs automatically once a week and downloads and
parses pages from the USCF’s web site.“