Linux Gazette: Downloading LinuxToday links and Linux Gazette’s TOC with Python (and Perl)

“I wanted to add Linux Today’s links to my website GNUJobs.com,
just for the fun of it. Later, I want to add more headlines from
other websites, and perhaps LG’s latest edition. I had a choice of
Perl or Python. I choose Python because I have been using it
for quite a while for a mathematical project, and it has proven
quite useful. I want to make a habit of using Python now. It tends
to be easier for me to program in Python than Perl. Also, in the
future, I wish to use threading to download many webpages at the
same time, which Python does very well. I might as well do it
in Python now since I know I will use it later.”

“Both Perl and Python will let you download webpages off of the
internet. You can do more than just download webpages, such as ftp,
gopher, and connect to other services. Downloading a webpage is
just one thing these languages can do.

“There are several things the programming language has to
do:

• Download the webpage
• Parse the data correctly to reformat the data
• Reformat the data
• Replace the old file with the new file only if it contains
  valid data

This article isn’t going to be too long. I commented the Python
code.

Complete
Story