Linux Today: Linux News On Internet Time.

Linux Gazette: Downloading LinuxToday links and Linux Gazette's TOC with Python (and Perl)

Feb 17, 2001, 19:38 (0 Talkback[s])
(Other stories by Mark Nielsen)

"I wanted to add Linux Today's links to my website GNUJobs.com, just for the fun of it. Later, I want to add more headlines from other websites, and perhaps LG's latest edition. I had a choice of Perl or Python. I choose Python because I have been using it for quite a while for a mathematical project, and it has proven quite useful. I want to make a habit of using Python now. It tends to be easier for me to program in Python than Perl. Also, in the future, I wish to use threading to download many webpages at the same time, which Python does very well. I might as well do it in Python now since I know I will use it later."

"Both Perl and Python will let you download webpages off of the internet. You can do more than just download webpages, such as ftp, gopher, and connect to other services. Downloading a webpage is just one thing these languages can do.

"There are several things the programming language has to do:

  • Download the webpage
  • Parse the data correctly to reformat the data
  • Reformat the data
  • Replace the old file with the new file only if it contains valid data
This article isn't going to be too long. I commented the Python code.

Complete Story

Related Stories: