Linux Today: Linux News On Internet Time.
Search Linux Today
Linux News Sections:  Developer -  High Performance -  Infrastructure -  IT Management -  Security -  Storage -
Linux Today Navigation
LT Home
Contribute
Contribute
Link to Us
Linux Jobs


More on LinuxToday


Net Grabbing and Mirroring Tools

May 09, 1999, 18:31 (7 Talkback[s])
(Other stories by Jeff Alami)

By Jeff Alami

Everyone's experienced it before. You're downloading an arbitrary 60 MB file via FTP. For purposes of illustration, let's call this file StarOffice. Netscape has 58 MB, and then--bam!--a timeout cuts off your connection with no chance of restarting. It's not a good feeling. Downloading an entire directory structure isn't a piece of cake either.

I was having quite a time trying to download the ISO image for Red Hat 6.0 until I found out about these great Internet downloading and mirroring tools.

GREED [GPL]

GREED (Get and Resume Elite EDition) is the Linux equivalent of GetRight on the Windows platform. GREED is probably the best way to get a single file from an HTTP or FTP source. It offers several features, including connection retries in the case of a busy FTP servers, download resumes, proxy support. An X11-based graphical frontend is in the works.

GREED home page
GREED example
Download GREED 0.8 source

Mirror [Open Source]

Mirror is a tool written in Perl which will fetch an entire directory hierarchy through the FTP protocol and effectively mirror it on the client machine. It also checks the size and timestamps on files already downloaded to avoid downloading files unnecessarily when doing a mirror update.

Mirror home page Download Mirror 2.9

Wget [GPL]

GNU Wget is a free download utility that lets the user get files through HTTP or FTP protocols. Wget is especially useful for making mirrors of FTP sites and Web sites through recursive retrieval. It works in the background, enabling it to be used without being logged in. Wget is especially well suited for cron jobs and other non-interactive tasks.

Wget home page
Wget manual
Download wget (FTP)

Copyright (C) 1999 Zeta Publications. (Reproduced from Bleeding Edge Magazine)