Net Grabbing and Mirroring Tools

By Jeff Alami

Everyone’s experienced it before. You’re downloading an
arbitrary 60 MB file via FTP. For purposes of illustration, let’s
call this file StarOffice. Netscape has 58 MB, and then–bam!–a
timeout cuts off your connection with no chance of restarting. It’s
not a good feeling. Downloading an entire directory structure isn’t
a piece of cake either.

I was having quite a time trying to download the ISO image for
Red Hat 6.0 until I found out about these great Internet
downloading and mirroring tools.


GREED (Get and Resume Elite EDition) is the Linux equivalent of
GetRight on the Windows platform. GREED is probably the best way to
get a single file from an HTTP or FTP source. It offers several
features, including connection retries in the case of a busy FTP
servers, download resumes, proxy support. An X11-based graphical
frontend is in the works.

GREED home


GREED 0.8 source

Mirror [Open Source]

Mirror is a tool written in Perl which will fetch an entire
directory hierarchy through the FTP protocol and effectively mirror
it on the client machine. It also checks the size and timestamps on
files already downloaded to avoid downloading files unnecessarily
when doing a mirror update.

Mirror home
Mirror 2.9

Wget [GPL]

GNU Wget is a free download utility that lets the user get files
through HTTP or FTP protocols. Wget is especially useful for making
mirrors of FTP sites and Web sites through recursive retrieval. It
works in the background, enabling it to be used without being
logged in. Wget is especially well suited for cron jobs and other
non-interactive tasks.

Wget home


Download wget

(C) 1999 Zeta Publications. (Reproduced from Bleeding Edge Magazine)

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends, & analysis