Linux Today: Linux News On Internet Time.

More on LinuxToday

Net Grabbing and Mirroring Tools

May 09, 1999, 18:31 (7 Talkback[s])
(Other stories by Jeff Alami)

WEBINAR: On-Demand

No-Size-Fits-All! An Application-Down Approach for Your Cloud Transformation REGISTER >

By Jeff Alami

Everyone's experienced it before. You're downloading an arbitrary 60 MB file via FTP. For purposes of illustration, let's call this file StarOffice. Netscape has 58 MB, and then--bam!--a timeout cuts off your connection with no chance of restarting. It's not a good feeling. Downloading an entire directory structure isn't a piece of cake either.

I was having quite a time trying to download the ISO image for Red Hat 6.0 until I found out about these great Internet downloading and mirroring tools.


GREED (Get and Resume Elite EDition) is the Linux equivalent of GetRight on the Windows platform. GREED is probably the best way to get a single file from an HTTP or FTP source. It offers several features, including connection retries in the case of a busy FTP servers, download resumes, proxy support. An X11-based graphical frontend is in the works.

GREED home page
GREED example
Download GREED 0.8 source

Mirror [Open Source]

Mirror is a tool written in Perl which will fetch an entire directory hierarchy through the FTP protocol and effectively mirror it on the client machine. It also checks the size and timestamps on files already downloaded to avoid downloading files unnecessarily when doing a mirror update.

Mirror home page Download Mirror 2.9

Wget [GPL]

GNU Wget is a free download utility that lets the user get files through HTTP or FTP protocols. Wget is especially useful for making mirrors of FTP sites and Web sites through recursive retrieval. It works in the background, enabling it to be used without being logged in. Wget is especially well suited for cron jobs and other non-interactive tasks.

Wget home page
Wget manual
Download wget (FTP)

Copyright (C) 1999 Zeta Publications. (Reproduced from Bleeding Edge Magazine)