Net Grabbing and Mirroring ToolsMay 09, 1999, 18:31 (7 Talkback[s])
(Other stories by Jeff Alami)
How You Can Improve Your Machine Learning with Spark REGISTER >
By Jeff Alami
Everyone's experienced it before. You're downloading an arbitrary 60 MB file via FTP. For purposes of illustration, let's call this file StarOffice. Netscape has 58 MB, and then--bam!--a timeout cuts off your connection with no chance of restarting. It's not a good feeling. Downloading an entire directory structure isn't a piece of cake either.
I was having quite a time trying to download the ISO image for Red Hat 6.0 until I found out about these great Internet downloading and mirroring tools.
GREED (Get and Resume Elite EDition) is the Linux equivalent of GetRight on the Windows platform. GREED is probably the best way to get a single file from an HTTP or FTP source. It offers several features, including connection retries in the case of a busy FTP servers, download resumes, proxy support. An X11-based graphical frontend is in the works.
Mirror [Open Source]
Mirror is a tool written in Perl which will fetch an entire directory hierarchy through the FTP protocol and effectively mirror it on the client machine. It also checks the size and timestamps on files already downloaded to avoid downloading files unnecessarily when doing a mirror update.
GNU Wget is a free download utility that lets the user get files through HTTP or FTP protocols. Wget is especially useful for making mirrors of FTP sites and Web sites through recursive retrieval. It works in the background, enabling it to be used without being logged in. Wget is especially well suited for cron jobs and other non-interactive tasks.
0 Talkback[s] (click to add your comment)