Re: Wget

[Fraser Farrell <fraser@trilobytes.com.au>]

Thomas Droege wrote:
> I have obviously done something wrong.   I have read all the wget man page
> and note that it even give examples contrary to linux practice, but I have
> no clue as to what to do next.


Tom,

Have you tried:

  wget --mirror --convert-links --html-extension  http://whatever...   ?

I use this to get site snapshots. It drops everything into a subfolder named 
"whatever", converts all the links (after all is downloaded) so that they 
make sense on your computer, and turns any CGI / ASP / PHP etc pages into 
rendered HTML equivalents. Note that it doesn't get the actual scripts (and 
associated backend code) unless you (a) run wget in FTP mode and (b) the 
webmaster of the site is careless. Nor does it get the banner ads :-)

If you want to be polite, also use the --limit-rate or --wait options. Many 
websites impose controls on file leeches to give everyone else a fair go.


cheers,

-- 

Fraser Farrell

----------------------------------
http://astronomy.trilobytes.com.au
----------------------------------