ScanGraphics

Download as EXE files (1.7Mb zip file, runs on Windows 98 series)

Download as Perl scripts (5Kb zip file, requires perl, runs on any OS)

ScanGraphics is a suite of programs which allows you to 'spider' a website, loading each HTML page and searching for links. It will follow all links on the site, and return a list of all graphics files and external links found.

The program has been re-written in perl, which provides many advantages over the old visual-basic program

• Much more accurate; can follow ../ links and such like.
• Follows the robot-exclusion protcol
• Does not download more frequently than 10 seconds from the same site, to avoid flooding the site and getting your IP blacklisted
• Uses content/type headers to distinguish between different file types, rather than just file-extensions
• Ability to pause and resume a scan
• Ability to use partial results while the scan is progressing

The program produces a number of useful outputs:

• A list of pages searched
• A list of pages yet to search. (This means you can use the 'continue' option to resume a search)
• A list of external links found
• A list of graphics found

A download script is included, which looks at the list of graphics, and downloads each of them into a /downloads/ directory. This program can be resumed at any point, and can check datestamps to avoid downloading the same file more than once.

A conversion script is included, which converts the list of graphics into an HTML file which references those graphics. Load the page in Mozilla (or your fave browser), and it will download and display all the graphics. You can then save the page with graphics included.

Using the Windows EXE files

I have used perl2exe to compile the programs into executable files: you can now run them directly in windows without having to download perl!

Perl2Exe inserts it's own delay and advertising message each time you run the program: this is a feature of the free version. If you'd like this message removed, send me the $50 or to buy Perl2Exe, and I'll buy the full version and compile with that!

Using the perl version

To download just the perl scripts (5K download), click here.
If you have Linux, you probably already have a copy of perl.
If you have Windows, You will need a copy of activeperl, available to download for free
If you have one of the other 80 operating systems supported by perl, have a look on perl.com for the download
If you have an operating system not mentioned above simply compile perl from source. You will need a C compiler.

Infrequently asked questions

Link

Alternative programs

For downloading web-pages recursively, you might also try GNU WGet, with the --recursive --span-hosts options.

GNU WGet is included as standard on most GNU/Linux distributions, and has even been ported to Windows.