Hawler, the Ruby crawler, 0.3 released

I received an email yesterday from ET LoWNOISE, a Metasploit contributor, regarding adding proxy support to Hawler. Apparently the hope is to be able utilize Hawler for the crawling duties within WMAP, the new web application scanning framework in Metasploit.

Since it has been several months since I've had to do anything to Hawler, I figured this was a good time to go in an do some much needed cleanup and improvements. Chief among the changes are:

• Proxy support ("-P [IP:PORT]")
• Documentation cleanup
• Support crawling frame and form tags
• Add a useful default banner to calling scripts if none provided
• Print out defaults when help is called

Thanks to ET for his proxy contributions.

As usual, the following will get you up and running with Hawler:

gem install --source http://spoofed.org/files/hawler/ hawler

Using Hawler? Comments? Complaints? Suggestions? Drop me a line - I'd like to hear it.