Name-based Virtual Hosting and Web Application Security

Over the past several weeks I've had the privilege of beginning evaluations of the best that the commercial security sector has to offer in the realm of web application security auditing. Arguments about whether simply buying a web application security firewall or pursuing this initiative from a code auditing point of view instead aside, the offerings from the top names in this space run the gamut from extremely impressive to downright depressing.

For purposes of this post, I am just focusing on the tools out there that do your traditional blackbox based approach to auditing. Given an IP address or a URL and some other minimal hand-holding, nearly all of the big names in this arena will do fairly good job of identifying the bulk of the "low hanging fruit."

Being the studious and meticulous geek that I am, one of my first requirements when evaluating these products was that they properly support the auditing of web applications that live on a host utilizing named-based virtual hosting. Simple, right? RFC2616 makes this sort of thing very clear and one would expect that not only would tools currently available be able to employ this knowledge to further leverage their way into a web application, but when asked about these features a given vendor would understand how they are useful. Needless to say, the HTTP/1.1 "Host" header plays a pretty important role on the majority of modern websites.

Well, I hate to break it to you, folks, but this is sadly not the case. Of all the vendors on the market, not a single one fully supports this functionality. Yes, you read that right -- no vendor in the blackbox web application security auditing space truly supports HTTP/1.1's "Host" header.

Pausing here for a second, I can sense the heated emails and comments aimed in my general direction. Please, hear me out.

So, just a hint of background. Many products utilize readily available HTTP client libraries written in their language of choice; rightfully so -- why reinvent the wheel! At least one even manipulates IE or Firefox directly to do their bidding, which I find particularly interesting.

And this is where the problem starts to manifest itself. When a human being makes a web request using a browser, they typically enter the address either using an IP address or a host name of some manner. Under the hood, as many of us know, the host name (if present) is resolved and the IP address that results is used as the destination address of the network socket. Once this connection is established using a modern browser, the host name (and port!) from the original URL is sent along as part of the request in the "Host" header of an HTTP/1.1 request. Simple, right? This allows a number of 1999-esque features such as being able to host more than one website on a single IP address or do fun SEO things like determining what URL a visitor originally used to arrive on your property.

Whats the big problem, right? Of course, all vendors that I've dealt with properly support sending HTTP/1.1 requests with a "Host" header, but they don't exploit all of its beauties. Imagine the following two situations.

One. Imagine two hosts: 192.168.0.1 (PRODUCTION) and 192.168.0.2 (TEST). Both PRODUCTION and TEST webservers contain name-based virtual hosts for www.example.com and seoftw.example.com. DNS for www.example.com and seoftw.example.com point to 192.168.0.1. How do you audit TEST accurately and completely? The obvious and hacky answer that most vendors gave was to diddle DNS before each audit so that the host performing the audit can correctly resolve the target host names to the IP addresses of the particular environment (PRODUCTION or TEST) that you are attempting to audit. If that was your answer, please hold while I drop the >$200k some of these tools cost for a single user into a bag and thwap some sense into you.

Two. Same two hosts as above, but toss a couple dozen other named-based virtual hosts, years of HTTP configurations gone wrong and a little luck into the mix, and you just might hit the proverbial pay-dirt if you grok the Host header. All manner of fun things can be discovered if you play around with it, such as:

• Other virtual hosts that you might not have known about. Hint: send a 1.1 request with an invalid Host header.
• Previously "protected" areas of the website now exposed by accessing it using a different Host header

Much like many of my other ideas which come while sleeping or in other prominent thinking spots, I've decided to whack together some code to explore these things more. The resulting abomination is vhinfo, which given a URL and/or an IP address will play around with various HTTP/1.1 Host headers and see what other VHOSTs it can find based on information that the server provides or is otherwise freely available. Like I said, its gross, partially unfinished and needs cleanup, but the results are fun:

Looks like Facebook really likes to strip the first portion of the hostname out and replace it with a 'www'.

$  vhinfo  www.facebook.com
HTTP/1.1 Host: www.facebook.com on 69.63.176.140 -> 200: OK
HTTP/1.0  on 69.63.176.140 -> HTTP/1.1 200 OK ()
HTTP/1.0 www.facebook.com on 69.63.176.140 -> HTTP/1.1 302 Found (http://www.facebook.com/common/browser.php)
HTTP/1.1 Host: 69.63.176.140 on 69.63.176.140 -> 200: OK
HTTP/1.0  on 69.63.176.140 -> HTTP/1.1 200 OK ()
HTTP/1.0 69.63.176.140 on 69.63.176.140 -> HTTP/1.1 302 Found (http://www.63.176.140/common/browser.php)
HTTP/1.1 Host: localhost on 69.63.176.140 -> 200: OK
HTTP/1.0  on 69.63.176.140 -> HTTP/1.1 200 OK ()
HTTP/1.0 localhost on 69.63.176.140 -> HTTP/1.1 302 Found (http://www.ocalhost/common/browser.php)
HTTP/1.1 Host: 127.0.0.1 on 69.63.176.140 -> 200: OK
HTTP/1.0  on 69.63.176.140 -> HTTP/1.1 200 OK ()
HTTP/1.0 127.0.0.1 on 69.63.176.140 -> HTTP/1.1 302 Found (http://www.0.0.1/common/browser.php)
HTTP/1.1 Host: www.04.05.sf2p.facebook.com on 69.63.176.140 -> 200: OK
HTTP/1.0  on 69.63.176.140 -> HTTP/1.1 200 OK ()
HTTP/1.0 www.04.05.sf2p.facebook.com on 69.63.176.140 -> HTTP/1.1 302 Found (http://www.04.05.sf2p.facebook.com/common/browser.php)

Sun likes to throw 500's on certain virtual hosts (these should probably actually be 404's):

$  vhinfo www.sun.com                   
HTTP/1.1 Host: www.sun.com on 72.5.124.61 -> 200: OK
HTTP/1.0  on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.0 www.sun.com on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.1 Host: 72.5.124.61 on 72.5.124.61 -> 200: OK
HTTP/1.0  on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.0 72.5.124.61 on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.1 Host: localhost on 72.5.124.61 -> 500: Server Error
HTTP/1.0  on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.0 localhost on 72.5.124.61 -> HTTP/1.1 500 Server Error 
HTTP/1.1 Host: 127.0.0.1 on 72.5.124.61 -> 500: Server Error
HTTP/1.0  on 72.5.124.61 -> HTTP/1.1 200 OK 
HTTP/1.0 127.0.0.1 on 72.5.124.61 -> HTTP/1.1 500 Server Error 

And Yahoo, not to be outdone, throws standards to the wind and just returns raw HTML text sans HTTP response headers (and breaks my code :().

$  vhinfo yahoo.com
HTTP/1.1 Host: yahoo.com on 206.190.60.37 -> 301: Moved Permanently (http://www.yahoo.com/)
HTTP/1.1 Host: www.yahoo.com on 206.190.60.37 -> 301: Moved Permanently (http://www.yahoo.akadns.net/)
Connection to http://206.190.60.37/ failed! -- wrong status line: "<!doctype html public \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\">"
/home/warchild/bin/vhinfo:128:in `check': undefined method `[]' for nil:NilClass (NoMethodError)

In summary, the HTTP/1.1 "Host" header is an important part of accurately and completely performing web application security audits and has a role in any vulnerability assessment that is definitely worth considering.