"Etherleak" -- Old dog, old tricks

On 04/27/2002, I disclosed on the Linux Kernel Mailing list, a vulnerability that would be come known as the 'etherleak' bug. In various situations an ethernet frame must be padded to reach a specific size or fall on a certain boundary. This task is left up to the driver for the ethernet device. The RFCs state that this padding must consist of NULLs. The bug is that at the time and still to this day, many device drivers do not pad will NULLs, but rather pad with unsanitized portions of kernel memory, oftentimes exposing sensitive information to remote systems or those savvy enough to coerce their targets to do so.

Proof of this can be found by googling for "remote memory reading using arp", or by visiting the original posting. Admittedly, I did not realize at the time the exact cause of the bug nor the scope, but to my knowledge this was the first public disclosure.

This was ultimately fixed in the Linux kernel, but over time this vulnerability reared its head numerous times, but at the core the vulnerability was the same as the one I originally published. The most public of these was CVE-2003-0001, which was assigned to address an official @stake advisory which did not credit my findings.

So, now, nearly 5 years later, I'm publishing the POC exploit code I've recently revamped that can demonstrate the issue. The good news is that the bug has been addressed by numerous vendors, but is still rampant and should be considered a threat.

Enjoy.

Akamai -- A blessing in disguise?

For the past week or so I've been looped into various projects that, for one reason or another, required analysis of web logs. This little side trip has (re)taught me a number of things about large, production websites and how attackers work.

Perhaps the most intruiging was an analysis of HTTP return codes from our production web tier as compared to what Akamai had logged for the same time period. I had initially assumed that, for the really strange error codes, the numbers between what Akamai had recorded and what we saw should match. Well, I was wrong. For example, for HTTP 500 ("Internal Server Error"), despite the fact that our production web tier received 25% less traffic than what Akamai handled for us, it saw 10% more HTTP 500 errors than what Akamai saw. The reverse was true for 503s and 504s, which are arguably worse than 500s. Production web tier recorded no HTTP 504s, whereas Akamai showed roughly 3500. For 503s, the production web tier recorded roughly 10% more than what Akamai had logged.

This got me to thinking about why the error code distribution varied so widely. I could explain away some of the HTTP 503 and 504s based on global network wonkiness and with some handwaving, but the 500s and most of the 40x series codes eluded me temporarily. On the way home, it hit me and I spent the rest of my drive in a dead stare thinking about this.

The variations in HTTP error codes on Akamaized sites can be largely explained by the nature of the way most web-based attacks work. I'd venture to say that 90% of the HTTP attacks that any publicly exposed web server faces are not targeted at their host in particular, but rather that their IP address(es) were simply part of a massive attack that someone somewhere had launched in the hopes of getting lucky. So, in theory, for Akamaized sites no "valid" traffic should ever trickle its way to your web tier unless it comes through Akamai. Anything that bypasses Akamai is likely malicious and should be dropped, or at least screwy enough to warrant investigation. Preventing such attacks should be fairly easy, as Akamai clearly identifies itself by way of Akamai-Origin-Hop, Via and X-Forwarded-For headers. Sure, those headers can be forged, but we are not looking to hinder attackers that are intent on attacking our specific web presence. The downside is that you must do L7 inspection for this to work, so processing time and power can become an issue.

If anyone has considered this approach and/or actually implemented it, I'd like your input -- leave a comment or drop me an email.

Cisco -- Making log analysis more difficult?

Much of my work at my current job as of late has been in a consulting capacity more than anything. Instead of driving security projects or being the primary workhorse behind a given project, my role has been to provide security advice, help the given business unit prioritize a risk in the grand security scheme, and do some shepherding along its path to completion.

Not too long ago we had a fairly visible outage in one of our environments, and some groups were left scrambing in an effort to try and explain what happened and how it could be prevented in the future. Unfortunately, there was minimal logging going on, no correlation, no alerting, and little if any monitoring in place. I was asked to address these issues, and within a few hours had replicated the SEC log analysis and correlation setup we have in other environments.

In my haste I appear to have partially botched the configuration, and I was ignoring large swaths of FWSM syslog messages. Fortunately, none of those syslog message types have occured since my initial screw-up, so my bacon was saved. When I discovered this earlier today, I figured now was a good time to do some more detailed filtering on the FWSM messages we get alerted on.

After many difficult google searches and trudging through Cisco's site, I found this link. "Great!", I thought, learning that FWSM (and PIX) messages are prefixed by their severity. I was this close to implementing something that would immediately alert on severity 1 and 2 messages, and pile up larger amounts of 3 and 4 for bulk alerting, and can the rest. Well, Cisco apparently thought better than to make my life that easy, and tossed the notion of "severity" right out the window.

As an example of some of Cisco's severity ratings, behold the following gems:

• Failing over (%FWSM-1-104002) is just as severe as receiving a spoofed packet (%FWSM-1-106022)
• Dropping a UDP packet (%FWSM-2-106006) is just as bad as if your device horks itself so bad (%FWSM-2-215001), you need a TAC case..
• Dropping ICMP packets (%FWSM-3-106014) is on-par with your Cisco device failing to malloc() memory ("RAM System Memory"), aka %FWSM-3-211001

The good news here is that I was able to warp my mind in such a way that I too could understand the logic here, and accurate alerting is now in place.

Golf clap.

Saving bandwidth by ... removing comments?

I fixed up a number of shortcomings in htcomment the other day. As part of the development of this tool, I regularly run it against sites I frequent. I started to notice what appeared to be an inordinate amount of comments on some sites, so that got me thinking -- on average, what percentage of a site's web content is comments?

Some quick analysis gave some interesting results:

$  for site in boston.com latimes.com cnn.com neu.edu mit.edu
www.whitehouse.gov www.fbi.gov sony.com slashdot.org
google.com reddit.com craigslist.org  blogger.com myspace.com
amazon.com w3c.org php.net; do
full=`lynx -source http://$site | wc -c`    
comments=`./htcomment -q http://$site |wc -c`                               
echo "$site is"`echo "scale=4; ($comments/$full)*100" | bc`"% comments"   
done                                                 
boston.com is 16.9600 % comments
latimes.com is 5.5300 % comments
cnn.com is 1.6200 % comments
neu.edu is 15.2600 % comments
mit.edu is 2.6600 % comments
www.whitehouse.gov is 5.8500 % comments
www.fbi.gov is 4.5900 % comments
sony.com is .7300 % comments
slashdot.org is 3.7300 % comments
google.com is 0 % comments
reddit.com is 0 % comments
craigslist.org is 0 % comments
blogger.com is 2.7500 % comments
myspace.com is 17.0700 % comments
amazon.com is .2600 % comments
w3c.org is 0 % comments
php.net is .3400 % comments

Unfortunately these numbers are not 100% accurate -- htcomment can't differentiate between "kjdflakjfdaf " and just "", so the numbers for the sites that do have comments can be a bit skewed in some respects, but it is a good first order approximation. It is no coincidence, in my opinion, that google, w3c and craigslist have 0 comments on their frontpage. For sites that have >5% comments on their frontpage alone, you can't help but wonder how the behavior of their site or their bandwidth expenses would change if those comments were filtered out at their edge, or never put there in the first place.

The Art of Software Security Assessment -- Page corruption

I've been reading The Art of Software Security Assessment for quite some time now. I had originally picked a particular track through the book, but when I finished that I went back through to read the remaining chapters. A week or two back I was in what I thought was an interesting chapter -- 14. Network Protocols. All of the sudden the content started to not fit together and I felt like I had already read tthis stuff before.

Well, sure enough, I had. My book has been owned. Pages 843 - 890 were replaced with pages 795 - 842. OMG, arbitrary memory overwrite of 47 pages!: