Archive for the ‘Site Related’ category

Blog revised with cloudy goodness

May 20th, 2011

Greets all,

First, I want to apologize for being dark for so long. To make a loooong story short I was doing work for one of those huge organizations who’s lawyers can’t sleep at night if they don’t own every damn thought and neuron spark in your head. So while I was able to keep writing within the organization, it made public side blogging problematic. A resolution was always just around the corner, but alas it became obvious that it was never going to happen.

I’m happy to say that problem has been resolved. I’m now working for an extremely cool startup that doesn’t try to stifle the free exchange of ideas, but rather encourages it. My god what a crazy concept, eh? :D

With that said, going forward there will be a couple of changes:

  1. I’m a strong believer that hybrid cloud is poised to take over the world, so my writing will be primarily focused on this discipline.
  2. Rather than post entries here, I’ll be posting them at my new employer, CloudPassage.Just click on the “Blog” link at the top right of the page.


See you there,



If you can read this, you don’t work for SANS – part 2

August 5th, 2009

This issue appears to have been resolved. Kind of funny actually. I had been dealing with the Host Monster ticket system and it was taking 24 hours to get a reply. This morning I made a post to Matt Heaton’s blog (CEO of Blue Host) about the problem. It was resolved within hours and I’ve already received 3 follow ups from support.

Host Monster support states that the problem was D-Shield put themselves (and I assume Cisco as well) on their own ban list. I spoke with Johannes at D-Shield. I’ve known him for 10 years and he’s a real straight shooter. He had no clue what they are talking about and had not heard of this problem with anyone else. Sounds a little funny to me, because if they were actually using D-Shield to generate a ban list they would have known they were the good guys last Thursday when I first contacted support.

In any event it appears that all of the previously mentioned blocks have been cleared. Who says security and day time soap operas have nothing in common. ;)

If you can read this, you don’t work for SANS

August 4th, 2009

Anyone that has trained with me can tell you I’m real big on being able to read your own detects. While we have plenty of security devices that try to accurately describe what they think they see on the wire, they are programmed by humans and humans make mistakes. Try and automate the process and the mistakes become compounded. Even Cisco has backed off a bit on their grandiose claims of what a self defending network is actually capable of. Nothing replaces having a skilled analyst reviewing the findings.

Good help is hard to find

Of course the keyword in that last comment is skilled. I’ve dealt with plenty of senior security folks that have never seen a decode of an IP packet, let alone can tell you what a legit IP session should look like. One of the problems is when we need training we usually turn to the vendors. Vendors tend to focus on their pretty GUI, not what’s going on behind the scenes.

In a previous life I owned an ISP and had some very entertaining abuse reports submitted. One of my personal favorites was an admin reporting that one of my systems was “sending hostile ICMP packets” to one of his systems. When I reviewed my logs, I noted that one of my routers was in fact sending him ICMP host unreachable messages. This would happen every time his host probed the RPC port of an IP address that was not in use. I wrote back and explained that if his system would simply stop probing for non-existent systems, my router would stop telling him the host is off-line.

Another admin (at a rather large, well known company I might add) informed me that one of my systems was attacking him with Code Red via e-mail. If you remember Code Red, it only attacked IIS Web servers via HTTP. The “attacks” in question were users subscribed to a mailing list. Folks were talking about how to write good intrusion signatures to properly catch Code Red. If that was not ironic enough, the payload of the decode he sent me as evidence explained that the attacks were only HTTP based. If that twist is still not enough to make you chuckle, he later admitted that he was the one of the people subscribed to that list. :D

The more things change the more they stay the same

My hosting provider Host Monster (a subsidiary of Blue Host) has put a filter in place blocking all access to the SANS Institute mail server (SysAdmin, Audit, Network, Security Institute; provides computer security training), The Internet Storm Center (daily diary of Internet security threats) and DShield (an early warning system for Internet threats). I contacted support and they confirmed they are filtering these sites. I was unable to find out why beyond “due to suspicious activity”.

I know the folks that maintain the SANS and Dshield servers. They are hard core security folks with a serious clue. When I first signed up with my hosting provider I was impressed with the knowledge level of their support personnel. Lately however, I’ve found them to be lacking in even the basics. While I’m left to guess as to what actually caused the ban, I’m inclined to think that someone at Host Monster (or possibly Blue Host) saw an alert but didn’t have the skills to figure out its a false positive.

Communication is a two way street

Blue Host claims 1.5 million hosted sites through all of their holdings. So they now have 1.5 million clients that can’t:

  • Receive real time blocking alerts of malicious IP’s
  • Receive assessments on current Internet threats
  • Receive info on what’s going on in the security industry

So while attempting to protect themselves is a positive thing, the implementation has had a negative effect on the security of their clients.

How to verify a detect

So let’s pull something positive out of all of this and identify the proper procedure for verifying a security alert. We first need to start with good gear. Do not even consider an intrusion detection or prevention system that does not include:

  • Access to the signature language
  • Full decode of suspect packets

Without these features you are shooting in the dark.

Step 1: Understand the attack

When an alert gets triggered, make sure you understand the attack mechanism. What ports or services does it go after? Are there any known signatures? If you Google the attack’s name followed by the key words “false positive” and “spoofed”, does anything come up?

Step 2: Understand your intrusion system

No security product is perfect. They all have weaknesses or limitations. Does your intrusion system maintain state? If so, is it all the time or just some of the time? Does it properly validate CRC fields? How does it deal with fragmented traffic? Is it known to generate false positives? If so, are the false positives limited to only certain signatures or protocols, or is it all of the time?

Step 3: Sanity check the alert

Sometimes false positives can be weeded out from the limited amount of info presented in an alert. For example does the alert claim to have detected an HTTP attack coming from TCP/80 instead of going to it? If so, there is an obvious problem with the signature generating the alert.

Step 4: Check the signature

Some signatures are written very specifically so that there is little chance of a false positive. Some are more general however so its possible to have false positive fall out. Review the signature that generated the alert and make a judgement call. Does the signature check 3-4 different conditions or ten or more? Obviously the more parameters we are checking, the less likely we are to get a false positive.

Step 5: Check the decode

If you understand the attack pattern, you should already have an expectation of what will be in the attack decode. Does the packet match your expectations? I’ve seen plenty of false positives generated by people reading info on a Web site describing an HTTP based attack. These are easy to distinguish due to the extra HTML, proper agent and referrer fields, etc. In short, if the packet does not match a known decode of the real attack, figure out why.

Step 6: Research the source

I always take the time to make sure I understand who is sitting behind the source IP address. Sometimes this can go a long way towards identifying whether I can trust the alert. I’m reminded of a friend that banned a number of IP addresses his intrusion system had identified as hostile. Shortly after he started noticing that parts of the Internet were no longer reachable. Turns out someone spoofed a series of attacks from the IP addresses of the root name servers. Had he taken the time to look up the IP addresses first, he most certainly would not have blocked them.

Exec Summary

Blocking known to be hostile IP addresses can certainly be beneficial to security, but it must be implemented with caution. At the core of any network security system must be a knowledgeable security expert with good common sense. If that component is missing, the whole structure can fall apart like a house of cards.

Update: Since posting this I’ve found that Host Monster (Blue Host) is blocking access to one or more Cisco servers as well. Guess the list continues…

Invisible security bug spray

July 11th, 2009

Figured it would only be a matter of time before someone asked about the “Invisible security bug spray” comment at the top of each page. Didn’t think it would take less than one day. ;)

Here’s the scoop. Its a play on a quote from one of my all time favorite ISP responses to letting them know that one of the IPs within their network is hostile.  Here’s some quotes from their response:

Snip #1:
Hackers are like ants, chasing one away from the picnic table will not protect you from the thousands and thousands in the local ant hill that are still hungry. As soon as you rid yourself of one, you will still have an infinite risk available to you to get scanned or probed again.

Snip #2:
The point of the above statements are, as soon as you’ve had one hacker prosecuted and/or addressed, you’ve only swatted one gnat on a hot & humid summer afternoon. A few seconds to a min later, you will start feeling itchy again because there’s another one biting you.

Granted there is some Zen truth to their commentary (malicious IPs always make me itchy), but what an interesting choice of words. The response then went on to say:

Snip #3:
Firewall logs such as yours will often show the paper trail of the framed computer rather than the hacker itself. The real hacker hasn’t been and normally cannot be detected during those circumstances. We’re looking into the situation but we get dozens, if not hundreds of these a week.

Snip #4:
What you should focus on is not trying to kill every gnat, but rather wear invisible bug spray so that no matter how many of them there are, they cannot find you therefore there is no war. You cannot win the war, all you can do is stay out of it.  The better the firewall you use, the more likely you will remain invisible.

We apologize for any inconvenience but just bear in mind the broader perspective to this.

So there you have it, invisible security bug spray. :D

New site intro

July 10th, 2009

Greets all,

I’ve run this site for a number of years now by granting access to specific sections for only a limited number of people. While I will continue to do this for certain clients, I’ve decided its time to open up a large portion of it to public access. I also want to try my hand at integrating much of the advice I give via e-mail and private lists here as well. Figure that way the greatest number of folks will be able to benefit.

If you are used to having access to something and you don’t anymore, I apologize in advance. Please be patient as I try and integrate everything into WordPress. The system is very cool, but its also very different than what I was using in the past. Please give me time to cut the curve.

Yours in bits,