I’ve had the above question asked enough times that I felt it worthy of a blog post. While a few years back the answer may have been “less secure”, today the answer is “both”. I know, sounds like Chris being non-committal, but that answer really does most accurately describe the current state of the technology.

Virtualization Changes Everything

I’ve heard a few folks remark that virtualization is about to impact the industry the same way that the Internet did in the 90’s. To be honest, I think there is merit in that opinion. In the early 90’s most folks were running IPX, AppleTalk, NetBUI and a plethora of other protocols on closed networks. By the end of the 90’s, most folks were running IP exclusively with connectivity to the entire world. The way we did business, as well as the way we applied security, completely changed over that 10 years. Both network administration and security skills that were cutting edge in 1990 were all but useless by 1999.

Virtualization is starting to ramp up to have the same impact on the industry. Virtualization deployment requires a complete rethinking of how to apply security. Back in the 1990’s, admins who simply plugged into the Internet, without regard for how this would impact their network, got burned big time. We are lining up to see a similar outcome as folks adopt virtualization.

What Makes Virtualization Less Secure

The Achilles heel of virtualization is in the software itself. We are hoping we can trust the software to keep guest systems away from each other, as well as the host and/or hypervisor. There are two major problems with this expectation:

1. No software is bug free
2. Software can be misconfigured

A few years back Core Research showed they could break out of a guest and gain full control of the host OS. While a hypervisor is supposed to limit that type of exposure, we’ve certainly seen cases where even the hypervisor has been bypassed. We’ve even seen cases were software becomes exploitable only when run in a virtualized environment. These links show a small cross section of the virtualization problems that have been discovered over the last few years. Google can give you a more complete list if you are interested.

So a prudent security professional is going to be cautious of blindly trusting software to be secure. The problem is vendors do not always take this same approach. Take VMware with their ESX (soon to be ESXi) product as an example. Many of us were flabbergasted when a VMware representative announced at CanSecWest that it was theoretically impossible to attack the ESX hypervisor. When we simply assume something is unbreakable, someone more creative is going to figure out a way to punch through.

One of my biggest concerns with ESX/ESXi is that VMware has designed it to be modular (via VMSafe). On the plus side, this means that outside vendors can create products to help improve the hypervisor’s functionality and security. On the downside this dramatically increases the chances of bad code being introduced which can compromise security.

We’ve seen a great example of this in the past. Marcus Ranum created the Gauntlet firewall, which at that time was one of the most secure and kick butt security devices available. When three letter agencies wanted the best security, they turned to Gauntlet. Marcus sold Gauntlet to Network Associates (later became McAfee) who immediately started adding in features. It was not long before a steady string of vulnerabilities were being discovered, each introduced by these new “features”. From there, the product lost its security cred and slid off of the radar.

Now it is certainly possible to add features and keep things secure. The FreeBSD folks are an excellent example of how to do this correctly. To ensure security they maintain a very strict auditing process. Is it perfect? Absolutely not, but their auditing process has set the bar for secure software implementation. With any luck VMware will do similar, but I have not heard any buzz about this being the case.

Getting Your Head Straight

OK, so we can’t blindly trust virtualization software to keep attackers at bay. We can however still take precautions to help minimize the impact if the worst does occur. One of the biggest steps you can take is to carefully consider which servers get hosted, and what other guest systems are permitted to run on the same box. The security zone concept used by network architects is just as applicable here.

A security zone is simply a collection of systems that share the same relative level of risk. For example Web, name and SMTP servers are usually all located on a DMZ, because they all share similar risk from direct attack. On the internal portion of the network, desktops are usually placed in a different security zone than the servers. This is because servers have little to no access to the Internet while desktops are usually permitted to communicate directly. This places the desktops at higher risk of attack than the servers.

We can apply this same logic when implementing virtualization. A DMZ server and an internal server should not be guests on the same hardware (both CPU and disk array). Doing so could allow an attacker to create an alternate route into our network. Rather than having to pass through any firewall, NIDS, NIPS, etc. devices that has been deployed on the wire, an attacker may be able to gain access to internal resources via the virtualization software. Is it an easy attack? Not from what we have seen so far. Functional exploits have been discovered however, so why introduce unnecessary risk if we don’t have to.

By the way, these same security zone rules should be applied to your virtualized network gear. For example it is a bad idea to use the same physical switch to VLAN the DMZ and the internal network. I’ve seen a couple of clients get whacked that way.

What Makes Virtualization More Secure

Fortunately, from a security perspective, virtualization is not all bad news. In fact there are some very cool security practices you can apply in a virtualized environment that you simply cannot do without it. This was one of the reasons we started using virtualization within the Honeynet as early as 2000.

One of the biggest security issues we face today is kernel level rootkits. What makes this strain of malware so insidious is it effectively turns the operating system itself into malware. This makes detection extremely difficult, as all security checks must pass through the kernel. If the kernel itself is compromised, we can’t rely on the kernel to accurately report security information. We end up having to shutdown the system, mount the drive on a known to be clean OS, and performing our forensic checks from there. Oh course the problem with this process is that it does not scale well. If we have dozens or hundreds of servers, there simply is not enough time in a day to check them all properly.

As mentioned earlier, VMware is now permitting third party vendors access to the hypervisor API via VMSafe. This permits access to privileged state information, such as memory and network traffic, on each of the guest OSs. By plugging into the hypervisor, some extremely cool security options become possible.

For example let’s say a guest OS is attacked by a kernel level rootkit. By analyzing guest memory, the rootkit can be detected from outside of the virtual operating system. When performing the checks via the hypervisor, there is far less of a chance that a rootkit can stealth its activities and go undetected. As mentioned earlier, there is no comparable option with a non-virtualized system.

The API plug also creates new possibilities for dealing with encrypted traffic. When end to end encryption is employed (like a VPN), network based checks of the application layer are easily bypassed. Your only real option was to run agent software on the end point, so security could be implemented after the decryption process. Of course the problem here is that if the agent is attacked, all bets are off. Again, by plugging into the hypervisor we are in a better position to more safely scrutinize this data.

We are just starting to see new products that leverage the VMSafe API plug. Since all of the products are relatively new, the jury is still out on how effective they can be. Offerings run the gambit from replacing host based firewall and IDS protection to full policy enforcement. It will be interesting to see how this product niche shakes out over the next year.

Summary

So as I mentioned at the beginning of this post, virtualization has the ability to make your environment either more or less secure, depending on how you deploy it. If you simply start running everything on a single box, you are probably going to get whacked. If you extend the best practices that have been developed over the years into the virtualization realm, as well as leverage some of the new security features that are being released, you can actually create a better overall security posture.