There have been some excellent papers written on how to detect if an operating system is running as a Vmware guest image. Automated tools have even been released to help expedite the process. All assume however that terminal or command prompt access is required to perform the detection. Few people realize that it is possible to passively detect a Vmware guest without any level of system access. In fact, if you know what to look for, you can even identify which version of VMware is hosting the virtual system.

Passive fingerprinting

Passive fingerprinting is a technique where by a remote operating system can be identified by the nuances in the IP packets it produces. For example the payload of Echo-Request packets are pretty unique from operating system to operating system. In a previous post I discussed how different operating systems use different time to live values. By analyzing these variations in the IP packets, you can do a pretty accurate job of identifying the source operating system. In fact many times you can even identify the patch reversion level of the system.

VMware virtual systems

VMware supports two modes for connecting a guest operating system to a network. You can NAT it behind the host, or connect it in bridging mode. Many people think that bridging mode gives the guest OS direct access to the network card, but they forget that VMware is still sitting in between the guest OS and the host system’s drivers to run the NIC. It is not uncommon for this layer to make changes in the IP packets before passing them onto the wire. So if you know what changes to look for, you can detect when VMware software is sitting between an operating system and the local network.

VMware makes multiple changes to the IP datastream. In this post we will only look at two of these changes (window size and scaling), and I’ll leave it to the reader to try and find the others. Further, different versions of VMware will modify the IP datastream in different ways. If you know what changes are unique to each version, you can identify the release level of VMware sitting between the guest OS and the network.

Window size and window scaling

TCP uses window size to identify how much data can be transmitted before a remote system must pause and wait for an acknowledgement. For example if “System A” tells “System B”, “window size = 1400”, “System B” is permitted to send just under one full size Ethernet packet before it must wait. After the 1400 bytes are transmitted, “System B” is not permitted to send any more data till it receives an acknowledgement of that data from “System A”. Window size is adjusted on the fly, so “System A” may eventually advertise “window size = 1600”. This means “System B” can now send a full size packet followed by one small packet before it must pause for an acknowledgement. Window size is a method of flow control to help insure that the receiving system does not see more data than it’s buffers can handle.

Window size dates back to the original implementation of TCP. At the time 16 bits were reserved (bytes 14 and 15 in the TCP header) to specify how many bytes could be transferred before pausing for an acknowledgement. This means we could specify a maximum window size of 65,535 bytes. That’s more than big enough by 1980 standards, but not so great when you are talking about 1 Gb and faster networks. In the early 90’s window scaling was added as a TCP option. Window scaling is only advertised in the first TCP packet sent by a system (SYN or SYN/ACK packet) and is a multiplier for the advertised window size. For example, assume a window size of 1400 bytes. If the window scale is 3, the math would be:

(2*2*2) * 1,400 = 11,200

If the window scale is 4, the math becomes:

(2*2*2*2) * 1,400 = 22,400

So window scaling permits us to now advertise much larger window sizes. This explanation is rather simplistic, but should give the reader a good idea of how window size and scale works. For more info see RFC 1323 or the TCP Wiki.

Backtrack 3 final as a stand-alone host

Here’s a SYN/ACK response to a connection request returned by a Backtrack 3 system (Linux 2.6.21.5 kernel) running on a dedicated host:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.4.80 > 192.168.100.50.44943: S, cksum 0×9552 (correct), 2760872392:2760872392(0) ack 4071399005 win 5792 <mss 1460,sackOK,timestamp 5362662 198395,nop,wscale 5>

Note the initial window scale is 5, which means the actual window size is 32 * 5792 = 185,344. This is pretty common for Linux systems running a post 2.6.17 kernel.

Now let’s take a look at how the window size for Backtrack 3 gets incremented:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.4     5792      5

192.168.100.4     245

192.168.100.4     336

192.168.100.4     426

192.168.100.4     517

192.168.100.4     607

192.168.100.4     698

192.168.100.4     788

192.168.100.4     879

192.168.100.4     969

192.168.100.4     1060

I used tshark to clean up the output so we could focus on the fields we actually care about. The above trace is a large data transfer to 192.168.100.4 (BT3 stand alone) via TCP/80, as show in the above tcpdump trace. Note that we can pretty clearly identify how the window size was incremented over the data session.

Backtrack 3 final, guest on Windows XP Workstation

So let’s take a look at how window size and scaling get affected when the operating system is sitting behind VMware. In this test I ran BT 3 as a guest operating system on a Windows XP workstation host. I used VMware player 2.5.3 and wrote a custom VMX file to run the BT 3 iso.

Here’s an initial SYN/ACK packet:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.7.80 > 192.168.100.50.35245: S, cksum 0x670d (correct), 3795217952:3795217952(0) ack 4108648760 win 5792 <mss 1460,sackOK,timestamp 5210512 208486,nop,wscale 4>

Note that while the initial window size matches the stand-alone BT 3 system, VMware has dropped the window scaling from 5 (32x) to 4 (16x).

Here’s the stream of packets being returned by the guest system:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.7     5792      4

192.168.100.7     490

192.168.100.7     671

192.168.100.7     852

192.168.100.7     1033

192.168.100.7     1214

192.168.100.7     1395

192.168.100.7     1576

192.168.100.7     1757

192.168.100.7     1938

192.168.100.7     2119

Note that after the first packet the negotiated window size appears larger, but when you factor in the window scale multiplier the overall window size ends up being slightly smaller than BT 3 when it is run on a dedicated system. Not all that surprising as virtual systems typically run slower than dedicated systems, but we’ve just tagged two differences we can look for to determine if the host is stand-alone or a guest virtual image.

Backtrack 3 final, guest on Fedora 11

Of course the big question is, “was the window size change due to the Windows XP host or VMware?”. VMware circumvents the Windows IP stack when transmitting on the wire, so the host OS should have no effect on the packet stream. To verify this, let’s try the same test using the same version of VMware player, only this time we’ll use Linux as the hosting operating system. Here’s the SYN/ACK:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.13.80 > 192.168.100.50.43363: S, cksum 0x6ef7 (correct), 3423175369:3423175369(0) ack 483782889 win 5792 <mss 1460,sackOK,timestamp 579990 366817,nop,wscale 4>

Here’s the stream of packets being returned from TCP/80 on the guest BT 3 system:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.13    5792      4

192.168.100.13    490

192.168.100.13    671

192.168.100.13    852

192.168.100.13    1033

192.168.100.13    1214

192.168.100.13    1395

192.168.100.13    1576

192.168.100.13    1757

192.168.100.13    1938

192.168.100.13    2119

Note the results are identical to when BT3 was the guest operating system on Windows XP. So while this methodology will not identify the host operating system being used, we can consistently determine if BT3 is running on a dedicated system or as a guest via VMware. While it is entirely possible to modify a dedicated system to appear to be a virtual guest, the opposite does not appear to be feasible.

Exec Summary

By having an in-depth understanding of IP and the nuances generated by different operating systems, it is possible to determine if an OS is running on a dedicated system or as a guest via VMware. Modifications made by the VMware wrapper leave behind tell tale clues. A smart packet weenie can leverage these clues to determine the virtualization status of the source operating system.