Archive for the ‘2-alert’ category

TCP Options Challenge – clues

November 18th, 2009

Earlier I posted a challenge to write a tcpdump/Windump filter that would capture packets that have the TCP option “Window Scale” set. Some folks are close, but I wanted to post a few hints. Also, I have no problem with you e-mailing me directly, but to win the challenge you have to post the answer to the comments section. That way there is no question as to who found the answer first.

All TCP options are specified by a “Registry Kind” value (similar to the ICMP “Type” field). In the case of Window Scale, that value is 3. Also, all TCP options except for NOP contain a secondary field called “Length”. This defines how many bytes the options is using, including the “Registry Kind” byte. In the case of Window Scale the length value is always 3. So we have:

  • 1 byte for Registry Kind
  • 1 byte for Length
  • 1 byte for the actual WScale value

Hint 2: If you have never drilled into the TCP Options field, tshark has a cool option:

tshark -n -r capture-file.cap -T fields -e tcp.options

This will output the entire TCP options field in Hex so you can at least see what it looks like.

Final clue, I’ve posted a Linux SYN packet which sets WScale to 5 for you to use to check your filter.

linux-syn

Good luck!

Chris

TCP Options Challenge

November 18th, 2009

I’ve had a bit going on including the release of a new reference tool for the Apple iPhone and iPod. The tool is call Packet Decode and I’ve setup an alternate site to help maintain it.

I feel bad at neglecting this site over the last few weeks, so I’ve decided to throw out another challenge. The winner will receive a free copy of the above mentioned Packet Decode tool. OK, so you will not be able to retire on the proceeds, but hopefully the tool will make your life a bit easier. ;)

So here’s the challenge:

Write a tcpdump or Windump filter that will only capture packets that have the TCP “Window Scale” option set. All other TCP option settings can be ignored.

Pretty simple, eh? First person to post the answer in the comments section gets the prize.

Tshark Challenge – Uber-geek Answer

October 13th, 2009

In my last post I left you with a question: Given what we have seen in the decode file with tshark, what impact (if any) would there be if we place a stateful inspection firewall between the attacker and the Web server? In other words, if the attacker is running a packet sniffer, would they still see the Web server leaking out 404 errors?

And the answer is…

Maybe. :)

Not all stateful inspection firewalls are equal. Some handle packets slightly differently than others. For example some (like Checkpoint, Netscreen and others) let non-SYN packets to permit rules generate new state table entries. Some (Cisco, Netfilter and others) will only generate a state table with proper session establishment (must see TCP three packet handshake).

The NIPS sent a valid reset packet to the attacker on the Internet. Each of the above firewalls would see the reset packet and remove it’s entry from the state table. When the Web server continued to communicate however, only the first set of firewalls would let the packets leak out to the attacker. The second set of firewalls would simply drop the traffic. In fact if we set them up with a deny rule instead of a drop rule, they would kill the session on the Web server thus fixing the problem created by the NIPS.

Why do some vendors let acknowledgement packets generate new state table entries? This seems a bit counterintuitive as a legitimate session is always going to start with a SYN packet. There are two reasons this makes good functional sense:

  • Updating the firewall rules does not kill active sessions
  • Active-active setup will pass traffic prior to state table sync

Of course we’ve increased functionality at the cost of security. Unfortunately that is the typical trade off.

Hope you had fun with this challenge. If there is interest I will post more in the future.

Tshark Challenge – The Final Answers

October 9th, 2009

In my last post we had determined that the trace file contained a session where a single honed network based intrusion prevention system had attempted to stop an attack from an HTTP client to a Web server. We concluded that the client’s data request did look rather suspicious, and proved a third party system (the NIPS) was responsible for the reset packets transmitted during the session.

In this post we still need to answer two questions:

  1. Was the attack successful?
  2. Why did the Web server ignore the TCP reset packet transmitted by the NIPS?

Was the attack successful?

The attacker was looking to identify if the file “startstop.html” was located in the “Administrator” directory. Let’s take a look at one of the packets sent to the attacker after the reset packets were transmitted:

tshark -n -r challenge1.cap -x frame.number==11

11   0.711825  12.33.247.4 -> 148.78.247.10 TCP [TCP Retransmission] [TCP segment of a reassembled PDU]

0000  00 50 8b ea 20 ab 00 b0 d0 20 7d e3 08 00 45 00   .P.. …. }…E.

0010  01 a7 37 47 40 00 40 06 73 8b 0c 21 f7 04 94 4e   ..7G@.@.s..!…N

0020  f7 0a 00 50 69 2a 3a 88 39 cd 6a 97 b9 4c 80 19   …Pi*:.9.j..L..

0030  19 20 8c d4 00 00 01 01 08 0a 3d 76 0e d3 00 ec   . ……..=v….

0040  48 4b 48 54 54 50 2f 31 2e 31 20 34 30 34 20 4e   HKHTTP/1.1 404 N

0050  6f 74 20 46 6f 75 6e 64 0d 0a 44 61 74 65 3a 20   ot Found..Date:

0060  54 75 65 2c 20 32 35 20 4a 75 6e 20 32 30 30 32   Tue, 25 Jun 2002

0070  20 30 30 3a 33 34 3a 35 38 20 47 4d 54 0d 0a 53    00:34:58 GMT..S

0080  65 72 76 65 72 3a 20 41 70 61 63 68 65 0d 0a 43   erver: Apache..C

0090  6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65   onnection: close

00a0  0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20   ..Content-Type:

00b0  74 65 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73   text/html; chars

00c0  65 74 3d 69 73 6f 2d 38 38 35 39 2d 31 0d 0a 0d   et=iso-8859-1…

00d0  0a 3c 21 44 4f 43 54 59 50 45 20 48 54 4d 4c 20   .<!DOCTYPE HTML

00e0  50 55 42 4c 49 43 20 22 2d 2f 2f 49 45 54 46 2f   PUBLIC “-//IETF/

00f0  2f 44 54 44 20 48 54 4d 4c 20 32 2e 30 2f 2f 45   /DTD HTML 2.0//E

0100  4e 22 3e 0a 3c 48 54 4d 4c 3e 3c 48 45 41 44 3e   N”>.<HTML><HEAD>

0110  0a 3c 54 49 54 4c 45 3e 34 30 34 20 4e 6f 74 20   .<TITLE>404 Not

0120  46 6f 75 6e 64 3c 2f 54 49 54 4c 45 3e 0a 3c 2f   Found</TITLE>.</

0130  48 45 41 44 3e 3c 42 4f 44 59 3e 0a 3c 48 31 3e   HEAD><BODY>.<H1>

0140  4e 6f 74 20 46 6f 75 6e 64 3c 2f 48 31 3e 0a 54   Not Found</H1>.T

0150  68 65 20 72 65 71 75 65 73 74 65 64 20 55 52 4c   he requested URL

0160  20 2f 63 66 69 64 65 2f 41 64 6d 69 6e 69 73 74    /cfide/Administ

0170  72 61 74 6f 72 2f 73 74 61 72 74 73 74 6f 70 2e   rator/startstop.

0180  68 74 6d 6c 20 77 61 73 20 6e 6f 74 20 66 6f 75   html was not fou

0190  6e 64 20 6f 6e 20 74 68 69 73 20 73 65 72 76 65   nd on this serve

01a0  72 2e 3c 50 3e 0a 3c 2f 42 4f 44 59 3e 3c 2f 48   r.<P>.</BODY></H

01b0  54 4d 4c 3e 0a 00 03 00 07                             TML>…..

So the answer to the attacker’s question; “Is the file located on the server?, is answered in every packet the server set to the client after the reset packets were issued. Now the question becomes, did the attacker see this information?

When the reset packet was sent to the attacker, the reset should have killed the TCP session. This means that when this data arrived, the receiving port (TCP/26922) should have been in a closed state. Had this in fact been true, I would expect to see a reset/ACK coming back from the attacker’s system telling us that TCP/26922 is now closed. We never see those packets.

So why didn’t the remote port close? The reset packet is valid, so it should have killed the session. A savvy attacker is going to run a packet sniffer in the background while performing their attack. It is possible that the attacker would have figured out what was going on and simply created a firewall rule on their attack system filtering out all inbound reset packets. This would have permitted their tool to continue functioning. Even without the firewall filter their packet sniffer would contain the answers they seek. So there is a very good chance the attacker has figured out what we are vulnerable to, despite the fact a NIPS is protecting the network.

Why did the Web server ignore the reset packet?

Our final question is why did the Web server ignore the reset packet sent by the NIPS. This is causing two problems:

  • The info the attacker wanted is continuing to leak out
  • If the attacker does multiple file probes, they could fill up the session table on the Web server

The second item is kind of scary because it would actually be the NIPS causing the DoS attack by only killing one side of the connection properly. So even if we were patched and up to date, vendor gear we paid money for would end up killing our Web server. Gee I hate it when I spend money to DoS myself. ;)

Let’s take another look at those reset packets:

tshark -n -r challenge1.cap frame.number==6 or frame.number==7

6   0.031319  12.33.247.4 -> 148.78.247.10 TCP 80 > 26922 [RST, ACK] Seq=1 Ack=222 Win=0 Len=0

7   0.031385 148.78.247.10 -> 12.33.247.4  TCP [TCP ACKed lost segment] 26922 > 80 [RST, ACK] Seq=1 Ack=222 Win=0 Len=0

Notice that tshark is telling us packet 7 is a lost segment. In other words, tshark is telling us that the TCP sequence number is not within the window the host is expecting. A look at the sequence and acknowledge numbers explains why. The values are identical to the ones in packet 6. You may remember from the last post that packet 6 and 7 also had the same IP ID value (45298).

So it appears that here is what happened:

  • Client sent an HTTP attack to our Web server
  • NIPS detected the attack
  • NIPS sent a valid TCP reset to the attacker to kill the session
  • NIPS swapped the source and destination IP addresses in the packet, recalculated the CRCs, and then sent the same reset to the Web server
  • Web server ignores the reset because it does not contain an acceptable sequence number

You may be asking yourself, “How did the vendor miss this?”. My best guess is that the vendor ran some simulated attacks; the attack tool did not show them vulnerable files, so they assumed everything was working OK. In other words, a vendor who created a product to protect networks on the wire never bothered to look at what their product was actually doing on the wire. Hummm…

Bonus question

OK, if you are having fun with this one, here’s a bonus question to answer that will definitively prove your uber-geek status upon the top of the secret pyramid:

Assume we place a stateful inspection firewall between this subnet and the attacker. Will the attacker still be able to see the answers with a packet sniffer?

I’ll post the answer next week.

Tshark Challenge – Hints 4

October 8th, 2009

In my last post we identified that the client system was probably running some kind of tool to probe for known to be vulnerable files. We still have to explain the reset packets however, as well as why the server was ignoring them.

Right now we don’t even know where this packet capture was taken in relation to the two end points. I’m going to run tshark using the “-T fields” switch to print out the TTL information. By analyzing the TTLs, we should be able to determine where we are sniffing in relationship to the client and the server. I’m also going to print out the IP IDs to see if there are any abnormalities in the packet order.

Here’s the command and output:

tshark -r challenge1.cap -T fields -e frame.number -e ip.src -e tcp.srcport -e ip.ttl -e ip.id

1       148.78.247.10   26922   49      0x2a6b

2       12.33.247.4     80      64      0×0000

3       148.78.247.10   26922   49      0x2a95

4       148.78.247.10   26922   49      0x2a96

5       12.33.247.4     80      64      0×3743

6       12.33.247.4     80      255     0xb0f2

7       148.78.247.10   26922   255     0xb0f2

8       12.33.247.4     80      64      0×3744

9       12.33.247.4     80      64      0×3745

10      12.33.247.4     80      64      0×3746

11      12.33.247.4     80      64      0×3747

12      12.33.247.4     80      64      0×3748

13      12.33.247.4     80      64      0×3749

14      12.33.247.4     80      64      0x374a

15      12.33.247.4     80      64      0x374b

16      12.33.247.4     80      64      0x374c

17      12.33.247.4     80      64      0x374d

148.78.247.10 is the client. We know that because it initiated the session and its communicating using an upper source port. 12.33.247.10 is our HTTP server communicating from port 80.

In packet 1 we see the client has a TTL of 49. The closest starting TTL match for a known operating system is 64, so this tells us the client is a Linux, UNIX, or Mac system sitting 15 hops away. In packet 2 we see the server with a TTL value of 64, so it must be one of the previously mentioned operating systems sitting on the local network. So, we are on the same collision domain as the server, but sitting 15 hops away from the client.

There is a problem however. Look at packets 6 and 7. Here we see the TTL of both systems changing to 255. An OS will never change it’s TTL during a session, so this looks extremely suspect. Further, we already determined that the client is sitting 15 hops away from us. 255 is the largest possible TTL value as the TTL field is only a single byte (byte 8 in the IP header). So even though the source IP address claims to be the remote client, there is no way that packet could have originated anywhere but from the local network. If the packet had crossed one or more routers, the TTL value would be lower.

The IP ID information does not look right either. In the first three packets from the client we see the IP ID values 0x2a6b (10859), 0x2a95 (10901), and 0x2a96 (10902). This tells us the client is possibly using a +1 incremental IP ID, we just didn’t see 42 of the packets between the first and second packet transmitted. The next IP ID we see from the client is 0xb0f2 (45298). OK, unless we missed over 34,000 packets, this IP ID does not jive.

Note the above analysis would only be theoretical were it not for the inconsistent TTL values. With only three packets to look at, we cannot precisely identify the IP ID increment. It is also possible, but very highly unlikely, that the IP ID increment is completely random and the system just happened to assign two incremental IP IDs, one right after the other. Still, combine the TTL and IP ID information together and they indicate that this final packet could not have originated from the same system.

We have more IP ID data from the server to work with, so let’s take a look at that. The IDs are:

  • 0 (0)
  • 0×3743 (14147)
  • 0xb0f2 (45298)
  • 0×3744 (14148)
  • 0×3745 (14149)
  • 0×3746 (14150)
  • 0×3747 (14151)
  • 0×3748 (14152)
  • 0×3749 (14153)
  • 0x374a (14154)
  • 0x374b (14155)
  • 0x374c (14156)
  • 0x374d (14157)

Take a look at the third listed IP ID, which is from packet 6 in the trace. Note that all of the other IP IDs are incremental +1, but this IP ID does not belong. In fact we can show that the server increments it’s IP ID by +1 and that there is no way this packet originated from the server.

Humm. I wonder if these suspect packets line up with the strange resets we saw:

tshark -r challenge1.cap -T fields -e frame.number -e ip.src -e tcp.srcport -e ip.ttl -e ip.id -e tcp.flags.reset

1       148.78.247.10   26922   49      0x2a6b  0

2       12.33.247.4     80      64      0×0000  0

3       148.78.247.10   26922   49      0x2a95  0

4       148.78.247.10   26922   49      0x2a96  0

5       12.33.247.4     80      64      0×3743  0

6       12.33.247.4     80      255     0xb0f2  1

7       148.78.247.10   26922   255     0xb0f2  1

8       12.33.247.4     80      64      0×3744  0

9       12.33.247.4     80      64      0×3745  0

10      12.33.247.4     80      64      0×3746  0

11      12.33.247.4     80      64      0×3747  0

12      12.33.247.4     80      64      0×3748  0

13      12.33.247.4     80      64      0×3749  0

14      12.33.247.4     80      64      0x374a  0

15      12.33.247.4     80      64      0x374b  0

16      12.33.247.4     80      64      0x374c  0

17      12.33.247.4     80      64      0x374d  0

Ah ha! So if we look at the packets with the strange TTL and IP ID values, these were the strange reset packets sent during the session.

It looks to me like some third system is jumping into the conversation in order to transmit the reset packets. Since these strange packets have a TTL of 255, we know it must be on the same collision domain as where we are sniffing. Since it is on the same collision domain, the Ethernet header information might be helpful:

tshark -r challenge1.cap -T fields -e frame.number -e eth -e tcp.flags.reset

1       Ethernet II, Src: HewlettP_ea:20:ab (00:50:8b:ea:20:ab), Dst: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3)     0

2       Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

3       Ethernet II, Src: HewlettP_ea:20:ab (00:50:8b:ea:20:ab), Dst: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3)     0

4       Ethernet II, Src: HewlettP_ea:20:ab (00:50:8b:ea:20:ab), Dst: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3)     0

5       Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

6       Ethernet II, Src: D-Link_8f:e0:0c (00:50:ba:8f:e0:0c), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)       1

7       Ethernet II, Src: D-Link_8f:e0:0c (00:50:ba:8f:e0:0c), Dst: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3)       1

8       Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

9       Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

10      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

11      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

12      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

13      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

14      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

15      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

16      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

17      Ethernet II, Src: DellComp_20:7d:e3 (00:b0:d0:20:7d:e3), Dst: HewlettP_ea:20:ab (00:50:8b:ea:20:ab)     0

So when the client comes in from the Internet, it passes through an HP system acting as either a router or a firewall. Our Web server is running on Dell hardware. Note our resets (packets 6 and 7) are both being generated by the same system using a D-Link NIC.

So why did the D-Link system try to kill the session? This occurred just after the client sent the suspicious file request. It looks to me like the D-Link system is actually a single honed intrusion prevention system (IPS). When an attack is detected, the IPS attempts to prevent the attack by transmitting reset packets to both ends of the connection.

But we still have some unanswered questions. Was the attack successful and why did the server appear to ignore the IPS’s attempt to kill the session?

Turn in to the next episode to find out.

Analyzing packets with tshark

October 1st, 2009

In an earlier post I discussed how to adjust the display output in tshark. The post generated a lot of interest, so I decided to add some additional information on using tshark to decode packets. This post assumes you have read the one linked to above.

Why use tshark instead of tcpdump/windump?

Many old time decoders swear by tcpdump, and it’s Windows counterpart windump. Both are great tools, but they have become a little dated. While patches are still released from time to time, little has been done to update or expand their decode capability. Wireshark on the other hand, as well as it’s included tools such as tshark, include decode support for hundreds of protocols and the list is growing all of the time. While you can certainly analyze packets without the decoders, they make the process go far quicker.

Why use tshark instead of Wireshark?

Wireshark is a great tool when you are doing an in-depth payload analysis. It can be a little tedious however if you wish to follow a specific field over multiple packets. For example let’s say we wish to watch the TCP sequence number increment over multiple packets. With Wireshark, I would have to note the sequence number location in the middle pane and page through each packet. Since there is no way to line up the value over multiple packets, I’m forced to remember previous values when performing my calculations. With tshark however, we can do something like this:

tshark -n -T fields -e ip.src -e tcp.seq -e tcp.len

64.111.96.38    0        0

64.111.96.38    1        0

64.111.96.38    1        363

64.111.96.38    364     1448

64.111.96.38    1812    1448

64.111.96.38    3260    1448

64.111.96.38    4708    1448

64.111.96.38    6156    1448

64.111.96.38    7604    1448

64.111.96.38    10500   310

64.111.96.38    9052    1448

64.111.96.38    10810   0

Remember that the TCP sequence number (the second field) should increment based on the size of the payload (the third field). Note that packets 10 and 11 where received out of order. This could mean there are multiple paths available between our location and the identified IP address. While Wireshark would show us this information as well, in this view it is a bit easier to follow the flow.

More on displaying fields

As discussed in my earlier post, as well as shown above, the “-T” switch can be used to manipulate the output being displayed. You can choose from XML, Postscript or plain text. The most useful option is “fields” as it lets you pick and choose which specific fields you want printed out. As shown above, the “-e” switch can then be used to identify which fields you wish to display. The complete list of filters can be found here. A nice cheat sheet of the most commonly used values can be found here.

If you define a specific protocol, tshark will display some of the more important fields from that header. For example to look at only the Ethernet header:

tshark -T fields -e eth

Ethernet II, Src: TyanComp_56:3b:14 (00:e0:81:56:3b:14), Dst: Dell_d1:fe:ef (00:12:3f:d1:fe:ef)

Ethernet II, Src: TyanComp_56:3b:14 (00:e0:81:56:3b:14), Dst: Dell_d1:fe:ef (00:12:3f:d1:fe:ef)

Ethernet II, Src: TyanComp_56:3b:14 (00:e0:81:56:3b:14), Dst: Dell_d1:fe:ef (00:12:3f:d1:fe:ef)

Note the type and CRC fields are not displayed, as they are not as “interesting” as the source and destination MAC address. We would have to define these fields specifically (ip.type and ip.trailer) if we wish to see them.

One side effect of printing fields is that tshark will add a blank line for any packet that does not contain the specified field. This can be a pain when analyzing HTTP packets as not every packet will contain a URI. An easy way to clean this up is to pipe it through grep. For example:

tshark -T fields -e http.request.uri | grep -v “^$”

/

/styles.css

/templates/classic/images/starsnstripes.gif

/templates/classic/images/unionjack.gif

/templates/classic/images/amazon.header.gif

/c_images/2009/05/07/71090.2.jpg

In my last post I discussed grep as well as where to grab a free version for Windows. The above grep command uses the “-v” switch to match all lines that do not contain the specified value. “^$” defines a blank line. So the above grep command filters out all blank lines.

More display options

Tshark has a number of other useful display options. For example you can print headers at the beginning of the output:

tshark -n -T fields -e ip.src -e ip.dst -E header=y

ip.src  ip.dst

64.111.96.38    12.5.200.100

64.111.96.38    12.5.200.100

64.111.96.38    12.5.200.100

If you plan on importing the information into a spreadsheet or database, you can define which character to use between fields:

tshark -T fields -e ip.src -e ip.dst -e tcp.dstport -E header=y -E separator=;

ip.src;ip.dst;tcp.dstport

64.111.96.38;12.5.200.100;34831

64.111.96.38;12.5.200.100;34831

64.111.96.38;12.5.200.100;34831

Packet statistics

Tshark has solid statistical capability as well. If you need to process a lot of files, sometimes it is help to start with looking at the raw stats. The “-z” switch is used to specify the statistics you wish to analyze. Normally these will be printed at the end of the decode information, but if you use the “-q” switch only the stats will be printed. Here’s an example:

C:\testing>tshark -q -z http,stat, -z http,tree -r test.cap

===============================================================

HTTP/Packet Counter           value            rate         percent

——————————————————————-

Total HTTP Packets            64915       0.048999

HTTP Request Packets           459       0.000346           0.71%

GET                                       24       0.000018           5.23%

HEAD                                   433       0.000327          94.34%

OPTIONS                                 2       0.000002           0.44%

HTTP Response Packets          448       0.000338           0.69%

???: broken                            0       0.000000           0.00%

1xx: Informational                    0       0.000000           0.00%

2xx: Success                         12       0.000009           2.68%

200 OK                                12       0.000009         100.00%

3xx: Redirection                   0       0.000000           0.00%

4xx: Client Error                436       0.000329          97.32%

404 Not Found                  432       0.000326          99.08%

403 Forbidden                      4       0.000003           0.92%

5xx: Server Error                  0       0.000000           0.00%

Other HTTP Packets           64008       0.048314          98.60%

===============================================================

===============================================================

HTTP Statistics

* HTTP Status Codes in reply packets

HTTP 200 OK

HTTP 403 Forbidden

HTTP 404 Not Found

* List of HTTP Request methods

GET  24

OPTIONS  2

HEAD  433

===============================================================

A couple of things stick out in this output. First, we have four 403 errors indicating that someone was attempting to access something they did not have permission to. Also, out of 459 HTTP requests, 432 of them were for non-existent files. We are also seeing a lot of “HEAD” requests which could be a proxy, or could be an attacker attempting to keep from being logged to the Web server’s access log. Clearly this capture file includes some suspect traffic that warrants further investigation.

Tshark can even produce general throughput statistics if you need them. This is an excellent way to check for DoS attacks:

tshark -q -z io,stat,10 -r test.cap

===============================================================

IO Statistics

Interval: 10.000 secs

Column #0:

|   Column #0

Time            |frames|  bytes

000.000-010.000     254    145081

010.000-020.000     145     80003

020.000-030.000     125     65527

030.000-040.000       4       264

===============================================================

Note tshark will print the frame and byte count for any interval specified, defined in seconds. The only problem is that if you are capturing packets off of the wire, stats are not displayed until the capture ends.

Exec Summary

Tshark is an extremely capable packet analysis tool that has surpassed it’s counterparts tcpdump and windump. Combine the extensive decode capability along with the flexible output display, and tshark has become the tool of choice for many packet decoders.

Passively Fingerprinting VMware Virtual Systems

September 15th, 2009

There have been some excellent papers written on how to detect if an operating system is running as a Vmware guest image. Automated tools have even been released to help expedite the process. All assume however that terminal or command prompt access is required to perform the detection. Few people realize that it is possible to passively detect a Vmware guest without any level of system access. In fact, if you know what to look for, you can even identify which version of VMware is hosting the virtual system.

Passive fingerprinting

Passive fingerprinting is a technique where by a remote operating system can be identified by the nuances in the IP packets it produces. For example the payload of Echo-Request packets are pretty unique from operating system to operating system. In a previous post I discussed how different operating systems use different time to live values. By analyzing these variations in the IP packets, you can do a pretty accurate job of identifying the source operating system. In fact many times you can even identify the patch reversion level of the system.

VMware virtual systems

VMware supports two modes for connecting a guest operating system to a network. You can NAT it behind the host, or connect it in bridging mode. Many people think that bridging mode gives the guest OS direct access to the network card, but they forget that VMware is still sitting in between the guest OS and the host system’s drivers to run the NIC. It is not uncommon for this layer to make changes in the IP packets before passing them onto the wire. So if you know what changes to look for, you can detect when VMware software is sitting between an operating system and the local network.

VMware makes multiple changes to the IP datastream. In this post we will only look at two of these changes (window size and scaling), and I’ll leave it to the reader to try and find the others. Further, different versions of VMware will modify the IP datastream in different ways. If you know what changes are unique to each version, you can identify the release level of VMware sitting between the guest OS and the network.

Window size and window scaling

TCP uses window size to identify how much data can be transmitted before a remote system must pause and wait for an acknowledgement. For example if “System A” tells “System B”, “window size = 1400”, “System B” is permitted to send just under one full size Ethernet packet before it must wait. After the 1400 bytes are transmitted, “System B” is not permitted to send any more data till it receives an acknowledgement of that data from “System A”. Window size is adjusted on the fly, so “System A” may eventually advertise “window size = 1600”. This means “System B” can now send a full size packet followed by one small packet before it must pause for an acknowledgement. Window size is a method of flow control to help insure that the receiving system does not see more data than it’s buffers can handle.

Window size dates back to the original implementation of TCP. At the time 16 bits were reserved (bytes 14 and 15 in the TCP header) to specify how many bytes could be transferred before pausing for an acknowledgement. This means we could specify a maximum window size of 65,535 bytes. That’s more than big enough by 1980 standards, but not so great when you are talking about 1 Gb and faster networks. In the early 90’s window scaling was added as a TCP option. Window scaling is only advertised in the first TCP packet sent by a system (SYN or SYN/ACK packet) and is a multiplier for the advertised window size. For example, assume a window size of 1400 bytes. If the window scale is 3, the math would be:

(2*2*2) * 1,400 = 11,200

If the window scale is 4, the math becomes:

(2*2*2*2) * 1,400 = 22,400

So window scaling permits us to now advertise much larger window sizes. This explanation is rather simplistic, but should give the reader a good idea of how window size and scale works. For more info see RFC 1323 or the TCP Wiki.

Backtrack 3 final as a stand-alone host

Here’s a SYN/ACK response to a connection request returned by a Backtrack 3 system (Linux 2.6.21.5 kernel) running on a dedicated host:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.4.80 > 192.168.100.50.44943: S, cksum 0×9552 (correct), 2760872392:2760872392(0) ack 4071399005 win 5792 <mss 1460,sackOK,timestamp 5362662 198395,nop,wscale 5>

Note the initial window scale is 5, which means the actual window size is 32 * 5792 = 185,344. This is pretty common for Linux systems running a post 2.6.17 kernel.

Now let’s take a look at how the window size for Backtrack 3 gets incremented:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.4     5792      5

192.168.100.4     245

192.168.100.4     336

192.168.100.4     426

192.168.100.4     517

192.168.100.4     607

192.168.100.4     698

192.168.100.4     788

192.168.100.4     879

192.168.100.4     969

192.168.100.4     1060

I used tshark to clean up the output so we could focus on the fields we actually care about. The above trace is a large data transfer to 192.168.100.4 (BT3 stand alone) via TCP/80, as show in the above tcpdump trace. Note that we can pretty clearly identify how the window size was incremented over the data session.

Backtrack 3 final, guest on Windows XP Workstation

So let’s take a look at how window size and scaling get affected when the operating system is sitting behind VMware. In this test I ran BT 3 as a guest operating system on a Windows XP workstation host. I used VMware player 2.5.3 and wrote a custom VMX file to run the BT 3 iso.

Here’s an initial SYN/ACK packet:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.7.80 > 192.168.100.50.35245: S, cksum 0x670d (correct), 3795217952:3795217952(0) ack 4108648760 win 5792 <mss 1460,sackOK,timestamp 5210512 208486,nop,wscale 4>

Note that while the initial window size matches the stand-alone BT 3 system, VMware has dropped the window scaling from 5 (32x) to 4 (16x).

Here’s the stream of packets being returned by the guest system:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.7     5792      4

192.168.100.7     490

192.168.100.7     671

192.168.100.7     852

192.168.100.7     1033

192.168.100.7     1214

192.168.100.7     1395

192.168.100.7     1576

192.168.100.7     1757

192.168.100.7     1938

192.168.100.7     2119

Note that after the first packet the negotiated window size appears larger, but when you factor in the window scale multiplier the overall window size ends up being slightly smaller than BT 3 when it is run on a dedicated system. Not all that surprising as virtual systems typically run slower than dedicated systems, but we’ve just tagged two differences we can look for to determine if the host is stand-alone or a guest virtual image.

Backtrack 3 final, guest on Fedora 11

Of course the big question is, “was the window size change due to the Windows XP host or VMware?”. VMware circumvents the Windows IP stack when transmitting on the wire, so the host OS should have no effect on the packet stream. To verify this, let’s try the same test using the same version of VMware player, only this time we’ll use Linux as the hosting operating system. Here’s the SYN/ACK:

IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.100.13.80 > 192.168.100.50.43363: S, cksum 0x6ef7 (correct), 3423175369:3423175369(0) ack 483782889 win 5792 <mss 1460,sackOK,timestamp 579990 366817,nop,wscale 4>

Here’s the stream of packets being returned from TCP/80 on the guest BT 3 system:

[root@fubar ~]# tshark -n -T fields -e ip.src -e tcp.window_size -e tcp.options.wscale_val dst host 192.168.100.50

Capturing on eth0

192.168.100.13    5792      4

192.168.100.13    490

192.168.100.13    671

192.168.100.13    852

192.168.100.13    1033

192.168.100.13    1214

192.168.100.13    1395

192.168.100.13    1576

192.168.100.13    1757

192.168.100.13    1938

192.168.100.13    2119

Note the results are identical to when BT3 was the guest operating system on Windows XP. So while this methodology will not identify the host operating system being used, we can consistently determine if BT3 is running on a dedicated system or as a guest via VMware. While it is entirely possible to modify a dedicated system to appear to be a virtual guest, the opposite does not appear to be feasible.

Exec Summary

By having an in-depth understanding of IP and the nuances generated by different operating systems, it is possible to determine if an OS is running on a dedicated system or as a guest via VMware. Modifications made by the VMware wrapper leave behind tell tale clues. A smart packet weenie can leverage these clues to determine the virtualization status of the source operating system.

Hiding A Backdoor Behind An Active Windows Listening Port

September 3rd, 2009

Its a common technique. You suspect one of your systems has been compromised, so you run a port scanner against the system. The hope is that if the system is backdoored you will identify an undocumented listening port. But what if a clever attacker hides the backdoor in plain site? What if they hide the backdoor behind an existing port you actually expect to see listening on the system? Unfortunately this is far too easy to accomplish if the host is running Windows.

The UNIX way

On a UNIX or Linux system, listening ports are always opened exclusively. In other words, if a mail process opens up TCP/25 to accept inbound mail connections, any other application that attempts to listen on TCP/25 will receive a bind error and be denied access. The only way the second application can bind to TCP/25, is to first kill off the mail process using the port. So only one application at a time is ever permitted access to a listening socket.

The Windows way

In Windows, exclusive access to a listening port is optional. Its perfectly legal to have multiple applications listening on the same TCP or UDP port. This of course can simplify the process of setting up a backdoor behind an existing application.

With Windows, exclusivity of socket access is actually a process that has been evolving over the years. Windows 95, 98 and ME actually had no way to open a port exclusively. There was nothing one application could do to prevent another from attaching to the same listening port. For versions of Windows between NT and XP, it is possible to request exclusive port access via the SO_EXCLUSIVEADDRUSE option. If exclusive access is not specifically requested however, the default is to fall back to providing shared access.

With Windows 2003 and later, the default is exclusive port access unless both applications specifically request shared access via the SO_REUSEADDR option. The exception is that if the same account is used to launch both applications, shared access is still possible unless exclusive access is specifically requested. This means that if a savvy attacker can launch their backdoor via the system account, this enhanced security will usually be defeated.

Try it yourself

To test how this works, you’ll need a copy of the Windows version of Netcat. Copy nc.exe into one of the directories in your path statement. Once you do, open a command prompt and try the following command:

C:\>nc -l -p 135

Can’t grab 0.0.0.0:135 with bind

We just told Netcat to listen (-l) on TCP port (-p) 135. When Windows first boots up, Microsoft RPC opens TCP/135 with the SO_EXCLUSIVEADDRUSE option set. Thus if any application (like Netcat) tries to listen on TCP/135, it receives the bind error shown above. This is the type of behavior we would like to see all of the time.

Now try opening a port that is not already in use:

C:\>nc -l -p 1234

It should appear as if the command is hanging (command prompt does not return). This tells you that Netcat is now listening on port TCP/1234. To verify this, open a second command prompt and type the command:

C:\>netstat -an | find “1234″

TCP 0.0.0.0:1234 0.0.0.0:0 LISTENING

This shows us that Netcat is now listening on TCP/1234 on all network interfaces.

When Netcat opens a listening port it does not request exclusive access to the port. This means it is possible to bind Netcat to an existing listening port that has not already been opened exclusively. To test this, open a third and fourth command prompt. Enter the same Netcat command into each that you run within the first command prompt. Each command should execute without error.

Does this mean Netcat is listening multiple times? Go back to the second command prompt window and try the following command:

C:\>netstat -aon | find “1234″

TCP 0.0.0.0:1234 0.0.0.0:0 LISTENING 448

TCP 0.0.0.0:1234 0.0.0.0:0 LISTENING 3044

TCP 0.0.0.0:1234 0.0.0.0:0 LISTENING 3016

We’ve added in the “o” switch which prints out the process ID for the application holding open the listening port. Note that a different instance of Netcat is responsible for each of the open listening ports. Because Netcat does not require exclusive access to the port, we can listen on the port multiple times.

So what happens when someone tries to connect to the port? Try it and find out using the following command:

C:\>nc 127.0.0.1 1234

This is a test

You should see the text “This is a test” appear in only one instance of the Netcat listeners. If you leave this session active and open (yet another) command prompt and type:

C:\>nc 127.0.0.1 1234

This is test 2

You should see this text appear in a different instance of the Netcat listener.

What have we learned

Since Windows does not require applications to have exclusive access to a listening port, its possible to have a malicious backdoor sitting behind a legitimate application, on the same listening port. This makes the backdoor undetectable when performing a port scan. In order to connect to the backdoor you must first busy out the legitimate application with an active session, but this is easy to achieve.

Spoofing Your IP Address During A Port Scan – Part 2

August 31st, 2009

In my last post I discussed an idle scan and how it can permit an attacker to mask their IP address during a port scan. In this installment we’ll look at some traces, as well as discuss how to identify when an idle scan has been used against your network.

Monitoring the IP ID increment

Let’s start by looking at the packets that were monitoring the IP ID field on the Windows system. Here is what our probe packet looked like:

07:22:15.367140 IP (tos 0×0, ttl 64, id 63897, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.1.2203 > 192.168.100.2.0: ., cksum 0xeca2 (correct), win 512

A few of these fields are kind of interesting. The TTL value is set to “64”, which suggests a Linux or UNIX system. Since we are raw writing the packet directly to the wire this value is actually controlled by hping. 64 just happen to be the program default, but we could change the value to anything we wish with the “-t” switch.

The target address is printed as “192.168.100.2.0”, which means the packet was sent to TCP port 0 at IP address 192.168.100.2. Windows does not offer services on TCP port 0 but that’s OK. We’re not actually trying to connect to the Windows system. We just need it to send us an IP packet so we can check the IP ID value. If we wanted to hit a different port, we could use hping’s “-p” switch.

The “.,” after the target specification means that no TCP flags were set within the packet. This is referred to as a null packet and would never occur during normal IP communications. For this reason, may NIDS, NIPS and firewalls will flag it. We created a null packet because we did not tell hping to sent any of the TCP flags. For example adding the “-S” switch would have turned on the SYN flag, “-A” would turn on the acknowledgment flag, and so on.

All of our probe packets to the Windows systems would look similar to this one. The only values that would change are the time stamp (obviously), the TCP source port and the CRC checksum value.

Here’s what the responses look like coming back from the Windows system:

07:22:15.367296 IP (tos 0×0, ttl 128, id 108, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.0 > 192.168.100.1.2203: R, cksum 0xc431 (correct), 0:0(0) ack 918250228 win 0

 

07:22:16.367453 IP (tos 0×0, ttl 128, id 109, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.0 > 192.168.100.1.2204: R, cksum 0xfa78 (correct), 0:0(0) ack 2127488152 win 0

 

07:22:17.367763 IP (tos 0×0, ttl 128, id 110, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.0 > 192.168.100.1.2205: R, cksum 0x2b9f (correct), 0:0(0) ack 1611256374 win 0

The important value here is the IP ID, identified as “id”. In the first packet it is set to 108, and then the value increments by +1 for each subsequent packet (109 then 110). As long as we continue to see an uninterrupted sequence in the IP ID, we know the Windows system is not transmitting any other packets except the ones that are part of this session.

Probing a closed port

Remember that as part of our scan, we will be spoofing the source IP address of the Windows system when target ports on a remote system. A change in the IP ID increment is what tells us we found an open port.

Let’s take a look at one of these can packets:

10:30:28.852602 IP (tos 0×0, ttl 64, id 41256, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.1266 > 192.168.100.3.79: S, cksum 0x97a6 (correct), 1704542340:1704542340(0) win 512

Note the source IP address is that of the Windows system (192.168.100.2). We know this could not have come from the Windows system however because the TTL is set to 64. So this is a packet we generated from 192.168.100.1 using a second instance of hping.

Also note we have targeted TCP port 79 and have turned on the SYN flag. Here is the response we received from the remote target:

10:30:28.852839 IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) 192.168.100.3.79 > 192.168.100.2.1266: R, cksum 0xbb1a (correct), 0:0(0) ack 1704542341 win 0

Note the “R” value that tells us the reset flag was turned on in the TCP header. We also see “ack” which tells us the acknowledgement flag was turned on as well. This is an error packet informing the source that the TCP port is in a closed state. Also note the packet is being transmitted to the Windows system since that was the source IP address in the probe packet.

So how will the Windows system respond? You may remember from my last post that the RFCs state you should never respond to an error packet. Even though the Windows system has no idea what the target is talking about, it will quietly discard this error packet. In other words, the Windows system WILL NOT send a response. This means we should see no change in the IP ID increment.

Probing an open port

Now let’s take a look at what happens when we target a port that is actually in an open state. Here’s the probe packet. Note it is pretty similar to the last one, except now we are checking TCP port 80.

10:29:46.947964 IP (tos 0×0, ttl 64, id 15249, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.1984 > 192.168.100.3.80: S, cksum 0x476b (correct), 947341260:947341260(0) win 512

Here is the response from the target. Note that we see an “S” instead of an “R” which means they SYN flag is turned on rather than the reset flag. This is the targets way of informing the source that TCP port 80 is open and has an application sitting behind it servicing connections.

10:29:46.953333 IP (tos 0×0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 44) 192.168.100.3.80 > 192.168.100.2.1984: S, cksum 0x0de6 (correct), 262246932:262246932(0) ack 947341261 win 5840 <mss 1460>

So this time the Windows system receives a packet saying “Sure, you can connect to TCP port 80”. This is a problem for the Windows system because it has no connection entry saying that it actually tried to connect to port 80 on the target. Without the connection entry, Windows has no way of knowing which application requested the session. With this in mind, it does the only thing it can do; transmit a reset packet to the target killing the session. Here is what the packet looked like:

10:29:46.953439 IP (tos 0×0, ttl 128, id 176, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.1984 > 192.168.100.3.80: R, cksum 0x5df1 (correct), 947341261:947341261(0) win 0

Note that Windows stamped the next available IP ID in the IP header (176). So had we been monitoring the IP ID increment on the Windows system in a different session, we would see: 173, 174, 175, 177, 178, 179. In other words, we would have seen that 176 was missing (+2 from the previous packet). This would be our indication that the TCP/80 probe sent to the target hit an open port.

Identifying an idle scan

So an idle scan does an excellent job of masking the true source IP address of a port scan. This means that without access to log information on the Windows host (something that will not exist if the attack has done their homework), we will never be able to discover the true source IP address of the attack.

Is there a way to tell an idle scan was performed rather than a straight forward port scan so we know if it is worth investigating the source IP address? There are in fact a couple of clues we can look for.

Probe retries

If the targeted system is sitting behind a firewall, you will notice a difference in the number of probe retries vs. a normal port scanner. When someone runs a normal port scanner, the scanner will typically hit open ports only once but firewalled ports multiple times. This is because the firewalled ports return no response. The scanner will make multiple attempts to ensure the packets are not just getting lost. Since the open port will actually respond, only one probe packet is required.

When an idle scan is performed, the opposite is true. If a firewalled port is probed there will be no IP ID increment change on the Windows system. So only one check is required to confirm this is not an active listening port. When an open port is probed however, we’ll detect a change in the Windows system’s IP ID increment. While we think this is due to us finding an open port, it could just as likely be because the Windows system sent a broadcast. So we will want to perform multiple probes against the open port to ensure the Windows IP ID data remains constant.

So:

  • Probe firewalled ports more often than open ports = normal port scanner
  • Probe open ports more often that firewalled ports = idle scan

Timing

Normal port scans be extremely fast. It is not uncommon to see hundreds of probe packets per second. With an idle scan however, we have to take things a bit slower. This is because we must check the IP ID of the spoofed address, send the probe packet, permit enough time for the probe to elicit a response, give the spoofed system enough time to respond with a reset if required (thus using up an IP ID), and then check the IP ID again to see if the increment has changed.

Try to perform all of these steps too quickly, and you will pay the price in accuracy.  For example nmap supports idle scans with the “-sI” switch. The default extremely fast scanning speed works fine if all of the systems are on the same switched network. If the hosts are 15 hops away from each other on the Internet however, my experience has been the accuracy rating drops off considerably unless you slow down the scanning speed.

So:

  • Hundreds of packets per second = normal port scanner
  • A dozen or less packets per second = possible idle scan, could just be a slow scan

TTL field

Let’s look again at the spoofed probe packet as well as the legitimate reset response sent by the Windows system:

10:29:46.947964 IP (tos 0×0, ttl 64, id 15249, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.1984 > 192.168.100.3.80: S, cksum 0x476b (correct), 947341260:947341260(0) win 512

 

10:29:46.953439 IP (tos 0×0, ttl 128, id 176, offset 0, flags [none], proto TCP (6), length 40) 192.168.100.2.1984 > 192.168.100.3.80: R, cksum 0x5df1 (correct), 947341261:947341261(0) win 0

You may have noticed the TTLs are way off from each other. If both of these packets had in fact originated from 192.168.200.2 then they would both have the same starting TTL value. The fact that they are different tells us that one of them is spoofed.

If you are monitoring the perimeter of the target system and notice that the SYN and reset packets have different TTL values, that’s a clue that you may be experiencing an idle scan. Now, as mentioned earlier, it is entirely possible to set the TTL to what ever we wish in our spoofed packet. So a smart attacker is simply going to set their starting TTL to 128 to better mimic the Windows system. If they do this, your only hope is that the attacker and the Windows system are a different number of hops away. In other words, if you consistently see the SYN packets with a TTL of 115, but the resets consistently have a TTL of 113, you are most likely looking at an idle scan.

If the TTLs do match, we can’t entirely rule out an idle scan. The attacker may just be that good.

So:

  • TTLs of SYNs and RSTs are consistent but do not match = idle scan
  • TTLS of SYNs and RSTs match = normal port scanner or smart idle scanner

IP ID value

The best way to tag an idle scan is leverage the same thing the attacker uses, namely the IP ID value. Take another look at those last two decoded packets. Note that the IP ID in the first one is 15249 but in the second it is 176. If you are seeing an idle scan, you will notice the rest packets consistently have an IP ID of +2 (same thing the attacker sees), but the SYN packets will have some completely unrelated value. The IP ID increment in the SYN packets might be predictable or it might be random. Quite honestly it does not matter. What does matter is that you can spot some pattern in IP ID of the rest packets that does not jive with the IP IDs in the SYN packets.

So:

  • IP ID of SYN and RST packets appear related = normal port scanner
  • IP ID increment of RST does not match SYNs = idle scan

Exec Summary

Hopefully you now have a better understanding of what is happening on the wire when an attacker performs an idle scan. If we are targeted with an idle scan, we cannot determine the true source IP address of the attacker. The best we can hope to accomplish is to determine that the scan is an idle scan vs. a normal port scan. There are a number of tell tale clues in packets, the best of which is the IP ID value.

Network Mapping Through A Firewall – Part 3

August 26th, 2009

In my last two posts I talked about two different methods that can be used to map a network through a firewall. The first leveraged ICMP time exceeded in transit errors, while the second used the IP record route option. In both posts I also gave possible solutions for preventing an attacker from using these techniques against your network.

In both cases however, supported features available in commercial grade firewalls limited our security options. In this third and final part of the series, I will cover how to properly prevent these attacks if you are using an open source firewall. I will specifically be using Netfilter, but many of the techniques are applicable to pf as well.

What is Netfilter?

Netfilter is the stateful inspection firewall that is included in every modern distribution of Linux. If you have a copy of Linux, you also have a copy of Netfilter. Netfilter is sometimes referred to as iptables, but this is because iptables is the name of the binary you use to manipulate the Netfilter rulebase. Netfilter is an extremely capable firewall with too many features to cover in this post. I highly recommend you check out some of the FAQs and tutorials as they do an excellent job of describing many of the features.

Controlling tcptraceroute

In the first post I described how tools like tcptraceroute could punch through an open firewall rule to map the network sitting behind it. With commercial firewalls, we were limited to controlling the flow of outbound ICMP time exceeded in transit errors.

With Netfilter, we have the ability to control traffic based on the TTL value. We can look for a specific value, or a value above or below a certain threshold. The supported switches are:

  • -m ttl –ttl-eq = Match packets with a TTL of a specified value
  • -m ttl –ttl-gt = Math packets with a TTL higher than a specified value
  • -m ttl –ttl-lt = Match packets with a TTL below a specified value

Here is a possible Netfilter rule we can use:

iptables -A FORWARD -m ttl –ttl-lt 5 -j DROP

This rule would be processed prior to any permit rules in the rulebase. The rule simply checks the TTL value to see if it is less than 5. If so, the packet is dropped. Since the lowest TTL used by a modern OS is 64, and most systems are about 15 hops away from each other on the Internet, we should never inadvertently filter out legitimate traffic.

Here’s tcptraceroute running though a regular firewall:

[root@fubar ~]# tcptraceroute -n -f 1 -m 5 -q 1 -S 10.1.4.10 80
Selected device eth0, address 10.1.1.10, port 39142 for outgoing packets
Tracing the path to 10.1.4.10 on TCP port 80 (http), 5 hops max
1 10.1.2.2 0.353 ms
2 10.1.4.1 0.450 ms
3 10.1.4.10 [open] 0.586 ms

And here is what tcptraceroute sees once we implement the above Netfilter rule:

[root@fubar ~]# tcptraceroute -n -f 1 -q 1 -S 10.1.4.10 80
Selected device eth0, address 10.1.1.10, port 54531 for outgoing packets
Tracing the path to 10.1.4.10 on TCP port 80 (http), 30 hops max
1 10.1.2.2 10.175 ms
2 10.1.4.1 0.464 ms
3 *
4 *
5 *
6 *
7 10.1.4.10 [open] 1.007 ms

Note that once we start filtering on TTL value, the appearance of our perimeter changes. Without the rule an attacker could enumerate or IP addressing scheme. Even if we filtered outbound TimeX packets, they would still know the proper hop count. The Netfilter rule makes it much more difficult to accurately identify our network layout.

Adding in some deception

One of the more powerful features of Netfilter is the ability to customize reject messages. While most firewalls reject packets by returning an administratively prohibited error message, Netfilter lets you choose from a number of different unreachable error codes. This makes for some interesting possibilities. For example, consider the following rule:

iptables -A FORWARD -m ttl –ttl-lt 5 -j REJECT –reject-with icmp-host-unreachable

This rule tells Netfilter that whenever it sees a packet with a TTL less than 5, it should return an ICMP destination host unreachable packet. In other words, Netfilter will impersonate a router and tell the transmitting system that the target host is off-line. Here’s an example of tcptraceroute output once this rule has been implemented:

[root@fubar ~]# tcptraceroute -n -f 1 -q 1 -S 10.1.4.10 80
Selected device eth0, address 10.1.1.10, port 47555 for outgoing packets
Tracing the path to 10.1.4.10 on TCP port 80 (http), 30 hops max
1 10.1.2.2 0.299 ms
2 10.1.4.1 0.450 ms
3 10.1.4.1 0.403 ms !H

Compare this output to the first tcptraceroute output shown above. Note that line 3 is now different. With a regular firewall, hop three was a response from the target host. In this output however, it appears the upstream router is returning an ICMP host unreachable (designated as “!H”) signifying the host is off-line. Since tcptraceroute thinks the host is off-line, it gives up trying and never actually reaches the target host.

So while this technique is a bit of security through obscurity, it is effective at disabling a tool that would normally punch right through a firewall. Since regular traffic would not have an abnormally low TTL value, it does not match this rule and is unaffected.

Controlling record route

In my second post in this series I talked about record route and how it can be leveraged to map through a firewall. I discussed that the range of the tool is limited (max 8 hops, 3 if you want hop info in both directions), but that there are ways for an attacker to get around this restriction. I also mentioned that commercial firewalls typically do not give you the ability to control record route traffic.

With Netfilter, there is support for controlling IP options via the ipv4options module (http://netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.html). The supported switches are:

  • -m ipv4options –ssrr = Match packets with strict source routing set
  • -m ipv4options –lsrr = Match packets with loose source routing set
  • -m ipv4options –rr = Match packets with record route set
  • -m ipv4options –ts = Match packets with timestamp set
  • -m ipv4options –ra = Match packets with router-alert set
  • -m ipv4options –any-opt = Match packets with at least one IP option set

Here’s an example of a rule that would block packets with the source route option set:

iptables -A FORWARD -m ipv4options –rr -j REJECT –reject-with icmp-host-unreachable

Note we are sending back an ICMP host unreachable in response. This is in order to shutdown the tool mapping our network.

Exec Summary

While commercial firewall excel at centralized management and selecting pleasing colors for their graphical interface, they usually pale in comparison to open source firewalls with regards to controlling traffic on the wire. In order to protect their networks, firewall administrators need greater control of the IP header than simply scrutinizing the source and destination IP address.