The Basics of Tunneling

When I first started my CCNP, the topic that was most difficult for me to grasp was tunneling.  I understood the general idea, but had a very difficult time putting it all together.  The part about being able to form neighbor relationships between routers separated by one, or many, Layer 3 devices (routers, firewalls, encryptors, etc.) I completely understood.  It is a bit embarrassing to admit, but it took a pretty long time for me to really understand the tunnel source and tunnel destination portions.  Both are actually pretty simple concepts.

For a simple (GRE) tunnel, all you need is:

• IP address
• Tunnel Source
• Tunnel Destination

When things really stuck for me was when I boiled a tunnel down to what it basically is: an additional Layer 3 header.  Here is a scenario to illustrate:

In the above scenario, I have an instance of EIGRP routing the 192.168.1.0 and 192.168.1.2 networks.  I have a second instance of EIGRP routing the loopbacks (1.1.1.1, 3.3.3.3) and the tunnel interface (10.10.10.0/31).  I’ve used arrows to depict the tunnel source and tunnel destination.  So that you can “see” the difference between normal traffic and from traffic that goes over a tunnel I started a capture on the FastEthernet interface of R1 (router on the left).  After I started the capture, I sent a ping to 192.168.1.2 (which was sent with a source of 192.168.1.0) and a second ping to 3.3.3.3 with a source of 1.1.1.1.  The first ping did not go through the tunnel, the second ping did.  Here is what the first ping looked like:

Under the IPv4 field, you can see the source and the destination is as a described above.  Lets take a look at the ping that went between the loopbacks and through the tunnel:

There are multiple things you should notice about the capture above.  The first is there are multiple IPv4 headers.  The second is that there is an added GRE header. The third major detail is that for the same packet (simple ping), the packet that goes through the tunnel is 24 bytes longer than the packet that did not (114 vs 138).  Those 24 bytes come from the additional IPv4 header (20 bytes) and the GRE header (4 bytes).  The last major thing I want to mention is that tunnel is not synonymous with encrypted.  You can very clearly see the IP addresses that are “inside” of the tunnel (1.1.1.1,3.3.3.3) and Wireshark can tell what type of traffic is going through the tunnel (ICMP).

Now that you are armed with all of this tunnel goodness, what are some of the “gotchas”?  This is not an all inclusive list, but when it is time to start troubleshooting, here are a couple of good places to start.

• The tunnels should use the same IPs for sources and destinations.  Specifically, my source is your destination and my destination is your source.
• Every router from the tunnel source to the tunnel destination MUST have a route to the source and the destination of the tunnel.
• Your tunnel interface should have an adjusted MTU.  1500 bytes is the default.  For the GRE tunnel we configured above, the MTU should be adjusted to 1476 to compensate for the IP and GRE headers that were added.  Depending on the type of tunnel(s) that you configure this number may vary.  The worst thing about this is it will “work” without being changed.  The symptoms may be difficult to notice and eventually strange things will happen that will be difficult to troubleshoot.  If you are unsure what to set your MTU at, send a ping across the tunnel with the DF (Don’t Fragment) bit set.  Continually change the MTU until the pings succeed.  Here is what that would look like:

• I’ve mentioned this in previous posts, but be very careful about recursive routing (getting to the end of a tunnel by going through a tunnel).  This will cause your tunnel to flap and your network to become very unstable.  For this post, I avoided this by using separate instances of EIGRP.
• For your tunnel to show “up/up” you only need to configure your tunnel IP, source, destination and have a route to the tunnel destination in your routing table.  Said another way, “up,up” requires additional verification.  Pinging across the tunnel would work (or if you for an EIGRP neighbor relationship).