What Does it Mean to be a Certain Layer Protocol?

When I used to teach the CompTIA courses at the cyclic rate, one of the concepts that students often struggled with was: “What Layer of the OSI model does the ____ protocol operate at?”  It sometimes felt like it was just rote memorization, but it is important to remember the questions actually have a functional reason behind why they were asked.  Let’s take ARP (Address Resolution Protocol) for example.  ARP was one of the protocols that always tripped me up when I was a student because, ARP goes out and asks the network: “If I have IP address X.X.X.X, which MAC address belongs to it?”  I’m pretty sure as a student I would mentally toss a coin because IP addresses are Layer 3, but MAC addresses are Layer 2.  When I got to know the protocol interaction better, I learned it was a Layer 2 protocol, but it would have been a lot easier to just look at the packets (frames) to find out the answer.  Reason number 6,745,522 why I wish I got into Wireshark much MUCH earlier in my career.  Here’s what ARP looks like:

The important part for determining which layer a protocol operates at is the bottom left.  Where it says “Frame” that is the overall size.  Where it says “Ethernet II” that is your Layer 2 protocol.  After that is the ARP information, but there is no Layer 3 info in there.  If there was, it would look like this:

Looking back at the bottom left, you can see that in addition to “Ethernet II” which is your Layer 2 protocol, we now have “Internet Protocol Version 4” which is our Layer 3 protocol.  The protocol we were looking at in the previous graphic was ARP (Layer 2 protocol), where the graphic above is ICMP (Layer 3 protocol).  So what does it all mean?  Why do I care if something is a Layer 2 or a Layer 3 protocol?  ARP being a Layer 2 protocol means it will not leave the local subnet.  Just like other protocols, ARP follows a sequence: Question and Answer, like I described earlier in the post.  Look at this specific portion:

This is a successful ARP conversation.  Who is X, followed by X is here.  So how can you use this?  First, it should get you thinking along the lines of how the protocol works, and where it could break down.  I was working with some Marines troubleshooting an issue once, which was a CallManager issue.  The CallManager and the phones were on the same subnet (don’t ask), and the phones could register and call each other, but they couldn’t call out and none of the external trunks would come up.  For whatever reason, we were not allowed access to any of the gear, including the CallManager so we used the tool we had: Wireshark.  I had the Marines call between phones, and I could see all the traffic I expected to see.  Then I asked them to try an external call.  This is when I noticed an ARP request from the CallManager go out with no response.  It seemed to be the IP address of a default gateway, so I took the capture to one of the Marines working the switch (which was employed as a Layer 3 switch).  When I asked to verify the default gateway was present and properly configured, we realized the plan had changed from when the CallManager was originally given its IP address and the default gateway was no longer on the CallManager’s VLAN (this is something you would normally want, as off-subnet gateways are a bad thing).  When the CallManager was given an appropriate gateway and the mask was updated, everything started working.  Knowing how ARP worked, and what a capture was supposed to look like saved us a lot of troubleshooting time.

We’ve seen Layer 2 and Layer 3 protocols, let’s take a look at a Layer 7 protocol:

I fired up a quick telnet session between the two lab routers.  Looking at the bottom left corner again, notice that we now have a Layer 4 protocol added to the mix: “Transmission Control Protocol”.

Hopefully this gave you a better understanding of the different OSI model Layers protocols can operate at.  For a little extra fun, here is why you should NEVER use telnet:

That’s right.  In the above screencap, you can see the logon password, enable secret password, and the salted MD5 of the enable secret.  If I would have put a bunch of local users with MD5 passwords, you would have seen the salted versions of all those passwords.