PRI E-mail Notifications – Unadulterated Nerdery

The focus of this blog post is the use of EEM for automated e-mail notification. The idea (and command examples) came from @ucgod, who you should follow on twitter if you are not already. We are going to configure a voice router to send an e-mail when a PRI T-1 goes up and down at layer 2.

PRI T-1s are capable of carrying voice and/or data traffic, but for the purposes of this post, I am just going to concentrate on the voice side. Your question might be: why send an e-mail notification for something that will show up on your router’s command line? In the Marine Corps, and most commercial organizations, the engineers in charge of routing and switching are not in charge of voice (and vice versa). Given that access to the command line is restricted, it is important that the voice engineers are given automated notification. The PRI T-1s (if you are not using IP trunking) are the way your voice calls leave your organization, which would be similar to a WAN connection. Just as you would want to know if your WAN connection is down, voice engineers would want to know if the T-1s are down.

First, we will take a look at what the layer 2 messages look like normally. Q921 is the layer 2 for your ISDN (PRI) connection. You will see below each switchboard sends layer 2 keepalives, very similar to the hello messages our routing protocols use. In this case, my voice router (on behalf of the CallManager) is sending to and receiving from a REDCOM switch. The REDCOM switch sends the same type of messages you would see from your local TELCO. It looks like this:

What we will do is have an e-mail sent to our voice engineer when these keepalives stop being received, and when they are restored following an outage.

We need to start things off by defining some parameters for the e-mail to work with EEM. The examples below show us saying who the e-mail will go to, which e-mail server we will use and who the e-mail will be from. Sadly I did not have access to an e-mail server when I labbed this out, so we won’t actually get to see the e-mails.

event manager environment _email_to address@domain.com
event manager environment _email_server 192.168.100.10
event manager environment _email_from router@domain.com

Next we will need to define the CLI username who will execute the EEM commands. Make sure this username is in the local user database. I also gave this username level 15 access. Tying the local username to EEM will look like:

event manager session cli username “aishelpdesk”

Now we will define our EEM settings. EEM is based off of an “event” which triggers an “action”. You can have multiple actions per event if you would like (I have examples of this in a previous post). The events and actions are defined within an EEM applet. Here is an example of the EEM applet for when the PRI goes down at layer 2. Below you can see the action is “down at layer 2”, which triggers an e-mail that says what has happened and who you should contact.

event manager applet ISDN_L2_Down
event syslog pattern “%ISDN-6-LAYER2DOWN: Layer 2 for Interface”
action 1.0 mail server “$_email_server” to “$_email_to” from “$_email_from” subject “PRI went Down.” body “PRI has gone down. Contact support at 877-XXX-XXXX for circuit ID 38/HCFU/XXXXX//XX.”

The next applet will define an event of the PRI coming back up at layer 2. The action defined is to send an e-mail reminding you to close your trouble ticket if you had opened one.

event manager applet ISDN_L2_Up
event syslog pattern “%ISDN-6-LAYER2UP: Layer 2 for Interface”
action 1.0 mail server “$_email_server” to “$_email_to” from “$_email_from” subject “PRI Is Back Up” body “PRI has come back up. If a ticket with provider was opened, please close it.”

Now that we have completed our config, it’s time to verify it works. As I mentioned before, I do not have access to a mail server, so we will just be able to verify that the EEM event triggers the action. We can verify this by debugging EEM. The command would be:

debug event-manager

Next, we will need to do something to bring the T-1 down. You could unplug the T-1 from the VWIC card, or shutdown the D channel, either one would work. Here is what the debugging messages looked like when I shutdown the D channel:

As you can see from the output, EEM is triggered but fails because it cannot contact my (fictional) mail server. Since there is no mail server, EEM did exactly what we expected. You can even see it lets you know the execution of the applet failed.

Lastly, we will check and ensure the second applet we wrote works. In this case, you bring the D channel back up or reconnect the cable you unplugged. As with any other debug, be sure you turn it off when you are done. Here is what this should look like:

undebug all

Again, EEM fails to contact my mail server and tells us it failed to execute. I’ll update this post when we get a mail server to play with, but this should give you a good idea of what your configuration would look like.

***UPDATE***

I setup Exchange 2010 and tested these configs. Originally it failed. After getting an error message, I decided to dig a bit deeper using “debug event manager action mail”. The EEM script was good to go, but I kept receiving a “user not authenticated” error. After some google engineering, it appeared I needed to create a receive connector for the router to use. I specified the router’s IP address, unchecked all authentication and it STILL failed. Knowing how strange Windows can be sometimes, I checked into the “permission groups” tab and clicked the “anonymous” and “exchange servers”. Finally success!

Leave a Reply Cancel reply