Saturday, October 3, 2015

Python - Using Paramiko to SSH to routers and switches

In the previous post, I demonstrated how to telnet to a router or a switch ( or any other telnet-able device) using python. But since most of the production network devices uses SSH, I'll be showing how to do that using a library using Paramiko module for Python.

There are many other way to SSH to a router or a switch, one of them is pexpect  but if you're using windows, you'll have to either install cgwin or using Paramiko which has SSH built in right in the module.

Step 1 - Installing Paramiko

There are two way to install Paramiko, the easiest one is using PIP command and this is simple done by opening the command prompt in windows or terminal screen and issuing the following command.

pip install paramiko



Since I already have Paramiko installed, it will just notify me about the needed dependencies and confirm that everything is OK.

One thing though, always make sure that you're connected to the internet directly without being behind a firewall or proxy. I faced many issued not being able to install Paramiko when my PC was behind a proxy.

Step 2 - Using Paramiko in Python

import paramiko
import time
username = 'lab'
password = 'lab123'
IP = '192.168.70.100'
ssh_pre = paramiko.SSHClient()
ssh_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_pre.connect(IP, username=username, password=password)
ssh = ssh_pre.invoke_shell()
ssh.send('show version | no-more \n')
time.sleep(3)
output = ssh.recv(65535)
ssh.close()
print output


Here's the output

--- JUNOS 14.1R1.10 built 2014-06-07 09:37:07 UTC
show version | no-more 
lab@vMX> show version | no-more 
Hostname: vMX
Model: vmx
Junos: 14.1R1.10
JUNOS Base OS Software Suite [14.1R1.10]
JUNOS Base OS boot [14.1R1.10]
JUNOS Crypto Software Suite [14.1R1.10]
JUNOS Online Documentation [14.1R1.10]
JUNOS Kernel Software Suite [14.1R1.10]
JUNOS Packet Forwarding Engine Support (M320) [14.1R1.10]
JUNOS Packet Forwarding Engine Support (M/T/EX Common) [14.1R1.10]
JUNOS Routing Software Suite [14.1R1.10]
JUNOS Runtime Software Suite [14.1R1.10]
JUNOS Services AACL PIC package [14.1R1.10]
JUNOS Services Application Level Gateway [14.1R1.10]
JUNOS Services Application Level Gateway (xlp64) [14.1R1.10]
JUNOS Services Application Level Gateway (xlr64) [14.1R1.10]
JUNOS AppId Services PIC Package [14.1R1.10]
JUNOS Services AppId PIC package (xlr64) [14.1R1.10]
JUNOS Border Gateway Function PIC package [14.1R1.10]
JUNOS Services Captive Portal and Content Delivery PIC package [14.1R1.10]
JUNOS Services HTTP Content Management PIC package [14.1R1.10]
JUNOS Services HTTP Content Management PIC package (xlr64) [14.1R1.10]
JUNOS IDP Services PIC Package [14.1R1.10]
JUNOS Packet Forwarding Engine Trio Simulation Package [14.1R1.10]
JUNOS Services JFLOW PIC package [14.1R1.10]
JUNOS Services JFLOW PIC package (xlp64) [14.1R1.10]
JUNOS Services LL-PDF PIC package [14.1R1.10]
JUNOS MobileNext PIC package [14.1R1.10]
JUNOS MobileNext PIC package (xlr64) [14.1R1.10]
JUNOS Services Mobile Subscriber Service Container package [14.1R1.10]
JUNOS Services Mobile Subscriber Service PIC package (xlr64) [14.1R1.10]
JUNOS Services NAT PIC package [14.1R1.10]
JUNOS Services NAT PIC package (xlp64) [14.1R1.10]
JUNOS Services NAT PIC package (xlr64) [14.1R1.10]
JUNOS Services PTSP PIC package [14.1R1.10]
JUNOS Services RPM PIC package [14.1R1.10]
JUNOS Services RPM PIC package (xlp64) [14.1R1.10]
JUNOS Services Stateful Firewall PIC package [14.1R1.10]
JUNOS Services Stateful Firewall PIC package (xlp64) [14.1R1.10]
JUNOS Services Stateful Firewall PIC package (xlr64) [14.1R1.10]
JUNOS BSG PIC package [14.1R1.10]
JUNOS Services Crypto Base PIC package [14.1R1.10]
JUNOS Services Crypto Base PIC package [14.1R1.10]
JUNOS Services Crypto Base PIC package(xlr64) [14.1R1.10]
JUNOS Services IPSec PIC package [14.1R1.10]
JUNOS Services IPSec PIC package [14.1R1.10]
JUNOS Services IPSec PIC(xlr64) package [14.1R1.10]
JUNOS Services SSL PIC package [14.1R1.10]

lab@vMX> >>> 


as you can see, the script is fairly easy a few lines of code and you're done.

let's take the code line by line


First you import Paramiko module and time module ( which I will explain why in just a moment)
import paramiko 
import time

then you define your variables which are the username, password and the router IP

username = 'lab'password = 'lab123'IP = '192.168.70.100'

Then we made the variable ssh_pre to resemble the SSHClient class in the paramiko module

ssh_pre = paramiko.SSHClient()

now this is why I like paramiko, when ever you first connect to a ssh server, there has to be a key exchange and you have to accept the key in order for the session to be established. What's nice about paramiko is that it can accept whatever the key that is being negotiated with the server automatically. of course from the security prespective this isn't the safest thing to do, but hey, when was the last time you saw someone read the host key?! ;)

ssh_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())

Now we instruct paramiko to connect to the router using the preconfigured usename and password

ssh_pre.connect(IP, username=username, password=password)

The next line is pretty interesting, by default if you're going to use the standard input/out library with paramiko, you'll only be able to send a single line or a command and read it's output. if you need to send another command, you'll have to close the session, reconnect and issue another command. what this command do is give you the ability to send multiple commands and receive the output at once or you can issue a single command and get it's output and another one without closing the session. this is pretty useful since connecting and disconnecting multiple times isn't a really nice thing to do to you're server eh..

so what we did is assign the already configures ssh_pre with the ability to send multiple lines to a new variable called ssh for simplicity

ssh = ssh_pre.invoke_shell()

we now can send a line to the router ( don't forget the \n at the end of the command)

ssh.send('show version | no-more \n')

Then we tell the script to settle down for three seconds using the time module, and the reason we do that is that sometimes the line is very slow and you need to wait for a second before the whole output is printed on the screen. this is how you tell that to a computer otherwise it will send the line and read the buffer in a mere milliseconds while the router is still sending the output.

time.sleep(3)

we assign a variable names output to receive all of the info received by the buffer

output = ssh.recv(65535)

now we close the SSH connection and print the buffer output

ssh.close()
print output


If you found this to be useful for you as a network engineer, please +1 the post and share it to your colleagues. you can also subscribe to my blog via RSS as I'll be posting many scripts that are specifically targeting network engineers






Friday, September 25, 2015

Python - Simple Example Telnet to a Cisco Router

Python has been gaining ground in the networking world for its simplicity. Network engineers rely heavily on utilities that makes planning, provisioning and fact gathering easier. The more the routers and switches you deal with, the more you need a tool to help to accomplish your day to day tasks.

The next example uses telnetlib ( short for telnet library) to connect to other devices via telnet. The device could be a Cisco, Juniper, linux.. virtually anything that has telnet server enabled on it

You can also check how to SSH using paramiko module here

let's first see the whole code

import getpass
import sys
import telnetlib

HOST = "192.168.1.28"user = raw_input("Enter your remote account: ")
password = getpass.getpass()

telnet = telnetlib.Telnet(HOST)

telnet.read_until("sername: ",3)
telnet.write(user + "\n")
if password:
   telnet.read_until("assword:",3)
   telnet.write(password + "\n")

telnet.write("show ip int br\n")
telnet.write("exit\n")
print telnet.read_all()

and here's the output for it

>>> Enter your remote account: cisco
Warning (from warnings module):  File "C:\Python27\lib\getpass.py", line 92    return fallback_getpass(prompt, stream)GetPassWarning: Can not control echo on the terminal.Warning: Password input may be echoed.Password: cisco 
Router#show ip int br
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet1       unassigned      YES unset  administratively down down    
GigabitEthernet2       192.168.1.29    YES DHCP   up                    up      
GigabitEthernet0       192.168.1.28    YES manual up                    up      
Router#exit

>>> 
so here's what happened,

telnetlib has a class in it called Telnet, which is responsible for all the telnet operations.

to ease things up, we're only interested in 4 lines of the code

telnet = telnetlib.Telnet(HOST) abbreviating the telnetlib.Telnet(host) to a single word making it easier to work with

telnet.read_until("something",3) keeps reading the buffer until it reaches the text in green. the integer in blue states the timeout value in seconds

telnet.write("something \n") this writes to the buffer that is sent to the remote device the text in green, always remember to type \n as an enter, otherwise the script will be stuck in that command indefinitly

print telnet.read_all() the difference between this and read_until is that this one keeps reading until the buffer empty, the print command before is to make it appear on your screen
here's the script lines one by one.

  1. We imported three modules which are sys , getpass and telnetlib. the later is the one we're interested in. the telnet module abstracts opening a socket and establishing the telnet session between the PC and the router
  2. The script asks the user to manually input a string to assign to the variable named username.
  3. We assign a variable named telnet to handle telnetlib.Telnet(HOST).
  4. HOST is a string variable containing the IP address of the host router we want to telnet to
  5. We instruct the script to keep reading the output of the telnet session until it reaches "sername" which is username without the leading "U" letter. the reason I do this is because sometimes the letter is capital and sometimes it is small. with that we can just ignore the first letter.
  6. We try to input the password using the getpass module. the reason we use it is that it can control the vty line, no password will show up when you're putting the password but in our case it did.
  7. we then instruct the telnetlib to write to the line the command we want to pass to the router. the \n at the end is the equivilant of pressing the enter key or "new line"
  8. we then instruct the script to read all the text in the receive buffer.

At this moment you'd say, what am I supposed to do with this?! secure-CRT can do this and even more!

well, the point is you managed to get the script to communicate with your router. What you can do next is a lot lot more than you can imagine. in the next blog-post, i'll show you how you can connect to multiple routers and get some info from it, maybe the version of the software, modules, interfaces.. etc..

Feel free adding me to your RSS reader or subscribing directly to blogger as I will be doing a whole series about python. also sharing  this blog for people interested in networking and scripting would be a nice thing to do :)



Friday, March 6, 2015

IOS-XR - Upgrading rommonA


The below is the procedure to upgrade rommonA of a MSC in IOS-XR. be sure that when you upgrade, you only upgrade one type of rommon then reload the location before upgrading to the other. meaning, if you decided to upgrade all rommonA, then you should reload all MSC/PLIM after upgrading rommonA then you can proceed with upgrading the other rommon.

Of course if you don't know what you're doing you might screw up the MSC and PLIM. make sure you consult Cisco if you don't know what's going on.

The only problem with upgrading rommonA is that the command is hidden, don't know why exactly. anyhow, here's how you do it

I decided to print a log of all goodies before till after the upgrade

RP/0/RP0/CPU0:CRS(admin)#show hw-module fpd location all 
Wed Mar  4 17:47:24.650 UTC

===================================== ==========================================
                                      Existing Field Programmable Devices
                                      ==========================================
                                        HW                       Current SW Upg/
Location     Card Type                Version Type Subtype Inst   Version   Dng?
============ ======================== ======= ==== ======= ==== =========== ====
0/0/SP       MSC_B                      0.9   lc   rommonA 0       2.10     No 
                                              lc   rommon  0       2.10     No 
--------------------------------------------------------------------------------
0/0/CPU0     CRS1-SIP-800               0.88  lc   fpga1   0       6.00     No 
                                              lc   rommonA 0       2.10     No 
                                              lc   rommon  0       2.10     No 
--------------------------------------------------------------------------------
0/0/1        SPA-1X10GE-L-V2            1.2   spa  fpga1   1       1.11     No 
--------------------------------------------------------------------------------
0/0/2        SPA-1X10GE-L-V2            1.2   spa  fpga1   2       1.11     No 
--------------------------------------------------------------------------------
0/4/CPU0     FP-140G                    0.6   lc   rommonA 0       2.07     Yes
                                              lc   rommon  0       2.10     No 
                                              lc   fpga1   0       0.08     No 
                                              lc   fpga2   0       0.36     No 
--------------------------------------------------------------------------------

RP/0/RP0/CPU0:CRS(admin)#upgrade hw-module fpd ? 
  all     All FPD
  bios    BIOS FPD
  cpld1   CPLD FPD #1
  cpld2   CPLD FPD #2
  cpld3   CPLD FPD #3
  cpld4   CPLD FPD #4
  cpld5   CPLD FPD #5
  cpld6   CPLD FPD #6
  fabldr  Fabric Downloader FPD
  fpga    All FPGA FPD
  fpga1   FPGA FPD
  fpga10  FPGA FPD #10
  fpga11  FPGA FPD #11
  fpga12  FPGA FPD #12
  fpga13  FPGA FPD #13
  fpga14  FPGA FPD #14
  fpga2   FPGA FPD #2
  fpga3   FPGA FPD #3
  fpga4   FPGA FPD #4
  fpga5   FPGA FPD #5
  fpga6   FPGA FPD #6
  fpga7   FPGA FPD #7
  fpga8   FPGA FPD #8
  fpga9   FPGA FPD #9
  ibmc    IBMC FPD
  rommon  Rommon FPD
  rxpod   Rx POD FPD
  txpod   Tx POD FPD
RP/0/RP0/CPU0:CRS(admin)#upgrade hw-module fpd rommonA ? 
                                                     ^
% Invalid input detected at '^' marker.

****you can see that the command doesn't auto-complete or show available commands, you'll have to type it right manually ****


RP/0/RP0/CPU0:CRS(admin)#upgrade hw-module fpd rommonA location 0/4/CPU0
Wed Mar  4 17:48:34.023 UTC

***** UPGRADE WARNING MESSAGE: *****
  *  This upgrade operation has a maximum timout of 160 minutes.  *
  *  If you are executing the cmd for one specific location and  *
  *  card in that location reloads or goes down for some reason  *
  *  you can press CTRL-C to get back the RP's prompt.           *
  *  If you are executing the cmd for _all_ locations and a node *
  *  reloads or is down please allow other nodes to finish the   *
  *  upgrade process before pressing CTRL-C.                     *

% RELOAD REMINDER:
  - The upgrade operation of the target module will not interrupt its normal
    operation. However, for the changes to take effect, the target module
    will need to be manually reloaded after the upgrade operation. This can
    be accomplished with the use of "hw-module <target> reload" command.
  - If automatic reload operation is desired after the upgrade, please use
    the "reload" option at the end of the upgrade command.
  - The output of "show hw-module fpd location" command will not display
    correct version information after the upgrade if the target module is
    not reloaded.
NOTE: Chassis CLI will not be accessible while upgrade is in progress.
Continue? [confirm]
RP/0/RP0/CPU0:Mar  4 17:48:52.391 : upgrade_fpd_cli[65858]: %PLATFORM-UPGRADE_FPD-6-STATUS : FPD upgrade started. 
RP/0/RP0/CPU0:Mar  4 17:48:52.402 : upgrade_fpd_cli[65858]: %PLATFORM-UPGRADE_FPD-6-STATUS : FPD upgrade started. 


FPD upgrade in progress on some hardware, reload/configuration change 
on those is not recommended as it might cause HW programming failure 
and result in RMA of the hardware.


Starting the upgrade/download of following FPD:
=========== ==== ======= ======= =========== =========
                                   Current    Upg/Dng
Location    Type Subtype Upg/Dng   Version    Version
=========== ==== ======= ======= =========== =========
0/4/CPU0    lc   rommonA upg         2.07        2.10   
------------------------------------------------------
RP/0/RP0/CPU0:Mar  4 17:48:52.458 : upgrade_fpd_cli[65858]: %PLATFORM-UPGRADE_FPD-6-STATUS_LOC : Upgrade is going on: FPD upgrade sent to location  node0_4_CPU0 
LC/0/4/CPU0:Mar  4 17:48:52.493 : lc_fpd_upgrade[250]: %PLATFORM-UPGRADE_FPD-6-START : Starting to upgrade rommonA subtype image from 2.07 to 2.10 for on this card on location 0/4/CPU0 
LC/0/4/CPU0:Mar  4 17:49:31.893 : lc_fpd_upgrade[250]: %PLATFORM-UPGRADE_FPD-6-PASSED : Successfully upgrade rommonA subtype image for on this card on location 0/4/CPU0 
Successfully upgraded   rommonA for              FP-140G on location    0/4/CPU0 from  2.07 to  2.10
RP/0/RP0/CPU0:Mar  4 17:49:31.896 : upgrade_fpd_cli[65858]: %PLATFORM-UPGRADE_FPD-6-STATUS_LOC : Upgrade is going on: FPD upgrade completed for location  node0_4_CPU0 
RP/0/RP0/CPU0:Mar  4 17:49:34.469 : upgrade_fpd_cli[65858]: %PLATFORM-UPGRADE_FPD-6-STATUS : FPD upgrade completed. 


FPD upgrade has ended.

#####AFTER THE upgrade is done, you should reload the location#####

RP/0/RP0/CPU0:CRS(admin)#reload location 0/4/CPU0
Wed Mar  4 17:50:03.802 UTC

Preparing system for backup. This may take a few minutes especially for large configurations.
[Done]
Proceed with reload? [confirm]
RP/0/RP0/CPU0:Mar  4 17:50:06.377 : shelfmgrv2[392]: %PLATFORM-SHELFMGRV2-4-BRINGDOWN_REQUEST_RCVD : process reload running on node0_RP0_CPU0 requested node node0_4_CPU0 to be PROGRAM RELOAD. reason: [User initiated reload] 
RP/0/RP0/CPU0:CRS(admin)#LC/0/4/CPU0:Mar  4 17:50:11.384 : ingressq[237]: %FABRIC-INGRESSQ-6-LINK_DOWN : Ingressq: Link 0 of Asic Instance 0 has been administratively shut down. 
LC/0/4/CPU0:Mar  4 17:50:11.399 : ingressq[237]: %PLATFORM-CIH-1-ASIC_ERROR_HARD_RESET_START : ingressq[0]: HARD_RESET needed 0x130a000b 
LC/0/4/CPU0:Mar  4 17:50:11.399 : ingressq[237]: %FABRIC-INGRESSQ_DLL-3-RESET : HARD RESET Resetting IQM ASIC Device with halt status 10 due to int_lns_all_link_down 
LC/0/4/CPU0:Mar  4 17:50:11.415 : ingressq[237]: %PLATFORM-CIH-2-ASIC_ERROR_HARD_RESET : ingressq[0]: A bp-err error has occurred causing  halt. 0x130a000b   
LC/0/4/CPU0:Mar  4 17:50:11.436 : fabricq_mgr[181]: %PLATFORM-CIH-1-ASIC_ERROR_RETRAIN_LINK : fabricq[1]: A link-err error has occurred causing  packet drop persistent. 0x111d0014   
RP/0/RP0/CPU0:Mar  4 17:50:16.381 : invmgr[259]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/4/CPU0, state: BRINGDOWN 
LC/0/4/CPU0:Mar  4 17:50:16.424 : i2c_server[58]: %PLATFORM-I2C-7-HANDLE_PARK_MUX : Putting the I2C mux in default state in preparation for card reload: started 
LC/0/4/CPU0:Mar  4 17:50:16.424 : i2c_server[58]: %PLATFORM-I2C-7-HANDLE_PARK_MUX : Putting the I2C mux in default state in preparation for card reload: finished 
LC/0/4/CPU0:Mar  4 17:50:16.424 : i2c_server[58]: %PLATFORM-I2C-7-HANDLE_PARK_MUX : Putting the I2C mux in default state in preparation for card reload: replied back after 
RP/0/RP0/CPU0:Mar  4 17:50:31.398 : shelfmgrv2[392]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/4/CPU0: MBI tftp:/disk0/hfr-os-mbi-5.1.3/lc/0x500064/mbihfr-lc-x86e.vm validated 
RP/0/RP0/CPU0:Mar  4 17:51:36.805 : invmgr[259]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/4/CPU0, state: IOS XR RUN 
LC/0/4/CPU0:Mar  4 17:50:49.794 : i2c_server[58]: %PLATFORM-I2C-7-BOARD_POWERUP : Sending DCDC Enable to shelfmgr. 
LC/0/4/CPU0:Mar  4 17:51:08.928 : i2c_server[58]: %PLATFORM-I2C-7-BOARD_POWERUP : Received DCDC Enable from shelfmgr. 
LC/0/4/CPU0:Mar  4 17:51:12.222 : i2c_server[58]: %PLATFORM-I2C-7-BOARD_POWERUP : MotherBoard Powered Up. 
LC/0/4/CPU0:Mar  4 17:51:12.344 : pciesvr[69]: %PLATFORM-PCIE-7-INIT_INFO : PCIe server bus enumeration started. 
LC/0/4/CPU0:Mar  4 17:51:12.360 : pciesvr[69]: %PLATFORM-PCIE-7-INIT_INFO : PCIe server initialization completed. 
LC/0/4/CPU0:Mar  4 17:51:12.399 : init[65540]: %OS-INIT-7-MBI_STARTED : total time 25.463 seconds 
LC/0/4/CPU0:Mar  4 17:51:34.413 : sysmgr[82]: %OS-SYSMGR-5-NOTICE : Card is COLD started  
LC/0/4/CPU0:Mar  4 17:51:34.770 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 47.834 seconds 
LC/0/4/CPU0:Mar  4 17:51:35.038 : sysmgr[375]: %OS-SYSMGR-6-INFO : Backup system manager is ready  
LC/0/4/CPU0:Mar  4 17:51:56.523 : sysmgr[82]: %OS-SYSMGR-7-DEBUG : sysmgr_admin_plane_check:SYSMGR_PLANE_ADMIN Notification sent.   
LC/0/4/CPU0:Mar  4 17:52:07.201 : rsi_agent[315]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Invalid to Customer Facing 
LC/0/4/CPU0:Mar  4 17:52:07.207 : rsi_agent[315]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Invalid to Not Interested 
LC/0/4/CPU0:Mar  4 17:52:27.581 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 4 
LC/0/4/CPU0:Mar  4 17:52:31.822 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 0 
LC/0/4/CPU0:Mar  4 17:52:35.967 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 1 
LC/0/4/CPU0:Mar  4 17:52:40.113 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 2 
LC/0/4/CPU0:Mar  4 17:52:44.258 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 3 
LC/0/4/CPU0:Mar  4 17:52:48.404 : plim_xge_otn_flex[295]: %L2-PLIM-6-XFP_OIR : Optic Interface Module inserted for port 5 
LC/0/4/CPU0:Mar  4 17:52:53.172 : egressq[160]: %PLATFORM-CIH-1-ASIC_ERROR : egressq[0]: A link-err error has occurred causing  packet drop transient. 0x151d0022   
LC/0/4/CPU0:Mar  4 17:53:10.330 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/5, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.333 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/4, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.333 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/3, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.333 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/2, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.333 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/1, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.333 : ifmgr[197]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/4/0/0, changed state to Administratively Down 
LC/0/4/CPU0:Mar  4 17:53:10.335 : cfgmgr-lc[142]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/4/0' has been restored. 
LC/0/4/CPU0:Mar  4 17:53:23.309 : 6-10ge-wlo-flex[245]: %L2-SPA-5-STATE_CHANGE : SPA in bay 2 type 10xGE SPA Initing   
RP/0/RP0/CPU0:Mar  4 17:53:30.057 : invmgr[259]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/4/2, state: OK 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/9, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/9, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/8, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/8, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/7, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/7, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/6, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/6, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/5, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/5, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/4, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/4, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/3, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/3, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/2, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/2, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/1, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/1, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/4/2/0, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.099 : ifmgr[197]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/4/2/0, changed state to Down 
LC/0/4/CPU0:Mar  4 17:53:31.101 : cfgmgr-lc[142]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/4/2' has been restored. 
LC/0/4/CPU0:Mar  4 17:53:31.750 : spa_ge_v2(2)[329]: %L2-OPTICS-2-TRANSCEIVER_LOC : Transceiver location is int Gige/4/2/1 
LC/0/4/CPU0:Mar  4 17:53:31.751 : spa_ge_v2(2)[329]: %L2-OPTICS-2-XCVR_CODE : xcvr code is 0x8 
LC/0/4/CPU0:Mar  4 17:53:31.751 : spa_ge_v2(2)[329]: %L2-OPTICS-2-VID : VID is  
LC/0/4/CPU0:Mar  4 17:53:31.751 : spa_ge_v2(2)[329]: %L2-OPTICS-2-PID : PID is  
LC/0/4/CPU0:Mar  4 17:53:31.751 : spa_ge_v2(2)[329]: %L2-OPTICS-2-PN : PN is  
LC/0/4/CPU0:Mar  4 17:53:31.853 : spa_ge_v2(2)[329]: %PLATFORM-XCVR-6-ENABLE : Transceiver(GigabitEthernet :rack:0/slot:4/bay:2/port:1) ENABLED 
LC/0/4/CPU0:Mar  4 17:53:32.024 : rsi_agent[315]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Customer Facing to Core Facing 
LC/0/4/CPU0:Mar  4 17:53:32.027 : rsi_agent[315]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Not Interested to Core Facing 
RP/0/RP0/CPU0:Mar  4 17:53:35.080 : invmgr[259]: %PLATFORM-INV-6-CARD_OIRIN : OIR: Card 0/4/2 inserted 

***The PLIM is up now and all SPAs and ports are detected. Now let's check the SW ver of the MSC***

RP/0/RP0/CPU0:CRS(admin)#show hw-module fpd location all 
Wed Mar  4 17:54:05.002 UTC

===================================== ==========================================
                                      Existing Field Programmable Devices
                                      ==========================================
                                        HW                       Current SW Upg/
Location     Card Type                Version Type Subtype Inst   Version   Dng?
============ ======================== ======= ==== ======= ==== =========== ====
0/0/SP       MSC_B                      0.9   lc   rommonA 0       2.10     No 
                                              lc   rommon  0       2.10     No 
--------------------------------------------------------------------------------
0/0/CPU0     CRS1-SIP-800               0.88  lc   fpga1   0       6.00     No 
                                              lc   rommonA 0       2.10     No 
                                              lc   rommon  0       2.10     No 
--------------------------------------------------------------------------------
0/0/1        SPA-1X10GE-L-V2            1.2   spa  fpga1   1       1.11     No 
--------------------------------------------------------------------------------
0/0/2        SPA-1X10GE-L-V2            1.2   spa  fpga1   2       1.11     No 
--------------------------------------------------------------------------------
0/4/CPU0     FP-140G                    0.6   lc   rommonA 0       2.10     No 
                                              lc   rommon  0       2.10     No 
                                              lc   fpga1   0       0.08     No 
                                              lc   fpga2   0       0.36     No 
--------------------------------------------------------------------------------


now you're good to go. 

Saturday, January 17, 2015

BGP fast-external-fallover

By default, BGP fast external fallover is enabled on Cisco IOS. What this feature do is allow BGP neighbors on a directly connected interface to annound the neighbor down as soon as the carrier signal is lost on that interface. This allows faster convergence in case of link failure since the failure will be detected instantiously, but the down side for it is that a flapping link might introduce a problem for that matter.


Examining the simple topology above. R1 and R2 are EBGP peers using the directly connected interface between them.

R1#show ip bgp summary

BGP router identifier 10.1.2.1, local AS number 1

BGP table version is 1, main routing table version 1

 

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd

10.1.2.2        4            2       9      10        1    0    0 00:05:23        0



The peering is up and everything seems fine. As mentioned earlier, the fast-external-failover feature is enabled by default, so when the link between those two routers goes down, the peering will instantly go down.

R1(config)#int f0/0
R1(config-if)#shut
R1(config-if)#
*Jan 17 01:36:11.143: BGP: tbl IPv4 Unicast:base Service reset requests
*Jan 17 01:36:11.143: BGP: tbl IPv4 Multicast:base Service reset requests
*Jan 17 01:36:11.147: BGP: 10.1.2.2 reset due to Interface flap
*Jan 17 01:36:11.151: %BGP-5-ADJCHANGE: neighbor 10.1.2.2 Down Interface flap
*Jan 17 01:36:11.151: %BGP_SESSION-5-ADJCHANGE: neighbor 10.1.2.2 IPv4 Unicast topology base removed from session  Interface flap
*Jan 17 01:36:13.115: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
*Jan 17 01:36:13.419: BGP: Regular scanner timer event
*Jan 17 01:36:13.419: BGP: Performing BGP general scanning
*Jan 17 01:36:13.419: BGP: tbl IPv4 Unicast:base Performing BGP Nexthop scanning for general scan
*Jan 17 01:36:13.419: BGP(0): Future scanner version: 14, current scanner version: 13
*Jan 17 01:36:13.419: BGP: tbl IPv4 Multicast:base Performing BGP Nexthop scanning for general scan
*Jan 17 01:36:13.419: BGP(6): Future scanner version: 15, current scanner version: 14
*Jan 17 01:36:14.115: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down



The reason that the BGP  peering went down was the interface flap between R1 and R2, this happened in mere milliseconds.
Now we’ll enable the interface between them and we’ll disable this feature. As a note, the hold time interval for BGP = 60 x 3 which is 180 sec

R1(config)#router bgp 1
R1(config-router)#no bgp fast-external-fallover
R1(config)#interface f0/0
R1(config-if)#shut
*Jan 17 01:43:45.659: BGP: 10.1.2.2 reset due to BGP Notification sent
*Jan 17 01:43:45.663: %BGP-5-ADJCHANGE: neighbor 10.1.2.2 Down BGP Notification sent
*Jan 17 01:43:45.663: %BGP-3-NOTIFICATION: sent to neighbor 10.1.2.2 4/0 (hold time expired) 0 bytes
*Jan 17 01:43:45.671: BGP: tbl IPv4 Unicast:base Service reset requests
*Jan 17 01:43:45.671: BGP: tbl IPv4 Multicast:base Service reset requests
*Jan 17 01:43:45.675: %BGP_SESSION-5-ADJCHANGE: neighbor 10.1.2.2 IPv4 Unicast topology base removed from session  BGP Notification sent
After three minutes, BGP peering went down due to hold time expiration.

So the logical question arise, should I leave this feature enabled or should I disable it? Well, that is dependent on many factors.
Depending on the link quality, if the feature is enabled and the link flaps a lot, that will cause many some instability of the BGP routing table, even more; remote service providers might dampen the routes you’re advertising and the more it flaps the greater the penalty on those routes.