Reliable Policy-based routing

Here’s another lab taking on how to set up reliable policy-based routing. The basic configuration is identical to a previous lab I did about policy-routing, so if you’re interested in the basics I suggest you also take a look there for an easier intro. This one ends up using some pretty nifty functions within route-maps to set up a secondary next-hop tied in with IP SLA-tracking & CDP that I haven’t seen earlier.

Lab requirements:

Basic routing:

  1. Configure a default-route on R3 towards R1s local interface (Gi1.13)
  2. Configure a default-route on R4 & R6 towards R1s local interface (Gi1.146)
  3. Configure a default-route on R5 towards R1s DMVPN-interface (Tu0)
  4. Configure static routes thru the DMVPN-cloud on R3 for R5’s Lo0, and vice versa on R5 for R3’s lo0

Policy-routing:

  1. Configure policy-routing on R1 so that traffic sourced from R4s G1.146 is routed to R3 over Gi1.13
  2. Configure policy-routing on R1 so that traffic sourced from R6s Gi1.146 is routed to R5 over DMVPN

Reliability:

  1. Configure CDP between R1 & R5 over DMVPN-cloud
  2. Configure IP SLA on R1 that confirms connection to R3 every 5 seconds
  3. Modify R1’s policy-routing so that if R1 loses ICMP reachability to R3, traffic from R4 is rerouted to R5 over the DMVPN-cloud
  4. Modify R1’s policy-routing so that if R1 loses R5 as CDP-neighbor, traffic from R6 is rerouted to R3 over the ethernet link (Gi1.13)

Let’s start with the easy stuff, the basic routing is very… basic, no explanation needed.

! Basic-1 R3

ip route 0.0.0.0 0.0.0.0 Gi1.13 155.1.13.1

! Basic-2
! R4

ip route 0.0.0.0 0.0.0.0 Gi1.146 155.1.146.1

! R6

ip route 0.0.0.0 0.0.0.0 Gi1.146 155.1.146.1

! Basic-3 R5

ip route 0.0.0.0 0.0.0.0 Tu0 155.1.0.1

! Basic-4
! R3
ip route 150.1.5.5 255.255.255.255 Tu0 155.1.0.5

! R5
ip route 150.1.3.3 255.255.255.255 Tu0 155.1.0.3

As in the previous lab, R1 has no routes so we won’t be able to pass much traffic yet.  Let’s take a look at setting up the policy-routing prerequisites so we can start testing basic functionality. First we’ll need an access-list to match the source-address of R4 & R6, then we manually set next-hop with a route-map and attach it to our incoming interface.

! Policy-routing R1

ip access-list extended R4
 permit ip host 155.1.146.4 any

ip access-list extended R6
 permit ip host 155.1.146.6 any

route-map RPR permit 10
 match ip address R4
 set ip next-hop 155.1.13.3

route-map RPR permit 20
 match ip address R6
 set ip next-hop 155.1.0.5

interface Gi1.146
 ip policy route-map RPR

We should now have basic connectivity finally, packets from R4 should be routed to R3 first whatever the destination:.

R4#ping 150.1.5.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.1.5.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/6/6 ms

R4#traceroute 150.1.5.5
Type escape sequence to abort.
Tracing the route to 150.1.5.5
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 5 msec 4 msec 4 msec
 2 155.1.13.3 5 msec 5 msec 5 msec
 3 155.1.0.5 5 msec * 5 msec

Packets from R6 should be routed to R5 first whatever the destination:

R6#ping 150.1.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 150.1.3.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/12/27 ms

R6#traceroute 150.1.3.3
Type escape sequence to abort.
Tracing the route to 150.1.3.3
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 5 msec 4 msec 4 msec
 2 155.1.0.5 5 msec 5 msec 5 msec
 3 155.1.0.3 5 msec * 5 msec

Everything is looking fine so far! Now to the tricky part with setting up reliability, cdp is easy though. We just enable it on both of our tunnel interfaces:

! Reliability-1 R1 & R5
cdp run
interface Tu0
 cdp enable

Next requirement was to set up IP SLA that tracks the connection between R1 & R3’s Gi1.13 every 5 seconds. We’ll do this in R1 as it will be the router doing the policy-routing later.

! Reliability-2 R1
ip sla 100
 icmp-echo 155.1.13.3 source-interface Gi1.13
 frequency 5

ip sla schedule start-time now life forever

track 1 ip sla 100 state

Let’s verify that everything’s OK this far:

R1#sh ip sla statistics 100
IPSLAs Latest Operation Statistics
IPSLA operation id: 100
 Latest RTT: 4 milliseconds
Latest operation start time: 17:00:09 UTC Wed Apr 11 2018
Latest operation return code: OK
Number of successes: 63
Number of failures: 0
Operation time to live: Forever

R1#sh track
Track 1
 IP SLA 100 state
 State is Up
 3 changes, last change 00:16:29
 Latest operation return code: OK
 Latest RTT (millisecs) 3
 Tracked by:
 Route Map 0

R5#sh cdp neighbors detail 
-------------------------
Device ID: R1
Entry address(es): 
 IP address: 155.1.0.1
Platform: cisco CSR1000V, Capabilities: Router IGMP 
Interface: Tunnel0, Port ID (outgoing port): Tunnel0
Holdtime : 138 sec

Version :
Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(3)S6, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2017 by Cisco Systems, Inc.
Compiled Mon 24-Jul-17 20:01 by mcpre

advertisement version: 2
Management address(es): 
 IP address: 155.1.0.1

Next step on how to use tracking with policy-routing had me stuck for quite a while, scrolling thru the Cisco Docs I finally found a page that had some examples of what we’re trying to do at:

Configuration Guides -> IP Routing: Protocol-Independent Configuration Guide – > PBR Support for Multiple Tracking Options

First requirement was to use our IP SLA 100 / Track 1 that’s currently verifying connection is up between R1 & R3. Our route-map currently looks like this for reference:

route-map RPR permit 10
 match ip address R4
 set ip next-hop 155.1.13.3
route-map RPR permit 20
 match ip address R6
 set ip next-hop 155.1.0.5

We’ll modify the first entry by removing the static next-hop for R4 and replace it with the command “set ip next-hop verify-availability [next-hop-address sequence track object]”. We also have to add something as a fallback next-hop if the tracker goes down, here the “set ip default” command looks like what we want to use. You can find more info here:

Configuration Guides -> IP Routing: Protocol-Independent Configuration Guide – > Policy-Based Routing Default Next-Hop Routes

“set ip default next-hop ip-address [ip-address]”- Specifies the next hop for routing packets if there is no explicit route for this destination.

Our final route-map for policy-routing R4 will look like this:

route-map RPR permit 10
 match ip address R4
 set ip-next verify availability 155.1.13.3 1 track 1
 set ip default next-hop 155.1.0.5

For R6 I had much bigger problems however. Here we were supposed to use CDP for tracking but I couldn’t manage to find any specific information about it more than that “set ip-next verify availability” has CDP-support. A quick google-search showed that it’s really basic however, we only use the actual command “set ip-next verify availability” and IOS will automatically do CDP-lookups before setting the chosen next-hop, in this case we still need the static set ip next-hop however. The final result will look something like this:

route-map RPR permit 20
 match ip address R6
 set ip next-hop 155.1.0.5
 set ip next-hop verify-availability
 set ip default next-hop 155.1.13.3

Let’s verify again:

R1#sh route-map
route-map RPR, permit, sequence 10
 Match clauses:
 ip address (access-lists): R4 
 Set clauses:
 ip next-hop verify-availability 155.1.13.3 1 track 1 [up]
 ip default next-hop 155.1.0.5
 Policy routing matches: 6 packets, 276 bytes
route-map RPR, permit, sequence 20
 Match clauses:
 ip address (access-lists): R6 
 Set clauses:
 ip next-hop 155.1.0.5
 ip next-hop verify-availability
 ip default next-hop 155.1.13.3
 Policy routing matches: 18 packets, 828 bytes

A traceroute from R4 should always take the route via R3 as long as there’s reachability between R1 & R3:

R4#traceroute 150.1.5.5
Type escape sequence to abort.
Tracing the route to 150.1.5.5
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 8 msec 5 msec 5 msec
 2 155.1.13.3 5 msec 5 msec 5 msec
 3 155.1.0.5 5 msec * 5 msec

Let’s see what happens when we take down the connection between R1 & R3:

R1#debug ip policy 
Policy routing debugging is on

R3(config)#int gi1.13
R3(config-subif)#shut

PBR Nexthop: Unregister tracking for 155.1.13.3, handle: 7F74CD6A6768
PBR Nexthop Callback invoked: 7F74F117B6C0, (155.1.13.3) tableid 0, status: 0, type: SET NEXTHOP TRACKINGmap: RPR, sequence: 10
PBR Control Plane Notification: 155.1.0.5 PBR_CP_SET_DEFAULT_NEXTHOP
Policy NextHop Inquiry: RPR seq: 10, type: SET DEFAULT NEXTHOP Nexthop: 155.1.0.5SW_OBJ_TYPE: 1E, SW_HANDLE: 7F74F1108FE8
PBR CP Notification sent: Type:SET DEFAULT NEXTHOP, 155.1.0.5SW_OBJ_TYPE: 1E, SW_HANDLE: 7F74F1108FE8

Let’s do another traceroute, the traffic should jump directly to R5 instead:

R4#traceroute 150.1.5.5
Type escape sequence to abort.
Tracing the route to 150.1.5.5
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 4 msec 4 msec 4 msec
 2 155.1.0.5 4 msec * 4 msec

Sweet! Let’s try the same for R6, all traffic should always first be redirected to R5:

R6#traceroute 150.1.3.3
Type escape sequence to abort.
Tracing the route to 150.1.3.3
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 5 msec 5 msec 4 msec
 2 155.1.0.5 5 msec 5 msec 5 msec
 3 155.1.0.3 5 msec * 5 msec

Looking good so far. Lets disable CDP on R5s Tunnel0 interface and wait out the hold-timer (180 seconds), not a very fast switchover with default timers…

! R5
interface Tu0
 no cdp enable

After a long wait R5 is no longer showed in cdp-table:

R1#sh cdp neighbors 
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
 S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone, 
 D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID Local Intrfce Holdtme Capability Platform Port ID
R10 Gig 1 172 R I CSR1000V Gig 1
R2 Gig 1 126 R I CSR1000V Gig 1
R3 Gig 1 131 R I CSR1000V Gig 1
R6 Gig 1 132 R I CSR1000V Gig 1
R7 Gig 1 166 R I CSR1000V Gig 1
R4 Gig 1 151 R I CSR1000V Gig 1
R8 Gig 1 150 R I CSR1000V Gig 1
R9 Gig 1 158 R I CSR1000V Gig 1

A traceroute should now go directly to R3 instead (don’t forget to open the interface we previously shutdown).

R6#traceroute 150.1.3.3
Type escape sequence to abort.
Tracing the route to 150.1.3.3
VRF info: (vrf in name/id, vrf out name/id)
 1 155.1.146.1 6 msec 5 msec 3 msec
 2 155.1.0.5 5 msec 5 msec 5 msec
 3 155.1.0.3 5 msec * 5 msec

Not what we were hoping for.. Traffic is for some reason still routed over to R5 even though we have no CDP-reachability. I tried troubleshooting this forever but finally found a notation that there’s apparently a bug in the CSR-1000V routers when it comes to verify-availability & CDP tracking(!!). The config looks OK though from other examples I could find so this is how it’s “supposed” to work at least.. Fun times. 🙂

MDH Lab – IP SLA ICMP & Jitter

Topologi

lab-ipsla

Objectives

  • Configure trunking, VTP, and SVIs.
  • Implement IP SLAs to monitor various network performance characteristics.

Background

Cisco IOS IP service level agreements (SLAs) allow users to monitor network performance between Cisco devices (switches or routers) or from a Cisco device to a remote IP device.

Cisco IOS IP SLAs can be applied to VoIP and video applications as well as monitoring end-to-end IP network performance.In this lab, you configure trunking, VTP, and SVIs. You configure IP SLA monitors to test ICMP echo network  and jitter performance between S1 and each switch.

Genomförande

Hoppar över grundkonfigen för Etherchannels/Vlan/VTP/Trunkar så all L2-konfig är redan klart. Se tidigare inlägg om detta skulle vara intressant..

Vi börjar med att konfa upp L3 för samtliga switchar.

S1

S1(config)#int vlan 1
 S1(config-if)#ip add 172.16.1.1 255.255.255.0
 S1(config-if)#no shut
 S1(config-if)#int vlan 100
 S1(config-if)#ip add 172.16.100.1 255.255.255.0
 S1(config-if)#int vlan 200
 S1(config-if)#ip add 172.16.200.1 255.255.255.0
 S1(config-if)#exit
 S1(config)#ip routing

S2

S2(config)#inte vlan 1
 S2(config-if)#ip add 172.16.1.102 255.255.255.0
 S2(config-if)#no shut

S3

S3(config)#inte vlan 1
 S3(config-if)#ip add 172.16.1.103 255.255.255.0
 S3(config-if)#no shut

Då återstår det endast att sätta upp IP SLA. Enligt labben ska vi konfigurera upp ICMP-Echo & Jitter på S1 mot både S2 & S3.

För att kunna testa just jitter behöver vi konfigurera IP SLA-responders på S2 & S3, detta behövs dock inte om vi endast vill använda ICMP.

S2 & S3

S2(config)#ip sla responder
 S2(config)#ip sla responder udp-echo ipaddress 172.16.1.1 port 2525
S3(config)#ip sla responder
 S3(config)#ip sla responder udp-echo ipaddress 172.16.1.1 port 2525

Vi konfar sedan upp SLA-mätningen på S1. Vi behöver en SLA-instans per mätnings-funktion, då vi vill testa både ICMP & Jitter behövs därför två mot S2 och två mot S3.

S1

S1(config)#ip sla 20
 S1(config-ip-sla-echo)#icmp-echo 172.16.1.102
 S1(config-ip-sla-echo)#frequency 5
 S1(config-ip-sla-echo)#timeout 1000
 S1(config-ip-sla-echo)#exit
 S1(config)#
S1(config)#ip sla 25
 S1(config-ip-sla)#udp-jitter 172.16.1.102 2525
 S1(config-ip-sla-jitter)#frequency 20
 S1(config-ip-sla-jitter)#precision milliseconds
 S1(config-ip-sla-jitter)#timeout 1000
 S1(config-ip-sla-jitter)#exit
 S1(config)#
S1(config)#ip sla 30
 S1(config-ip-sla)#icmp-echo 172.16.1.103
 S1(config-ip-sla-echo)#frequency 5
 S1(config-ip-sla-echo)#timeout 1000
 S1(config-ip-sla-echo)#exit
 S1(config)#
S1(config)#ip sla 35
 S1(config-ip-sla)#udp-jitter 172.16.1.103 2525
 S1(config-ip-sla-jitter)#frequency 20
 S1(config-ip-sla-jitter)#precision milliseconds
 S1(config-ip-sla-jitter)#timeout 1000
 S1(config-ip-sla-jitter)#exit
 S1(config)#end

Vi behöver sedan bara schemalägga och starta våra IP SLA-mätningar.

S1(config)#ip sla schedule 20 life forever start-time now
S1(config)#ip sla schedule 25 life forever start-time now
S1(config)#ip sla schedule 30 life forever start-time now
S1(config)#ip sla schedule 35 life forever start-time now

Vi kan verifiera vår IP-SLA konfig med “sh ip sla configuration”.

S1#sh ip sla configuration 25
IP SLAs, Infrastructure Engine-II.
Entry number: 25
Owner: 
Tag: 
Type of operation to perform: udp-jitter
Target address/Source address: 172.16.1.102/0.0.0.0
Target port/Source port: 2525/0
Type Of Service parameter: 0x0
Request size (ARR data portion): 32
Operation timeout (milliseconds): 1000
Packet Interval (milliseconds)/Number of packets: 20/10
Verify data: No
Vrf Name: 
Control Packets: enabled
Schedule:
 Operation frequency (seconds): 20
 Next Scheduled Start Time: Start Time already passed
 Group Scheduled : FALSE
 Randomly Scheduled : FALSE
 Life (seconds): Forever
 Entry Ageout (seconds): never
 Recurring (Starting Everyday): FALSE
 Status of entry (SNMP RowStatus): Active
Threshold (milliseconds): 5000
Distribution Statistics:
 Number of statistic hours kept: 2
 Number of statistic distribution buckets kept: 1
 Statistic distribution interval (milliseconds): 20
Enhanced History:

Det går även att verifiera på S2 & S3 så att de verkligen tar emot förfrågningar från S1.

S2#sh ip sla responder
IP SLAs Responder is: Enabled
Number of control message received: 8 Number of errors: 0 
Recent sources:
 172.16.1.1 [01:13:22.643 UTC Mon Mar 1 1993]
 172.16.1.1 [01:13:02.644 UTC Mon Mar 1 1993]
 172.16.1.1 [01:12:42.638 UTC Mon Mar 1 1993]
 172.16.1.1 [01:12:22.639 UTC Mon Mar 1 1993]
 172.16.1.1 [01:12:02.641 UTC Mon Mar 1 1993]
Recent error sources:
udpEcho Responder: 
 IP Address Port
 172.16.1.1 2525

För att se resultatet av våra mätningar, använd “show ip sla statistics”.

S1#sh ip sla statistics
Round Trip Time (RTT) for Index 20
 Latest RTT: 8 ms
Latest operation start time: *01:14:11.767 UTC Mon Mar 1 1993
Latest operation return code: OK
Number of successes: 40
Number of failures: 1
Operation time to live: Forever

Round Trip Time (RTT) for Index 25
 Latest RTT: 2 ms
Latest operation start time: *01:13:55.904 UTC Mon Mar 1 1993
Latest operation return code: OK
RTT Values
 Number Of RTT: 10
 RTT Min/Avg/Max: 2/2/4 ms
Latency one-way time milliseconds
 Number of Latency one-way Samples: 0
 Source to Destination Latency one way Min/Avg/Max: 0/0/0 ms
 Destination to Source Latency one way Min/Avg/Max: 0/0/0 ms
Jitter time milliseconds
 Number of SD Jitter Samples: 9
 Number of DS Jitter Samples: 9
 Source to Destination Jitter Min/Avg/Max: 0/1/2 ms
 Destination to Source Jitter Min/Avg/Max: 0/1/1 ms
Packet Loss Values
 Loss Source to Destination: 0 Loss Destination to Source: 0
 Out Of Sequence: 0 Tail Drop: 0 Packet Late Arrival: 0
Voice Score Values
 Calculated Planning Impairment Factor (ICPIF): 0
 Mean Opinion Score (MOS): 0
Number of successes: 11
Number of failures: 0
Operation time to live: Forever

Round Trip Time (RTT) for Index 30
 Latest RTT: 1 ms
Latest operation start time: *01:14:15.038 UTC Mon Mar 1 1993
Latest operation return code: OK
Number of successes: 40
Number of failures: 0
Operation time to live: Forever

Round Trip Time (RTT) for Index 35
 Latest RTT: 2 ms
Latest operation start time: *01:14:04.217 UTC Mon Mar 1 1993
Latest operation return code: OK
RTT Values
 Number Of RTT: 10
 RTT Min/Avg/Max: 1/2/3 ms
Latency one-way time milliseconds
 Number of Latency one-way Samples: 0
 Source to Destination Latency one way Min/Avg/Max: 0/0/0 ms
 Destination to Source Latency one way Min/Avg/Max: 0/0/0 ms
Jitter time milliseconds
 Number of SD Jitter Samples: 9
 Number of DS Jitter Samples: 9
 Source to Destination Jitter Min/Avg/Max: 0/1/1 ms
 Destination to Source Jitter Min/Avg/Max: 0/1/2 ms
Packet Loss Values
 Loss Source to Destination: 0 Loss Destination to Source: 0
 Out Of Sequence: 0 Tail Drop: 0 Packet Late Arrival: 0
Voice Score Values
 Calculated Planning Impairment Factor (ICPIF): 0
 Mean Opinion Score (MOS): 0
Number of successes: 10
Number of failures: 0
Operation time to live: Forever

IP SLA går dock att använda till så mycket mer än att bara köra lite jitter/udp-test sen, exempelvis PBR/HSRP-tracking etc, vilket jag kommer ta upp i senare inlägg. 🙂