English

Understanding the Layered MPLS Troubleshooting Model

date_range 23-Mar-20

Problem

Description: The layered MPLS troubleshooting model is a disciplined approach to investigating problems with an MPLS network. Figure 1 illustrates the layers in the model, and the commands you can use to structure your investigation. Because of the complexity of the MPLS network, you can obtain much better results from your investigations if you progress through the layers and verify the functioning of each layer on the ingress, egress, and transit routers before moving on to the next layer.

Solution

Figure 1 shows the layered MPLS troubleshooting model that you can use to troubleshoot problems with your MPLS network.

Layered MPLS Network Troubleshooting
Model

As you move from one layer of the model to the next, you verify the correct functioning of a different component of the MPLS network and eliminate that layer as the source of the problem.

Physical Layer

When you investigate the physical layer, you check that the routers are connected, and the interfaces are up and configured correctly. To check the physical layer, enter the show interfaces, show interfaces terse, and ping commands. If there is a problem in the physical layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command. For more information on checking the physical layer, see Checklist for Verifying the Physical Layer.

Data Link Layer

When you investigate the data link layer, you check the encapsulation mode, for example, Point-to-Point Protocol (PPP) or Cisco High-Level Data Link Control (HDLC); PPP options, for example, header encapsulation; frame check sequence (FCS) size; and whether keepalive frames are enabled or disabled.To check the data link layer, enter the show interfaces extensive command. If there is a problem in the data link layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command. For more information on checking the data link layer, see Checking the Data Link Layer and the Junos Interfaces Operations Guide.

IP Layer

When you investigate the IP layer, you verify that interfaces have correct IP addressing, and that the interior gateway protocol (IGP) neighbor adjacencies are established. To check the IP layer, enter the show interfaces terse, show ospf neighbor extensive, and show isis adjacency extensive commands. If there is a problem in the IP layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command.

IGP Layer

When you investigate the IGP layer, you verify that the the Open Shortest Path First (OSPF) or Intermediate System-to-Intermediate System (IS-IS) protocols are configured and running correctly.

If you have the OSPF protocol configured, you must check the IP layer first, and then the OSPF configuration. When you investigate the OSPF layer, you check that the protocol, interfaces, and traffic engineering are configured correctly. To check the OSPF layer, enter the show configuration protocols ospf and show ospf interface commands. If the problem exists in the OSFP layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command. For more information about checking the OSPF layer, see Verifying the OSPF Protocol.
If you have the IS-IS protocol configured, because IS-IS and IP are independent of each other, it doesn’t matter which one you check first. When you check the IS-IS configuration, you verify that IS-IS adjacencies are up, and the interfaces and IS-IS protocol are configured correctly. To check the IS-IS layer, enter the show isis adjacency, show configuration protocols isis, and show isis interfaces commands. If the problem exists in the IS-IS layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command. For more information about checking the IS-IS layer, see Verifying the IS-IS Protocol.

Note
The IS-IS protocol has traffic engineering enabled by default.

RSVP and MPLS Layers

After you have both the IP and IGP layers functioning and the problem is still not solved, you can begin to check the Resource Reservation Protocol (RSVP) and MPLS layers to determine if the problem is in one of these layers.

When you investigate the RSVP layer, you are checking that dynamic RSVP signaling is occurring as expected, neighbors are connected, and interfaces are configured correctly for RSVP. To check the RSVP layer, enter the show rsvp session, show rsvp neighbor, and show rsvp interface commands. If there is a problem in the RSVP layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command.
When you investigate the MPLS layer, you are checking whether the LSP is up and functioning correctly. To check the MPLS layer, enter the show mpls lsp, show mpls lsp extensive, show route table mpls.0,show route address, traceroute address, and ping mpls rsvp lsp-name detail commands. If there is a problem in the MPLS layer, take appropriate action to fix it; then check that the LSP is operating as expected using the show mpls lsp extensive command.

BGP Layer

If the problem persists after you have checked the RSVP and MPLS layers, you must verify that the Border Gateway Protocol (BGP) is working correctly. There is no point in checking the BGP layer unless the LSP is established because BGP uses the MPLS LSP to forward traffic. When you check the BGP layer, you verify that the route is present and active, and more importantly, you ensure that the next hop is the LSP. To check the BGP layer, enter the traceroute host-name, show bgp summary, show configuration protocols bgp, show route destination-prefix detail, and show route receive protocol bgp neighbor-addresscommands. For more information on checking the BGP layer, see Checking the BGP Layer.

In reality, you could start at any level of the MPLS model to investigate a problem with your MPLS network. However, a disciplined approach, as the one described here, produces more consistent and reliable results.

Figure 2 illustrates the basic network topology used in the following topics that demonstrate how to troubleshoot an MPLS network:

The MPLS network consists of the following components:

Router-only network with SONET interfaces
MPLS protocol enabled on all routers, with interfaces selectively deactivated to illustrate a particular problem scenario
All interfaces configured with MPLS
A full-mesh IBGP topology, using AS 65432
IS-IS or OSPF as the underlying IGP, using one level (IS-IS Level 2) or one area (OSPF area 0.0.0.0)
A send-statics policy on routers R1 and R6, allowing a new route to be advertised into the network
Two LSPs between routers R1 and R6, allowing for bidirectional traffic.

After you have configured an LSP, it is considered best practice to issue the show mpls lsp command to verify that the LSP is up, and to investigate further if you find an error message in the output. The error message can indicate a problem at any layer of the MPLS network.

The LSPs can be ingress, transit, or egress. Use the show mpls lsp command for quick verification of the LSP state, with the extensive option (show mpls lsp extensive) as a follow-up if the LSP is down. If your network has numerous LSPs, you might consider specifying the name of the LSP, using the name option (show mpls lsp name name or show mpls lsp name name extensive).

Action

To begin the investigation of an error in your MPLS network, from the ingress router, enter some or all of the following Junos OS command-line interface (CLI) operational mode commands:

content_copy zoom_out_map
user@host> show mpls lsp
user@host> show mpls lsp extensive
user@host> show mpls lsp name name
user@host> show mpls lsp name name extensive

Sample Output 1

content_copy zoom_out_map
user@R1> show mpls lsp 
Ingress LSP: 1 sessions
To              From            State Rt ActivePath       P     LSPname
10.0.0.6        10.0.0.1        Up     1                  *     R1-to-R6
Total 1 displayed, Up 1, Down 0

Egress LSP: 1 sessions
To              From            State Rt Style Labelin Labelout LSPname 
10.0.0.1        10.0.0.6        Up     0 1 FF       3        - R6-to-R1
Total 1 displayed, Up 1, Down 0

Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0

Sample Output 2

content_copy zoom_out_map
user@R1> show mpls lsp extensive 
Ingress LSP: 1 sessions

0.0.6
  From: 10.0.0.1, State: Up, ActiveRoute: 1, LSPname: R1-to-R6
  ActivePath:  (primary)
  LoadBalance: Random
  Encoding type: Packet, Switching type: Packet, GPID: IPv4
 *Primary                    State: Up
    Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 20)
1.13.2 S 10.1.36.2 S 
    Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt):
          10.1.13.2 10.1.36.2
Dec 28 13:47:29 Selected as active path
Dec 28 13:47:29 Record Route:  10.1.13.2 10.1.36.2
Dec 28 13:47:29 Up
Dec 28 13:47:29 Originate Call
Dec 28 13:47:29 CSPF: computation result accepted
Dec 28 13:46:59 CSPF failed: no route toward 10.0.0.6
Dec 28 13:46:39 Deselected as active
Dec 28 13:46:39 CSPF failed: no route toward 10.0.0.6
Dec 28 13:46:39 Clear Call
Dec 28 13:46:39 ResvTear received
Dec 28 13:46:39 Down
Dec 28 13:46:39 10.1.13.2: Session preempted
Dec 28 13:42:07 Selected as active path
Dec 28 13:42:07 Record Route:  10.1.13.2 10.1.36.2
Dec 28 13:42:07 Up
Dec 28 13:42:07 Originate Call
Dec 28 13:42:07 CSPF: computation result accepted
Dec 28 13:41:37 CSPF failed: no route toward 10.0.0.6
Dec 28 13:41:16 Deselected as active
Dec 28 13:41:16 CSPF failed: no route toward 10.0.0.6
Dec 28 13:41:16 Clear Call
Dec 28 13:41:16 ResvTear received
Dec 28 13:41:16 Down
Dec 28 13:41:16 10.1.13.2: Session preempted
Dec 13 11:50:15 Selected as active path
Dec 13 11:50:15 Record Route:  10.1.13.2 10.1.36.2
Dec 13 11:50:15 Up
Dec 13 11:50:15 Originate Call
Dec 13 11:50:15 CSPF: computation result accepted
Dec 13 11:49:45 CSPF failed: no route toward 10.0.0.6[6 times]
---(more)---[abort]

Sample Output 3

content_copy zoom_out_map
user@R1>            show mpls lsp name R1-to-R6
Ingress LSP: 1 sessions
To              From            State Rt ActivePath       P     LSPname
10.0.0.6        10.0.0.1        Up     1                  *     R1-to-R6
Total 1 displayed, Up 1, Down 0

Egress LSP: 1 sessions
Total 0 displayed, Up 0, Down 0

Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0

Sample Output 4

content_copy zoom_out_map
user@R1>            show mpls lsp name R1-to-R6 extensive
Ingress LSP: 1 sessions

10.0.0.6
  From: 10.0.0.1, State: Up, ActiveRoute: 1, LSPname: R1-to-R6
  ActivePath:  (primary)
  LoadBalance: Random
  Encoding type: Packet, Switching type: Packet, GPID: IPv4
 *Primary                    State: Up
    Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 20)
 10.1.13.2 S 10.1.36.2 S 
    Received RRO (ProtectionFlag 1=Available 2=InUse 4=B/W 8=Node 10=SoftPreempt):
          10.1.13.2 10.1.36.2
   30 Dec 28 13:47:29 Selected as active path
   29 Dec 28 13:47:29 Record Route:  10.1.13.2 10.1.36.2
   28 Dec 28 13:47:29 Up
   27 Dec 28 13:47:29 Originate Call
   26 Dec 28 13:47:29 CSPF: computation result accepted
   25 Dec 28 13:46:59 CSPF failed: no route toward 10.0.0.6
   24 Dec 28 13:46:39 Deselected as active
   23 Dec 28 13:46:39 CSPF failed: no route toward 10.0.0.6
   22 Dec 28 13:46:39 Clear Call
   21 Dec 28 13:46:39 ResvTear received
   20 Dec 28 13:46:39 Down
   19 Dec 28 13:46:39 10.1.13.2: Session preempted
   18 Dec 28 13:42:07 Selected as active path
   17 Dec 28 13:42:07 Record Route:  10.1.13.2 10.1.36.2
   16 Dec 28 13:42:07 Up
   15 Dec 28 13:42:07 Originate Call
   14 Dec 28 13:42:07 CSPF: computation result accepted
   13 Dec 28 13:41:37 CSPF failed: no route toward 10.0.0.6
   12 Dec 28 13:41:16 Deselected as active
   11 Dec 28 13:41:16 CSPF failed: no route toward 10.0.0.6
   10 Dec 28 13:41:16 Clear Call
    9 Dec 28 13:41:16 ResvTear received
    8 Dec 28 13:41:16 Down
    7 Dec 28 13:41:16 10.1.13.2: Session preempted
    6 Dec 13 11:50:15 Selected as active path
    5 Dec 13 11:50:15 Record Route:  10.1.13.2 10.1.36.2
    4 Dec 13 11:50:15 Up
    3 Dec 13 11:50:15 Originate Call
    2 Dec 13 11:50:15 CSPF: computation result accepted
    1 Dec 13 11:49:45 CSPF failed: no route toward 10.0.0.6[6 times]
  Created: Mon Dec 13 11:47:19 2004
Total 1 displayed, Up 1, Down 0

Egress LSP: 1 sessions
Total 0 displayed, Up 0, Down 0

Transit LSP: 0 sessions
Total 0 displayed, Up 0, Down 0

Meaning

The sample output from the ingress router R1 shows that the label-switched path is traversing the network as intended, from R1 through R3 to R6, and another LSP in the reverse direction, from R6 through R3 to R1.

If your network has numerous LSPs, you might consider using the show mpls lsp command for quick verification of the LSP state. and the show mpls lsp name name extensive command to continue your investigation if you find that the LSP is down.

For more information about the status and statistics of the show mpls lsp command, see Checklist for Determining LSP Status. For more information on the availability and valid use of an LSP, see Checklist for Verifying LSP Use.

In all the following topics, the network topology is broken at different layers of the network so that you can investigate various MPLS network problems. The problems presented are not inclusive. Instead, the problems serve to illustrate one possible process of investigation into the different layers of the troubleshooting model.