Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Troubleshooting Network Issues

Working with Problems on Your Network

Problem

Description

This checklist provides links to troubleshooting basics, an example network, and includes a summary of the commands you might use to diagnose problems with the router and network.

Solution

Table 1: Checklist for Working with Problems on Your Network

Tasks

Command or Action
Isolating a Broken Network Connection  

  1. Identifying the Symptoms of a Broken Network Connection

ping (ip-address | hostname) show route (ip-address | hostname) traceroute (ip-address | hostname)

  1. Isolating the Causes of a Network Problem

show < configuration | interfaces | protocols | route >

  1. Taking Appropriate Action for Resolving the Network Problem

[edit] delete routing options static route destination-prefix commit and-quit show route destination-prefix

  1. Evaluating the Solution to Check Whether the Network Problem Is Resolved

show route (ip-address | hostname) ping (ip-address | hostname) count 3 traceroute (ip-address | hostname)

Isolating a Broken Network Connection

By applying the standard four-step process illustrated in Figure 1, you can isolate a failed node in the network. Note that the functionality described in this section is not supported in versions 15.1X49, 15.1X49-D30, or 15.1X49-D40.

Figure 1: Process for Diagnosing Problems in Your NetworkProcess for Diagnosing Problems in Your Network

Before you embark on the four-step process, however, it is important that you are prepared for the inevitable problems that occur on all networks. While you might find a solution to a problem by simply trying a variety of actions, you can reach an appropriate solution more quickly if you are systematic in your approach to the maintenance and monitoring of your network. To prepare for problems on your network, understand how the network functions under normal conditions, have records of baseline network activity, and carefully observe the behavior of your network during a problem situation.

Figure 2 shows the network topology used in this topic to illustrate the process of diagnosing problems in a network.

Figure 2: Network with a ProblemNetwork with a Problem

The network in Figure 2 consists of two autonomous systems (ASs). AS 65001 includes two routers, and AS 65002 includes three routers. The border router (R1) in AS 65001 announces aggregated prefixes 100.100/24 to the AS 65002 network. The problem in this network is that R6 does not have access to R5 because of a loop between R2 and R6.

To isolate a failed connection in your network, follow the steps in these topics:

Identifying the Symptoms of a Broken Network Connection

Problem

Description

The symptoms of a problem in your network are usually quite obvious, such as the failure to reach a remote host.

Solution

To identify the symptoms of a problem on your network, start at one end of your network and follow the routes to the other end, entering all or one of the following Junos OS command-line interfaces (CLI) operational mode commands:

Sample Output

Meaning

The sample output shows an unsuccessful ping command in which the packets are being rejected because the time to live is exceeded. The output for the show route command shows the interface (10.1.26.1) that you can examine further for possible problems. The traceroute command shows the loop between 10.1.26.1 (R2) and 10.1.26.2 (R6), as indicated by the continuous repetition of the two interface addresses.

Isolating the Causes of a Network Problem

Problem

Description

A particular symptom can be the result of one or more causes. Narrow down the focus of your search to find each individual cause of the unwanted behavior.

Solution

To isolate the cause of a particular problem, enter one or all of the following Junos OS CLI operational mode command:

Your particular problem may require the use of more than just the commands listed above. See the appropriate command reference for a more exhaustive list of commonly used operational mode commands.

Sample Output

The following sample output is from R2:

Meaning

The sample output shows that all interfaces on R6 are up. The output from R2 shows that a static route [Static/5] configured on R2 points to R6 (10.1.26.2) and is the preferred route to R5 because of its low preference value. However, the route is looping from R2 to R6, as indicated by the missing reference to R5 (10.1.15.2).

Taking Appropriate Action for Resolving the Network Problem

Problem

Description

The appropriate action depends on the type of problem you have isolated. In this example, a static route configured on R2 is deleted from the [routing-options] hierarchy level. Other appropriate actions might include the following:

Solution

  • Check the local router’s configuration and edit it if appropriate.

  • Troubleshoot the intermediate router.

  • Check the remote host configuration and edit it if appropriate.

  • Troubleshoot routing protocols.

  • Identify additional possible causes.

To resolve the problem in this example, enter the following Junos OS CLI commands:

Sample Output

Meaning

The sample output shows the static route deleted from the [routing-options] hierarchy and the new configuration committed. The output for the show route command now shows the BGP route as the preferred route, as indicated by the asterisk (*).

Evaluating the Solution to Check Whether the Network Problem Is Resolved

Problem

Description

If the problem is solved, you are finished. If the problem remains or a new problem is identified, start the process over again.

You can address possible causes in any order. In relation to the network in Isolating a Broken Network Connection, we chose to work from the local router toward the remote router, but you might start at a different point, particularly if you have reason to believe that the problem is related to a known issue, such as a recent change in configuration.

Solution

To evaluate the solution, enter the following Junos OS CLI commands:

Sample Output

Meaning

The sample output shows that there is now a connection between R6 and R5. The show route command shows that the BGP route to R5 is preferred, as indicated by the asterisk (*). The ping command is successful and the traceroute command shows that the path from R6 to R5 is through R2 (10.1.26.1), and then through R1 (10.1.12.1).

Checklist for Tracking Error Conditions

Problem

Description

Table 2 provides links and commands for configuring routing protocol daemon tracing, Border Gateway Protocol (BGP), Intermediate System-to-Intermediate System (IS-IS) protocol, and Open Shortest Path First (OSPF) protocol tracing to diagnose error conditions.

Solution

Table 2: Checklist for Tracking Error Conditions

Tasks

Command or Action

Configure Routing Protocol Process Tracing

  1. Configure Routing Protocol Process Tracing

[edit] edit routing-options traceoptions set file filename size size files number show commit run show log filename

  1. Configure Routing Protocol Tracing for a Specific Routing Protocol

[edit] edit protocol protocol-name traceoptions set file filename size size files number show commit run show log filename

  1. Monitor Trace File Messages Written in Near-Real Time

monitor start filename

  1. Stop Trace File Monitoring

monitor stop filename

Configure BGP-Specific Options

  1. Display Detailed BGP Protocol Information

[edit] edit protocol bgp traceoptions set flag update detail show commit run show log filename

  1. Display Sent or Received BGP Packets

[edit] edit protocol bgp traceoptions set flag update (send | receive) show commit run show log filename

  1. Diagnose BGP Session Establishment Problems

[edit] edit protocol bgp set traceoptions flag open detail show commit run show log filename

Configure IS-IS-Specific Options

  1. Displaying Detailed IS-IS Protocol Information

[edit] edit protocol isis traceoptions set flag hello detail show commit run show log filename

  1. Displaying Sent or Received IS-IS Protocol Packets

[edit] edit protocols isis traceoptions set flag hello (send | receive) show commit run show log filename

  1. Analyzing IS-IS Link-State PDUs in Detail

[edit] edit protocols isis traceoptions set flag lsp detail show commit run show log filename

Configure OSPF-Specific Options

  1. Diagnose OSPF Session Establishment Problems

[edit] edit protocols ospf traceoptions set flag hello detail show commit run show log filename

  1. Analyze OSPF Link-State Advertisement Packets in Detail

[edit] edit protocols ospf traceoptions set flag lsa update detail show commit run show log filename

Configure Routing Protocol Process Tracing

Action

To configure routing protocol process (rpd) tracing, follow these steps:

  1. In configuration mode, go to the following hierarchy level:

  2. Configure the file, file size, number, and flags:

    For example:

  3. Verify the configuration:

    For example:

  4. Commit the configuration:

Note:

Some traceoptions flags generate an extensive amount of information. Tracing can also slow down the operation of routing protocols. Delete the traceoptions configuration if you no longer require it.

  1. View the contents of the file containing the detailed messages:

    For example:

Meaning

Table 3 lists tracing flags and example output for Junos-supported routing protocol daemon tracing.

Table 3: Routing Protocol Daemon Tracing Flags

Tracing Flag

Description

Example Output

all

All operations

Not available.

general

Normal operations and routing table change

Not available.

normal

Normal operations

Not available.

policy

Policy operations and actions

Nov 29 22:19:58 export: Dest 10.0.0.0 proto Static Nov 29 22:19:58 policy_match_qual_or: Qualifier proto Sense: 0 Nov 29 22:19:58 policy_match_qual_or: Qualifier proto Sense: 0 Nov 29 22:19:58 export: Dest 10.10.10.0 proto IS-IS

route

Routing table changes

Nov 29 22:23:59 Nov 29 22:23:59 rtlist_walker_job: rt_list walk for RIB inet.0 started with 42 entries Nov 29 22:23:59 rt_flash_update_callback: flash KRT (inet.0) start Nov 29 22:23:59 rt_flash_update_callback: flash KRT (inet.0) done Nov 29 22:23:59 rtlist_walker_job: rt_list walk for inet.0 ended with 42 entries Nov 29 22:23:59 Nov 29 22:23:59 KRT Request: send len 68 v14 seq 0 CHANGE route/user af 2 addr 172.16.0.0 nhop-type unicast nhop 10.10.10.33 Nov 29 22:23:59 KRT Request: send len 68 v14 seq 0 ADD route/user af 2 addr 172.17.0.0 nhop-type unicast nhop 10.10.10.33 Nov 29 22:23:59 KRT Request: send len 68 v14 seq 0 ADD route/user af 2 addr 10.149.3.0 nhop-type unicast nhop 10.10.10.33 Nov 29 22:24:19 trace_on: Tracing to "/var/log/rpdlog" started Nov 29 22:24:19 KRT Request: send len 68 v14 seq 0 DELETE route/user af 2 addr 10.10.218.0 nhop-type unicast nhop 10.10.10.29 Nov 29 22:24:19 RELEASE 10.10.218.0 255.255.255.0 gw 10.10.10.29,10.10.10.33 BGP pref 170/-101 metric so-1/1/0.0,so-1/1/1.0 <Release Delete Int Ext> as 65401 Nov 29 22:24:19 KRT Request: send len 68 v14 seq 0 DELETE route/user af 2 addr 172.18.0.0 nhop-type unicast nhop 10.10.10.33

state

State transitions

Not available.

task

Interface transactions and processing

Nov 29 22:50:04 foreground dispatch running job task_collect for task Scheduler Nov 29 22:50:04 task_collect_job: freeing task MGMT_Listen (DELETED) Nov 29 22:50:04 foreground dispatch completed job task_collect for task Scheduler Nov 29 22:50:04 background dispatch running job rt_static_update for task RT Nov 29 22:50:04 task_job_delete: delete background job rt_static_update for task RT Nov 29 22:50:04 background dispatch completed job rt_static_update for task RT Nov 29 22:50:04 background dispatch running job Flash update for task RT Nov 29 22:50:04 background dispatch returned job Flash update for task RT Nov 29 22:50:04 background dispatch running job Flash update for task RT Nov 29 22:50:04 task_job_delete: delete background job Flash update for task RT Nov 29 22:50:04 background dispatch completed job Flash update for task RT Nov 29 22:50:04 background dispatch running job Flash update for task RT Nov 29 22:50:04 task_job_delete: delete background job Flash update for task RT

timer

Timer usage

Nov 29 22:52:07 task_timer_hiprio_dispatch: ran 1 timer Nov 29 22:52:07 main: running normal priority timer queue Nov 29 22:52:07 main: ran 1 timer Nov 29 22:52:07 task_timer_hiprio_dispatch: running high priority timer queue Nov 29 22:52:07 task_timer_hiprio_dispatch: ran 1 timer Nov 29 22:52:07 main: running normal priority timer queue Nov 29 22:52:07 main: ran 1 timer Nov 29 22:52:07 main: running normal priority timer queue Nov 29 22:52:07 main: ran 2 timers

Configure Routing Protocol Tracing for a Specific Routing Protocol

Action

To configure routing protocol tracing for a specific routing protocol, follow these steps:

  1. In configuration mode, go to the following hierarchy level:

  2. Configure the file, file size, number, and flags:

    For example:

  3. Verify the configuration:

    For example:

  4. Commit the configuration:

  5. View the contents of the file containing the detailed messages:

    For example:

Meaning

Table 4 lists standard tracing options that are available globally or that can be applied to specific protocols. You can also configure tracing for a specific BGP peer or peer group. For more information, see the Junos System Basics Configuration Guide.

Table 4: Standard Trace Options for Routing Protocols

Tracing Flag

Description

all

All operations

general

Normal operations and routing table changes

normal

Normal operations

policy

Policy operations and actions

route

Routing table changes

state

State transitions

task

Interface transactions and processing

timer

Timer usage

Monitor Trace File Messages Written in Near-Real Time

Purpose

To monitor messages in near-real time as they are being written to a trace file.

Action

To monitor messages in near-real time as they are being written to a trace file, use the following Junos OS command-line interface (CLI) operational mode command:

Sample Output

command-name

Stop Trace File Monitoring

Action

To stop monitoring a trace file in near-real time, use the following Junos OS CLI operational mode command after you have started monitoring:

Sample Output