Resolve Hardware Issues

Interface Damping

Interface damping and hold-times suppress rapid state changes. There are two methods to configure port damping:

• Static Hold-Time Up/Down Timer

• Exponential Backoff Damping

Static Damping

Configure static hold-time timers per interface and per link state (up and down). These timers are defined in milliseconds and help control physical flapping and reduce its impact on routing protocols. Apply these timers to physical interfaces such as ge-, xe-, et-.

Recommendations:

• Set hold-time up to a value greater than or equal to the IGP SPF hold‑down timer (for example, 5000 ms for IS‑IS by default). This setting prevents short, repeated flaps from reaching the routing process and reduces unnecessary flooding.
• Keep hold-time down at the default value of 0 ms, unless a specific use case requires a delay. Report link‑down events to routing protocols immediately.
• During recovery, hold-time up suppresses subsequent rapid flaps and prevents additional events

Dynamic Damping

Configure interface damping to suppress periodic, long‑duration flaps by using an exponential backoff mechanism similar to BGP.

• Each interface down event adds a penalty of 1000.
• The penalty decays by 50% every half‑life interval while the interface remains stable.
• When the penalty exceeds the suppress threshold, Junos OS suppresses the interface and does not advertise state changes to routing protocols.
• The system restores the interface when the penalty drops below the reuse threshold or when the maximum suppress timer expires.

Configure the following parameters:

• Enable or disable damping (enabled by default)
• half-life—Interval for penalty decay (1–30 seconds, default 5 seconds)
• reuse—Penalty value to exit the suppressed state (1–20,000, default 750)
• suppress—Penalty value to enter the suppressed state (1–20,000, default 3000)
• max-suppress-time—Maximum duration to keep the interface suppressed (1–255 seconds, default 60 seconds)

Example:

Resolve Data-plane Hardware Issues

Hardware Redundancy

The number of forwarding ASICs Packet Forwarding Engine varies by line card model. Each Packet Forwarding Engine operates independently from a forwarding perspective.

For example, an MX10004 chassis with two LC9600 line cards includes six Trio 6 ASICs per line card. Design redundancy based on the hardware architecture and apply cabling guidelines accordingly.

In this example, apply the following rules for the upstream path:

• Distribute interfaces across at least two line cards
• Limit connections to a maximum of two ports per PFE on the same line card

LAG Cabling Rules

Configure a 6 × 400‑GbE LAG between R1 and R10. Distribute three interfaces on each LC9600 line card. On each line card, place two interfaces on one PFE and the remaining interface on a different PFE.

This design improves fault isolation. If a PFE or an entire line card experiences a hardware issue that causes packet loss, you can isolate the affected component without disabling the entire path. You can continue forwarding traffic on the remaining PFEs and minimize routing changes.

Use the same approach for downstream connectivity by distributing interfaces across different PFEs. This design limits the impact of a single failure and maintains network stability.

If available bandwidth becomes insufficient after a failure, shut down the LAG to avoid congestion. Use the minimum-links option to control this behavior. In this example, losing half of the LAG capacity requires shutting down the entire bundle.

LAG configuration example:

Note:

If the number of active interfaces drops below the configured minimum (4), the LAG goes down even if some interfaces remain operational. The LAG stays up with 4 active links and goes down when only 3 links remain.

Error Handling

On MX platforms, the cmerror subsystem manages hardware errors. Errors are classified into two scopes:

• board—Line card components other than the PFE
• pfe—Forwarding ASIC components

Junos categorizes errors by severity:

• Minor—Transient errors that typically do not affect packet forwarding
• Major—Errors that can impact services and affect transit traffic
• Fatal—Errors that indicate hardware failure and can cause traffic loss

By default, Junos OS applies actions based on severity. Use the following command to view default behavior. The system also groups alarms by category.

This command displays the following information:

• threshold—Number of errors required to trigger an action
• action—Actions taken when the system detects the alarm

Supported actions include:

• get-state—Collect additional diagnostic data and store it in /var/tmp
• CM-ALARM—Generate an internal CMERROR notification
• LOG—Generate a syslog message
• RESET—Restart the affected component (Packet Forwarding Engine or line card)
• DISABLE-PFE—Disable the affected Packet Forward Engine

By default, the system disables a Packet Forwarding Engine for major Packet Forwarding Engine errors. Adjust this behavior as needed. Configure actions per line card, severity level, or category under chassis fpc <slot> error.

In this example, configure the system to keep the line card offline for fatal errors and to isolate a faulty ASIC by using the disable-pfe action for major errors. For detailed options, refer to the product documentation.

With this configuration, the system isolates a faulty Packet Forwarding Engine and, based on the design, removes up to two links from the LAG. If another Packet Forwarding Engine that carries upstream traffic fails, the system also disables that Packet Forwarding Engine to prevent traffic impact. In this case, the minimum-links threshold is reached, and the LAG transitions to the down state.

In this scenario, a PFE encounters a hardware issue and the system automatically disables it. The system logs events similar to the following:

You also should check the active alarms under:

You can gather more error information by issuing the next command (referring to the "Identifier" field from the previous command):

Override the severity of specific alarms when required. Use this option to adjust how the system handles errors based on operational experience.

For example, treat a minor alarm as major if it has a greater impact in your environment, or promote a specific major alarm to fatal severity. Configure severity remapping as shown below:

The MX platform (like PTX) offers fine-grained control for handling hardware events at the forwarding ASIC level. Combined with strict engineering design rules, this dramatically increases your network's resiliency and quality.