Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation

Aggregated Ethernet Interfaces Overview

Link aggregation of Ethernet interfaces is defined in the IEEE 802.3ad standard. The Junos OS implementation of 802.3ad balances traffic across the member links within an aggregated Ethernet bundle based on the Layer 3 information carried in the packet. This implementation uses the same load-balancing algorithm used for per-flow load balancing.

Note: For information about configuring circuit cross-connects over aggregated Ethernet, see Circuit and Translational Cross-Connects Overview.

Platform Support for Aggregated Ethernet Interfaces

You configure an aggregated Ethernet virtual link by specifying the link number as a physical device and then associating a set of ports that have the same speed and are in full-duplex mode. The physical interfaces can be Fast Ethernet, Tri-Rate Ethernet copper, Gigabit Ethernet, Gigabit Ethernet IQ, 10-Gigabit Ethernet IQ, Gigabit Ethernet IQ2 and IQ2-E, or 10-Gigabit Ethernet IQ2 and IQ2-E. Generally, you cannot use a combination of these interfaces within the same aggregated link; however, you can combine Gigabit Ethernet and Gigabit Ethernet IQ interfaces in a single aggregated Ethernet bundle.

Starting with Junos OS Release 13.2, aggregated Ethernet supports the following mixed rates and mixed modes on T640, T1600, T4000, and TX Matrix Plus routers:

  • Member links of different modes (WAN and LAN) for 10-Gigabit Ethernet links.
  • Member links of different rates: 10-Gigabit Ethernet, 40-Gigabit Ethernet, 50-Gigabit Ethernet, 100-Gigabit Ethernet, and OC192 (10-Gigabit Ethernet WAN mode)
  • Member links of 50-Gigabit Ethernet can only be configured using the 50-Gigabit Ethernet interfaces of 100-Gigabit Ethernet PIC with CFP (PD-1CE-CFP-FPC4).
  • Starting with Junos OS Release 13.2, 100-Gigabit Ethernet member links can be configured using the two 50-Gigabit Ethernet interfaces of 100-Gigabit Ethernet PIC with CFP. This 100-Gigabit Ethernet member link can be included in an aggregated Ethernet link that includes member links of other interfaces as well. In releases before Junos OS Release 13.2, the 100-Gigabit Ethernet member link configured using the two 50-Gigabit Ethernet interfaces of 100-Gigabit Ethernet PIC with CFP cannot be included in an aggregated Ethernet link that includes member links of other interfaces.

Going forward:

  • Aggregated Ethernet link with member links of different modes will be referred as 10-Gigabit Ethernet mixed mode aggregated Ethernet link.
  • Aggregated Ethernet link with member links of different rates will be referred as mixed rate aggregated Ethernet link.
  • These aggregated Ethernet links will generically be referred as mixed aggregated Ethernet links.

Table 1 lists the platforms and corresponding hardware components that support mixed aggregated Ethernet bundles.

Table 1: Platform Support Matrix for Mixed Aggregated Ethernet Bundles

Rate and Mode

Supported Platform

Supported FPCs

Supported PICs

10-Gigabit Ethernet LAN and WAN

(WAN rate: OC192)

T640, T1600, T4000, and TX Matrix Plus routers

  • T4000 FPC5 (T4000-FPC5-3D)
  • 10-Gigabit Ethernet LAN/WAN PIC with Oversubscription and SFP+ (PF-24XGE-SFPP)
  • 10-Gigabit Ethernet LAN/WAN PIC with SFP+ (PF-12XGE-SFPP)
  • Enhanced Scaling FPC3 (T640-FPC3-ES)
  • 10-Gigabit Ethernet PIC with XENPAK (PC-1XGE-XENPAK)
  • Enhanced Scaling FPC4 (T640-FPC4-ES)
  • Enhanced Scaling FPC4-1P (T640-FPC4-1P-ES)
  • T1600 Enhanced Scaling FPC4 (T1600-FPC4-ES)
  • 10-Gigabit Ethernet LAN/WAN PIC with SFP+ (PD-5-10XGE-SFPP)
  • 10-Gigabit Ethernet LAN/WAN PIC with XFP (PD-4XGE-XFP)

40-Gigabit Ethernet, 100-Gigabit Ethernet

T4000 and TX Matrix Plus routers

  • T4000 FPC5 (T4000-FPC5-3D)
  • 100-Gigabit Ethernet PIC with CFP (PF-1CGE-CFP)
 

T640, T1600, T4000, and TX Matrix Plus routers

  • Enhanced Scaling FPC4 (T640-FPC4-ES)
  • Enhanced Scaling FPC4-1P (T640-FPC4-1P-ES)
  • T1600 Enhanced Scaling FPC4 (T1600-FPC4-ES)
  • 100-Gigabit Ethernet PIC with CFP (PD-1CE-CFP-FPC4)

    Note: This PIC is available packaged only in an assembly with the T1600-FPC4-ES FPC.

  • 40-Gigabit Ethernet PIC with CFP (PD-1XLE-CFP)

The following routers support a maximum of 16 physical interfaces per single aggregated Ethernet bundle:

  • M120
  • M320
  • All MX Series 3D Universal Edge Routers
  • All T Series routers

All other routers support a maximum of eight physical interfaces per aggregated Ethernet bundle.

On M Series and T Series routers, you can create a maximum of 1024 logical interfaces on an aggregated Ethernet interface.

Aggregated Ethernet interfaces can use interfaces from different FPCs, DPCs, PICs, or MPCs.

Configuration Guidelines for Aggregated Ethernet Interfaces

  • Simple filters are not supported for interfaces in aggregated Ethernet bundles:
    • On M Series routers, simple filters are supported in Gigabit Ethernet Enhanced Intelligent Queuing interfaces only, except when the interface is part of an aggregated Ethernet bundle.
    • On MX Series routers, simple filters are supported in Enhanced Queuing Dense Port Concentrator (EQ DPC) interfaces only, except when the interface is part of an aggregated Ethernet bundle.

    For more information about simple filters, see the Junos OS Class of Service Library for Routing Devices.

  • On the aggregated Ethernet bundle, no IQ-specific capabilities such as MAC accounting, VLAN rewrites, and VLAN queuing are available. For more information about IQ-specific capabilities, see Gigabit Ethernet Accounting and Policing Overview.
  • Aggregated Ethernet interfaces can be either tagged or untagged, with LACP enabled or disabled. Aggregated Ethernet interfaces on MX Series routers support the configuration of flexible-vlan-tagging and native-vlan-id on dual-tagged frames, which consist of the following configuration statements:

    In all cases, you must set the number of aggregated Ethernet interfaces on the chassis. You can also set the link speed and the minimum links in a bundle.

  • When configuring mixed aggregated Ethernet bundles on T640, T1600, T4000, and TX Matrix Plus routers, consider the following:
    • A maximum of 16 member links can be configured to form a mixed aggregated Ethernet link.
    • Link Aggregation Control Protocol (LACP), aggregated Ethernet link protection, and LACP link protection are supported only on mixed aggregated Ethernet link configured on a 100-Gigabit Ethernet PIC with CFP (PD-1CE-CFP-FPC4).
    • Traffic distribution is based on the hash calculated on the egress packet header. Hash range is fairly distributed according to member links’ speed. This guarantees hash fairness but it does not guarantee fair traffic distribution depending on the rate of the egress streams.
    • Packets are dropped when the total throughput of the hash flow exiting a member link (or multiple hash flows exiting a single member link) exceeds the link speed of the member link. This can happen when egress member link changes because of a link failure and the hash flow switches to a member link of speed that is less than the total throughput of the hash flow.
    • Rate-based CoS components such as scheduler, shaper, and policer are not supported on mixed rate aggregated Ethernet links. However, the default CoS settings are supported by default on the mixed rate aggregated Ethernet links.
    • Load balancing is performed at the ingress Packet Forwarding Engine. Therefore, you must ensure that the egress traffic on the aggregated Ethernet link enters through the hardware platforms that support mixed aggregated Ethernet bundles. Table 1 lists the platforms and corresponding hardware components that support mixed aggregated Ethernet bundles.
    • Mixed aggregated Ethernet links can interoperate with non-Juniper Networks aggregated Ethernet member links provided that mixed aggregated Ethernet load balancing is configured at egress.
    • Load balancing of the egress traffic across the member links of a mixed rate aggregated Ethernet link is proportional to the rates of the member links.
    • Egress multicast load balancing is not supported on mixed aggregated Ethernet interfaces.
    • Changing the edit interfaces aex aggregated-ether-options link-speed configuration of a mixed aggregated Ethernet link, which is configured on the supported interfaces of on T640, T1600, T4000, and TX Matrix Plus routers, leads to aggregated Ethernet link flapping.
    • When configuring a mixed aggregated Ethernet link on a 100-Gigabit Ethernet PIC with CFP (PD-1CE-CFP-FPC4), ensure that you add both the 50-Gigabit Ethernet interfaces of the 100-Gigabit Ethernet PIC with CFP to the aggregated Ethernet bundle. Moreover, both these 50-Gigabit Ethernet interfaces must be included in the same aggregated Ethernet bundle.
    • When a mixed aggregated Ethernet link is configured on a 100-Gigabit Ethernet PIC with CFP, changing aggregated Ethernet link protection or LACP link protection configurations leads to aggregated Ethernet link flapping.
    • For a single physical link event of an aggregated Ethernet link configured on a 100-Gigabit Ethernet PIC with CFP, the packet loss performance value is twice the original value because of the two 50-Gigabit Ethernet interfaces of the 100-Gigabit Ethernet PIC with CFP.
    • The show interfaces aex command displays the link speed of the aggregated Ethernet interface, which is the sum of the link speeds of all the active member links.
  • Use the show interfaces aggregate-interface extensive and show interfaces aggregate.logical-interface commands to show the bandwidth of the aggregate. Also, the SNMP object identifier ifSpeed/ifHighSpeed shows the corresponding bandwidth on the aggregate logical interface if it is configured properly.

Published: 2013-11-21