Understanding Graceful Restart for BGP
Understanding the Long-Lived BGP Graceful Restart Capability
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
Historically, routing protocols and BGP, in particular, have been designed with a focus on correctness, where a significant aspect of the "correctness" is for each network element's forwarding state to converge toward the current state of the network as quickly as possible. For this reason, the protocol was designed to remove state advertised by routers which went down (from a BGP perspective) as promptly as possible. Using BGP Graceful Restart defined in RFC 4724, the fast convergence functionality has been an attempt to rapidly remove "stale" state from the network.
Over a period of time, two contributing factors have caused this method of rapid removal of stale states to be modified and enhanced. The first is the widespread adoption of tunneled forwarding infrastructures, for example MPLS. Such infrastructures eliminate the risk of some types of forwarding loops that can arise in hop-by-hop forwarding, and thereby reduce one of the motivations for strong consistency between forwarding elements. The second is the increasing use of BGP as a transport for data less closely associated with packet forwarding than was originally the case. Examples include the use of BGP for autodiscovery (VPLS [RFC4761]) and filter programming (FLOWSPEC [RFC5575]). In these cases, BGP data assumes a characteristic that is not in line with traditional routing.
It was important to offer network operators the ability to choose to retain BGP data for a longer period when the BGP control plane fails for some reason. Although the properties of BGP Graceful Restart are close to this desired requirement to preserve BGP information for a longer duration, several gaps exist, most notably in maximum time for which "stale" information can be retained—graceful restart imposes a 4095-second upper-bound limitation. Junos OS supports a BGP capability called long-lived graceful restart capability so that stale information can be retained for a longer time across a session reset. It also supports a new BGP community, "LLGR_STALE", to mark such information. Such stale information is to be treated as least-preferred, and its advertisement limited to BGP speakers that support the new capability.
BGP long-lived graceful restart (LLGR) allows a network operator to choose to maintain stale routing information from a failed BGP peer much longer than the existing BGP Graceful Restart facility. This functionality to maintain the BGP routes for a longer time period is in accordance with the IETF draft, Support for Long-lived BGP Graceful Restart—draft-uttaro-idr-bgp-persistence-03. According to this draft, long-lived graceful restart (LLGR) must be explicitly configured per NLRI, and it includes provisions to prevent the spread of stale information to other peers that do not recognize and validate LLGR. The following benefits and operations are caused by LLGR:
Routes from failed nodes are retained for a configured time period (on the order of days).
You can examine per-NLRI LLGR negotiation states using appropriate show commands.
You can view whether LLGR is currently in effect for a peer, and if it is effective, the period after which it expires.
Stale routes retained by LLGR are explicitly marked in the output of the
show bgp neighborcommand.Stale routes learned from other neighbors are explicitly marked in the output of the
show bgp neighborcommand (using well-defined communities).
Although the LLGR methodology can be applied to a number of different scenarios, one specific scenario is the salient objective of this feature. In a scenario in which a loss of connectivity between a route reflector and a client occurs, including intermittent connectivity which can cause a connection to be reset before the entire RIB can be transmitted, such a failure does not result in a restart. Also, such a phenomenon does not imply that any sort of connectivity problem between the clients and the next-hops advertised by the route reflector exists. It is anticipated that a typical long-lived restart time is in the order of 12 hours.
All of the behavioral guidelines and operational points described in the IETF draft, draft-uttaro-idr-bgp-persistence-03, for LLGR are supported. Also, backward compatibility with existing Junos OS features in releases earlier than Release 15.1, specifically graceful restart and nonstop routing (NSR), is supported. When LLGR is configured, graceful restart operates in the existing manner, except as explicitly illustrated in the Internet draft. You can also configure both LLGR and NSR at the same time, and achieve the complete LLGR functionality. As a prerequisite for LLGR, support for the IETF draft, Notification Message support for BGP Graceful Restart—draft-ietf- idr-bgp-gr-notification-01, is implemented. This draft extends the behavior of ordinary GR to allow it to protect against communications interruptions and protocol errors.
See Also
Understanding Maximum Period Configuration for Automatic Generation of BGP Keepalives by Kernel Timers After Switchover
In Junos OS, nonstop active routing (NSR) uses the same infrastructure as graceful Routing Engine switchover (GRES) to preserve interface and kernel information. However, NSR also saves routing protocol information by running the routing protocol process (rpd) on the backup Routing Engine. By saving this additional information, NSR is self-contained and does not rely on helper routers (or switches) to assist the routing platform in restoring routing protocol information. NSR is advantageous in networks where neighbor routers (or switches) do not support graceful restart protocol extensions. As a result of this enhanced functionality, NSR is a natural replacement for graceful restart.
Nonstop active routing automerge is one of the kernel components of the socket replication. On switchover, this component merges the socket pairs automatically from the backup to the primary Routing Engine. NSR switchover from backup to primary happens when rpd issues a merge call for each secondary socket pair to merge them to a single socket, which could result in a delay. To avoid this delay, an automerge module in the kernel decouples the secondary socket merge from rpd and automatically merges secondary sockets on switchover so that the rpd high priority thread takes advantage of this and generates faster keepalive to sustain TCP connections on switchover.
By default, BGP does not register for the automatic keepalive
generation service provided by the kernel right after the switchover
event from backup to primary. For this, you need to enable the nonstop-routing-options statement at [edit routing-options] hierarchy level and configure precision timers in BGP. Configuring
precision timers in BGP allows BGP to register all of its sessions
with the automatic keepalive generation service provided by the
kernel. Once registered, the kernel automatically generates keepalives
using its timers on behalf of BGP for its control sessions just after
the switchover event from backup to primary. This allows generation
of more reliable keepalives for control sessions with very small timers
during the switchover event.
See Also
Interoperation of Functionalities With BGP Long-Lived Graceful Restart
This topic contains the following sections that describe the working behavior of different functionalities with BGP long-lived graceful restart and the various system conditions:
Starting in Junos OS Release 15.1, Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
- Limitations on Supported NLRIs
- LLGR Restarter Mode Under NSR
- LLGR Capability At Global, BGP Group, and BGP Neighbor Levels
Limitations on Supported NLRIs
LLGR configuration and capability negotiation is supported for the following BGP network layer reachability information (NLRI) families:
l2vpn
inet labeled-unicast
inet flow
route-target
inet-vpn unicast
inet-vpn flow
inet6-vpn unicast
LLGR configuration and capability negotiation is prevented for the following families:
inet-mvpn
inet6-mvpn
inet-mdt
For the NLRI families for which LLGR capability is prevented, it indicates that an attempt to commit a configuration that includes an LLGR configuration for these families is rejected, and such settings are not saved. The NLRIs associated with these families are not included in an LLGR capabilities advertisement, and are disregarded in a received LLGR capabilities advertisement.
LLGR configuration and capability negotiation is permitted, but hidden, for other families.
LLGR Restarter Mode Under NSR
When NSR and LLGR are configured together, the router negotiates the LLGR capability in the usual, regular manner, including a long-lived stale time to trigger LLGR receiver mode in its peers. However, full LLGR restarter functionality (delaying the transmission of End of RIB markers until EoRs are received from all peers) does not function under NSR. During a full system (both Routing Engines) restart, the routing protocol daemon (rpd) does not wait for EoRs from other peers before sending its own EoR. It transmits the EoR as soon as it has transmitted the current RIB contents. This condition can cause transient outages when the network reconverges. NSR is considered to be adequate to handle all single-Routing Engine restart scenarios. The restarter mode restriction effects only scenarios where both Routing Engines (or both copies of rpd) restart simultaneously. Ordinary restarter mode configuration is not enabled with NSR.
Ordinary graceful-restart restarter mode configuration continues to be not supported with NSR.
LLGR Capability At Global, BGP Group, and BGP Neighbor Levels
Long-lived graceful restart receiver mode is enabled by default,
unless ordinary graceful restart receiver mode is disabled. To enable
the BGP long-lived graceful restart (LLGR) capability, include the long-lived receiver enable statement at the [edit protocols
bgp graceful-restart] hierarchy level. Apart from enabling BGP
LLGR at the global or system-wide level, you can also include the
long-lived receiver enable statement at the [edit protocols bgp
group group-name graceful- restart] hierarchy
level to configure LLGR for a particular BGP group and at the [edit protocols bgp group group-name neighbor neighbor-address graceful-restart] hierarchy level
to configure LLGR for a particular BGP neighbor. To disable the BGP
LLGR mechanism, include the long-lived receiver disable option the [edit protocols bgp graceful-restart], [edit protocols bgp group group-name graceful-restart], or [edit protocols bgp group-group-name neighbor neighbor-address
graceful-restart] hierarchy level. Disabling LLGR deactivates all
of the LLGR capabilities (both receiver and restarter modes) for all
NLRI families. This property is inherited by groups from the global
configuration, and by neighbors from the group configuration.
See Also
Monitoring and Administering BGP Long-Lived Graceful Restart
This topic describes the operational commands and their significance to enable you analyze and view the parameters related to BGP long-lived graceful restart. You can analyze the statistical counters and metrics related to any traffic loss and take an appropriate corrective measure. The fields displayed in the output of the show commands help in diagnosing and debugging network performance and traffic-handling efficiency problems.
The clear bgp neighbor neighbor-address stale-routes causes any stale routes currently being held
for the specified neighbor because of graceful restart (GR) or long-lived
graceful restart (LLGR) receiver mode operations. The clear
bgp neighbor neighbor-address gracefully command is the same as clear bgp neighbor hard (the
default for clear bgp neighbor), but it does not use the
new Hard Reset subcode on the Notify and Cease messages that are
sent. This allows the neighbor to enter GR or LLGR helper mode, if
negotiated. The session is still cleared on this router, and this
router does not enter GR or LLGR helper mode.
A hidden clear command is available added for the
BGP long-lived graceful restart capability for debugging purposes:
clear bgp neighbor neighbor-address socket.
This command breaks the TCP connection for an established peering session. This is the only direct implication of the command and all other implications are side effects of the connection being broken. The resultant effect is that (unless GR notification extensions have been disabled) both sides of the connection will enter GR or LLGR helper mode, if negotiated, and the TCP connection will be reestablished.
The output of the show bgp neighbor command is enhanced
to display the following additional information:
The long-lived graceful restart option
The LLGR parameters that the peer has negotiated
The LLGR parameters that the restart router has negotiated
Times are displayed using the routing protocol daemon (rpd) %#0T format:
<weeks>w<days>d <hours>:<minutes>:<seconds>Zero leading elements are omitted, for example, a value less than one week do not include the weeks.
If long-lived graceful restart is completely disabled for a neighbor, the following is displayed:
user@router> show bgp neighbor
Peer: 10.6.128.225+45824 AS 100 Local: 10.255.255.14+44542 AS 100
Type: Internal State: Established Flags: <Sync>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Options: <Preference LocalAddress AddressFamily Rib-group Refresh>
Options: <LLGRHelperDisabled> {The LLGRHelperDisabled value for the Options field denotes that long-lived BGP graceful restart is completely disabled for a neighbor}
If a neighbor does not support LLGR entirely, the following is displayed:
user@router> show bgp neighbor
...
Peer does not support LLGR Restarter or Receiver functionality {BGP neighor or peer does not support long-lived BGP graceful restart restarter or receiver
functionality}
While LLGR receiver mode is active (a peer that negotiated LLGR
has disconnected and not yet reconnected), the output of the show bgp neighbor command displays the amount of time left
until the LLGR expires, the time remaining on the GR stale timer,
and RIB details:
user@router> show bgp neighbor
Peer: 10.4.12.11 AS 100 Local: 10.6.128.225 AS 100
Type: Internal State: Active Flags: <>
Last State: Idle Last Event: Start
Last Error: None
Export: [ foo ]
Options: <Preference LocalAddress Refresh GracefulRestart>
Options: <LLGR>
Local Address: 10.6.128.225 Holdtime: 90 Preference: 170
Number of flaps: 3
Last flap event: Restart
Error: 'Cease' Sent: 0 Recv: 1
Time until long-lived stale routes deleted: inet-vpn-unicast 10:00:22 route-target 10:00:22
Table bgp.l3vpn.0
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: not advertising
Active prefixes: 0
Received prefixes: 7
Accepted prefixes: 7
Suppressed due to damping: 0
Table foo.inet.0 Bit: 30000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: not in sync
Active prefixes: 0
Received prefixes: 7
Accepted prefixes: 7
Suppressed due to damping: 0
When BGP graceful restart receiver mode is active for a neighbor,
additional information is displayed in the output of the show
bgp neighbor command. These details include the list of NLRIs
that stale routes are held for (NLRI we are holding stale routes for
field), the time remaining on the restart timer (Time until stale
routes are deleted or become long-lived stale field), the time remaining
on the stale timer (Time until end-of-rib is assumed for stale routes),
and the RIB details. Time is displayed in Coordinated Universal Time
(UTC) format (YYYY-MM-DD-HH:MM:SS). Note that the stale timer display
(‘Time until end-of-rib is assumed’) is also present
when a session is active, but the neighbor as not yet sent all of
the end-of-rib indications.
When graceful restart or LLGR helper mode is active, the RIB
information is now displayed by the show bgp summary command.
If a BGP session is established on the main routing device, the field
shows the number of active, received, accepted, and damped routes
that are received from a neighbor and appear in the inet.0 (main)
and inet.2 (multicast) routing tables. For example, 8/10/10/2 and
2/4/4/0 indicate the following:
8 active routes, 10 received routes, 10 accepted routes, and 2 damped routes from a BGP peer appear in the inet.0 routing table.
2 active routes, 4 received routes, 4 accepted routes, and no damped routes from a BGP peer appear in the inet.2 routing table.
The show route detail command (with and without the receive-protocol bgp option) is enhanced to identify routes
that are held in the long-lived stale state. The LongLivedStale flag indicates that the route was marked LLGR-stale by this router,
as part of the operation of LLGR receiver mode. The LongLivedStaleImport flag indicates that the route was marked LLGR-stale when it was
received from a peer, or by import policy. One or both of these flags
may be displayed for a route. Neither of these flags will be displayed
at the same time as the Stale (ordinary GR stale) flag. When a route
is de-preferenced because it is long-lived stale, the Inactive reason
field in the output of the show route detail command displays LLGR
stale. The new LLGR stale inactive reason fits into the route selection
hierarchy between Preference and Local preference.
user@router> show route receive-protocol bgp 10.4.12.11 detail
bgp.l2vpn.0: 38 destinations, 39 routes (37 active, 0 holddown, 1 hidden)
* 1.1.1.4:100:1.1.1.4/96 AD (1 entry, 1 announced)
Accepted LongLivedStale LongLivedStaleImport
Nexthop: 10.4.12.11
Localpref: 100
AS path: I
According to the Juniper Technical Assistance Center (JTAC),
one helpful command to help troubleshoot issues related to BGP long-lived
graceful restart is the show route table bgp.l2vpn.0 detail hidden command. The output of the command helps you detect if the BGP routes
still exist after the BGP session has ended. Use of the hidden option enables you to see the routes during and after an incident,
and discover information that explains why the routes are hidden.
Other clues that help you troubleshoot this scenario include the appearance
of stale BGP log entries (such as bgp_mark_route_stale),
and hidden routes showing up in the output of the show bgp summary command.
See Also
Increasing the Duration for Preserving BGP Routes Across Slowly-Restarting Peers By BGP Long-Lived Graceful Restart
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
Long-lived graceful restart receiver mode is enabled by default,
unless ordinary graceful restart receiver mode is disabled. To enable
the BGP long-lived graceful restart (LLGR) capability, include the long-lived receiver enable statement at the [edit protocols
bgp graceful-restart] hierarchy level. Apart from enabling BGP
LLGR at the global or system-wide level, you can also include the
long-lived receiver enable statement at the [edit protocols bgp
group group-name graceful-restart] hierarchy
level to configure LLGR for a particular BGP group and at the [edit protocols bgp group group-name neighbor neighbor-address graceful-restart] hierarchy level
to configure LLGR for a particular BGP neighbor. To disable the BGP
LLGR mechanism, include the long-lived receiver disable option the [edit protocols bgp graceful-restart], [edit protocols bgp group group-name graceful-restart], or [edit protocols bgp group-group-name neighbor neighbor-address
graceful-restart] hierarchy level. Disabling LLGR deactivates all
of the LLGR capabilities (both receiver and restarter modes) for all
NLRI families. This property is inherited by groups from the global
configuration, and by neighbors from the group configuration.
BGP neighbors can be configured at the following hierarchy levels:
[edit protocols bgp group group-name]—Default logical system and default routing instance.[edit routing-instances instance-name protocols bgp group group-name]—Default logical system with a specified routing instance.[edit logical-systems logical-system-name protocols bgp group group-name]—Configured logical system and default routing instance.[edit logical-systems logical-system-name routing-instances instance-name protocols bgp group group-name]—Configured logical system with a specified routing instance.
The long-lived receiver enable overrides a disable
option inherited from a higher level in the configuration. It does
not enable long-lived graceful restart restarter mode for all families—restarter
mode must be configured explicitly for each family.
To enable LLGR-stale routes to be advertised to neighbors that
do not advertise the LLGR capability, include the advertise-to-non-llgr-neighbor statement at the [edit protocols bgp graceful-restart long-lived], [edit protocols bgp group group-name graceful-restart
long-lived], or [edit protocols bgp group group-name neighbor neighbor-address graceful-restart
long-lived] hierarchy level. This setting applies to both routes
that were marked LLGR-stale by this router, and LLGR-stale routes
received from neighbors. Ideally, all routers in an autonomous system
support the IETF draft specification before it was enabled. However,
to facilitate incremental deployment, stale routes might be required
to be advertised to neighbors that have not advertised the long-lived
graceful restart capability under the following conditions: The neighbors
must be internal (IBGP or Confederation) neighbors. The NO_EXPORT
community must be attached to the stale routes. The stale routes must
have their LOCAL_PREF attribute set to zero. If this technique for
partial deployment is used, you must set LOCAL_PREF to zero for all
LLGR routes throughout the autonomous system. This configuration trades
off a small reduction in flexibility (ordering may not be preserved
between competing LLGR routes) for consistency between routers that
support and do not support this specification. Because consistency
of route selection can be important for preventing forwarding loops,
the latter consideration of routers that do not support this specification
precedes.
To avoid the no-export BGP community from being automatically
added to routes advertised to external BGP neighbors (presumed to
be CE routers), include the omit- no-export statement at
the [edit protocols bgp graceful-restart long-lived], [edit protocols bgp group group-name graceful-restart
long-lived], or [edit protocols bgp group group-name neighbor neighbor-address graceful-restart
long-lived] hierarchy level. In VPN deployments, for example,
BGP is often used as a PE-CE protocol. It might be a practical necessity
in such deployments to accommodate interoperation with CEs that cannot
easily be upgraded to support specifications such as this one. This
requirement causes a problem while ensuring that "stale" routing
information does not leak beyond the perimeter of routers that support
these procedures where one or more IBGP routers are not upgraded.
In the VPN PE-CE case, the protocol in use is EBGP, and the LOCAL_PREF,
an IBGP-only path attribute, is used. The principal motivation for
restricting the propagation of "stale" routing information is the
reason to prevent it from spreading without limit once it exits the
BGP confederation boundary. VPN deployments are typically topologically
constrained, removing this concern. For this reason, an implementation
might advertise stale routes over a PE-CE session, when explicitly
configured. In such a scenario, the implementation must attach the
NO_EXPORT community to the routes in question by default, as an additional
protection against stale routes spreading without limit. Attachment
of the NO_EXPORT community can be disabled explicitly to accommodate
exceptional cases. It might be necessary to advertise stale routes
to a CE in some VPN deployments, even if the CE does not support
this specification. In that case, if you configure the PE routers
to advertise such routes, you must notify the operator of the CE receiving
the routes, and the CE must be configured to depreference the routes.
Typical BGP implementations perform this operation by matching on
the LLGR_STALE community, and setting the LOCAL_PREF for matching
routes to zero.
When the LLGR receiver mode is enabled or disabled, the session
is reset. This behavior enables the new capability value to be sent
to the neighbor. When the advertise-to-non-llgr-neighbor option is enabled or disabled, export policy is reevaluated, and
LLGR stale routes might be advertised or withdrawn. When the omit-no-export option is added or removed, the session is reset.
This rest of a session enables LLGR stale routes to be readvertised
with or without the no- export community (which is added outside of
the export policy).
To enable the BGP long-lived graceful restart capability at the system or global level and configure its properties:
[edit]
protocols {
bgp {
graceful-restart {
long-lived {
receiver {
enable:
disable;
}
advertise-to-non-llgr-neighbor {
omit-no-export;
}
}
}
}
}
To enable the BGP long-lived graceful restart capability at the BGP group level and configure its properties:
[edit]
protocols {
bgp {
group group-name {
graceful-restart {
long-lived {
receiver {
enable:
disable;
}
advertise-to-non-llgr-neighbor {
omit-no-export;
}
}
}
}
}
}
To enable the BGP long-lived graceful restart capability at the neighbor or peer group level and configure its properties:
[edit]
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
long-lived {
receiver {
enable:
disable;
}
advertise-to-non-llgr-neighbor {
omit-no-export;
}
}
}
}
}
}
}
See Also
Configuring BGP Long-Lived Graceful Restart Communities in Routing Policies
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
Two new well-known communities are introduced. These new BGP communities can be used in any of the configuration hierarchy levels as other symbolic well-known communities (such as no-advertise, no-export, and no-export-subconfed) in the community attribute of static route definitions or in a policy-options community definition. The two new communities are as follows:
llgr-stale—Adds a community to a long-lived stale route when it is readvertised.no-llgr—Marks routes which a BGP speaker does not want to be retained by LLGR. The Notification message feature does not have any associated configuration parameters.
You can include the llgr-stale and no-llgr options with the community name members statement to associate BGP community information with a static,
aggregate, or generated route at the following hierarchy levels:
[edit dynamic policy-options], [edit logical-systems logical-system-name policy-options], [edit policy-options]
To configure the BGP long-lived graceful restart communities for use in a routing policy match condition:
[edit policy-options]
community name {
members [ llgr-stale | nollgr];
}
Configuring LLGR does not require that BGP graceful restart also be configured. The values for the llgr-stale and no-llgr well-known communities are 0xFFFF0006 and 0xFFFF0007 respectively. The privileges are the same as for protocols bgp. The long-lived-graceful-restart section is visible only for families l2vpn, inet labeled-unicast, inet flow and route-target. It is prohibited for inet-mvpn, inet6-mvpn and inet-mdt. It is hidden for other families.
Junos OS also provides support for configuring a BGP export policy that matches the state of a route for BGP long-lived graceful restart. You can associate the community that you previously defined and a list of address prefixes in a routing policy to selectively accept or reject the routes for long-lived graceful restart for the specified prefixes, as follows:
policy-options {
prefix-list name;
community name members [ llgr-stale | nollgr];
policy-statement name{
from {
prefix-list name;
community name;
}
then {
(accept | reject)
}
}
}
Two hidden configuration statements are added under the [edit protocols bgp graceful-restart] hierarchy level for global,
group-level, and neighbor group-level configuration.
The disable-notification-flag statement at the [edit protocols bgp graceful-restart], [edit protocols
bgp group group-name graceful-restart],
or [edit protocols bgp group group-name neighbor neighbor-address graceful-restart] hierarchy level
disables the transmission of the N flag in the graceful restart capability
negotiation. The disable-notification-extensions statement
at the [edit protocols bgp graceful-restart], [edit
protocols bgp group group-name graceful-restart], or [edit protocols bgp group group-name neighbor neighbor-address graceful-restart] hierarchy level also disables the transmission of the N flag in
the graceful restart capability negotiation, but in addition, it disables
the new rules for invoking graceful restart receiver mode as specified
in the IETF bgp-gr-notification draft, and disables the transmission
of the Hard Reset subcode. The Hard Reset subcode is continued to
be observed when received in a Notify or a Cease message.
To disable the transmission of N flags and to disable rules for triggering graceful restart at the global or system-wide level:
[edit]
protocols {
bgp {
graceful-restart {
disable-notification-flag;
disable-notification-extensions;
}
}
}
To disable the transmission of N flags and to disable rules for triggering graceful restart at the group level:
[edit]
protocols {
bgp {
group group-name {
graceful-restart {
disable-notification-flag;
disable-notification-extensions;
}
}
}
}
To disable the transmission of N flags and to disable rules for triggering graceful restart at the neighbor or peer level:
[edit]
protocols {
bgp {
group group-name {
graceful-restart {
disable-notification-flag;
disable-notification-extensions;
}
}
}
}
See Also
Configuring Long-Lived Graceful Restarter Mode Negotiation for a Specific Address Family in Logical Systems and Routing Instances
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
You can also configure the BGP long-lived graceful restarter
mode negotiation mechanism for a particular address family instead
of configuring this capability for all address families in a system,
logical system, or routing instance. To enable BGP LLGR for a specific
address family, include the graceful-restart long-lived restarter
stale-time interval statement at one of
the following hierarchy levels.
Each routing table is identified by the protocol family or address
family indicator (AFI) and a subsequent address family identifier
(SAFI). The AFI parameter can be one of the (l2vpn | inet |
route-target) protocols and the SAFI parameter can be either
of the (flow | labeled-unicast) protocols for inet family
and one of the (auto-discovery-mspw | auto-discovery-only | signaling) protcols for L2VPN family..
Configuring LLGR does not require that BGP graceful restart also be configured. The long-lived-graceful-restart section is visible only for families l2vpn, inet labeled-unicast, inet flow and route-target. It is prohibited for inet-mvpn, inet6-mvpn and inet-mdt. It is hidden for other families.
[edit logical-systems logical-system-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)] [edit routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)],
The stanzas in the per-family graceful-restart long-lived restarter
configuration section enables LLGR restarter mode negotiation for
BGP globally, or for a group or neighbor. The values are inherited
by groups from the global configuration, and by neighbors from the
group configuration. The disable attribute is used to override configuration
inherited from a higher level. It does not disable LLGR receiver
mode; you must disable LLGR receiver mode explicitly for all families
as necessary. A hidden enable attribute can be used to
override an inherited disable attribute. Configuring graceful-restart
long-lived restarter at the neighbor level (when it is not configured
at the containing group level or globally) causes an internal group
to be split. When LLGR restarter is enabled or disabled for a family
or the stale- time is changed, the session is reset so that the new
capability can be sent to the neighbor.
The range of values for stale-time is from 1 to 16777215 (2^24 – 1) seconds. The value is a simple integer giving the number of seconds by default, but it can also be specified using the following notation:
[<weeks>w][<days>d][<hours>h][<minutes>m][<seconds>s] For example, you can specify 27 days as 27d, 648h, 38880m or 2332800s. 90 minutes can be configured as 1h30m, 90m or 5400s. The specified number of days is multiplied by 86400, the number of hours by 3600 and the number of minutes by 60; these are added to the seconds to get the total. A combined format of days and hours, in different time period units, such as 1d36h are permitted, as long as the specified total does not exceed the maximum stale time.
In addition, times can also be configured using the following notation: <hours>:<minutes>:<seconds> For example, 12:00:00 specifies twelve hours. The hours and minutes are optional.
The two notations can be combined, for example, 2w1d 12:00:02
specifies two weeks, one day, twelve hours and two seconds (1339202
seconds). (Note that the CLI requires double-quotes around a value
like this with spaces in it.) Expressed in this notation, the maximum
stale time is 27w5d 04:20:15 (27 weeks, 5 days, 4 hours, 20 minutes
and 15 seconds). While the show configuration command displays the
actually configured values, when the associated timers are displayed
in run-time show commands such as show bgp neighbor, the
values are normalized, such as 1d36h becoming 2d 12:00:00. The full
rules for displaying normalized LLGR times depend on the clear
bgp neighbor neighbor-address gracefully command configuration.
To configure the BGP long-lived graceful restart characteristics per-address family and per-subsequent address family at the global level for a logical system or a routing instance:
Configuring BGP Long-Lived Graceful Restart Per Address Family At the Global Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
Configuring BGP Long-Lived Graceful Restart Per Address Family At the Global Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
To configure the BGP long-lived graceful restart characteristics per-address family and per-subsequent address family at the BGP group level for a logical system or a routing instance:
Configuring BGP Long-Lived Graceful Restart Per Address Family At the BGP Group Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
}
Configuring BGP Long-Lived Graceful Restart Per Address Family At the BGP Group Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
}
To configure the BGP long-lived graceful restart characteristics per-address family and per-subsequent address family at the BGP neighbor group level for a logical system or a routing instance:
Configuring BGP Long-Lived Graceful Restart Per Address Family At the BGP Neighbor Group Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
}
}
Configuring BGP Long-Lived Graceful Restart Per Address Family At the BGP Neighbor Group Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
long-lived {
restarter {
disable;
stale-time interval;
}
}
}
}
}
}
}
See Also
Informing the BGP Helper Router or Peer About Retaining Routes By Configuring the Forwarding State Bit for All Address Families and for a Specific Address Family
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
After a BGP session goes down and before the session is reestablished, stale routes might be retained for up to two consecutive periods, controlled by the restart time and long-lived stale time parameters, respectively. During the first period routing modifications are prevented but with potential null-route filtering of traffic. During the second period, possible null-route filtering of traffic might be reduced but routing changes ares visible throughout the network. In your network environment, the setting of the relevant parameters for a particular application must consider the tradeoffs, the network dynamics and potential failure scenarios. If necessary, the first period can be bypassed either by local configuration or by setting the restart time in the graceful restart capability to zero, not listing the address family indicators (AFI) and a subsequent address family identifiers (SAFI) in that capability.
The setting of the F bit (and the "Forwarding State" bit of the accompanying GR capability) depends in part on deployment considerations. The F bit can be interpreted to indicate the helper router needs to flush associated routes (if the bit is left clear). An important scenario in which LLGR is used is for routes that are more similar to configuration than to traditional routing (hop-by-hop forwarding instead of tunnel-based routing). For such routes, it might be useful to always set the F bit, regardless of other considerations. Similarly, for control-plane-only entities such as dedicated route reflectors, that do not participate in the forwarding plane, it is preferred that the F bit be always set. Overall, the guideline to be adopted is that if loss of state on the restarting router can reasonably be expected to cause a forwarding loop or null route, the F bit must be set judiciously, depending on whether state has been retained. You can determine whether the F bit needs to be set or not, based on your deployment needs and configured settings. It might be necessary to advertise stale routes to a CE in some VPN deployments, even if the CE does not support this specification. In such a scenario, the network operator configuring their PE to advertise such routes must notify the operator of the CE receiving the routes, and the CE must be configured to depreference the routes. Typically, BGP implementations perform this behavior by matching on the LLGR_STALE community, and setting the LOCAL_PREF for matching routes to zero.
You can specify the Forwarding State bit, which is a BGP configuration
option that can be defined at the global, group and neighbor levels,
for any logical system or routing instance. To specify the Forwarding
State bit at the global, BGP group, or BGP neighbor level, include
the forwarding-state-bit (as-rr-client | from-fib) statement
at the [edit protocols bgp graceful-restart], [edit
protocols bgp group-group-name graceful-restart], or [edit
protocols bgp group-group-name neighbor neighbor-address graceful-restart] hierarchy level. The forwarding-state-bit attribute controls how
the Forwarding State bit is set in both graceful restart and long-lived
graceful restart capability advertisements. By default, the value
depends on whether the neighbor is a route reflector client. If the
neighbor is not a route reflector client, the value is set according
to the state of the associated FIB in compliance with RFC 4724. If
the neighbor is a route reflector client, the value is set to 1 for
all families except inet unicast and inet6 unicast, which use the
state of the associated FIB. The as-rr-client option sets
the behavior for all address families to be the same as the functionality
for a route reflector client. The from-fib option forces
the behavior for all address families to be as they would be for
a non-route-reflector client.
To configure the forwarding-state flag negotiation at the global level:
[edit]
protocols {
bgp {
graceful-restart {
forwarding-state-bit (as-rr-client | from-fib);
}
}
}
To configure the forwarding-state flag negotiation at the group level:
[edit]
protocols {
bgp {
group group-name {
graceful-restart {
forwarding-state-bit (as-rr-client | from-fib);
}
}
}
}
To configure the forwarding-state flag negotiation at the neighbor or peer group level:
[edit]
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
forwarding-state-bit (as-rr-client | from-fib);
}
}
}
}
}
In addition to the global setting for the Forwarding State bit,
the Forwarding State bit behavior can be specified for individual
families. Changing the forwarding-state-bit setting has no effect
on any existing sessions. To specify the Forwarding State bit for
a particular address family, include the forwarding-state-bit
(set | from-fib) statement at the [edit protocols bgp graceful-restart
family address-family subsequent-address-family], [edit protocols bgp group-group-name graceful-restart
family address-family subsequent-address-family], or [edit protocols bgp group-group-name neighbor neighbor-address
graceful-restart family address-family subsequent-address-family] hierarchy level on a
logical system and a routing instance. Per-family BGP configuration
options are added to control the Forwarding State bit in graceful
restart and long-lived graceful restart capability advertisements.
They can be specified for the default logical system or for a specific
logical system, and for the primary routing instance or a specific
routing instance. The per-family forwarding-state-bit attribute
overrides the default rules or the global configuration for setting
the Forwarding State bit. The set option forces the Forwarding
State bit to be set to 1. The from-fib option causes the
value to be set according to the state of the associated FIB. Changing
the per-family forwarding-state-bit setting has no effect on any existing
sessions.
The following are the complete configuration hierarchy levels
at which you can include the forwarding-state-bit (set | from-fib) statement to configure the forwarding state bit per address family:
[edit logical-systems logical-system-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)] [edit routing-instances routing-instance-name protocols bgp family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name family inet (labeled-unicast | unicast | multicast)], [edit routing-instances routing-instance-name protocols bgp group group-name neighbor address family inet (labeled-unicast | unicast | multicast)],
To configure the forwarding state bit for BGP long-lived graceful restart per-address family and per-subsequent address family at the global level for a logical system or a routing instance:
Configuring the Forwarding State Bit Per Address Family At the Global Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
Configuring the Forwarding State Bit Per Address Family At the Global Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
To configure the forwarding state bit for BGP long-lived graceful restart per-address family and per-subsequent address family at the BGP group level for a logical system or a routing instance:
Configuring the Forwarding State Bit Per Address Family At the BGP Group Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
}
Configuring the Forwarding State Bit Per Address Family At the BGP Group Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
}
To configure the forwarding state bit for BGP long-lived graceful restart per-address family and per-subsequent address family at the BGP neighbor group level for a logical system or a routing instance:
Configuring the Forwarding State Bit Per Address Family At the BGP Neighbor Group Level for Logical Systems
[edit logical-systems logical-system-name routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
}
}
Configuring the Forwarding State Bit Per Address Family At the BGP Neighbor Group Level for Routing Instances
[edit routing-instances routing-instance-name protocols bgp family address-family subsequent-address-family
protocols {
bgp {
group group-name {
neighbor neighbor-address {
graceful-restart {
forwarding-state-bit (set | from-fib);
}
}
}
}
}
See Also
Example: Preserving Route Details for Slow and Latent BGP Peers By Using BGP Long-Lived Graceful Restart
Junos OS supports the mechanism to preserve BGP routing details for a longer period from a failed BGP peer than the duration for which such routing information is maintained using the BGP graceful restart functionality.
Historically, routing protocols and BGP, in particular, have been designed with a focus on correctness, where a significant aspect of the "correctness" is for each network element's forwarding state to converge toward the current state of the network as quickly as possible. For this reason, the protocol was designed to remove state advertised by routers which went down (from a BGP perspective) as promptly as possible. Using BGP Graceful Restart defined in RFC 4724, the fast convergence functionality has been an attempt to rapidly remove "stale" state from the network.
BGP long-lived graceful restart (LLGR) allows a network operator to choose to maintain stale routing information from a failed BGP peer much longer than the existing BGP Graceful Restart facility. This functionality to maintain the BGP routes for a longer time period is in accordance with the IETF draft, Support for Long-lived BGP Graceful Restart—draft-uttaro-idr-bgp-persistence-03. According to this draft, long-lived graceful restart (LLGR) must be explicitly configured per NLRI, and it includes provisions to prevent the spread of stale information to other peers that do not recognize and validate LLGR.
This example describes how to configure BGP long-lived graceful restart functionality on MX Series routers, and contains the following sections:
Requirements
This example uses the following hardware and software components:
One MX Series router with an MPC.
Junos OS Release 15.1R1 or later for MX Series routers
Before you configure BGP long-lived graceful restart, make sure you:
Configure the device interfaces.
Configure BGP.
Overview
Graceful restart allows a routing device undergoing a restart to inform its adjacent neighbors and peers of its condition. During a graceful restart, the restarting device and its neighbors continue forwarding packets without disrupting network performance. Because neighboring devices assist in the restart (these neighbors are called helper routers), the restarting device can quickly resume full operation without recalculating algorithms.
Long-lived graceful restart receiver mode is enabled by default,
unless ordinary graceful restart receiver mode is disabled. To enable
the BGP long-lived graceful restart (LLGR) capability, include the long-lived receiver enable statement at the [edit protocols
bgp graceful-restart] hierarchy level. Apart from enabling BGP
LLGR at the global or system-wide level, you can also include the
long-lived receiver enable statement at the [edit protocols bgp
group group-name graceful-restart] hierarchy
level to configure LLGR for a particular BGP group and at the [edit protocols bgp group group-name neighbor neighbor-address graceful-restart] hierarchy level
to configure LLGR for a particular BGP neighbor. To disable the BGP
LLGR mechanism, include the long-lived receiver disable option the [edit protocols bgp graceful-restart], [edit protocols bgp group group-name graceful-restart], or [edit protocols bgp group-group-name neighbor neighbor-address
graceful-restart] hierarchy level. Disabling LLGR deactivates all
of the LLGR capabilities (both receiver and restarter modes) for all
NLRI families. This property is inherited by groups from the global
configuration, and by neighbors from the group configuration.
Topology
Consider a sample scenario in which you want to increase the time period for which stale routes are maintained for a BGP peer or neighbor with the address of 1.2.3.4. Besides specifying the duration for which the routes must be retained for stale sessions and when a graceful restart of a peer occurs, you can also configure BGP routers from certain address prefixes to be disregarded when you define the long-lived graceful restart mechanism. You can define a list of IPv4 or IPv6 address prefixes for use in a routing policy statement and a BGP community to be included in the routing policy. If you set the action modifier to reject routes from a particular prefix, such BGP routes are not maintained for the increased time period.
You can also configure the BGP long-lived graceful restarter
mode negotiation mechanism for a particular address family instead
of configuring this capability for all address families in a system,
logical system, or routing instance. To enable BGP LLGR for a specific
address family, include the graceful-restart long-lived restarter
stale-time interval statement at one of
the following hierarchy levels.
Each routing table is identified by the protocol family or address
family indicator (AFI) and a subsequent address family identifier
(SAFI). The AFI parameter can be one of the (l2vpn | inet |
route-target) protocols and the SAFI parameter can be either
of the (flow | labeled-unicast) protocols for inet family
and one of the (auto-discovery-mspw | auto-discovery-only | signaling) protcols for L2VPN family..
Configuring LLGR does not require that BGP graceful restart also be configured. The long-lived-graceful-restart section is visible only for families l2vpn, inet labeled-unicast, inet flow and route-target. It is prohibited for inet-mvpn, inet6-mvpn and inet-mdt. It is hidden for other families.
Configuration
CLI Quick Configuration
To quickly configure this example, copy the
following commands, paste them into a text file, remove any line breaks,
change any details necessary to match your network configuration,
copy and paste the commands into the CLI at the [edit] hierarchy
level, and then enter commit from configuration mode.
Configuring the Address Prefix List, BGP Community, and BGP Routing Policy
set policy-options prefix-list special 44.44.44.44/32 set policy-options community llgr-community llgr-stale set policy-options policy-statement llgr-import from prefix-list special set policy-options policy-statement llgr-import from community llgr-community set policy-options policy-statement llgr-import then reject
Configuring the BGP Group, NLRI, and Long-Lived Graceful Restart
set protocols bgp group ibgp-group type internal set protocols bgp group ibgp-group import llgr-import set protocols bgp group ibgp-group family inet unicast set protocols bgp group ibgp-group family inet unicast graceful-restart long-lived restarter stale-time 12h
Configuring the BGP Neighbor Group
set protocols bgp group ibgp-group neighbor 1.2.3.4
Configuring Long-Lived Graceful Restart for Restarter Mode
Step-by-Step Procedure
The following example requires that you navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
Configure the address prefix list, BGP community, and the match condition and action modifier for the BGP routing policy.
[edit] user@ host# set policy-options prefix-list special 44.44.44.44/32 user@ host# set policy-options community llgr-community llgr-stale user@ host# set policy-options policy-statement llgr-import from prefix-list special user@ host# set policy-options policy-statement llgr-import from community llgr-community user@ host# set policy-options policy-statement llgr-import then reject
Configure the BGP group, address family, and long-lived graceful restart functionality for restarter mode with the stale time for flows.
[edit] user@ host# set protocols bgp group ibgp-group type internal user@ host# set protocols bgp group ibgp-group import llgr-import user@ host# set protocols bgp group ibgp-group family inet unicast user@ host# set protocols bgp group ibgp-group family inet unicast graceful-restart long-lived restarter stale-time 12h
Configure the BGP neighbor group.
[edit] user@ host# set protocols bgp group ibgp-group neighbor 1.2.3.4
Results
From configuration mode, confirm your configuration by entering the show policy-options and show protocols commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.
[edit]
user@host# show policy-options
policy-options {
prefix-list special 44.44.44.44/32;
community llgr-community llgr-stale;
policy-statement llgr-import {
from {
prefix-list special;
community llgr-community;
}
then {
reject;
}
}
}
user@host# show protocols
protocols {
bgp {
group ibgp-group {
type internal;
import llgr-import;
family inet unicast {
graceful-restart {
long-lived {
restarter {
stale-time 12h;
}
}
}
}
neighbor 1.2.3.4;
}
}
}
Verification
Confirm that the configuration is working properly.
Verifying That the Long-Lived Graceful Restart Capability is Enabled
Purpose
Verify the BGP long-lived graceful restart capability configured for BGP neighbor level
Action
While LLGR receiver mode is active (a peer that negotiated
LLGR has disconnected and not yet reconnected), the output of the show bgp neighbor command displays the amount of time left
until the LLGR expires, the time remaining on the GR stale timer,
and RIB details:
user@router> show bgp neighbor
Peer: 10.4.12.11 AS 100 Local: 10.6.128.225 AS 100
Type: Internal State: Active Flags: <>
Last State: Idle Last Event: Start
Last Error: None
Export: [ foo ]
Options: <Preference LocalAddress Refresh GracefulRestart>
Options: <LLGR>
Local Address: 10.6.128.225 Holdtime: 90 Preference: 170
Number of flaps: 3
Last flap event: Restart
Error: 'Cease' Sent: 0 Recv: 1
Time until long-lived stale routes deleted: inet-vpn-unicast 10:00:22 route-target 10:00:22
Table bgp.l3vpn.0
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: not advertising
Active prefixes: 0
Received prefixes: 7
Accepted prefixes: 7
Suppressed due to damping: 0
Table foo.inet.0 Bit: 30000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: not in sync
Active prefixes: 0
Received prefixes: 7
Accepted prefixes: 7
Suppressed due to damping: 0
Meaning
The output shows information about BGP neighbors.