Supported Platforms
Related Documentation
Example: Configuring Graceful Restart
To enable graceful restart, include the graceful-restart statement at the [edit routing-instance instance-name routing-options] or [edit routing-options] hierarchy level as well as in the protocol level.
For example:
Figure 1 shows a standard MPLS VPN network. Routers CE1 and CE2 are customer edge routers, PE1 and PE2 are provider edge routers, and P0 is a provider core router. Several Layer 3 VPNs are configured across this network, as well as one Layer 2 VPN. Interfaces are shown in the diagram and are not included in the configuration example that follows.
Figure 1: Layer 3 VPN Graceful Restart Topology

Router CE1
On Router CE1, configure the following protocols on the logical interfaces of t3-3/1/0: OSPF on unit 101, RIP on unit 102, BGP on unit 103, and IS-IS on unit 512. Also configure graceful restart, BGP, IS-IS, OSPF, and RIP on the main instance to be able to connect to the routing instances on Router PE1.
Router PE1
On Router PE1, configure graceful restart in the master instance, along with BGP, OSPF, MPLS, and LDP. Next, configure several protocol-specific instances of graceful restart. By including instances for BGP, OSPF, Layer 2 VPNs, RIP, and static routes, you can observe the wide range of options available when you implement graceful restart. Configure the following protocols in individual instances on the logical interfaces of t3-0/0/0: a static route on unit 100, OSPF on unit 101, RIP on unit 102, BGP on unit 103, and Frame Relay on unit 512 for the Layer 2 VPN instance.
Router P0
On Router P0, configure graceful restart in the main instance, along with OSPF, MPLS, and LDP. This allows the protocols on the PE routers to reach one another.
Router PE2
On Router PE2, configure BGP, OSPF, MPLS, LDP, and graceful restart in the master instance. Configure the following protocols in individual instances on the logical interfaces of t1-0/1/3: a static route on unit 200, OSPF on unit 201, RIP on unit 202, BGP on unit 203, and Frame Relay on unit 612 for the Layer 2 VPN instance. Also configure protocol-specific graceful restart in all routing instances, except the Layer 2 VPN instance.
Router CE2
On Router CE2, complete the Layer 2 and Layer 3 VPN configuration by mirroring the protocols already set on Routers PE2 and CE1. Specifically, configure the following on the logical interfaces of t1-0/0/3: OSPF on unit 201, RIP on unit 202, BGP on unit 203, and IS-IS on unit 612. Finally, configure graceful restart, BGP, IS-IS, OSPF, and RIP on the main instance to be able to connect to the routing instances on Router PE2.
Router PE1 Status Before a Restart
The following example displays neighbor relationships on Router PE1 before a restart happens:
user@PE1> show bgp neighbor
Peer: 10.96.103.2+3785 AS 65100 Local: 10.96.103.1+179 AS 65103 Type: External State: Established Flags: <> Last State: OpenConfirm Last Event: RecvKeepAlive Last Error: None Export: [ BGP-INET-import ] Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily PeerAS Refresh> Address families configured: inet-unicast Local Address: 10.96.103.1 Holdtime: 90 Preference: 170 Number of flaps: 0 Peer ID: 10.96.110.1 Local ID: 10.96.103.1 Active Holdtime: 90 Keepalive Interval: 30 Local Interface: t3-0/0/0.103 NLRI for restart configured on peer: inet-unicast NLRI advertised by peer: inet-unicast NLRI for this session: inet-unicast Peer supports Refresh capability (2) Restart time configured on the peer: 120 Stale routes from peer are kept for: 300 Restart time requested by this peer: 120 NLRI that peer supports restart for: inet-unicast NLRI peer can save forwarding state: inet-unicast NLRI that peer saved forwarding for: inet-unicast NLRI that restart is negotiated for: inet-unicast NLRI of all end-of-rib markers sent: inet-unicast Table BGP-INET.inet.0 Bit: 30001 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Last traffic (seconds): Received 8 Sent 3 Checked 3 Input messages: Total 15 Updates 0 Refreshes 0 Octets 321 Output messages: Total 18 Updates 2 Refreshes 0 Octets 450 Output Queue[2]: 0 Peer: 10.245.14.182+4701 AS 69 Local: 10.245.14.176+179 AS 69 Type: Internal State: Established Flags: <> Last State: OpenConfirm Last Event: RecvKeepAlive Last Error: None Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily Rib-group Refresh> Address families configured: inet-vpn-unicast l2vpn Local Address: 10.245.14.176 Holdtime: 90 Preference: 170 Number of flaps: 1 Peer ID: 10.245.14.182 Local ID: 10.245.14.176 Active Holdtime: 90 Keepalive Interval: 30 NLRI for restart configured on peer: inet-vpn-unicast l2vpn NLRI advertised by peer: inet-vpn-unicast l2vpn NLRI for this session: inet-vpn-unicast l2vpn Peer supports Refresh capability (2) Restart time configured on the peer: 120 Stale routes from peer are kept for: 300 Restart time requested by this peer: 120 NLRI that peer supports restart for: inet-vpn-unicast l2vpn NLRI peer can save forwarding state: inet-vpn-unicast l2vpn NLRI that peer saved forwarding for: inet-vpn-unicast l2vpn NLRI that restart is negotiated for: inet-vpn-unicast l2vpn NLRI of all end-of-rib markers sent: inet-vpn-unicast l2vpn Table bgp.l3vpn.0 Bit: 10000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Table bgp.l2vpn.0 Bit: 20000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 1 Received prefixes: 1 Suppressed due to damping: 0 Table BGP-INET.inet.0 Bit: 30000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Table OSPF.inet.0 Bit: 60000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Table RIP.inet.0 Bit: 70000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Table STATIC.inet.0 Bit: 80000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 0 Suppressed due to damping: 0 Table L2VPN.l2vpn.0 Bit: 90000 RIB State: BGP restart is complete RIB State: VPN restart is complete Send state: in sync Active prefixes: 1 Received prefixes: 1 Suppressed due to damping: 0 Last traffic (seconds): Received 28 Sent 28 Checked 28 Input messages: Total 2 Updates 0 Refreshes 0 Octets 86 Output messages: Total 13 Updates 10 Refreshes 0 Octets 1073 Output Queue[0]: 0 Output Queue[1]: 0 Output Queue[2]: 0 Output Queue[3]: 0 Output Queue[4]: 0 Output Queue[5]: 0 Output Queue[6]: 0 Output Queue[7]: 0 Output Queue[8]: 0 user@PE1> show route instance detail master: Router ID: 10.245.14.176 Type: forwarding State: Active Restart State: Complete Path selection timeout: 300 Tables: inet.0 : 17 routes (15 active, 0 holddown, 1 hidden) Restart Complete inet.3 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete iso.0 : 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete mpls.0 : 19 routes (19 active, 0 holddown, 0 hidden) Restart Complete bgp.l3vpn.0 : 10 routes (10 active, 0 holddown, 0 hidden) Restart Complete inet6.0 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete bgp.l2vpn.0 : 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete BGP-INET: Router ID: 10.96.103.1 Type: vrf State: Active Restart State: Complete Path selection timeout: 300 Interfaces: t3-0/0/0.103 Route-distinguisher: 10.245.14.176:103 Vrf-import: [ BGP-INET-import ] Vrf-export: [ BGP-INET-export ] Tables: BGP-INET.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden) Restart Complete L2VPN: Router ID: 0.0.0.0 Type: l2vpn State: Active Restart State: Complete Path selection timeout: 300 Interfaces: t3-0/0/0.512 Route-distinguisher: 10.245.14.176:512 Vrf-import: [ L2VPN-import ] Vrf-export: [ L2VPN-export ] Tables: L2VPN.l2vpn.0 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete OSPF: Router ID: 10.96.101.1 Type: vrf State: Active Restart State: Complete Path selection timeout: 300 Interfaces: t3-0/0/0.101 Route-distinguisher: 10.245.14.176:101 Vrf-import: [ OSPF-import ] Vrf-export: [ OSPF-export ] Tables: OSPF.inet.0 : 8 routes (7 active, 0 holddown, 0 hidden) Restart Complete RIP: Router ID: 10.96.102.1 Type: vrf State: Active Restart State: Complete Path selection timeout: 300 Interfaces: t3-0/0/0.102 Route-distinguisher: 10.245.14.176:102 Vrf-import: [ RIP-import ] Vrf-export: [ RIP-export ] Tables: RIP.inet.0 : 6 routes (6 active, 0 holddown, 0 hidden) Restart Complete STATIC: Router ID: 10.96.100.1 Type: vrf State: Active Restart State: Complete Path selection timeout: 300 Interfaces: t3-0/0/0.100 Route-distinguisher: 10.245.14.176:100 Vrf-import: [ STATIC-import ] Vrf-export: [ STATIC-export ] Tables: STATIC.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden) Restart Complete __juniper_private1__: Router ID: 0.0.0.0 Type: forwarding State: Active user@PE1> show route protocol l2vpn inet.0: 16 destinations, 17 routes (15 active, 0 holddown, 1 hidden) Restart Complete inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete BGP-INET.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden) Restart Complete OSPF.inet.0: 7 destinations, 8 routes (7 active, 0 holddown, 0 hidden) Restart Complete RIP.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden) Restart Complete STATIC.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden) Restart Complete iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete mpls.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden) Restart Complete + = Active Route, - = Last Active, * = Both 800003 *[L2VPN/7] 00:06:00 > via t3-0/0/0.512, Pop Offset: 4 t3-0/0/0.512 *[L2VPN/7] 00:06:00 > via t1-0/1/0.0, Push 800003, Push 100004(top) Offset: -4 bgp.l3vpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) Restart Complete inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete L2VPN.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete + = Active Route, - = Last Active, * = Both 10.245.14.176:512:512:611/96 *[L2VPN/7] 00:06:01 Discard bgp.l2vpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete
Router PE1 Status During a Restart
Before you can verify that graceful restart is working, you must simulate a router restart. To cause the routing process to refresh and simulate a restart, use the restart routing operational mode command:
user@PE1> restart routing
Routing protocol daemon started, pid 3558
The following sample output is captured during the router restart:
user@PE1> show bgp neighbor
Peer: 10.96.103.2 AS 65100 Local: 10.96.103.1 AS 65103 Type: External State: Active Flags: <ImportEval> Last State: Idle Last Event: Start Last Error: None Export: [ BGP-INET-import ] Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily PeerAS Refresh> Address families configured: inet-unicast Local Address: 10.96.103.1 Holdtime: 90 Preference: 170 Number of flaps: 0 Peer: 10.245.14.182+179 AS 69 Local: 10.245.14.176+2131 AS 69 Type: Internal State: Established Flags: <ImportEval> Last State: OpenConfirm Last Event: RecvKeepAlive Last Error: None Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily Rib-group Refresh> Address families configured: inet-vpn-unicast l2vpn Local Address: 10.245.14.176 Holdtime: 90 Preference: 170 Number of flaps: 0 Peer ID: 10.245.14.182 Local ID: 10.245.14.176 Active Holdtime: 90 Keepalive Interval: 30 NLRI for restart configured on peer: inet-vpn-unicast l2vpn NLRI advertised by peer: inet-vpn-unicast l2vpn NLRI for this session: inet-vpn-unicast l2vpn Peer supports Refresh capability (2) Restart time configured on the peer: 120 Stale routes from peer are kept for: 300 Restart time requested by this peer: 120 NLRI that peer supports restart for: inet-vpn-unicast l2vpn NLRI peer can save forwarding state: inet-vpn-unicast l2vpn NLRI that peer saved forwarding for: inet-vpn-unicast l2vpn NLRI that restart is negotiated for: inet-vpn-unicast l2vpn NLRI of received end-of-rib markers: inet-vpn-unicast l2vpn Table bgp.l3vpn.0 Bit: 10000 RIB State: BGP restart in progress RIB State: VPN restart in progress Send state: in sync Active prefixes: 10 Received prefixes: 10 Suppressed due to damping: 0 Table bgp.l2vpn.0 Bit: 20000 RIB State: BGP restart in progress RIB State: VPN restart in progress Send state: in sync Active prefixes: 1 Received prefixes: 1 Suppressed due to damping: 0 Table BGP-INET.inet.0 Bit: 30000 RIB State: BGP restart in progress RIB State: VPN restart in progress Send state: in sync Active prefixes: 2 Received prefixes: 2 Suppressed due to damping: 0 Table OSPF.inet.0 Bit: 60000 RIB State: BGP restart is complete RIB State: VPN restart in progress Send state: in sync Active prefixes: 2 Received prefixes: 2 Suppressed due to damping: 0 Table RIP.inet.0 Bit: 70000 RIB State: BGP restart is complete RIB State: VPN restart in progress Send state: in sync Active prefixes: 2 Received prefixes: 2 Suppressed due to damping: 0 Table STATIC.inet.0 Bit: 80000 RIB State: BGP restart is complete RIB State: VPN restart in progress Send state: in sync Active prefixes: 1 Received prefixes: 1 Suppressed due to damping: 0 Table L2VPN.l2vpn.0 Bit: 90000 RIB State: BGP restart is complete RIB State: VPN restart in progress Send state: in sync Active prefixes: 1 Received prefixes: 1 Suppressed due to damping: 0 Last traffic (seconds): Received 0 Sent 0 Checked 0 Input messages: Total 14 Updates 13 Refreshes 0 Octets 1053 Output messages: Total 3 Updates 0 Refreshes 0 Octets 105 Output Queue[0]: 0 Output Queue[1]: 0 Output Queue[2]: 0 Output Queue[3]: 0 Output Queue[4]: 0 Output Queue[5]: 0 Output Queue[6]: 0 Output Queue[7]: 0 Output Queue[8]: 0 user@PE1> show route instance detail master: Router ID: 10.245.14.176 Type: forwarding State: Active Restart State: Pending Path selection timeout: 300 Tables: inet.0 : 17 routes (15 active, 1 holddown, 1 hidden) Restart Pending: OSPF LDP inet.3 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Pending: OSPF LDP iso.0 : 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete mpls.0 : 23 routes (23 active, 0 holddown, 0 hidden) Restart Pending: LDP VPN bgp.l3vpn.0 : 10 routes (10 active, 0 holddown, 0 hidden) Restart Pending: BGP VPN inet6.0 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete bgp.l2vpn.0 : 1 routes (1 active, 0 holddown, 0 hidden) Restart Pending: BGP VPN BGP-INET: Router ID: 10.96.103.1 Type: vrf State: Active Restart State: Pending Path selection timeout: 300 Interfaces: t3-0/0/0.103 Route-distinguisher: 10.245.14.176:103 Vrf-import: [ BGP-INET-import ] Vrf-export: [ BGP-INET-export ] Tables: BGP-INET.inet.0 : 6 routes (5 active, 0 holddown, 0 hidden) Restart Pending: VPN L2VPN: Router ID: 0.0.0.0 Type: l2vpn State: Active Restart State: Pending Path selection timeout: 300 Interfaces: t3-0/0/0.512 Route-distinguisher: 10.245.14.176:512 Vrf-import: [ L2VPN-import ] Vrf-export: [ L2VPN-export ] Tables: L2VPN.l2vpn.0 : 2 routes (2 active, 0 holddown, 0 hidden) Restart Pending: VPN L2VPN OSPF: Router ID: 10.96.101.1 Type: vrf State: Active Restart State: Pending Path selection timeout: 300 Interfaces: t3-0/0/0.101 Route-distinguisher: 10.245.14.176:101 Vrf-import: [ OSPF-import ] Vrf-export: [ OSPF-export ] Tables: OSPF.inet.0 : 8 routes (7 active, 1 holddown, 0 hidden) Restart Pending: OSPF VPN RIP: Router ID: 10.96.102.1 Type: vrf State: Active Restart State: Pending Path selection timeout: 300 Interfaces: t3-0/0/0.102 Route-distinguisher: 10.245.14.176:102 Vrf-import: [ RIP-import ] Vrf-export: [ RIP-export ] Tables: RIP.inet.0 : 8 routes (6 active, 2 holddown, 0 hidden) Restart Pending: RIP VPN STATIC: Router ID: 10.96.100.1 Type: vrf State: Active Restart State: Pending Path selection timeout: 300 Interfaces: t3-0/0/0.100 Route-distinguisher: 10.245.14.176:100 Vrf-import: [ STATIC-import ] Vrf-export: [ STATIC-export ] Tables: STATIC.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden) Restart Pending: VPN __juniper_private1__: Router ID: 0.0.0.0 Type: forwarding State: Active
user@PE1> show route instance summary
Instance Type Primary rib Active/holddown/hidden master forwarding inet.0 15/0/1 iso.0 1/0/0 mpls.0 35/0/0 l3vpn.0 0/0/0 inet6.0 2/0/0 l2vpn.0 0/0/0 l2circuit.0 0/0/0 BGP-INET vrf BGP-INET.inet.0 5/0/0 BGP-INET.iso.0 0/0/0 BGP-INET.inet6.0 0/0/0 L2VPN l2vpn L2VPN.inet.0 0/0/0 L2VPN.iso.0 0/0/0 L2VPN.inet6.0 0/0/0 L2VPN.l2vpn.0 2/0/0 OSPF vrf OSPF.inet.0 7/0/0 OSPF.iso.0 0/0/0 OSPF.inet6.0 0/0/0 RIP vrf RIP.inet.0 6/0/0 RIP.iso.0 0/0/0 RIP.inet6.0 0/0/0 STATIC vrf STATIC.inet.0 4/0/0 STATIC.iso.0 0/0/0 STATIC.inet6.0 0/0/0 __juniper_private1__ forwarding __juniper_priva.inet.0 0/0/0 __juniper_privat.iso.0 0/0/0 __juniper_priv.inet6.0 0/0/0 user@PE1> show route protocol l2vpn inet.0: 16 destinations, 17 routes (15 active, 1 holddown, 1 hidden) Restart Pending: OSPF LDP inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Pending: OSPF LDP BGP-INET.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden) Restart Pending: VPN OSPF.inet.0: 7 destinations, 8 routes (7 active, 1 holddown, 0 hidden) Restart Pending: OSPF VPN RIP.inet.0: 6 destinations, 8 routes (6 active, 2 holddown, 0 hidden) Restart Pending: RIP VPN STATIC.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden) Restart Pending: VPN iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Restart Complete mpls.0: 24 destinations, 24 routes (24 active, 0 holddown, 0 hidden) Restart Pending: LDP VPN + = Active Route, - = Last Active, * = Both 800001 *[L2VPN/7] 00:00:13 > via t3-0/0/0.512, Pop Offset: 4 t3-0/0/0.512 *[L2VPN/7] 00:00:13 > via t1-0/1/0.0, Push 800003, Push 100004(top) Offset: -4 bgp.l3vpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) Restart Pending: BGP VPN inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete L2VPN.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Pending: VPN L2VPN + = Active Route, - = Last Active, * = Both 10.245.14.176:512:512:611/96 *[L2VPN/7] 00:00:13 Discard bgp.l2vpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Restart Pending: BGP VPN