Appendix: Test Case Example Information

Virtual Test Lab

The examples in this appendix to the JVDE are evaluated in a virtual test lab consisting of a vJunos-switch , a vMX router, and vSRX V3.0 firewalls. We did not create a pair of virtual service block switches but ensured that both types of WAN routers (router or firewall) were available as redundant pairs. This is not the same lab used for testing that required the use physical hardware. We use this as an example and something you can potentially build yourself with environments such as EVE-NG to build your own labs to test the configuration examples. The fabric, in this case, was configured as IP Clos.

L2 Exit with Stretched VLAN

Note:

When you create any VLAN or VRF creation with campus fabric remember the following best practices:

• Do not use this method in a production environment You must use the transport VLAN method instead.
• Create all VLANs in a switch template and then import them in the campus fabric dialogue. Creating the VLANs anywhere else in the Mist GUI ultimately leads to inconsistency which makes it hard to resolve issues.
• If needed, the fabric creates any required VRFs. Do not create VRFs manually elsewhere in the Mist GUI.
• We recommend that you create port profiles within switch templates so that any changes are in sync on all switches in a fabric.

The following configuration is the exported version of the switch template used in this fabric. Use this to review your setup when importing. As you can see, there is only one VLAN per VRF hence a stretched approach is required.

When inside the Campus Fabric Configuration dialogue there is a page called Configure Networks. This is where you import your VLANs from the switch template. In this case, see Figure 1 for the result.

The next step is to create all 3 VRFs and attach one of the networks to each as shown in Figure 2 .

Next, you go to each VRF and add a manual route to the VIP that your WAN router has in each subnet. The VRF configurations are shown in the following three images.

Continue with the Fabric Configuration dialogue until it starts building the fabric.

Core1 and Core2 Switch Configuration

The service block function is virtual and co-located on the core switch, hence we must configure the two core switches. This following pseudo-code is a description of what to configure on switches core1 and core2:

The following three screenshots show a major part of the above-described configuration. We start with the port profile. Keep in mind that only the stretched VLANs used are included.

The second uplink configuration is very similar. Only the AE Index changes from 11 to 12.

Note:

You must ensure that the AE Indexes on each service block function are in sync with each other towards the same WAN router and that you define them each as ESI-LAG. You must also ensure that you don’t reuse an AE Index that is already defined elsewhere in the fabric.

Juniper MX as the WAN Router

The following is the configuration of the interfaces, the VRRP gateway redundancy, and the static routes as example. You might need to add default routes and interfaces to complete the configuration.

On the second WAN router, the notable configuration changes are usually AE keys, indexes, and the static IP addresses.

You may wonder about those static routes in the 172.16.19x.0 range. Remember that IP Clos is an anycast fabric. As such, you must have the static routes to prepare for when the DHCP relay will use IP addresses in the fabric overlay. You can see the definition in Figure 6 :

These overlay loopback IPs are assigned to each VRF on a switch as a /24 range. You can figure them by looking at a fabric access switch as shown in Figure 7 . You must map them back as any other additional VLAN attached to the VRF to achieve the required reachability.

L2 Exit with Transport VLAN

Note:

When doing any VLAN or VRF creation with campus fabric remember the following best practices:

1. Create all VLANs in a switch template and then import them in the Campus Fabric dialogue. Creating the VLANs anywhere else in the Mist GUI ultimately leads to inconsistency which makes it hard to resolve issues.
2. With the exception of the service block functions, do not create VRFs outside of the Campus Fabric dialogue.
3. The transport VLAN method requires you to create VRFs manually on the service block function and add the transport VLAN and routes locally to the VRFs. Do not create the VRFs or routes in the Campus Fabric dialogue.
4. We recommend that you create port profiles within switch templates so that any changes are in sync on all switches in the fabric.

When defining the transport VLANs in the switch template, do not set the subnet information. You configure this information later as Additional IP Subnet on each service block function. See Figure 8 , Figure 9 , and Figure 10 .

The following CLI configuration shows the exported version of the switch template used in the transport VLAN fabric. This allows you to review our setup when importing. As you can see, there is a minimum of two VLANs per VRF plus an additional transport VLAN per VRF.

Within the Campus Fabric Configuration dialogue, there is a section called Configure Networks. This is where you import your six access VLANs from the switch template. When finished, the configuration should be as shown in Figure 11 and the result in our case will look as shown below. Since the three transport VLANs are not part of the access layer, they are not defined in the service block function.

Next, you create 3 VRFs and attach two of the access networks to each VRF as shown in Figure 12 .

Next, go to each VRF and confirm that you only have access networks defined with no default route. You will define the transport VLANs and default routes later in the service block function. See Figure 13, Figure 14 , and Figure 15 .

Core1 and Core2 Switch Configuration

In the transport VLAN attach example, the service block function is virtual and co-located on the core switch. Therefore, you must configure the two core switches. The following pseudocode represents the configuration you must apply to the core1 and core2 switches:

The following four images display the Mist GUI configuration that results from the previous pseudocode starting with the additional IP configuration required to assign the local IP addresses to each transport VLAN.

Next, you define the Port Profile used for the uplinks. It is critical that you only include the transport VLAN in the Trunk Networks definition since only those VLANs are used and visible to the WAN router.

Next, you assign the port profiles to each uplink port.

Figure 22 shows the configuration of the first uplink to the first WAN router.

Figure 23 shows the configuration of the second uplink to the first WAN router.

Note:

You must ensure that the AE Indexes on each service block function are in sync with each other towards the same WAN router and that you define them each as ESI-LAG. You must also ensure that you don’t reuse an AE Index that is already defined elsewhere in the fabric service block.

Next you create and modify local VRFs. Remember this is an exception made only for the transport VLAN exit method. Usually, the fabric creates the VRFs automatically. In this case we must enable the Override Site/Template Settings checkbox in the VRF configuration. Figure 24 shows the required configuration in the Mist GUI.

Next you must perform the following three configurations in each of your three VRS instances:

• Enable Override Template Defined VRF Instance checkbox
• Add your transport VLAN to the pre-populated list of access VLANs
• Add a default route where the gateway IP address is the VRRP-VIP address of your WAN router.
• Figure 25 , Figure 26 , and Figure 27 show the override configurations for each of the three VRFs.

Now you must configure additional CLI to modify the transport VLANs to use VGA configuration to help avoid excess hair-pin routing of traffic within the fabric. In the switch configuration for each of your service block function switches, locate the CLI Configuration section in the Mist GUI. You must paste the required configuration into the field indicated in Figure 28 .

The example CLI configuration for your core1 switch, is shown in the following code block. We have configured the static IP address as the virtual gateway IP address + 1 (10.99.1.2).

For your core2 switch, only the static IP addresses of the transport VLAN are changed to be the virtual gateway IP address + 2 (10.88.1.3).

Note:

Keep in mind that our test lab used virtual EX9214 switches as core switches. In most production environments you will not use an EX92xx switch. Therefore, you must uncomment the two lines that are commented out in the previous configuration snippet:

Note:

Service block for each transport VLAN used per VRF you must manually set the MAC address of the virtual gateway address used on the IRB interface. In our example, we used different MAC addresses per transport VLAN because it’s easier to debug.

Juniper MX as WAN Router

The following CLI snippet example contains the configuration of the interfaces, the VRRP gateway redundancy, and the static routes for the first WAN router. You may need to add default routes and interfaces to complete the configuration.

On the second WAN router, the notable configuration changes are the AE keys and indexes, and the static IP addresses.

You may wonder about those static routes in the 172.16.19x.0 range. Remember that IP Clos is an anycast fabric. As such, you must have th static routes to prepare for when the DHCP relay will use IP addresses in the fabric overlay. See Figure 29 for an example definition.

The overlay Loopbacks IPs are assigned to each VRF on a switch as a /24 range. You can figure them out by looking at a fabric access switch as shown in Figure 30 . Hence, you must map them back like any other additional VLAN attached to the VRF to achieve the required reachability.

The following commands help to debug the connections on WAN router1.

The following commands help you to debug connections on WAN router2.

L3 Exit With eBGP Routing Protocol

Note:

When you create any VLAN or VRF creation with campus fabric remember the following best practices:

• Create all VLANs in a switch template and then import them into the Campus Fabric Dialogue. Creating the VLANs anywhere else in the Mist GUI ultimately leads to inconsistency which makes it hard to resolve issues.
• If needed, the fabric creates any required VRFs. Do not create VRFs manually elsewhere in the Mist GUI.
• We recommend that you create port profiles within switch templates so that any changes are in sync on all switches in a fabric.

Before you begin, you need a plan for:

• How to implement the routing protocol and route exchange
• How to configure the P2P links
• How to distribute the VLAN assignment that is indirectly used to identify the VRF

Even if the VRF already exists elsewhere in the fabric, such as on the access switch for IP Clos, the system will automatically re-create it on all service block functions when doing an L3 exit.

For each WAN router, you must reuse a VLAN name on a VRF to help the automatic creation of VRFs on the service block function. Keep in mind that when you define the local P2P links and reuse the VLAN, those definitions are purely local, so they do not conflict with the overlay VLAN definition. Additionally, you do not need to define special transport VLANs here. However, you can still use and define special transport VLANs for the P2P links if that better suits your needs.

When defining the P2P links, you must ensure that they are outside of the address range in use by the fabric. The default range used by the fabric for these links is 10.255.240.0/20. We recommend that you a /31 netmask. With that plan, you can use the even number IP addresses for the WAN router side and the odd IP addresses for the fabric side.

The system requires that you use a VLAN for each P2P link on a physical cable. This allows you to have multiple VRFs multiplexed on a single uplink cable. Remember the VLAN internally refers back to the VRF.

For eBGP you must also define your own private ASN for peering. By hardcoded default, the fabric uses 65000 ASN for the EVPN control plane and starts allocating configurable ASN at 65001. After that, it advances one digit for each node. Therefore, we recommend using ASN values below 65000 to avoid conflict with system assigned ASN. The QFX switch only allows 16 local ASN. Therefore, we recommend that you use a shared ASN among all VRFs. However, in our example, we decided to use a different ASN per WAN router.

Figure 31 shows how the two service block functions of the fabric would connect to the first WAN router.

Figure 32 shows how the two service block functions of the fabric would connect to the second WAN router. Notice that we now use the second block of VLANs from each VRF.

Table 1 displays the full configuration between the core1 and core2 switches, as service block function, and the two WAN routers. You can also see the ASN chosen for eBGP.

Table 1: MX WAN Router and Core Switch Configuration Summary for eBGP Exit

Switch	Switch AS	VRF	Core P2P IP	Core IF	WAN Router	WAN Router P2P IP	WAN Router AS	WAN Router IF	VLAN-ID
core1	64911	customera	10.255.224.1/31	ge-0/0/3.1091	wanrouter1	10.255.224.0/31	64901	ge-0/0/1.1091	1091
core1	64911	customerb	10.255.224.3/31	ge-0/0/3.1081	wanrouter1	10.255.224.2/31	64901	ge-0/0/1.1081	1081
core1	64911	devices	10.255.224.5/31	ge-0/0/3.1031	wanrouter1	10.255.224.4/31	64901	ge-0/0/1.1031	1031
core1	64911	customera	10.255.225.1/31	ge-0/0/4.1099	wanrouter2	10.255.225.0/31	64902	ge-0/0/1.1099	1099
core1	64911	customerb	10.255.225.3/31	ge-0/0/4.1088	wanrouter2	10.255.225.2/31	64902	ge-0/0/1.1088	1088
core1	64911	devices	10.255.225.5/31	ge-0/0/4.1033	wanrouter2	10.255.225.4/31	64902	ge-0/0/1.1033	1033
core2	64911	customera	10.255.226.1/31	ge-0/0/3.1091	wanrouter1	10.255.226.0/31	64901	ge-0/0/2.1091	1091
core2	64911	customerb	10.255.226.3/31	ge-0/0/3.1081	wanrouter1	10.255.226.2/31	64901	ge-0/0/2.1081	1081
core2	64911	devices	10.255.226.5/31	ge-0/0/3.1031	wanrouter1	10.255.226.4/31	64901	ge-0/0/2.1031	1031
core2	64911	customera	10.255.227.1/31	ge-0/0/4.1099	wanrouter2	10.255.227.0/31	64902	ge-0/0/2.1099	1099
core2	64911	customerb	10.255.227.3/31	ge-0/0/4.1088	wanrouter2	10.255.227.2/31	64902	ge-0/0/2.1088	1088
core2	64911	devices	10.255.227.5/31	ge-0/0/4.1033	wanrouter2	10.255.227.4/31	64902	ge-0/0/2.1033	1033

The code block below shows the exported version of the switch template used in this fabric. This allows you to review our setup when importing. As you can see, we have a minimum of two VLANs per VRF. Remember that the L3 exit model requires one VLAN per WAN router and VRF).

Within the Campus Fabric Configuration dialogue, there is a page called Configure Networks. This is where you import your six VLAN’s from the switch template. The resulting configuration is shown in the following figures.

Then you go to each VRF and delete all manual routes you may have. Make sure each VRF has a minimum of two VLAN’s attached as those are used to identify the VRF later.

Core1 Switch Configuration

In this example, the service block function is virtual and co-located on the core switch. Therefore, you must configure the two core switches. The following block of pseudocode describes what you need to configure on the core1 switch:

The following screenshots show the previous configuration translated into the Mist GUI. We start with the additional IP configuration. Notice that the VLAN IP addresses do not match the IP addresses that these VLANs had originally in the overlay. This is by design. You must always have the VLAN as a reference back to the VRF.

In the Port Configuration window, you must enable the L3-sub-interfaces and assign the first 3 sub-interfaces defined.

In the second Port Configuration window, towards the other WAN router, assign the second 3 sub-interfaces defined.

Next, you must enter the entire eBGP configuration with all six peers (three VRFs and two WAN routers. When finished, the overview page should be as shown in Figure 40 .

First, you define two routing policies, a summary of which is shown in the above table.

The export route policies contain a subnet for each VLAN in your VRFs and a definition for the loopback-per-VRF subnet that is part of the definition in the initial fabric dialogue. You can substitute a single 0.0.0.0/0-32 prefix for all six prefixes. Writing the prefix as 0.0.0.0/0-32 is a way of defining orlonger in the Junos OS.

The import policy imports the default route from the WAN router.

Figure 42 shows the configuration of a single BGP peering entry with the required entries called out.

You must also define the WAN router as a BGP neighbor.

You may see a warning message as shown in Figure 43 . It is safe to ignore those.

Core2 Switch Configuration

The following pseudocode represents the configuration you must apply to the core2 switch:

Aside from the P2P subnets and the BGP neighbours, the configuration on the core2 switch is the same as the configuration on the core1 switch.

Juniper MX as a WAN Router

The following several code blocks show the Junos OS CLI configuration of the P2P interfaces, the entire eBGP config with all BGP neighbours, and all import and export route policies for each WAN router. You may need to add default routes and interfaces to complete the configuration because those need to be signalled to the fabric but we don’t know how your device gets to know those.

CLI configuration for the first WAN router:

Configuration for the second WAN router:

You can use the following CLI commands to assist with debugging on WAN router1.

You can use the following CLI commands to assist with debugging on WAN router2.

Juniper SRX Series Firewall as WAN Router

The following example table and configurations show the differences between using an SRX Series Firewall in cluster mode and an MX router as the WAN router. On the fabric side, only the interface names of the SRX cluster change from the MX router configuration. Because the SRX Series Firewall runs in active/active cluster mode, there is only a single WAN router configuration and a single ASN to consider. That single configuration also includes cluster management and trust-zone management commands that are not present in a similar MX router-based configuration.

This SRX Series Firewall -based approach is less complicated than configuring redundant ethernet (reth) interfaces and link aggregation groups (LAG) on the MX router. In addition, there is need for additional CLI on the fabric side to insert virtual gateways, and so on.

Table 2 shows the configuration information for the core1 and core2 switches as service block function along with the WAN router configuration for the SRX cluster. We’ve marked the changes with respect to Table 1 (for MX WAN routers in bold).

Table 2: SRX Series Firewall WAN Router and Core Switch Configuration Summary

Switch	Switch AS	VRF	Core P2P IP	Core IF	WAN Router	WAN Router P2P IP	WAN Router AS	WAN Router IF	VLAN-ID
core1	64911	customera	10.255.224.1/31	ge-0/0/5.1091	node0	10.255.224.0/31	64901	ge-0/0/2.1091	1091
core1	64911	customerb	10.255.224.3/31	ge-0/0/5.1081	node0	10.255.224.2/31	64901	ge-0/0/2.1081	1081
core1	64911	devices	10.255.224.5/31	ge-0/0/5.1031	node0	10.255.224.4/31	64901	ge-0/0/2.1031	1031
core1	64911	customera	10.255.225.1/31	ge-0/0/6.1099	node1	10.255.225.0/31	64901	ge-7/0/2.1099	1099
core1	64911	customerb	10.255.225.3/31	ge-0/0/6.1088	node1	10.255.225.2/31	64901	ge-7/0/2.1088	1088
core1	64911	devices	10.255.225.5/31	ge-0/0/6.1033	node1	10.255.225.4/31	64901	ge-7/0/2.1033	1033
core2	64911	customera	10.255.226.1/31	ge-0/0/5.1091	node0	10.255.226.0/31	64901	ge-0/0/3.1091	1091
core2	64911	customerb	10.255.226.3/31	ge-0/0/5.1081	node0	10.255.226.2/31	64901	ge-0/0/3.1081	1081
core2	64911	devices	10.255.226.5/31	ge-0/0/5.1031	node0	10.255.226.4/31	64901	ge-0/0/3.1031	1031
core2	64911	customera	10.255.227.1/31	ge-0/0/6.1099	node1	10.255.227.0/31	64901	ge-7/0/3.1099	1099
core2	64911	customerb	10.255.227.3/31	ge-0/0/6.1088	node1	10.255.227.2/31	64901	ge-7/0/3.1088	1088
core2	64911	devices	10.255.227.5/31	ge-0/0/6.1033	node1	10.255.227.4/31	64901	ge-7/0/3.1033	1033

The following pseudocode describes what you need to configure on the core1 switch for this example:

The following pseudocode describes what you need to configure on the core2 switch for this example:

When finished with configuring the individual service block functions (here core1 and core2 Switch) your overview table should be as shown in Figure 46 .

Figure 46 shows an overview of how the BGP looks after you have configured the individual service block functions for the core1 and core2 switches.

The following Junos OS CLI represents the entire configuration needed on the Series Firewall cluster for this example.