Creating Layer 3 Fabrics

You can create and manage 3-stage Layer 3 Fabrics in Network Director by using the Create Layer 3 Fabrics wizard. Use the various pages of the wizard to specify the requirements and configurations for a Layer 3 Fabric. You can save the data that you have entered in one or more wizard pages, and come back later to specify the remaining details and complete the fabric creation.

CAUTION:

Ensure that you always create the Layer 3 Fabric using this wizard and perform the physical connections based on the cabling plan that Network Director generates for your fabric. Not following this set order might render your Layer 3 Fabric defunct.

Before you begin, ensure that you have the necessary privileges on the FTP and the file server that Network Director uses for Zero Touch Provisioning. For more details, see User Privileges Required for the DHCP and File Server While Using Zero Touch Provisioning.

You can do the following tasks from the Create Layer 3 Fabric wizard pages:

Specifying the Fabric Requirements

To specify the fabric requirements:

Enter a name for the Layer 3 Fabric. The fabric name must be unique and can contain alphanumerals, hyphens, and underscores.
Enter a description for the Layer 3 Fabric.
Note:
Network Director currently enables you to create 3-stage Layer 3 Fabrics and hence this is the default selection. You cannot modify the Fabric Type.
Select QFX10008, QFX10002-36Q, QFX10002-72Q, QFX5100-24Q-2P or QFX5200-32C-32Q as the device model for spine devices. All spine devices will be of the model that you select.
Note:
If you select QFX10008 as spine, all the line cards must be homogenous across spines. For example, if you are building an IP fabric with four QFX10008 spines containing L1, L2, and L3 line cards, all the four spines must have L1, L2, and L3 line cards only and in the same slots.
Enter the number of spine devices that you plan to have initially and the maximum number of devices that you plan to have in this fabric, in the Initial Capacity and Max Capacity boxes respectively. You can have a minimum of 2 and a maximum of 8 spine devices.
Note:
Initial capacity must be less than or equal to the maximum capacity. Maximum capacity must be greater than or equal to the initial capacity and must not be more than 8.
If you selected QFX10008 as the spine device you must build the device chassis using the chassis builder.
To do this:
1. Click Build New Chassis to create a new chassis.
  The Build Chassis window opens. The Build Chassis window has two panes— the Available line cards pane and the Chassis: FPC slots pane.
  
  The Available line cards pane lists the line cards that are supported on the selected aggregation device and the Chassis: FPC slots pane lists the available FPC slots on the device.
2. Drag and drop the line cards that you want to add to the chassis from the Available line cards pane to the appropriate FPC slots in the Chassis: FPS slots pane.
3. Click Set after you have added all the required line cards to the FPC slots. The Build Chassis window closes.
4. Mouse over Preview to preview the chassis with the line cards that you added.
In the Fabric leaves section, click a row in the table to select a leaf device model and specify the capacity of the selected model that you plan to have in the fabric. Click Add to add subsequent rows.
Note:
This is an optional step, however, it is mandatory to specify the maximum number of leaf devices you plan to have in this fabric. If you do not add any leaf devices, Network Director considers these devices as unknown and creates a cabling plan accordingly. After the fabric is deployed, you can plug and play any of the supported leaf device models to the fabric. After reaching the initial capacity for the spine devices, Network Director regenerates the cabling plan. Follow this plan to connect additional spine devices.

You can add one or more of the following device models as leaf devices in your Layer 3 Fabric:
- QFX5100-48S-6Q
- QFX5100-96S-8Q
- QFX5100-48T-6Q
- QFX5200-32C-32Q
- QFX5100-24Q-2P
- EX4300-32F
- EX4300-48P
- EX4300-24P
- EX4300-48T
- EX4300-24T
Note:
QFX5100-48T-6Q can be standalone or Virtual Chassis leaf devices. QFX10008, QFX10000-36Q, QFX10000-30C, QFX10000-60S-6Q, and QFX5200 are supported only as standalone devices.

Network Director supports a maximum of two members in a Virtual Chassis.

If you want to delete a device entry, select a row and click Remove.

If you want to include Virtual Chassis as a leaf device, select Include Virtual Chassis (VC) as a leaf.

Enter the number of Virtual Chassis that you want to deploy immediately in Initial Capacity and the total number of Virtual Chassis that will be part of the Layer 3 Fabric in Max. Capacity. The minimum number of devices you can specify in Initial Capacity is 0. The Max. Capacity is the maximum number of devices you can specify, which depends on the spine device that you have selected. See Table 1.

Table 1: Maximum Virtual Chassis Supported on Spine Devices
If you choose the spine device as...	then, the maximum number of virtual chassis leaf devices supported is...
QFX5100-24Q-2P	16
QFX10002-36Q	18
QFX5200-32C-32Q	16
QFX10002-72Q	36
QFX10008	~144 Note: Depends upon the type of line card connected.

Note:

Plug and play is not supported for Virtual Chassis leaf members. Therefore, before you physically connect the Virtual Chassis members, make sure that you add the Virtual Chassis leaf members by using this Layer 3 Fabric wizard.

Initial capacity must be less than or equal to the maximum capacity.

For example, if your selected spine model is QFX5100-24Q-2P and if all of the leaf device members are Virtual Chassis, each containing 2 members, then the maximum number of Virtual Chassis leaf devices is restricted to 16, as there is a connection from both the primary and backup member of the Virtual Chassis to each spine device. See Table 2 for the maximum number of devices supported on various spine devices.

Note:

You cannot modify the number of Virtual Chassis after the

Layer 3 Fabric is created.

Network Director helps in creating the access link aggregation group (LAG) between Virtual Chassis members and host access devices. Network Director creates the access LAG in either of the two ways.

Dynamic LAG creation—As the access devices are connected to the Virtual Chassis members, Network Director creates the LAG (if there are more than one connection between the access device and the Virtual Chassis members) dynamically. To identify the connected links for LAG creation, Network Director uses the Topology Discovery, which requires LLDP to be enabled in both the host and leaf members. For Network Director to create the LAG dynamically, ensure that LLDP is enabled in both the host and leaf (Virtual Chassis) devices.
Preprovisioning LAG configuration on Virtual Chassis members—If LLDP is not enabled in the access or host devices, Network Director generates the LAG configuration on the Virtual Chassis member devices during the workflow creation and pushes the configuration to the Virtual Chassis members when they are connected to the network. The LAG interfaces are depicted in the cabling plan graph and in the grid view generated by Network Director. You must connect the host devices to the Virtual Chassis member devices according to the cabling plan.

You can enable Network Director to create the LAG as the physical connections are established. Select the Dynamically create LAG when hosts are connected check box.

Enter the maximum number of leaf devices, which includes standalone and Virtual Chassis devices, that the fabric can accommodate in Max Capacity. The minimum value you can enter is 1 and the maximum value depends on the spine device that you choose. See Table 2.

Table 2: Maximum Number of Leaves
If you choose the spine device as...	then, the maximum number of leaves is...
QFX5100-24Q-2P	32
QFX10002-36Q	36
QFX10002-72Q	72
QFX10008 QFX10000-36Q QFX10000-60S-6Q QFX10000-30C	< 288 < 48 < 240
QFX5200-32C	32

Do one of the following:
- Click Next to open the Devices page where you can view and modify details of the spine and the leaf devices.
- Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

Specifying the Device Details

The Devices page displays the number of spine and leaf devices that you are provisioning as part of the initial capacity, enables you to edit the hostname for all the spine and leaf devices. Select a model for each member of Virtual Chassis if you have opted for Virtual Chassis leaves, and search for a specific device in the fabric.

Network Director prefixes the name of the fabric that you specified in the Fabric Requirements page to the name of all the spine and leaf devices. If required, you can modify this prefix in the Devices page. You can also use the search box to search for specific devices in the fabric.

To specify the device details:

Click Edit Host Name Prefix if you want to change the device name prefix to something other than the name of the fabric. The Edit Host Name window opens.
Enter the name that you want to use as the device name and click OK.
Network Director replaces the device name prefix with the name that you entered.

The Devices page displays the details of the hostname and the devices associated with it. See Table 3.

Note:

The details of the device in each row, which is colored blue are to be provisioned now, and those colored orange are reserved for future allocation.

Table 3: Devices Page Description
Column	Description
Host Name	Displays the hostname with the name of the fabric, which you specified in the Fabric Requirement page.
Model	Displays the model of the switch. If you have selected Virtual Chassis to be included in your Layer 3 Fabric in the Fabric Requirements page, the `Type` of the model will be `Virtual Chassis` and the `Model` is not displayed. You can select the switch model for the Virtual Chassis member from the drop-down list, which lists all supported Virtual Chassis members. Note: It is mandatory to select the switch model for Virtual Chassis member that you are provisioning now. For the Virtual Chassis members that are `Reserved for future` you may select the model later.
Type	Displays the type of switch—standalone, virtual chassis, virtual chassis member, or FPC.
Role	Displays the role being played by the switch model.

Do one of the following:
- Click Next to open the Configuration page, where you can specify the configuration details of the Layer 3 Fabric.
- Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

Specifying Configuration Details

To specify the configuration details for the Layer 3 Fabric:

Enter details in the Configuration page by following the descriptions given in Table 4.

Table 4: Layer 3 Fabric Configuration Details
Field	Description
Loopback Network Address	Specify the IP address block that you want to use for configuring the loopback interface in each member. Each device in the fabric is assigned one IP address from the block. This IP address can be used for troubleshooting and for checking connectivity between switches.
Interconnect Network Address	Specify the IP address block that you want to use for configuring the IP addresses for interconnect links between leaves and spines. Each interconnect links is assigned two IP addresses from this block.
VLAN Network Address	Specify the IP address block to be reserved for the virtual machines or hosts that you want to connect to the leaves. Network Director allocates each leaf device with a subnet from the given IP address block.
Start Management IP	Specify the management IP address that Network Director will use to manage each switch. Note: If you have provisioned for Virtual Chassis members in the Layer 3 Fabric, each Virtual Chassis member is initially treated as a standalone device and it goes through the ZTP process. The Management IP address block is sufficient to provide individual unique IP address for each of the Virtual Chassis member in the fabric.
Max Hosts/VMs per leaf	Specify the maximum IP addresses that are required in the subnet to be allocated from the VLAN Network Address.
Spine-BGP Autonomous System Number	Specify the starting autonomous system (AS) number to be assigned to the first spine device. Subsequent spine devices are assigned incremental AS numbers starting from the number you specified. Network Director updates the last AS number based on the number of spine devices that you plan to have in the fabric. You cannot modify the last AS number.
Leaf-BGP Autonomous System Number	Specify the starting autonomous system (AS) number to be assigned to the first leaf device. Subsequent leaf devices are assigned incremental AS numbers starting from the number you specified. Network Director updates the last AS number based on the number of leaf devices that you plan to have in the fabric. You cannot modify the last AS number.
Device Password	Specify the default password that you want to set for all the devices in the fabric.
Management Gateway	If Network Director and the Layer 3 Fabric devices are in different subnets, specify the gateway IP address that these devices can use to reach Network Director. Note: This is an optional field if the Layer 3 Fabric and Network Director are in the same subnet.

Do one of the following:
- Click Next to open the Cabling page where you can view the cabling plan for your Layer 3 Fabric. This might take some time depending on the fabric capacity.
- Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

Viewing the Cabling Plan

The Cabling Plan page displays the recommended cabling plan for the device that you select in the left pane. If you specify all the spine and leaf devices, the cabling plan displays the exact port numbers that you must use to connect your spine and leaf devices. However, if you have not specified any leaf devices and have only specified the maximum leaf count, the plan displays all the leaf devices as unknown. The leaf devices in this case are plug-and-play and you can use any of the uplink ports on your plug-and-play leaf device.

This holds good until you have reached the initial capacity of the spine devices. If you are adding an additional spine device, beyond the initial capacity, Network Director regenerates the cabling plan and you must follow the recommended cabling plan for all subsequent spine to leaf connections. Note that the connections to the existing devices need not be changed as part of this change.

Network Director regenerates the cabling plan, if one of the following occurs:

A spine device is added
A spine device is deleted
A leaf device is added
A leaf device is deleted

If the selected spine device model in the Fabric Requirements page is QFX10002-72Q, the cabling plan is represented as two chassis images. The first chassis image displays the connections for the ports in the first and second rows, and the second chassis image displays the connections for the ports in the third and fourth rows.

If the selected spine device model in the Fabric Requirements page is QFX10008, and selected line card model is QFX10000-60S-6Q in the Build New Chassis section, cabling plan is represented in two chassis images. The first image displays connections for the ports in first and third rows, and the second chassis image displays the connections for the ports in the middle row.

From the Cabling page, you can:

View the cable connectivity that you must follow for each device in your fabric. The device table is color coded to identify the devices that are provisioned now (identified by color), the devices reserved for future (identified by color), the Virtual Chassis connections (identified by green color), and access LAG ports (identified by pink color).
Click Grid View to view the cabling plan in a grid. Select a device in the left pane to view the cabling details and access LAG connections of the selected device.
Note:
The Access LAG ports are displayed if you have not selected Dynamically create LAG when hosts are connected in the Fabric Requirement page. Network Director preprovisions the LAG configuration in the Virtual Chassis members.
Click Graph View to view the graphical representation of the cabling plan.
Do one of the following:
- Click Next to open the ZTP page where you can specify the Zero Touch Provisioning (ZTP) details for the Layer 3 Fabric.
- Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

Specifying Zero Touch Provisioning Details

Zero Touch Provisioning (ZTP) enables you to provision devices in your network automatically, without manual intervention. When a device is physically connected, it boots up with factory-default configuration and auto installs a configuration file from the network. In Network Director, the ZTP is used to provision Layer 3 abric and all the configurations are pushed through OpenClos. To specify the ZTP details:

Note:

When you select QFX10008 as the spine model, only the leaf models are provisioned with ZTP configuration. For the spine model QFX10008, you must either copy the config file from Network Director or manually download it from the file server. To copy the config file from the file server, SSH or Telnet must be enabled on the device (QFX10008).

Specify the DHCP server settings by following the descriptions given in Table 5.

Table 5: DHCP Server Details
Field	Description
DHCP Server	IP address or the hostname of the DHCP server.
DHCP Server Type	The type of DHCP server that provides the necessary information to the switch. You can choose to use a CentOS DHCP server, an Ubuntu DHCP server, or any other DHCP server. Note: If you select Other, you must configure the DHCP server settings manually.
Manually Configure Server	Select to indicate that you want to manually configure the DHCP server. You can configure the CentOS and Ubuntu DHCP servers manually or from Network Director. If you want to use any other type of DHCP server, do the following: Select the Manually Configure Server check box. Network Director hides all the other details except the DHCP Server Type. Follow the instructions displayed in this box to configure the DHCP server manually.
DHCP User	Username to log in to the DHCP server.
DHCP Password	Password for the specified username.
Confirm Password	Confirm the DHCP server password.

Note:

When you are replacing a member device—If the member that is replaced is up, Network Director obtains the latest configuration from the replaced device and maps this configuration to the corresponding MAC or serial number in the DHCP server.

However, if the member that is replaced is down, Network Director is not able to reach the device to get its latest configuration. In such case, Network Director maps the configuration that is generated from OpenClos (Stage-2 for leaf devices) for the replaced device to the MAC or serial number of the new device in the DHCP server. Note that the mapped configuration in the DHCP server does not have any configuration that is pushed from Network Director to the device.

Note:

The DHCP server configuration file does not contain entries related to the spine device QFX10008 as the device does not go through ZTP.

Specify the File server settings by following the descriptions given in Table 6.

Table 6: File Server and Software Details
Field	Description
File Server Type	The type of file server where the software images are to be stored. You can choose to use an FTP, HTTP, or an TFTP file server.
File Server	IP address or hostname of the file server.
File Server Root Dir	The root directory of the file server.
Spine Image	The software image file that you want to use for your spine devices. Note: Ensure that the software image is uploaded to Network Director using the Image Management > Manage Image Repository in the Deploy mode. Else Network Director does not display the software image.
Leaf Image	The software image file that you want to use for your leaf devices. Note: Ensure that the software image is uploaded to Network Director using the Image Management > Manage Image Repository in the Deploy mode. Else Network Director does not display the software image. As Network Director supports two device models—EX4300 and QFX5100—and their variants as leaf devices, you can specify a software image for each of these leaf devices irrespective of the variant that you have selected for your fabric. The same software image applies to all the variants of a device.

ZTP process maps the management interface MAC address or the device chassis serial number of each spine device to the device-specific software image, IP address, hostname, and the configuration file stored on the file server. This mapping is stored in the DHCP server. When a spine device starts up, the device contacts the DHCP server to obtain the IP address and the software image location. The DHCP server looks up in its MAC address or serial number mapping database to identify the device and provide details about the file server that the device must contact to get the software image and configuration file. The device uses this information to contact the file server and obtain the software image and the configuration file for deploying on the device.
Do one of the following to specify the MAC address or the serial number of your spine devices:
- Enter the MAC address of the management interface (for example, the em0 interface) or the device chassis serial number of the spine devices in MAC Addresses or Serial Number in the table.
- Click Import MAC Address to import the MAC addresses of spine device in CSV format. You must enter the MAC addresses in the specified format. Click Download CSV format to download a sample CSV file that you can use to import MAC addresses.
  
  Note:
  When you use the Import option, you must specify either the MAC address or the Serial number, but not both.
- Click Import Serial Number to import the serial numbers of spine device in CSV format. You must enter the MAC addresses in the specified format. Click Download CSV format to download a sample CSV file that you can use to import serial numbers.
  
  Note:
  You can specify serial number only for spine devices running Junos OS Release 14.1X53D15 or later.
Note:
Entering serial number or MAC address of the spine device is not applicable for the device model QFX10008 as it is not provisioned through ZTP.
To view the configuration that is deployed on a spine device, click Actions > View Config.
To view the configuration that is deployed initially on the leaf devices, click View Leaf Config.
Do one of the following:
- Click Next to open the Review page where you can review the Layer 3 Fabric settings.
- Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

Reviewing the Layer 3 Fabric Settings

From the Review page you can:

View the DHCP configuration that will be deployed on to the DHCP server by clicking View DHCP Config in the ZTP Settings sub-tab.
The DHCP configuration opens in a new window.
Review the Layer 3 Fabric settings in the Review page, Devices sub-tab and the Configuration sub-tab.
Click Deploy to deploy the Layer 3 Fabric.
Click Save & Exit to save the changes and exit the Create Layer 3 Fabric wizard. You can resume this task later without losing any information that you entered.

ON THIS PAGE

Creating Layer 3 Fabrics

Specifying the Fabric Requirements

Specifying the Device Details

Specifying Configuration Details

Viewing the Cabling Plan

Specifying Zero Touch Provisioning Details

Reviewing the Layer 3 Fabric Settings

Related Documentation