Ubuntu KVM and libvirtd Server BMS Environment

Preparations Before You Deploy

Optional: Install KVM/Qemu if You Have Not Done It Yet

The following commands show how you install the KVM hypervisor and run the required checks. Ensure you are using root directory.

Optional: Create br0-bridge with DHCP Server

Once installed, the hypervisor automatically creates the default virbr0 Linux bridge in the 192.168.122.0/24 prefix. This bridge will source NAT the remaining network interfaces and manage the DHCP server leases. You can connect the vJunos-switch VM fxp0 interface for OOB management. A limitation of this default virbr0 Linux bridge is that the DHCP lease handout is random and unpredictable.

For your lab server, we recommend you create an additional br0 Linux bridge. This Linux bridge must NAT external interfaces to copy the same using the 192.168.10.0/24 prefix.

You’ve assigned a fixed IP to the vEX switches on fxp0 using DHCP, allowing you to SSH to the switches. The standard virbr0-bridge assigns a random IP address from the range 192.168.122.0/24. Instead, you create a simple DHCP server to provide static leases to known MAC addresses on br0 Linux bridge in the range 192.168.10.0/24. Then, start the vJunos-switch VM using one of the MAC addresses on the first interface to assign a predefined IP address to the fxp0 interface.

Create Linux Bridges to Simulate Network Links Between VMs

Use the default virbr0 or the new br0 Linux bridge when you connect the fxp0 interface of a vJunos-switch VM. As all other links need new bridges, create virtual links between devices. In this example, the bridge name tries to copy the Junos OS interface naming conventions.

Default Junos OS Configuration for vJunos-switch

You might need to include a custom Junos OS configuration executed at the starting the vJunos-switch VM for the following main reasons:

• Create a root/supervisor account with a known password for a management system such as, Juniper Apstra to login. This avoids the need for a local DHCP server to push the customized configuration and supports SSH login for better debugging.
• Apply an adopt configuration for the Mist Cloud so the new switch appears in the inventory.
• Load an existing valid configuration from a previous lab at the starting of a new lab.

Customized Factory-Default

Here is an example of minimal Junos OS configuration that you load when creating a file juniper.conf with the following configurations:

• Username=root
• Password=ABC123
• Expected to get a DHCP lease from fxp.0
• Enable LLDP on all interfaces

Optional: Adding Junos OS Configuration for Adoption to Mist Cloud

In this step, you’ll add Junos OS configuration to make the switch appear automatically in the Mist Cloud.

1. Go to Organization -> Inventory.

   

2. Select Switches and then click Adopt Switches.

   

3. Click Copy to Clipboard.

   

4. Open a file adopt-template.txt from your preferred editor and paste the gathered information in this file. Then, save and close the file.
   Note:

   You can use the same Mist Cloud adoption code to onboard all switches in the same organization.

5. Convert the information using a template to change it in a different Junos OS configuration format. First, you must extract five variables from the file adopt-template.txt using bash shell commands.

In this step, you add back the Junos OS configuration, which does not depend on set commands.

Creating Virtual Disk with Junos OS Configuration for VM

1. Create an HD image to load your custom configuration using the original bash script make-config.sh from the vJunos-switch support site.

   

2. Download the image through a link. For example: https://webdownload.juniper.net/swdl/dl/anon/site/1/record/168885.html.

   If you can’t find an image, use the copy below.

3. Create a qcow2-Image that includes your customized configuration using make-config.sh.
4. Copy your configuration image to the final destination for KVM VMs.
5. Insert the bold line as shown below to the VM virt-install configuration when you create a new vJunos-switch VM. An example is described in chapter Proxmox Virtual Environment.

Deployment of vJunos-switch VM Using virt-install CLI

For each vJunos-switch VM, you need an individual copy of the base image for the system to write to it. The configuration below uses the KVM backing file method to create an image with the changes made and reads from the original image. This method speeds up the VM start and saves storage. You might have to modify the file /etc/libvirt/qemu.conf to add yourself as a user and group. Then, run sudo systemctl restart libvirtd to use this feature.

Backup files must always allow you to copy the image for each VM.

Finally, you can successfully start the vJunos-switch VM in the configuration below without changing the parameters in bold. The VM first Ethernet interface is always fxp0. After you've set the MAC address and configured the DHCP server to assign 192.168.10.201 as static DHCP lease, you can directly start a remote shell later.

Deployment of vJunos-switch VM Using virt-manager GUI

For installing vJunos-switch through virt-manager GUI, you must manually change few XML based configuration files. So, the GUI does not provide any benefit for this VM.

1. Obtain a copy of the base VM to the libvirtd image directory as shown.

   

2. Click Edit -> Preferences to enable XML editing for this VM.

   

3. Select Enable XML editing.

   

4. Select File -> New Virtual Machine.

   

5. Select Import existing disk image for installation.

   

6. Click Browse.

   

7. Select the image that you have copied and click Choose Volume.

   

8. Select Generic default for OS and click Forward.

   

9. Click + to configure:
   • Memory=5120
   • CPUs=4

   

10. Then, configure:
    1. Name=spine1
    2. Select Customize configuration before install.
    3. Select Virtual network 'br0network': NAT for now. Network selection for the first interface (fxp0) must be a NAT network.
    4. Click Finish.

       

11. Click CPUs.

    The host CPU is configured. You can click XML to change the configuration later as the other options are not required.

    

12. Change the Device model as virtio for the first interface.

    

13. Add a new network interface using the add function for interfaces and configure:
    • Network source=Bridge ge000
    • Uncheck=MAC address
    • Device model=virtio

    

14. Repeat Step 13 to add network source Bridge ge001, Bridge ge002 and Bridge ge003.

    

    

    

15. Optionally, you can remove the default interface and options highlighted in yellow in the figure below as they are not required for vJunos-switch VM.

    

16. Select OS information and then XML for editing. Delete everything between <os> and </os> and replace it with the highlighted lines as show below. The replaced configuration instructs special BIOS command to ensure that the VM acts as a vEX9214 and not as a vMX.

    

    1. Insert the following configuration as shown below:

       The replaced configuration is shown as highlighted in yellow in the figure below.

       

    2. Insert the following configuration from <cpu as shown in the figure above.

       The replaced configuration is shown as highlighted in yellow in the figure below.

       

17. Click apply and Begin Installation.

The installation process might take three minutes and you’ll be prompted with a VM login.

Linux Bridge and VM Interface Post VM Creation Changes

After the VM is launched, you must change the interface and Linux bridge configuration. The bridges that you created with the default configuration in chapter Preparations Before You Deploy, are applicable for an EVPN VXLAN fabric.

The default Linux bridge does not support:

• LLDP message transport, which means you cannot see any link neighbors.
• LACP 802.3ad message transport, which prevents you from building LAG.
• Large MTUs, which causes fragmentation of VXLAN messages. Using VXLAN, the fabric must have MTUs at least 50 bytes larger than the attached clients. Image the attached Desktop VM and all fabric links with same default MTU of 1500, where the transport links always add extra 50 bytes. The small packets such as, ICMP Ping works well but not when the clients use the full MTU of 1500 bytes.
• 802.1X wired client authentication is blocked.

Hence, you need to change the virtual interface and Linux bridges after creating the VM to support these features. This change ensures that you build a virtual EVPN VXLAN fabric smoothly.

1. Create a bash script using the following commands to perform all the required functions.
2. Copy and paste the below configuration to your editor. Then, save and close.
3. Run this script on a demonstration VM, where you can monitor the commands that are required to apply the changes.

You’ve intentionally given a warning about the non-upgraded MTU on a certain interface. This warning implies that a Linux bridge always has the lowest MTU of any attached network interface. Hence, ensure to upgrade all interfaces attached to a single bridge. Else, your MTU changes are not implemented.

Note:

The script intentionally does not upgrade the first interface of a VM, which is usually fxp0. If you want to include this interface, change the second line to “virsh domiflist $1 | tail -n +3 > /tmp/vmbridgelist.txt”.

Note:

Tweaking Linux bridge works only with Ubuntu 20.04 or higher.

Optional: Optimizing Your KVM Server

Ultra Kernel Samepage Merging Kernel

Using a kernel that supports Ultra Kernel Samepage Merging can save 30 - 40% memory compared to the standard KSM support when launching multiple vJunos-switch VMs. This is beneficial for a lab and systems such as EVE-NG which includes this Kernel by default. If the system runs on Ubuntu 20.04.x, which does not support higher kernels, you can use the following configurations to build such a kernel from the official GitHub repository.

Note:

Ensure that you have disabled Secure Boot in the BIOS. If not, the unsigned kernel does not load.

1. Enter https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal/refs/ in a browser.

   1. If a normal 5.4.x kernel is installed on your server, then run the following commands:

   2. If a newer kernel such as, 5.15.x with Hardware Enablement Stack (HWE) support is installed on your server, then run the following commands:
2. Regardless of your kernel version, proceed with the following configuration to build and install the USKM kernel.
3. After the server is up again, check if you are running on the new kernel.

Turn Off Security Mitigations

Note:

Avoid these instructions if your server is in production or accessible through the Internet. You can revise the security checks to gain CPU performance in your lab. You are warned.

To avoid 20% CPU performance loss, you’ll disable all mitigations against meltdown/spectre vulnerabilities. This server does not host VMs from different users. As you manage all the VMs in your lab and it’s not a public offered production-grade system, you prioritize performance over security.