Known Behavior

CEM-14751 After release 2003 deployment with OpenStack Rocky you cannot view the dashboard in the Horizon UI. This behavior is attributed to the OpenStack bug, https://bugs.launchpad.net/horizon/+bug/1788631.

CEM-11497 Polling Interval should default to 0 while creating an sFlow profile.

CEM-14264 In release 2003, the Virtual Port Group create workflow no longer pre-populates the VLAN-ID with the existing value that was defined with the first VPG for a given virtual network. The field is editable unlike in previous releases. This issue occurs in a fabric that was provisioned with the Fabric-wide VLAN-ID significance checkbox enabled.

CEM-14542 vRouter to vRouter datapath encryption does not work as per design. When datapath encryption is enabled between computes, traffic is encrypted in one direction only.

CEM-14090 In release 2003, the OpenStack Rocky installation using Kolla OpenStack containers fails. This is due to the upstream breakage in centos-binary-kolla-toolbox container as noted in https://bugs.launchpad.net/kolla/+bug/1869613.

CEM-14045 In Contrail fabric deployments ERB+DC GW doesn’t work with LR.

CEM-14003, CEM-13976 vRouter offload with Mellanox NIC cards does not work. However the DPDK on Mellanox NICs without offload is supported.

CEM-13977 In Contrail fabric deployments, managed and unmanaged PNF cannot be used simultaneously on the leaf nodes.

CEM-13940 - Image upgrade through Contrail fabric manager on QFX10002-60C devices fails due to Junos PR 1498337. Customers using this device are advised to contact Juniper Technical Assistance Center (JTAC) for the Junos image with the fix for this PR.

CEM-13767 - Though Contrail fabric manager has the ability for the user to use custom image names for the fabric devices, for platforms like QFX10002-60C which runs on vmhost-based platforms, while uploading the image to CFM, the image name should be chosen in junos-vmhost-install-x.tgz format.

CEM-13685 - CX5: DPDK vRouter with MLNX CX5 takes about 10 minutes and also lcore crash is seen. This happens once during initial installation.

CEM-13628 - In Contrail Fabric deployments CRB-MCAST-Gateway role works only for QFX10k devices.

CEM-13533 - VPG update in Contrail Command fails when one of the Contrail Controllers is down (Contrail HA setup).

CEM-13380 - AppFormix Flows does not show up for multi homed devices on the fabric

CEM-13379 - StormControl profile config push to device fails if the name has " " <space>.

CEM-13360 - Uncaught exception observed when selecting a device before the GDO operation completes.

CEM-12752 - Data center interconnect between ERB and CRB fabrics or between 2 ERB fabrics doesn't work.

CEM-12861 - Flow to VN mapping using AppFormix sFlow does not work for any traffic involving BMS traffic end points.

CEM-12507 - Configuring DC-GW on spine without the RR role (border leaf) does not work.

CEM-11229 - PNF with DC-GW does not advertise static routes to PNF instance and hence traffic from private VN to internet does not go through. As a workaround, apply “set groups _contrail_overlay_service_chain_ policy-options policy-statement pnf-si_right-export term DIRECT from protocol static” on the DC-GW device handling PNF.

CEM-10929 - When AppFormix is querying LLDP table from a device through SNMP, if SNMP calls time out, AppFormix marks the device as invalidConfiguration and notifies the user to take a look. When the user verifies that snmpwalk is working and there are no network issues, click Edit and reconfigure that device from Settings > Network Devices to make AppFormix try to run LLDP discovery and add this device again.

CEM-11141 - Contrail fabric manager cannot configure PNF service chain on devices acting as ERB unicast gateway.

CEM-8701, CEM-8149 - Onboarding of multiple BMS in parallel on SP-style fabric does not work.

CEM-7424, CEM-6570 - MX acting as DC-GW cannot serve FIP for both Contrail VMs and Datacenter BMSes simultaneously.

CEM-5788 Installation fails if FQDN is used to deploy Contrail Cluster through Contrail Command with OpenStack orchestration.

CEM-5577 - Contrail fabric manager does not configure PIM RP on spine devices.

CEM-4370 - Additional links cannot be appended to service templates used to create PNF service chaining. If there is a need to add additional links, the service template needs to be deleted and re-added again.

CEM-4358 - In Contrail fabric deployments configuring QFX5110 as spine (CRB-Gateway) does not work.

CEM-11163 In Fortville X710 NIC: With TX and RX buffers performance degrade is observed as mbufs gets exhausted.

CEM-9979 During upgrade of DPDK computes deployed with OOO Heat Templates in RHOSP environment, vRouter coredumps are observed. This is due to the sequence in which the services are started during upgrade and does not have impact on cluster operation.

CEM-8701 While bringing up a BMS using the Life Cycle Management workflow, sometimes on faster servers the re-image does not go through and instance not moved from ironic vn to tenant vn. This is because if the PXE boot request from the BMS is sent before the routes are converged between the BMS port and the TFTP service running in Contrail nodes. As a workaround, the servers can be rebooted or the BIOS in the servers can be configured to have a delayed boot.

CEM-8149 BMS LCM with fabric set with enterprise_style=True is not supported. By default, enterprise_style is set to False. User should avoid using enterprise_style=True if the fabric object will onboard BMS LCM instance.

CEM-7874 User defined alarms may not be generated, when third stunnel/Redis service instance is down after the first two instances were restarted.

CEM-5141 For deleting compute nodes, the UI workflow will not work. Instead, update the instances.yaml with “ENABLE_DESTROY: True” and “roles:” (leave it empty) and run the following playbooks.

For example:

CEM-5043 VNI update on a LR doesnt update the RouteTable. As a workaround, delete the LogicalRouter and create a new LogicalRouter with the new VNI.

CEM-4370 After creating a PNF Service Instance, the fields like PNF eBGP ASN*, RP IP Address, PNF Left BGP Peer ASN*, Left Service VLAN*, PNF Right BGP Peer ASN* ,Right Service VLAN* cannot be modified. If there is a need to modify these values, delete and re-create the Service Instance with intended values.

CEM-3959 BMS movement across TORs is not supported. To move BMS across TORs the whole VPG need to be moved. That means if there are more than one BMS associated to one VPG, and one of the BMS need to be moved, the whole VPG need to be deleted and re-configured as per the new association.

JCB-187287 High Availability provisioning of Kubernetes master is not supported.

JCB-184776 When the vRouter receives the head fragment of an ICMPv6 packet, the head fragment is immediately enqueued to the assembler. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. Fragments get enqueued in the hold queue if agent does not write flow action by the time the assembler releases fragments to the flow module. A maximum of three fragments are enqueued in the hold queue at a time. The remaining fragments are dropped from the assembler to the flow module.

As a workaround, the head fragment is enqueued to assembler only after flow action is written by agent. If the flow is already present in non-hold state, it is immediately enqueued to assembler.

JCB-177787 In DPDK vRouter use cases such as SNAT and LBaaS that require netns, jumbo MTU cannot be set. Maximum MTU allowed: <=1500.

JCB-177541 When you receive an error message during Kolla provisioning, rerunning the code will not work. In order for the provisioning to work, restart provisioning from scratch.

CEM-14751 After release 2003 deployment with OpenStack Rocky you cannot view the dashboard in the Horizon UI. This behavior is attributed to the OpenStack bug, https://bugs.launchpad.net/horizon/+bug/1788631.

CEM-14542 vRouter to vRouter datapath encryption does not work as per design. When datapath encryption is enabled between computes, traffic is encrypted in one direction only.

CEM-11497 Polling Interval should default to 0 while creating an sFlow profile.

CEM-14264 In release 2003, the Virtual Port Group create workflow no longer pre-populates the VLAN-ID with the existing value that was defined with the first VPG for a given virtual network. The field is editable unlike in previous releases. This issue occurs in a fabric that was provisioned with the Fabric-wide VLAN-ID significance checkbox enabled.

CEM-14090 In release 2003, the OpenStack Rocky installation using Kolla OpenStack containers fails. This is due to the upstream breakage in centos-binary-kolla-toolbox container as noted in https://bugs.launchpad.net/kolla/+bug/1869613.

CEM-14045 In Contrail fabric deployments ERB+DC GW doesn’t work with LR.

CEM-14003, CEM-13976 vRouter offload with Mellanox NIC cards does not work. However the DPDK on Mellanox NICs without offload is supported.

CEM-13977 In Contrail fabric deployments, managed and unmanaged PNF cannot be used simultaneously on the leaf nodes.

CEM-13940 - Image upgrade through Contrail fabric manager on QFX10002-60C devices fails due to Junos PR 1498337. Customers using this device are advised to contact Juniper Technical Assistance Center (JTAC) for the Junos image with the fix for this PR.

CEM-13767 - Though Contrail fabric manager has the ability for the user to use custom image names for the fabric devices, for platforms like QFX10002-60C which runs on vmhost-based platforms, while uploading the image to CFM, the image name should be chosen in junos-vmhost-install-x.tgz format.

CEM-13685 - CX5: DPDK vRouter with MLNX CX5 takes about 10 minutes and also lcore crash is seen. This happens once during initial installation.

CEM-13628 - In Contrail Fabric deployments CRB-MCAST-Gateway role works only for QFX10k devices.

CEM-13533 - VPG update in Contrail Command fails when one of the Contrail Controllers is down (Contrail HA setup).

CEM-13380 - AppFormix Flows does not show up for multi homed devices on the fabric

CEM-13379 - StormControl profile config push to device fails if the name has " " <space>.

CEM-13360 - Uncaught exception observed when selecting a device before the GDO operation completes.

CEM-12752 - Data center interconnect between ERB and CRB fabrics or between 2 ERB fabrics doesn't work.

CEM-12861 - Flow to VN mapping using AppFormix sFlow does not work for any traffic involving BMS traffic end points.

CEM-12507 - Configuring DC-GW on spine without the RR role (border leaf) does not work.

CEM-11229 - PNF with DC-GW does not advertise static routes to PNF instance and hence traffic from private VN to internet does not go through. As a workaround, apply “set groups _contrail_overlay_service_chain_ policy-options policy-statement pnf-si_right-export term DIRECT from protocol static” on the DC-GW device handling PNF.

CEM-10929 - When AppFormix is querying LLDP table from a device through SNMP, if SNMP calls time out, AppFormix marks the device as invalidConfiguration and notifies the user to take a look. When the user verifies that snmpwalk is working and there are no network issues, click Edit and reconfigure that device from Settings > Network Devices to make AppFormix try to run LLDP discovery and add this device again.

CEM-11141 - Contrail fabric manager cannot configure PNF service chain on devices acting as ERB unicast gateway.

CEM-8701, CEM-8149 - Onboarding of multiple BMS in parallel on SP-style fabric does not work.

CEM-7424, CEM-6570 - MX acting as DC-GW cannot serve FIP for both Contrail VMs and Datacenter BMSes simultaneously.

CEM-5577 - Contrail fabric manager does not configure PIM RP on spine devices.

CEM-4370 - Additional links cannot be appended to service templates used to create PNF service chaining. If there is a need to add additional links, the service template needs to be deleted and re-added again.

CEM-4358 - In Contrail fabric deployments configuring QFX5110 as spine (CRB-Gateway) does not work.

CEM-11163 In Fortville X710 NIC: With TX and RX buffers performance degrade is observed as mbufs gets exhausted.

CEM-9979 During upgrade of DPDK computes deployed with OOO Heat Templates in RHOSP environment, vRouter coredumps are observed. This is due to the sequence in which the services are started during upgrade and does not have impact on cluster operation.

CEM-8701 While bringing up a BMS using the Life Cycle Management workflow, sometimes on faster servers the re-image does not go through and instance not moved from ironic vn to tenant vn. This is because if the PXE boot request from the BMS is sent before the routes are converged between the BMS port and the TFTP service running in Contrail nodes. As a workaround, the servers can be rebooted or the BIOS in the servers can be configured to have a delayed boot.

CEM-8149 BMS LCM with fabric set with enterprise_style=True is not supported. By default, enterprise_style is set to False. User should avoid using enterprise_style=True if the fabric object will onboard BMS LCM instance.

CEM-7874 User defined alarms may not be generated, when third stunnel/Redis service instance is down after the first two instances were restarted.

CEM-5788 Installation fails if FQDN is used to deploy Contrail Cluster through Contrail Command with OpenStack orchestration.

CEM-5141 For deleting compute nodes, the UI workflow will not work. Instead, update the instances.yaml with “ENABLE_DESTROY: True” and “roles:” (leave it empty) and run the following playbooks.

For example:

CEM-5043 VNI update on a LR doesnt update the RouteTable. As a workaround, delete the LogicalRouter and create a new LogicalRouter with the new VNI.

CEM-4370 After creating a PNF Service Instance, the fields like PNF eBGP ASN*, RP IP Address, PNF Left BGP Peer ASN*, Left Service VLAN*, PNF Right BGP Peer ASN* ,Right Service VLAN* cannot be modified. If there is a need to modify these values, delete and re-create the Service Instance with intended values.

CEM-3959 BMS movement across TORs is not supported. To move BMS across TORs the whole VPG need to be moved. That means if there are more than one BMS associated to one VPG, and one of the BMS need to be moved, the whole VPG need to be deleted and re-configured as per the new association.

JCB-187287 High Availability provisioning of Kubernetes master is not supported.

JCB-184776 When the vRouter receives the head fragment of an ICMPv6 packet, the head fragment is immediately enqueued to the assembler. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. Fragments get enqueued in the hold queue if agent does not write flow action by the time the assembler releases fragments to the flow module. A maximum of three fragments are enqueued in the hold queue at a time. The remaining fragments are dropped from the assembler to the flow module.

As a workaround, the head fragment is enqueued to assembler only after flow action is written by agent. If the flow is already present in non-hold state, it is immediately enqueued to assembler.

JCB-177787 In DPDK vRouter use cases such as SNAT and LBaaS that require netns, jumbo MTU cannot be set. Maximum MTU allowed: <=1500.

JCB-177541 When you receive an error message during Kolla provisioning, rerunning the code will not work. In order for the provisioning to work, restart provisioning from scratch.