ON THIS PAGE
Static Routes
SUMMARY Juniper Cloud-Native Contrail Networking (CN2) release 23.1 supports static routes for your cluster. This article provides information about how to configure static routes for your CN2 cluster.
Understanding Static Routes
You can use static routes when a network doesn't require the complexity of a dynamic routing protocol. Routes that are permanent fixtures in routing and forwarding tables are often configured as static routes.
The route consists of a destination prefix and a next-hop forwarding address. The static route is activated in the routing table and inserted into the forwarding table when the next-hop address is reachable. Traffic that matches the static route is forwarded to the specified next-hop address.
Static Routes in CN2
CN2 implements static routes through the following two custom resources (CRs):
-
RouteTable
: Contains a user-defined network destination prefix, along with an associated next hop (nextHop
). ThenextHop
IP address must be an IP address of a VMI object. A prefix defines the destination network that is reachable via the next hop. ARouteTable
lets you define a static route and next-hop pairing, which you can then associate with aRouteTable
and a virtual network (VN). The following is an example of aRouteTable
CR. Note that the namespace field is not required in this instance.apiVersion: core.contrail.juniper.net/v3 kind: RouteTable metadata: name: static-rt namespace: static-route spec: routes: route: - nextHop: 10.20.30.2 nextHopType: ip-address prefix: 10.20.30.0/24 communityAttributes: communityAttribute: - accept-own - no-advertise
Note that the field nextHopType must have the value ip-address. Any other value results in a user input error. The communityAttributes field allows you to set BGP communities on the route, which are advertised via BGP.
-
InterfaceRouteTable
: TheInterfaceRouteTable
configures static routing with the next hop of a virtual machine interface (VMI). AnInterfaceRouteTable
contains the destination prefix without the need for a statically-configured next hop entry. As with aRouteTable
, the prefix defines the destination network. Unlike aRouteTable
, you do not need to define anextHop
IP address because when you associate anInterfaceRouteTable
with a VMI, the associated VMI acts as the next hop for this prefix. -
The following is an example of an
InterfaceRouteTable
CR. Note that the namespace field is not required in this instance.apiVersion: core.contrail.juniper.net/v3 kind: InterfaceRouteTable metadata: name: static-rt namespace: static-route spec: interfaceRouteTableRoutes: route: - nextHopType: ip-address prefix: 10.20.30.0/24 communityAttributes: communityAttribute: - accept-own
Note that the field
nextHopType
must have the valueip-address
. Any other value results in a user input error.These CRs are scoped to their respective namespaces and enable you to configure required attributes for static routes.
Configure Static Routes for a Virtual Network
Configure the RouteTable
CR to apply static routes to a virtual network
(VN). A VN references a RouteTable
in it's spec. As a result, when the
RouteTable
is associated with that VN, the static route is applied. The
following is a VN object with an associated RouteTable
:
apiVersion: core.contrail.juniper.net/v3 kind: VirtualNetwork metadata: namespace: static-route name: vn-route spec: v4SubnetReference: apiVersion: core.contrail.juniper.net/v1alpha1 kind: Subnet namespace: static-route name: vn-subnet routeTableReferences: - apiVersion: core.contrail.juniper.net/v3 kind: RouteTable namespace: static-route name: static-rt
Configure Static Routes for a VMI
Configure the interface-route-table annotation on a VMI in order to apply an
InterfaceRouteTable
to a VMI.
apiVersion: v3 kind: VirtualMachineInterface metadata: name: static-route-pod namespace: static-route annotations: core.juniper.net/interface-route-table: '[{"name": "static-rt", "namespace": "static-route"}]' spec: <VMI_SPEC> status: interfaceRouteTableReferences: - apiVersion: core.contrail.juniper.net/v3 kind: InterfaceRouteTable namespace: static-route name: static-rt
The following is an example kubectl describe vmi
output:
Name: forwarder-6dff5888fd-59c9l-da8f57ef Namespace: static-route Labels: back-reference.core.juniper.net/28ffe95511d099080e315d0bb633ec5334f9250a40c8c9c61825d185=Tag_t2-kubernetes.io_metadata.name-static-route back-reference.core.juniper.net/5b136ca6d41e33dfb79a3f066e3db71186ade2514d1de62cf7d3a2b7=Tag_t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-ku back-reference.core.juniper.net/83323440b6198bd1f37a71f0ad4afde199fffd2742ed93cdeea56d4f=InterfaceRouteTable_static-route_to-zone-1 back-reference.core.juniper.net/84befcc656f5c3b04ece96e4d040cd168c3470825052cd9ef6903755=Tag_t1-core.juniper.net_namespace-static-route back-reference.core.juniper.net/92b2adff92e4dea99659da131af95ac5623ac10275e0a581fcab8c77=InterfaceRouteTable_static-route_to-right back-reference.core.juniper.net/9793fa23186c25d750b1a724d088a2cd46bff77cc54f819860847259=VirtualNetwork_static-route_left-vn back-reference.core.juniper.net/bcaf7cc535f4f34dd8d9e4a425cebeea7c229e4efc8e2f8ba3821bc8=VirtualMachine_contrail-k8s-kubemanager-kubernetes-forwarder-6d back-reference.core.juniper.net/ec2031060f3699dd2f7dd888d858d0a51de65fc2e5e62100b85df999=RoutingInstance_static-route_left-vn back-reference.core.juniper.net/f48f4e9c4f6c1dfccc439d752da4c9ee8eb7472a73d7bf5749a050af=Tag_t1-app-forwarder back-reference.core.juniper.net/f6b670baed4845c6640bb4db030b0d22cb3165a5e3929aad3a91fff6=Tag_t1-pod-template-hash-6dff5888fd Annotations: core.juniper.net/interface-route-table: [{"Namespace":"static-route","Name":"to-right"},{"Namespace":"static-route","Name":"to-zone-1"}] index: 1/5 interface: eth1 kube-manager.juniper.net/pod-cluster-name: contrail-k8s-kubemanager-kubernetes kube-manager.juniper.net/pod-name: forwarder-6dff5888fd-59c9l kube-manager.juniper.net/pod-namespace: static-route network: left-vn vmi-address-family: dualStack API Version: core.contrail.juniper.net/v3 Kind: VirtualMachineInterface Metadata: Creation Timestamp: 2023-05-04T06:05:15Z Finalizers: virtualmachineinterface.finalizers.core.juniper.net Generation: 2 Managed Fields: API Version: core.contrail.juniper.net/v3 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"virtualmachineinterface.finalizers.core.juniper.net": f:labels: f:back-reference.core.juniper.net/ec2031060f3699dd2f7dd888d858d0a51de65fc2e5e62100b85df999: f:spec: f:virtualMachineInterfaceMacAddresses: f:macAddress: Manager: manager Operation: Update Time: 2023-05-04T06:05:15Z API Version: core.contrail.juniper.net/v3 Fields Type: FieldsV1 fieldsV1: f:status: f:interfaceRouteTableReferences: f:observation: f:routingInstanceReferences: f:state: Manager: manager Operation: Update Subresource: status Time: 2023-05-04T06:05:16Z API Version: core.contrail.juniper.net/v3 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:core.juniper.net/interface-route-table: f:index: f:interface: f:kube-manager.juniper.net/pod-cluster-name: f:kube-manager.juniper.net/pod-name: f:kube-manager.juniper.net/pod-namespace: f:network: f:vmi-address-family: f:ownerReferences: .: k:{"uid":"a48fb78d-710b-4d66-9c29-73b77d5db8c6"}: f:spec: f:portSecurityEnabled: f:tagReferences: f:virtualMachineReferences: f:virtualNetworkReference: .: f:apiVersion: f:kind: f:name: f:namespace: f:resourceVersion: f:uid: Manager: kubemanager Operation: Update Time: 2023-05-04T06:05:17Z Owner References: API Version: core.contrail.juniper.net/v3 Block Owner Deletion: true Controller: true Kind: VirtualMachine Name: contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d UID: a48fb78d-710b-4d66-9c29-73b77d5db8c6 Resource Version: 43839 UID: bbe79a87-fda4-4a2d-be5f-52ecad9863e2 Spec: Allowed Address Pairs: Fq Name: default-domain static-route forwarder-6dff5888fd-59c9l-da8f57ef Parent: Port Security Enabled: true Properties: Tag References: API Version: core.contrail.juniper.net/v3 Fq Name: t1-core.juniper.net_namespace-static-route Kind: Tag Name: t1-core.juniper.net_namespace-static-route Resource Version: 43153 UID: c6bc3209-03c0-4c42-92fe-a71bcf5d2268 API Version: core.contrail.juniper.net/v3 Fq Name: t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-kubernetes Kind: Tag Name: t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-kubernetes Resource Version: 5169 UID: 72bf6479-90c2-4c57-97ca-214f0ddeec38 API Version: core.contrail.juniper.net/v3 Fq Name: t1-app-forwarder Kind: Tag Name: t1-app-forwarder Resource Version: 43297 UID: c31f28bb-efba-4793-bd2f-567fd3ff396d API Version: core.contrail.juniper.net/v3 Fq Name: t1-pod-template-hash-6dff5888fd Kind: Tag Name: t1-pod-template-hash-6dff5888fd Resource Version: 43298 UID: 7add9d84-8db2-4ddc-b32a-6f2ba06182ee API Version: core.contrail.juniper.net/v3 Fq Name: t2-kubernetes.io_metadata.name-static-route Kind: Tag Name: t2-kubernetes.io_metadata.name-static-route Resource Version: 43178 UID: ed588b5a-bc07-4ab4-b163-016ab16924ee Virtual Machine Interface Mac Addresses: Mac Address: 02:bb:e7:9a:87:fd Virtual Machine References: API Version: core.contrail.juniper.net/v3 Fq Name: contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d Kind: VirtualMachine Name: contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d Resource Version: 43434 UID: a48fb78d-710b-4d66-9c29-73b77d5db8c6 Virtual Network Reference: API Version: core.contrail.juniper.net/v3 Fq Name: default-domain static-route left-vn Kind: VirtualNetwork Name: left-vn Namespace: static-route Resource Version: 42804 UID: 41ab021d-0b55-4598-bc32-abf954708fd0 Status: Interface Route Table References: API Version: core.contrail.juniper.net/v3 Fq Name: static-route to-right Kind: InterfaceRouteTable Name: to-right Namespace: static-route Resource Version: 42473 UID: 912aa23d-fb26-4181-8c42-880ebe9f1a9b API Version: core.contrail.juniper.net/v3 Fq Name: static-route to-zone-1 Kind: InterfaceRouteTable Name: to-zone-1 Namespace: static-route Resource Version: 42475 UID: a2cfce5c-cb31-4a24-b432-16bdb8318bd6 Observation: Routing Instance References: API Version: core.contrail.juniper.net/v3 Attributes: Direction: both Fq Name: default-domain static-route left-vn left-vn Kind: RoutingInstance Name: left-vn Namespace: static-route UID: 300cda6a-a7fa-4f79-a815-dc3d7c0a46e7 State: Success Events: <none>]
Note the Interface Route Table References
section. This section shows the
association between the listed InterfaceRouteTables
and VMIs.
Configure Static Routes on Pod Interfaces
You can use the annotation section of a pod's manifest to configure static routes for a
pod's default or secondary interface. The pod reconciler processes the annotation section to
create a VMI object with an associated InterfaceRouteTable
. The reconciler
looks for the string key: "core.juniper.net/interface-route-table" in the annotation
section. The pod's VMI uses that string as a metadata label to associate with an
InterfaceRouteTable
.
The following is an example of a pod manifest with an InterfaceRouteTable
defined for the default interface:
apiVersion: v1 kind: Pod metadata: name: static-route-pod namespace: static-route annotations: core.juniper.net/interface-route-table: '[{"name": "vmi-rt", "namespace": "static-route"}]' spec: containers: - name: praqma image: <image-repository>:<tag> imagePullPolicy: Always securityContext: capabilities: add: - NET_ADMIN privileged: true
Note the securityContext
fields. These fields are necessary because the
pod must have permission to change it's routing table in order to utilize the configured
route.
The following is an example of a pod manifest with an InterfaceRouteTable
defined for the secondary interface:
apiVersion: v1 kind: Pod metadata: name: static-route-pod namespace: static-route annotations: k8s.v1.cni.cncf.io/networks: | [ { "name": "vn-route", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "[{\"name\": \"vmi-rt\", \"namespace\": \"static-route\"}]" } } ] spec: containers: - name: praqma image: <image-repository>:<tag> imagePullPolicy: Always securityContext: capabilities: add: - NET_ADMIN privileged: true
Note that the name for the primary interface InterfaceRouteTable
is
vmi-rt
and that the name for the secondary interface is
vn-route
. Defining two InterfaceRouteTables
with
different names in the same namespace automatically creates an
InterfaceRouteTable
for the primary and secondary interface of that
pod.
Configure Static Routes for a Virtual Network with a NAD
You can also specify static route properties in a network attachment definition (NAD)
object. After the NAD is reconciled or applied, a RouteTable
is created and
the resulting VN object references that RouteTable
. The following is an
example of a NAD with static route information defined:
apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: vn-route namespace: static-route labels: vn: vn-route annotations: juniper.net/networks: '{ "ipamV4Subnet": "108.108.2.0/24" "routeTableReferences": '[{"name": "vn-rt", "namespace": "static-route"}]' }' spec: config: '{ "cniVersion": "0.3.1", "name": "vn-route", "type": "contrail-k8s-cni" }'
Multiple Static Routes on Pod Interfaces
Using InterfaceRouteTable
, you can associate multiple static routes to a
single pod interface (VMI). This means that that VMI object has multiple next hop
destinations, depending on the IP prefix. You can specify multiple
InterfaceRouteTable
references using cluster service version (CSV) syntax
or JSON syntax annotations.
You must reference an InterfaceRouteTable
in a "namespace/name" format.
In the following example, static-route
is the namespace and
to-right
and to-zone-1
are the
InterfaceRouteTable
objects, or next hop destination for the
left-vn
VMI.
The following example is a Deployment
with multiple
InterfaceRouteTable
references:
apiVersion: apps/v1 kind: Deployment metadata: name: forwarder namespace: static-route labels: app: forwarder spec: replicas: 3 selector: matchLabels: app: forwarder template: metadata: labels: app: forwarder annotations: k8s.v1.cni.cncf.io/networks: | [ { "name": "left-vn", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "static-route/to-right,static-route/to-zone-1" } }, { "name": "right-vn", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "static-route/to-left" } }, { "name": "zone-1", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "static-route/to-left" } }, { "name": "zone-2", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "static-route/to-left" } } ] spec: containers: - name: praqma image: <repository>:<tag> securityContext: capabilities: add: - NET_ADMIN privileged: true
The following example is a pod manifest with multiple InterfaceRouteTable
references using JSON syntax:
apiVersion: v1 kind: Pod metadata: name: irt-right namespace: static-route annotations: k8s.v1.cni.cncf.io/networks: | [{ "name": "right-vn", "namespace": "static-route", "cni-args": { "core.juniper.net/interface-route-table": "[{\"namespace\": \"static-route\", "\name\": \"to-left\"}, {\"namespace\": \"static-route\", \"name\": \"to-zone-1\"}]" } }] spec: containers: - name: praqma image: <image-repository>:<tag> securityContext: capabilities: add: - NET_ADMIN privileged: true
You must use backward slashes in JSON syntax. Backward slashes are required to encode a JSON string inside another JSON string.
Troubleshooting RouteTable and InterfaceRouteTable
Dataplane Verification
-
In the vRouter introspect, verify that the VRF of the VN shows a row with a matching static route prefix specified in the RT using the following steps:
-
Access vRouter introspect at https://<vrouter_ip>:8085/Snh_VrfListReq
-
Verify that the VRF is associated with the VN.
-
Navigate to the ucindex column in the VRF unicast
RouteTable
. -
Verify that the table contains a row with the correct static route prefix.
-
- In the introspect, verify that the next hop properties of the VN are accurate and as
desired. In the introspect, the next hop column for the prefix should contain the
following:
-
The next hop interface name must be a valid tap interface.
-
The
label
must be a positive integer. -
The
resolved
value must betrue
. -
The
route-type:
value must beInterfaceStaticRoute
.
-
Config Plane Verification
-
Verify the state of the
RouteTable
andInterfaceRouteTable
objects.-
Check the status of the reconciler for the
InterfaceRouteTable
object.kubectl get interfaceroutetable -n
-
Check the status of the reconciler for the
RouteTable
object.kubectl get routetable -n
-
-
Verify the
RouteTable
reference in the associated VN. Verify theInterfaceRouteTable
reference in the associated VMI.-
Check the status of the reconciler for the VMI. You should see the
InterfaceRouteTable
in the VMI with an associated universally unique identifier (UUID) the Contrail FQ (meta info such asapiversion
,kind
,namespace
,name
) name.kubectl get vmi -n -oyaml | grep -i interfaceRouteTable
kubectl get vn -n -oyaml | grep -i routeTable
-