Customize JCNR Configuration
SUMMARY Read this topic to understand how to customize JCNR configuration using a ConfigMap.
JCNR ConfigMap
Starting with Juniper Cloud-Native Router (JCNR) Release 23.3, JCNR supports customizing configuration using a ConfigMap when deployed in L3 mode. In cloud-based deployments, in the event of a node failure, the JCNR pods may be spawned on newer or different nodes. A ConfigMap decouples the configuration parameters from node names and is based on node labels instead. This enables the JCNR CNI deployer to consume the configuration parameters, apply them to the cRPD configuration template and render the configuration, as long as a matching label is available for the node.A ConfigMap is an API object to store data in key-values
pairs. A ConfigMap defines per node variables that are consumed by nodes matching
the label. The key-value pairs are used to render the configuration via a go
template. The configured template must be available in the
Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory for the configuration to be applied to the cRPD pods.
You must apply the ConfigMap before installing JCNR to create cRPD pods with custom configuration. The cRPD pod must be deleted and respawned should you wish to apply the configuration parameters any time after JCNR installation. The configuration parameters are applied by default to any newly spawned cRPD pods. The JCNR customization via ConfigMap is optional.
JCNR also supports customization via node annotations for backward compatibility with previous releases. Considering that node annotations are coupled with node names, it is highly recommended to customize JCNR via ConfigMaps, specifically for cloud deployments. Refer to Customize JCNR Configuration using node annotations for more information.
Configuration Example
Sample ConfigMap and template files are available under
Juniper_Cloud_Native_Router_<release-number>/helmchart/cRPD_examples
directory.
You define the key-value pair for different node labels in your cluster. An example
of the jcnr-params-configmap.yaml file is provided below:
apiVersion: v1
kind: ConfigMap
metadata:
name: jcnr-params
namespace: jcnr
data:
jcnr1: |
{
"isoLoopbackAddr": "49.0004.1000.0000.0001.00",
"IPv4LoopbackAddr": "110.1.1.2",
"srIPv4NodeIndex": "2000",
"srIPv6NodeIndex": "3000",
"BGPIPv4Neighbor": "110.1.1.254",
"BGPLocalAsn": "64512"
}
jcnr2: |
{
"isoLoopbackAddr": "49.0004.1000.0000.0000.00",
"IPv4LoopbackAddr": "110.1.1.3",
"srIPv4NodeIndex": "2001",
"srIPv6NodeIndex": "3001",
"BGPIPv4Neighbor": "110.1.2.254",
"BGPLocalAsn": "64512"
}The key-value pairs you define in the annotations is used to render the cRPD
configuration via a go template. An example of the
jcnr-cni-custom-config-cm.tmpl template file is provided
below:
apply-groups [custom];
groups {
custom {
interfaces {
lo0 {
unit 0 {
{{if .Params.isoLoopbackAddr}}
family iso {
address {{.Params.isoLoopbackAddr}};
}
{{end}}
family inet {
address {{.Params.IPv4LoopbackAddr}};
}
}
}
}
routing-options {
router-id {{.Params.IPv4LoopbackAddr}}
route-distinguisher-id {{.Params.IPv4LoopbackAddr}}
}
protocols {
isis {
interface all;
{{if and .Env.SRGB_START_LABEL .Env.SRGB_INDEX_RANGE}}
source-packet-routing {
srgb start-label {{.Env.SRGB_START_LABEL}} index-range {{.Env.SRGB_INDEX_RANGE}};
node-segment {
{{if .Params.srIPv4NodeIndex}}
ipv4-index {{.Params.srIPv4NodeIndex}};
{{end}}
{{if .Params.srIPv6NodeIndex}}
ipv6-index {{.Params.srIPv6NodeIndex}};
{{end}}
}
}
{{end}}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then community add udp;
}
community udp members encapsulation:0L:13;
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address {{.Params.IPv4LoopbackAddr}};
local-as {{.Params.BGPLocalAsn}};
neighbor {{.Params.BGPIPv4Neighbor}};
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
}
}
}
routing-options {
dynamic-tunnels {
dyn-tunnels {
source-address {{.Params.IPv4LoopbackAddr}};
udp;
destination-networks {{.Params.BGPIPv4Neighbor}}/32;
}
}
}
}
}You can define additional cRPD configuration hierarchies in the template. The
values to be rendered from the ConfigMap defined in the
jcnr-params-configmap.yaml must be defined as
{{.Params.var-name}}. Any
environment variables, such as variables defined in
values.yaml, must be defined as
{{.Env.variable_name}}.
Complete the following steps to apply the customizations.
-
Label each node based on the keys used in the ConfigMap.
kubectl label nodes <node_name1> jcnr.juniper.net/params-profile=jcnr1 kubectl label nodes <node_name2> jcnr.juniper.net/params-profile=jcnr2
- Apply the ConfigMap to the cluster nodes using the command provided
below:
# kubectl apply -f jcnr-params-configmap.yaml configmap/jcnr-params created
- Once the template is configured, you must copy the
jcnr-cni-custom-config.tmplfile to theJuniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/directory.# cp Juniper_Cloud_Native_Router_release_number/helmchart/cRPD_examples/jcnr-cni-custom-config-cm.tmpl Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/ #
- Deploy the cloud-native router components, including the cRPD. Once the
installation completes, access the cRPD
CLI and issue the
show configuration | display setcommand in theclimode to view the custom configuration you applied.root@jcnr-01> show configuration ## Last commit: 2023-06-23 08:30:42 EDT by root version 20230608.143922_builder.r1342735; groups { base { /* OMITTED */ }; custom { interfaces { lo0 { unit 0 { family inet { address 110.1.1.2/32; } family iso { address 49.0004.1000.0000.0001.00; } } } } policy-options { # policy to signal dynamic UDP tunnel attributes to BGP routes policy-statement udp-export { then { community add udp; } } community udp members encapsulation:0L:13; } routing-options { route-distinguisher-id 110.1.1.2; router-id 110.1.1.2; dynamic-tunnels { dyn-tunnels { source-address 110.1.1.2; udp; destination-networks { 110.1.1.254/32; } } } } protocols { bgp { group jcnrbgp1 { type internal; local-address 110.1.1.2; family inet-vpn { unicast; } family inet6-vpn { unicast; } local-as 64512; neighbor 110.1.1.254; } } isis { interface all; source-packet-routing { srgb start-label 400000 index-range 4000; node-segment { ipv4-index 2000; ipv6-index 3000; } } level 1 disable; } ldp { interface all; } mpls { interface all; } } } cni { /* OMITTED */ }; internal { /* OMITTED */ }; } apply-groups [ custom base internal ];
Modifying the ConfigMap
If you wish to change the ConfigMap any time after JCNR installation, you must delete the cRPD pod and respawn it using the following command:kubectl patch sts kube-crpd-worker-sts -n jcnr -p '{"spec":{"template":{"metadata":{"annotations":{"configmap-reload/timestamp":"'$(date +%s)'"}}}}}'Troubleshooting
The cRPD pod continues to restart in CrashLoopBackOff state if
invalid configuration is rendered and applied via the go template. The rendered
configuration is saved in /config directory on the JCNR host as
juniper.conf.master. You can apply the rendered configuration
manually to a running cRPD pod to validate the configuration and identify issues.
For an AWS EKS deployment you can find the rendered config within the cRPD pod in
the /config directory.