Customize JCNR Configuration using Node Annotations
SUMMARY Read this topic to understand how to customize JCNR configuration using node annotations and custom configuration template.
Node Annotations
Starting with Juniper Cloud-Native Router (JCNR) Release 23.2, JCNR supports customizing the JCNR configurations using node annotations when deployed in L3 mode. Node annotations are key-value pairs. The key-value pairs are used to render the cRPD configuration via a go template. The configured template must be available in theJuniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory for the configuration to be applied to the cRPD pods.You must apply the node annotations before installing JCNR to create cRPD pods with custom configuration. The cRPD pod must be deleted and respawned should you wish to apply the annotations any time after JCNR installation. cRPD customization via node annotations is optional.
Configuration Example
Sample node annotation
and template files are available under
Juniper_Cloud_Native_Router_<release-number>/helmchart/cRPD_examples
directory.
You define the key-value pair for each worker node of your
cluster. An example of the node-annotations.yaml file is provided
below:
apiVersion: v1
kind: Node
metadata:
name: 5d8s1-node1 ---> Node name
annotations:
jcnr.juniper.net/params: '{
"isoLoopbackAddr": "49.0004.1000.0000.0001.00", ---> Key value pairs
"IPv4LoopbackAddr": "110.1.1.2",
"srIPv4NodeIndex": "2000",
"srIPv6NodeIndex": "3000",
"BGPIPv4Neighbor": "110.1.1.254",
"BGPLocalAsn": "64512"
}'
---
apiVersion: v1
kind: Node
metadata:
name: 5d8s1-node2
annotations:
jcnr.juniper.net/params: '{
"isoLoopbackAddr": "49.0004.1000.0000.0000.00",
"IPv4LoopbackAddr": "110.1.1.3",
"srIPv4NodeIndex": "2001",
"srIPv6NodeIndex": "3002",
"BGPIPv4Neighbor": "110.1.2.254",
"BGPLocalAsn": "64512"
}'The
key-value pairs you define in the annotations is used to render the cRPD
configuration via a go template. An example of the
jcnr-cni-custom-config.tmpl template file is provided
below:
apply-groups [custom];
groups {
custom {
interfaces {
lo0 {
unit 0 {
{{if .Node.isoLoopbackAddr}} ---> key is replaced by value, if available for the node
family iso {
address {{.Node.isoLoopbackAddr}};
}
{{end}}
family inet {
address {{.Node.IPv4LoopbackAddr}};
}
}
}
}
routing-options {
router-id {{.Node.IPv4LoopbackAddr}}
route-distinguisher-id {{.Node.IPv4LoopbackAddr}}
}
protocols {
isis {
interface all;
{{if and .Env.SRGB_START_LABEL .Env.SRGB_INDEX_RANGE}} ---> conditional statement based on environment variables
source-packet-routing {
srgb start-label {{.Env.SRGB_START_LABEL}} index-range {{.Env.SRGB_INDEX_RANGE}};
node-segment {
{{if .Node.srIPv4NodeIndex}}
ipv4-index {{.Node.srIPv4NodeIndex}};
{{end}}
{{if .Node.srIPv6NodeIndex}}
ipv6-index {{.Node.srIPv6NodeIndex}};
{{end}}
}
}
{{end}}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
bgp {
interface all;
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then community add udp;
}
community udp members encapsulation:0L:13;
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address {{.Node.IPv4LoopbackAddr}};
local-as {{.Node.BGPLocalAsn}};
neighbor {{.Node.BGPIPv4Neighbor}};
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
}
}
}
routing-options {
dynamic-tunnels {
dyn-tunnels {
source-address {{.Node.IPv4LoopbackAddr}};
udp;
destination-networks {{.Node.BGPIPv4Neighbor}}/32;
}
}
}
}
}You can define additional cRPD configuration hierarchies in the template. The
values to be rendered from the annotations defined in the
node-annotations.yaml must be defined as
{{.Node.key}}. Any environment
variables, such as variables defined in values.yaml, must be
defined as {{.Env.variable_name}}.
Once the template is configured, you must copy the
jcnr-cni-custom-config.tmpl file to the
Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory.
# cp Juniper_Cloud_Native_Router_release_number/helmchart/cRPD_examples/jcnr-cni-custom-config.tmpl Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/ cp: overwrite 'Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/jcnr-cni-custom-config.tmpl'? yes #
# kubectl apply -f node-annotation.yaml node/5d8s1-node1 configured node/5d8s1-node2 configured
Follow
the JCNR
installation instructions to deploy the cloud-native router components,
including the cRPD. Once the installation completes, access the cRPD shell and issue the show configuration | display
set command in the cli mode to view the custom
configuration you
applied.
root@jcnr-01> show configuration
## Last commit: 2023-06-23 08:30:42 EDT by root
version 20230608.143922_builder.r1342735;
groups {
base { /* OMITTED */ };
custom {
interfaces {
lo0 {
unit 0 {
family inet {
address 110.1.1.2/32;
}
family iso {
address 49.0004.1000.0000.0001.00;
}
}
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then {
community add udp;
}
}
community udp members encapsulation:0L:13;
}
routing-options {
route-distinguisher-id 110.1.1.2;
router-id 110.1.1.2;
dynamic-tunnels {
dyn-tunnels {
source-address 110.1.1.2;
udp;
destination-networks {
110.1.1.254/32;
}
}
}
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address 110.1.1.2;
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
local-as 64512;
neighbor 110.1.1.254;
}
}
isis {
interface all;
source-packet-routing {
srgb start-label 400000 index-range 4000;
node-segment {
ipv4-index 2000;
ipv6-index 3000;
}
}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
}
}
cni { /* OMITTED */ };
internal { /* OMITTED */ };
}
apply-groups [ custom base internal ]; Troubleshooting
The cRPD pod continues to remain in Pending state if invalid
configuration is rendered and applied via the go template. The rendered
configuration is saved in /etc/crpd directory on the JCNR host as
juniper.conf.master. You can apply the rendered configuration
manually to a running cRPD pod to validate the configuration and identify issues.
For an AWS EKS deployment you can find the rendered config within the cRPD pod in
the /config directory.