Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Troubleshoot Deployment Issues

SUMMARY This topic provides information about how to troubleshoot deployment issues using Kubernetes commands and how to view the cloud-native router configuration files.

Troubleshoot Deployment Issues

This topic provides information on some of the issues that might be seen during deployment of the cloud-native router components and provides a number of Kubernetes (K8s) and shell commands that you run on the host server to help determine the cause of deployment issues.

Table 1: Investigate Deployment Issues
Potential issue What to check Related Commands
Image not found Check if the images are uploaded to the local docker using the command docker images. If not, then the registry configured in values.yaml should be accessible. Ensure image tags are correct. 
  • kubectl -n jcnr describe pod <crpd-pod-name>
Initialization errors Check if jcnr-secrets is loaded and has a valid license key 
[root@jcnr-01]# kubectl get secrets -n jcnr
NAME                  TYPE                                  DATA   AGE
crpd-token-zp8kc      kubernetes.io/service-account-token   3      29d
default-token-zn6p9   kubernetes.io/service-account-token   3      29d
jcnr-secrets          Opaque                                2      29d
Confirm that root password and license key are present in /var/run/jcnr/juniper.conf
cRPD Pod in CrashLoopBackOff state
  • Check if startup/liveness probe is failing or vrouter pod not running
  • rpd-vrouter-agent gRPC connection not UP
  • Composed configuration is invalid or config template is invalid
 
  • kubectl get pods -A
    kubectl -n jcnr describe pod <crpd-pod-name>
    tail –f /var/log/jcnr/jcnr-cni.log​
    tail –f /var/log/jcnr/jcnr_notifications.json

  • See Access cRPD CLI to enter the cRPD CLI and run the following command:

    show krt state channel vrouter
  • cat /var/run/jcnr/juniper.conf

vRouter Pod in CrashLoopBackOff state

 Check the contail-k8s-deployer pod for errors 
kubectl logs contrail-k8s-deployer-<pod-hash> -n contrail-deploy

Verify Cloud-Native Router Controller Configuration

The cloud-native router deployment process creates a configuration file for the cloud-native router controller (cRPD) as a result of entries in the values.yaml file for L2 mode and custom configuration via node annotations in L3 mode. You can view this configuration file to see the details of the cRPD configuration. To view the cRPD configuration, navigate to the /var/run/jcnr folder to access the configuration file details and view the contents of the configuration file.

The cRPD configuration may be customized using node annotations. The cRPD pod will stay in pending state if the applied configuration is invalid.

You can view the rendered custom configuration in the /etc/crpd/ directory.

In an AWS EKS deployment you can review the rendered custom configuration by accessing the cRPD CLI and reviewing the contents of the /config directory.

View Log Files

You can view the jcnr log files in the default log_path directory, /var/log/jcnr/. You can change the location of the log files by changing the value of the log_path: or syslog_notifications: keys in the values.yaml file prior to deployment.

Navigate to the following path and issue the ls command to list the log files for each of the cloud-native router components.

Uninstallation Issues

After the triggering of helm uninstall command, please wait for all Kubernetes resources to be fully deleted before attempting a re-installation. Premature re-installation can lead to installation stalls and may require manual steps for recovery. ​The recovery steps are provided below: