How to Backup and Restore Contrail Databases in JSON Format
This document shows how to backup and restore the Contrail databases—Cassandra and Zookeeper—in JSON format.
Before You Begin
The backup and restore procedure must be completed for nodes running the same Contrail Networking release. The procedure is used to backup the Contrail Networking databases only; it does not include instructions for backing up orchestration system databases.
Database backups must be consistent across all systems because the state of the Contrail database is associated with other system databases, such as OpenStack databases. Database changes associated with northbound APIs must be stopped on all the systems before performing any backup operation. For example, you might block the external VIP for northbound APIs at the load balancer level, such as HAproxy.
Simple Database Backup in JSON Format
This procedure provides a simple database backup in JSON format.
This procedure is performed using the db_json_exim.py script located in the /usr/lib/python2.7/site-packages/cfgm_common on the controller node.
To perform this database backup:
Log into one of the config nodes. Create the /tmp/db-dump directory on any of the config node hosts.
mkdir /tmp/db-dump
On the same config node, copy the
contrail-api.conffile from the container to the host.Ansible Deployer:
docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/
Red Hat Openstack Deployer:
docker cp contrail_config_api:/etc/contrail/contrail-api.conf /tmp/db-dump/
The Cassandra database instance on any configuration node includes the complete Cassandra database for all configuration nodes in the cluster. Steps 1 and 2, therefore, only need to be performed on one configuration node.
Stop the following docker configuration services on all of the Contrail configuration nodes.
Ansible Deployer:
docker stop config_svcmonitor_1 docker stop config_devicemgr_1 docker stop config_schema_1 docker stop config_api_1
Red Hat Openstack Deployer:
docker stop contrail_config_svc_monitor docker stop contrail_config_device_manager docker stop contrail_config_schema docker stop contrail_config_api
This step must be performed on each individual config node in the cluster.
Return to the config node where you performed steps 1 and 2.
List the docker image to find the name or ID of the config api image..
docker image ls | grep config-apiExample:
docker image ls | grep config-api hub.juniper.net/contrail/contrail-controller-config-api 1909.30-ocata c9d757252a0c 4 months ago 583MBFrom the same config node, start the config api container pointing the
entrypoint.shscript to /bin/bash and mapping /tmp/db-dump from the host to the /tmp directory inside the container. You perform this step to ensure that the API services are not started on the config node.Enter the -v /etc/contrail/ssl:/etc/contrail/ssl:ro command option when
cassandra_use_sslis used as api-server configuration parameter to ensure TLS certificates are mounted to the Contrail SSL directory. This mounting ensures that the backup procedure succeeds in environments with endpoints that require TLS authentication.The registry_name and container_tag variables must match step 4.
docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash <registry_name>/contrail-controller-config_api:<container_tag>
Example:
docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata
Note:In case of RHOSP deployment with RedHat IDM (IPA) (only if CA certificate is different), mount the following file too:
-v /etc/ipa/ca.crt:/etc/ipa/ca.crt
From the docker container created on the config node in Step 5, use the
db_json_exim.pyscript to backup data in JSON format.. The db dump file will be saved in the /tmp/db-dump/ on this config node.cd /usr/lib/python2.7/site-packages/cfgm_common python db_json_exim.py --export-to /tmp/db-dump.json --api-conf /tmp/contrail-api.conf
The Cassandra database instance on any configuration node includes the complete Cassandra database for all configuration nodes in the cluster. You, therefore, only need to perform step 4 through 6 from one of the configuration nodes.
(Optional. Recommended) From the same config node, enter the
cat db-dump.json | python -m json.tool | lesscommand to view a more readable version of the file transfer.cat db-dump.json | python -m json.tool | less
From the same config node, exit out of the config api container. This will stop the container.
exit
Start the following configuration services on all of the Contrail configuration nodes.
Ansible Deployer:
docker start config_api_1 docker start config_schema_1 docker start config_svcmonitor_1 docker start config_devicemgr_1
Red Hat Openstack Deployer:
docker start contrail_config_api docker start contrail_config_schema docker start contrail_config_svc_monitor docker start contrail_config_device_manager
This step must be performed on each individual config node.
On each config node, enter the
contrail-statuscommand to confirm that services are in the active or running states.Note:Some command output and output fields are removed for readability. Output shown is from a node hosting config and analytics services.
contrail-status Pod Service Original Name State analytics api contrail-analytics-api running analytics collector contrail-analytics-collector running analytics nodemgr contrail-nodemgr running analytics provisioner contrail-provisioner running analytics redis contrail-external-redis running analytics-alarm alarm-gen contrail-analytics-alarm-gen running analytics-alarm kafka contrail-external-kafka running <some output removed for readability> == Contrail control == control: active nodemgr: active named: active dns: active == Contrail analytics-alarm == nodemgr: active kafka: active alarm-gen: active == Contrail database == nodemgr: active query-engine: active cassandra: active == Contrail analytics == nodemgr: active api: active collector: active == Contrail config-database == nodemgr: active zookeeper: active rabbitmq: active cassandra: active == Contrail webui == web: active job: active == Contrail analytics-snmp == snmp-collector: active nodemgr: active topology: active == Contrail config == svc-monitor: active nodemgr: active device-manager: active api: active schema: active
Examples: Simple Database Backups in JSON Format
These examples illustrate the process for creating a simple database backup in JSON format in both an Ansible deployer environment and a Red Hat Openstack deployer environment.
In each example, a cluster with three config nodes—control_config1, control_config2, and control_config3—is backed up. All tasks that need to be performed on a single config nodes are performed on control-config1. The tasks must be performed in the shown order.
Ansible Deployer Environment:
## control_config1 ## mkdir /tmp/db-dump docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/ docker stop config_svcmonitor_1 docker stop config_devicemgr_1 docker stop config_schema_1 docker stop config_api_1 ## control_config2 ## docker stop config_svcmonitor_1 docker stop config_devicemgr_1 docker stop config_schema_1 docker stop config_api_1 ## control_config3 ## docker stop config_svcmonitor_1 docker stop config_devicemgr_1 docker stop config_schema_1 docker stop config_api_1 ## control_config1 ## docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata cd /usr/lib/python2.7/site-packages/cfgm_common python db_json_exim.py --export-to /tmp/db-dump.json --api-conf /tmp/contrail-api.conf cat db-dump.json | python -m json.tool | less exit docker start config_api_1 docker start config_schema_1 docker start config_svcmonitor_1 docker start config_devicemgr_1 contrail-status ## control_config2 ## docker start config_api_1 docker start config_schema_1 docker start config_svcmonitor_1 docker start config_devicemgr_1 contrail-status ## control_config3 ## docker start config_api_1 docker start config_schema_1 docker start config_svcmonitor_1 docker start config_devicemgr_1 contrail-status
Red Hat Openstack Deployer Environment:
## control_config1 ## mkdir /tmp/db-dump docker cp contrail_config_api:/etc/contrail/contrail-api.conf /tmp/db-dump/ docker stop contrail_config_svc_monitor docker stop contrail_config_device_manager docker stop contrail_config_schema docker stop contrail_config_api ## control_config2 ## docker stop contrail_config_svc_monitor docker stop contrail_config_device_manager docker stop contrail_config_schema docker stop contrail_config_api ## control_config3 ## docker stop contrail_config_svc_monitor docker stop contrail_config_device_manager docker stop contrail_config_schema docker stop contrail_config_api ## control_config1 ## docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata cd /usr/lib/python2.7/site-packages/cfgm_common python db_json_exim.py --export-to /tmp/db-dump.json --api-conf /tmp/contrail-api.conf cat db-dump.json | python -m json.tool | less exit docker start contrail_config_api docker start contrail_config_schema docker start contrail_config_svc_monitor docker start contrail_config_device_manager contrail-status ## control_config2 ## docker start contrail_config_api docker start contrail_config_schema docker start contrail_config_svc_monitor docker start contrail_config_device_manager contrail-status ## control_config3 ## docker start contrail_config_api docker start contrail_config_schema docker start contrail_config_svc_monitor docker start contrail_config_device_manager contrail-status
Restore Database from the Backup in JSON Format
This procedure provides the steps to restore a system using the simple database backup JSON file that was created in Simple Database Backup in JSON Format.
To restore a system from a backup JSON file:
Copy the
contrail-api.conffile from the container to the host on any one of the config nodes.Ansible Deployer:
docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/
Red Hat Openstack Deployer:
docker cp contrail_config_api:/etc/contrail/contrail-api.conf /tmp/db-dump/
Stop the configuration services on all of the controllers.
Ansible Deployer:
docker stop config_svcmonitor_1 docker stop config_devicemgr_1 docker stop config_schema_1 docker stop config_api_1 docker stop config_nodemgr_1 docker stop config_database_nodemgr_1 docker stop analytics_snmp_snmp-collector_1 docker stop analytics_snmp_topology_1 docker stop analytics_alarm_alarm-gen_1 docker stop analytics_api_1 docker stop analytics_collector_1 docker stop analytics_alarm_kafka_1
Red Hat Openstack Deployer—Node hosting Contrail Config containers:
docker stop contrail_config_svc_monitor docker stop contrail_config_device_manager docker stop contrail_config_schema docker stop contrail_config_api docker stop contrail_config_nodemgr docker stop contrail_config_database_nodemgr
Red Hat Openstack Deployer—Node hosting Contrail Analytics containers:
docker stop contrail_analytics_snmp_collector docker stop contrail_analytics_topology docker stop contrail_analytics_alarmgen docker stop contrail_analytics_api docker stop contrail_analytics_collector docker stop contrail_analytics_kafka
Stop the Cassandra service on all the
config-dbcontrollers.Ansible Deployer:
docker stop config_database_cassandra_1
Red Hat Openstack Deployer:
docker stop contrail_config_database
Stop the Zookeeper service on all controllers.
Ansible Deployer:
docker stop config_database_zookeeper_1
Red Hat Openstack Deployer:
docker stop contrail_config_zookeeper
Backup the Zookeeper data directory on all the controllers.
Ansible Deployer:
cd /var/lib/docker/volumes/config_database_config_zookeeper/ cp -R _data/version-2/ version-2-save
Red Hat Openstack Deployer:
cd /var/lib/docker/volumes/config_zookeeper/ cp -R _data/version-2/ version-2-save
Delete the Zookeeper data directory contents on all the controllers.
rm -rf _data/version-2/*
Backup the Cassandra data directory on all the controllers.
Ansible Deployer:
cd /var/lib/docker/volumes/config_database_config_cassandra/ cp -R _data/ Cassandra_data-save
Red Hat Openstack Deployer:
cd /var/lib/docker/volumes/config_cassandra/ cp -R _data/ Cassandra_data-save
Delete the Cassandra data directory contents on all controllers.
rm -rf _data/*
Start the Zookeeper service on all the controllers.
Ansible Deployer:
docker start config_database_zookeeper_1
Red Hat Openstack Deployer:
docker start contrail_config_zookeeper
Start the Cassandra service on all the controllers.
Ansible Deployer:
docker start config_database_cassandra_1
Red Hat Openstack Deployer:
docker start contrail_config_database
List docker image to find the name or ID of the
config-apiimage on the config node.docker image ls | grep config-api
Example:
docker image ls | grep config-api hub.juniper.net/contrail/contrail-controller-config-api 1909.30-ocata c9d757252a0c 4 months ago 583MBRun a new docker container using the name or ID of the
config_apiimage on the same config node.Enter the -v /etc/contrail/ssl:/etc/contrail/ssl:ro command option when
cassandra_use_sslis used as api-server configuration parameter to ensure TLS certificates are mounted to the Contrail SSL directory. This mounting ensures that this backup procedure succeeds in environments with endpoints that require TLS authentication.Use the registry_name and container_tag from the output of the step 11.
docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash <registry_name>/contrail-controller-config_api:<container tag>
Example
docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata
Restore the data in new running docker on the same config node.
cd /usr/lib/python2.7/site-packages/cfgm_common python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf
Exit out of the config api container. This will stop the container.
exit
Start config services on all the controllers.
Ansible Deployer:
docker start config_svcmonitor_1 docker start config_devicemgr_1 docker start config_schema_1 docker start config_api_1 docker start config_nodemgr_1 docker start config_database_nodemgr_1 docker start analytics_snmp_snmp-collector_1 docker start analytics_snmp_topology_1 docker start analytics_alarm_alarm-gen_1 docker start analytics_api_1 docker start analytics_collector_1 docker start analytics_alarm_kafka_1
Red Hat Openstack Deployer—Node hosting Contrail Config containers:
docker start contrail_config_svc_monitor docker start contrail_config_device_manager docker start contrail_config_schema docker start contrail_config_api docker start contrail_config_nodemgr docker start contrail_config_database_nodemgr
Red Hat Openstack Deployer—Node hosting Contrail Analytics containers:
docker start contrail_analytics_snmp_collector docker start contrail_analytics_topology docker start contrail_analytics_alarmgen docker start contrail_analytics_api docker start contrail_analytics_collector docker start contrail_analytics_kafka
Enter the
contrail-statuscommand on each configuration node and, when applicable, on each analytics node to confirm that services are in the active or running states.Note:Output shown for a config node. Some command output and output fields are removed for readability.
contrail-status Pod Service Original Name State config api contrail-controller-config-api running config device-manager contrail-controller-config-devicemgr running config dnsmasq contrail-controller-config-dnsmasq running config nodemgr contrail-nodemgr running config provisioner contrail-provisioner running config schema contrail-controller-config-schema running config stats contrail-controller-config-stats running <some output removed for readability> == Contrail control == control: active nodemgr: active named: active dns: active == Contrail database == nodemgr: active query-engine: active cassandra: active == Contrail config-database == nodemgr: active zookeeper: active rabbitmq: active cassandra: active == Contrail webui == web: active job: active == Contrail config == svc-monitor: active nodemgr: active device-manager: active api: active schema: active
Example: How to Restore a Database Using the JSON Backup (Ansible Deployer Environment)
This example shows how to restore the databases for three controllers connected to the Contrail Configuration database (config-db). This example assumes a JSON backup file of the databases was previously created using the instructions provided in Simple Database Backup in JSON Format.The network was deployed using Ansible and the three controllers—nodec53, nodec54, and nodec55—have separate IP addresses.
## Make db-dump directory. Copy contrail-api.conf to db-dump directory. ## root@nodec54 ~]# mkdir /tmp/db-dump root@nodec54 ~]# docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/ ## Stop Configuration Services on All Controllers ## [root@nodec53 ~]# docker stop config_schema_1 [root@nodec53 ~]# docker stop config_svcmonitor_1 [root@nodec53 ~]# docker stop config_devicemgr_1 [root@nodec53 ~]# docker stop config_nodemgr_1 [root@nodec53 ~]# docker stop config_database_nodemgr_1 [root@nodec53 ~]# docker stop analytics_snmp_snmp-collector_1 [root@nodec53 ~]# docker stop analytics_snmp_topology_1 [root@nodec53 ~]# docker stop analytics_alarm_alarm-gen_1 [root@nodec53 ~]# docker stop analytics_api_1 [root@nodec53 ~]# docker stop analytics_collector_1 [root@nodec53 ~]# docker stop analytics_alarm_kafka_1 [root@nodec54 ~]# # docker stop config_schema_1 [root@nodec54 ~]# docker stop config_svcmonitor_1 [root@nodec54 ~]# docker stop config_devicemgr_1 [root@nodec54 ~]# docker stop config_nodemgr_1 [root@nodec54 ~]# docker stop config_database_nodemgr_1 [root@nodec54 ~]# docker stop analytics_snmp_snmp-collector_1 [root@nodec54 ~]# docker stop analytics_snmp_topology_1 [root@nodec54 ~]# docker stop analytics_alarm_alarm-gen_1 [root@nodec54 ~]# docker stop analytics_api_1 [root@nodec54 ~]# docker stop analytics_collector_1 [root@nodec54 ~]# docker stop analytics_alarm_kafka_1 [root@nodec55 ~]# docker stop config_schema_1 [root@nodec55 ~]# docker stop config_svcmonitor_1 [root@nodec55 ~]# docker stop config_devicemgr_1 [root@nodec55 ~]# docker stop config_nodemgr_1 [root@nodec55 ~]# docker stop config_database_nodemgr_1 [root@nodec55 ~]# docker stop analytics_snmp_snmp-collector_1 [root@nodec55 ~]# docker stop analytics_snmp_topology_1 [root@nodec55 ~]# docker stop analytics_alarm_alarm-gen_1 [root@nodec55 ~]# docker stop analytics_api_1 [root@nodec55 ~]# docker stop analytics_collector_1 [root@nodec55 ~]# docker stop analytics_alarm_kafka_1 ## Stop Cassandra ## [root@nodec53 ~]# docker stop config_database_cassandra_1 [root@nodec54 ~]# docker stop config_database_cassandra_1 [root@nodec55 ~]# docker stop config_database_cassandra_1 ## Stop Zookeeper ## [root@nodec53 ~]# docker stop config_database_zookeeper_1 [root@nodec54 ~]# docker stop config_database_zookeeper_1 [root@nodec55 ~]# docker stop config_database_zookeeper_1 ## Backup Zookeeper Directories Before Deleting Zookeeper Data Directory Contents ## [root@nodec53 _data]# cd /var/lib/docker/volumes/config_database_config_zookeeper/ [root@nodec53 config_database_config_zookeeper]# cp -R _data/version-2/ version-2-save [root@nodec53 config_database_config_zookeeper]# rm -rf _data/version-2/* [root@nodec54 _data]# cd /var/lib/docker/volumes/config_database_config_zookeeper/ [root@nodec54 config_database_config_zookeeper]# cp -R _data/version-2/ version-2-save [root@nodec54 config_database_config_zookeeper]# rm -rf _data/version-2/* [root@nodec55 _data]# cd /var/lib/docker/volumes/config_database_config_zookeeper/ [root@nodec55 config_database_config_zookeeper]# cp -R _data/version-2/ version-2-save [root@nodec55 config_database_config_zookeeper]# rm -rf _data/version-2/* ## Backup Cassandra Directory Before Deleting Cassandra Data Directory Contents ## [root@nodec53 ~]# cd /var/lib/docker/volumes/config_database_config_cassandra/ [root@nodec53 config_database_config_cassandra]# cp -R _data/ Cassandra_data-save [root@nodec53 config_database_config_cassandra]# rm -rf _data/* [root@nodec54 ~]# cd /var/lib/docker/volumes/config_database_config_cassandra/ [root@nodec54 config_database_config_cassandra]# cp -R _data/ Cassandra_data-save [root@nodec54 config_database_config_cassandra]# rm -rf _data/* [root@nodec55 ~]# cd /var/lib/docker/volumes/config_database_config_cassandra/ [root@nodec55 config_database_config_cassandra]# cp -R _data/ Cassandra_data-save [root@nodec55 config_database_config_cassandra]# rm -rf _data/* ## Start Zookeeper ## [root@nodec53 ~]# docker start config_database_zookeeper_1 [root@nodec54 ~]# docker start config_database_zookeeper_1 [root@nodec55 ~]# docker start config_database_zookeeper_1 ## Start Cassandra ## [root@nodec53 ~]# docker start config_database_cassandra_1 [root@nodec54 ~]# docker start config_database_cassandra_1 [root@nodec55 ~]# docker start config_database_cassandra_1 ## Run Docker Image & Mount Contrail TSL Certificates to Contrail SSL Directory ## [root@nodec54 ~]# docker image ls | grep config-api hub.juniper.net/contrail/contrail-controller-config-api 1909.30-ocata c9d757252a0c 4 months ago 583MB [root@nodec54 ~]# docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata ## Restore Data in New Docker Containers ## (config_api_1)[root@nodec54 /root]$ cd /usr/lib/python2.7/site-packages/cfgm_common/ (config_api_1)[root@nodec54 /usr/lib/python2.7/site-packages/cfgm_common]$ python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf ## Start Configuration Services ## [root@nodec53 ~]# docker start config_schema_1 [root@nodec53 ~]# docker start config_svcmonitor_1 [root@nodec53 ~]# docker start config_devicemgr_1 [root@nodec53 ~]# docker start config_nodemgr_1 [root@nodec53 ~]# docker start config_database_nodemgr_1 [root@nodec53 ~]# docker start contrail_config_api_1 [root@nodec53 ~]# docker start analytics_snmp_snmp-collector_1 [root@nodec53 ~]# docker start analytics_snmp_topology_1 [root@nodec53 ~]# docker start analytics_alarm_alarm-gen_1 [root@nodec53 ~]# docker start analytics_api_1 [root@nodec53 ~]# docker start analytics_collector_1 [root@nodec53 ~]# docker start analytics_alarm_kafka_1 [root@nodec54 ~]# docker start config_schema_1 [root@nodec54 ~]# docker start config_svcmonitor_1 [root@nodec54 ~]# docker start config_devicemgr_1 [root@nodec54 ~]# docker start config_nodemgr_1 [root@nodec54 ~]# docker start config_database_nodemgr_1 [root@nodec54 ~]# docker start contrail_config_api_1 [root@nodec54 ~]# docker start analytics_snmp_snmp-collector_1 [root@nodec54 ~]# docker start analytics_snmp_topology_1 [root@nodec54 ~]# docker start analytics_alarm_alarm-gen_1 [root@nodec54 ~]# docker start analytics_api_1 [root@nodec54 ~]# docker start analytics_collector_1 [root@nodec54 ~]# docker start analytics_alarm_kafka_1 [root@nodec55 ~]# docker start config_schema_1 [root@nodec55 ~]# docker start config_svcmonitor_1 [root@nodec55 ~]# docker start config_devicemgr_1 [root@nodec55 ~]# docker start config_nodemgr_1 [root@nodec55 ~]# docker start config_database_nodemgr_1 [root@nodec55 ~]# docker start contrail_config_api_1 [root@nodec55 ~]# docker start analytics_snmp_snmp-collector_1 [root@nodec55 ~]# docker start analytics_snmp_topology_1 [root@nodec55 ~]# docker start analytics_alarm_alarm-gen_1 [root@nodec55 ~]# docker start analytics_api_1 [root@nodec55 ~]# docker start analytics_collector_1 [root@nodec55 ~]# docker start analytics_alarm_kafka_1 ## Confirm Services are Active ## [root@nodec53 ~]# contrail-status [root@nodec54 ~]# contrail-status [root@nodec55 ~]# contrail-status
Example: How to Restore a Database Using the JSON Backup (Red Hat Openstack Deployer Environment)
This example shows how to restore the databases from an environment that was deployed using Red Hat Openstack and includes three config nodes—config1, config2, and config3—connected to the Contrail Configuration database (config-db). All steps that need to be done from a single config node are performed from config1.
The environment also contains three analytics nodes—analytics1, analytics2, and analytics3—to provide analytics services.
This example assumes a JSON backup file of the databases was previously created using the instructions provided in Simple Database Backup in JSON Format.
## Make db-dump directory. Copy contrail-api.conf to db-dump directory. ## [root@config1 ~]# mkdir /tmp/db-dump [root@config1 ~]# docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/ ## Stop Configuration Services on All Config Nodes ## [root@config1 ~]# docker stop contrail_config_svc_monitor [root@config1 ~]# docker stop contrail_config_device_manager [root@config1 ~]# docker stop contrail_config_schema [root@config1 ~]# docker stop contrail_config_api [root@config1 ~]# docker stop contrail_config_nodemgr [root@config1 ~]# docker stop contrail_config_database_nodemgr [root@config2 ~]# docker stop contrail_config_svc_monitor [root@config2 ~]# docker stop contrail_config_device_manager [root@config2 ~]# docker stop contrail_config_schema [root@config2 ~]# docker stop contrail_config_api [root@config2 ~]# docker stop contrail_config_nodemgr [root@config2 ~]# docker stop contrail_config_database_nodemgr [root@config3 ~]# docker stop contrail_config_svc_monitor [root@config3 ~]# docker stop contrail_config_device_manager [root@config3 ~]# docker stop contrail_config_schema [root@config3 ~]# docker stop contrail_config_api [root@config3 ~]# docker stop contrail_config_nodemgr [root@config3 ~]# docker stop contrail_config_database_nodemgr ## Stop Analytics Services on All Analytics Nodes ## [root@analytics1 ~]# docker stop contrail_analytics_snmp_collector [root@analytics1 ~]# docker stop contrail_analytics_topology [root@analytics1 ~]# docker stop contrail_analytics_alarmgen [root@analytics1 ~]# docker stop contrail_analytics_api [root@analytics1 ~]# docker stop contrail_analytics_collector [root@analytics1 ~]# docker stop contrail_analytics_kafka [root@analytics2 ~]# docker stop contrail_analytics_snmp_collector [root@analytics2 ~]# docker stop contrail_analytics_topology [root@analytics2 ~]# docker stop contrail_analytics_alarmgen [root@analytics2 ~]# docker stop contrail_analytics_api [root@analytics2 ~]# docker stop contrail_analytics_collector [root@analytics2 ~]# docker stop contrail_analytics_kafka [root@analytics3 ~]# docker stop contrail_analytics_snmp_collector [root@analytics3 ~]# docker stop contrail_analytics_topology [root@analytics3 ~]# docker stop contrail_analytics_alarmgen [root@analytics3 ~]# docker stop contrail_analytics_api [root@analytics3 ~]# docker stop contrail_analytics_collector [root@analytics3 ~]# docker stop contrail_analytics_kafka ## Stop Cassandra ## [root@config1 ~]# docker stop contrail_config_database [root@config2 ~]# docker stop contrail_config_database [root@config3 ~]# docker stop contrail_config_database ## Stop Zookeeper ## [root@config1 ~]# docker stop contrail_config_zookeeper [root@config2 ~]# docker stop contrail_config_zookeeper [root@config3 ~]# docker stop contrail_config_zookeeper ## Backup Zookeeper Directories Before Deleting Zookeeper Data Directory Contents ## [root@config1 _data]# cd /var/lib/docker/volumes/config_zookeeper/ [root@config1 config_zookeeper]# cp -R _data/version-2/ version-2-save [root@config1 config_zookeeper]# rm -rf _data/version-2/* [root@config2 _data]# cd /var/lib/docker/volumes/config_zookeeper/ [root@config2 config_zookeeper]# cp -R _data/version-2/ version-2-save [root@config2 config_zookeeper]# rm -rf _data/version-2/* [root@config3 _data]# cd /var/lib/docker/volumes/config_zookeeper/ [root@config3 config_zookeeper]# cp -R _data/version-2/ version-2-save [root@config3 config_zookeeper]# rm -rf _data/version-2/* ## Backup Cassandra Directory Before Deleting Cassandra Data Directory Contents ## [root@config1 ~]# cd /var/lib/docker/volumes/config_cassandra/ [root@config1 config_cassandra]# cp -R _data/ Cassandra_data-save [root@config1 config_cassandra]# rm -rf _data/* [root@config2 ~]# cd /var/lib/docker/volumes/config_cassandra/ [root@config2 config_cassandra]# cp -R _data/ Cassandra_data-save [root@config2 config_cassandra]# rm -rf _data/* [root@config3 ~]# cd /var/lib/docker/volumes/config_cassandra/ [root@config3 config_cassandra]# cp -R _data/ Cassandra_data-save [root@config3 config_cassandra]# rm -rf _data/* ## Start Zookeeper ## [root@config1 ~]# docker start contrail_config_zookeeper [root@config2 ~]# docker start contrail_config_zookeeper [root@config3 ~]# docker start contrail_config_zookeeper ## Start Cassandra ## [root@config1 ~]# docker start contrail_config_database [root@config2 ~]# docker start contrail_config_database [root@config3 ~]# docker start contrail_config_database ## Run Docker Image & Mount Contrail TSL Certificates to Contrail SSL Directory ## [root@config1 ~]# docker image ls | grep config-api hub.juniper.net/contrail/contrail-controller-config-api 1909.30-ocata c9d757252a0c 4 months ago 583MB [root@config1 ~]# docker run --rm -it -v /tmp/db-dump/:/tmp/ -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash hub.juniper.net/contrail/contrail-controller-config-api:1909.30-ocata ## Restore Data in New Docker Containers ## (config_api_1)[root@config1 /root]$ cd /usr/lib/python2.7/site-packages/cfgm_common/ (config_api_1)[root@config1 /usr/lib/python2.7/site-packages/cfgm_common]$ python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf ## Start Configuration Services on All Config Nodes ## [root@config1 ~]# docker start contrail_config_svc_monitor [root@config1 ~]# docker start contrail_config_device_manager [root@config1 ~]# docker start contrail_config_schema [root@config1 ~]# docker start contrail_config_api [root@config1 ~]# docker start contrail_config_nodemgr [root@config1 ~]# docker start contrail_config_database_nodemgr [root@config2 ~]# docker start contrail_config_svc_monitor [root@config2 ~]# docker start contrail_config_device_manager [root@config2 ~]# docker start contrail_config_schema [root@config2 ~]# docker start contrail_config_api [root@config2 ~]# docker start contrail_config_nodemgr [root@config2 ~]# docker start contrail_config_database_nodemgr [root@config3 ~]# docker start contrail_config_svc_monitor [root@config3 ~]# docker start contrail_config_device_manager [root@config3 ~]# docker start contrail_config_schema [root@config3 ~]# docker start contrail_config_api [root@config3 ~]# docker start contrail_config_nodemgr [root@config3 ~]# docker start contrail_config_database_nodemgr ## Start Configuration Services on All Analytics Nodes ## [root@analytics1 ~]# docker start contrail_analytics_snmp_collector [root@analytics1 ~]# docker start contrail_analytics_topology [root@analytics1 ~]# docker start contrail_analytics_alarmgen [root@analytics1 ~]# docker start contrail_analytics_api [root@analytics1 ~]# docker start contrail_analytics_collector [root@analytics1 ~]# docker start contrail_analytics_kafka [root@analytics2 ~]# docker start contrail_analytics_snmp_collector [root@analytics2 ~]# docker start contrail_analytics_topology [root@analytics2 ~]# docker start contrail_analytics_alarmgen [root@analytics2 ~]# docker start contrail_analytics_api [root@analytics2 ~]# docker start contrail_analytics_collector [root@analytics2 ~]# docker start contrail_analytics_kafka [root@analytics3 ~]# docker start contrail_analytics_snmp_collector [root@analytics3 ~]# docker start contrail_analytics_topology [root@analytics3 ~]# docker start contrail_analytics_alarmgen [root@analytics3 ~]# docker start contrail_analytics_api [root@analytics3 ~]# docker start contrail_analytics_collector [root@analytics3 ~]# docker start contrail_analytics_kafka ## Confirm Services are Active ## [root@config1 ~]# contrail-status [root@config2 ~]# contrail-status [root@config3 ~]# contrail-status [root@analytics1 ~]# contrail-status [root@analytics2 ~]# contrail-status [root@analytics3 ~]# contrail-status