Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Announcement: Try the Ask AI chatbot for answers to your technical questions about Juniper products and solutions.

close
header-navigation
keyboard_arrow_up
close
keyboard_arrow_left
list Table of Contents
file_download PDF
{ "lLangCode": "en", "lName": "English", "lCountryCode": "us", "transcode": "en_US" }
English
keyboard_arrow_right

How to Backup and Restore Contrail Databases in JSON Format in Openstack Environments Using the Openstack 16.1 Director Deployment

date_range 20-Sep-23

This document shows how to backup and restore the Contrail databases—Cassandra and Zookeeper—in JSON format when Contrail Networking is running in Openstack-orchestrated environments that were deployed using the RedHat Openstack 16.1 director deployment.

If you are deploying Contrail Networking in an Openstack-orchestrated environment that was deployed using an Openstack 13-based or Ansible deployer, see How to Backup and Restore Contrail Databases in JSON Format in Openstack Environments Using the Openstack 13 or Ansible Deployer.

Contrail Networking is initially supported in Openstack environments using the Openstack 16.1 director deployment in Contrail Networking Release 2008. See Contrail Networking Supported Platforms for a matrix of Contrail Networking release support within orchestration platforms and deployers.

Before You Begin

The backup and restore procedure must be completed for nodes running the same Contrail Networking release. The procedure is used to backup the Contrail Networking databases only; it does not include instructions for backing up orchestration system databases.

CAUTION:

Database backups must be consistent across all systems because the state of the Contrail database is associated with other system databases, such as OpenStack databases. Database changes associated with northbound APIs must be stopped on all the systems before performing any backup operation. For example, you might block the external VIP for northbound APIs at the load balancer level, such as HAproxy.

Simple Database Backup in JSON Format

This procedure provides a simple database backup in JSON format. This procedure is performed using the db_json_exim.py script located inside the config-api container in /usr/lib/python2.7/site-packages/cfgm_common on the controller node.

To perform this database backup:

  1. From a controller node, ensure the db_json_exim.py script is available:

    content_copy zoom_out_map
    (overcloud) [user@overcloud-contrailcontroller-0 heat-admin]# podman exec -it contrail_config_api bash
    (config-api)[user@overcloud-contrailcontroller-0 /]$ ls /usr/lib/python2.7/site-packages/cfgm_common/db_json_exim.py
    /usr/lib/python2.7/site-packages/cfgm_common/db_json_exim.py
  2. Log into one of the contrail controller nodes. Create the /tmp/db-dump directory on any of the contrail controller node hosts.

    content_copy zoom_out_map
    mkdir /tmp/db-dump
    Note:

    /tmp/db-dump is a user-defined directory name. You can assign any name to a directory.

  3. On the same contrail controller node, copy the contrail-api.conf file from the container to the host.

    content_copy zoom_out_map
    podman cp contrail_config_api:/etc/contrail/contrail-api-0.conf /tmp/db-dump/contrail-api.conf

    The Cassandra database instance on any contrail controller node includes the complete Cassandra database for all contrail controller nodes in the cluster. Steps 2 and 3, therefore, only need to be performed on one contrail controller node.

  4. On all Contrail controller nodes, stop the following Contrail configuration services:

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_config_svc_monitor.service
    systemctl stop tripleo_contrail_config_device_manager.service
    systemctl stop tripleo_contrail_config_schema.service
    systemctl stop tripleo_contrail_config_api.service
    systemctl stop tripleo_contrail_config_nodemgr.service
    systemctl stop tripleo_contrail_config_database_nodemgr.service

    This step must be performed on each individual controller node in the cluster.

  5. On all nodes hosting Contrail Analytics containers, stop the following analytics services:

    content_copy zoom_out_map
    systemctl  stop tripleo_contrail_analytics_kafka.service
    systemctl stop tripleo_contrail_analytics_snmp_nodemgr.service
    systemctl stop tripleo_contrail_analytics_alarmgen.service
    systemctl stop tripleo_contrail_analytics_alarm_nodemgr.service
    systemctl stop tripleo_contrail_analytics_topology.service
    systemctl stop tripleo_contrail_analytics_collector.service
    systemctl stop tripleo_contrail_analytics_nodemgr.service
    systemctl stop tripleo_contrail_analytics_snmp_collector.service
    systemctl stop tripleo_contrail_analytics_api.service

    This step must be performed on each individual analytics node in the cluster.

  6. Return to the contrail controller node where you performed steps 2 and 3.

    Use the podman images command to list the name or ID of the config api image.

    content_copy zoom_out_map
    podman images | grep config-api

    Example:

    content_copy zoom_out_map
    (overcloud) [user@overcloud-contrailcontroller-0 db-dump]# podman images | grep config-api
    192.168.24.1:8787/contrail/contrail-controller-config-api
    2011.L1.297   2dcd2feaeed5   2 months ago   876 MB
  7. From the same contrail controller node, start the config api container by pointing the entrypoint.sh script to the /bin/bash directory then mapping /tmp/db-dump directory from the host to the /tmp directory inside the container. You perform this step to ensure that the API services are not started on the contrail controller node.

    Enter the -v /etc/contrail/ssl:/etc/contrail/ssl:ro command option when cassandra_use_ssl is used as the api-server configuration parameter to ensure TLS certificates are mounted to the Contrail SSL directory. This mounting ensures that the backup procedure succeeds in environments with endpoints that require TLS authentication.

    The registry_name and container_tag variables must match step 6.

    content_copy zoom_out_map
    podman run --rm -it -v /tmp/db-dump/:/tmp:Z -v /etc/contrail/ssl:/etc/contrail/ssl:ro -v /etc/ipa:/etc/ipa:ro --network host --entrypoint=/bin/bash registry_name/contrail-controller-config-api:container_tag

    Example:

    content_copy zoom_out_map
    podman run --rm -it -v /tmp/db-dump/:/tmp:Z -v /etc/contrail/ssl:/etc/contrail/ssl:ro -v /etc/ipa:/etc/ipa:ro --network host --entrypoint=/bin/bash 192.168.24.1:8787/contrail-nightly/contrail-controller-config-api:21.4.L3.441
    Note:

    The cluster is TLS-enabled if TLS-related files are used during Contrail Networking deployment. The TLS-related files are part of the deployment script. These are custom files and can have any name. For example:

    content_copy zoom_out_map
     ~/tripleo-heat-templates/environments/contrail/contrail-tls.yaml \
     ~/tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml \
     ~/tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml \
     ~/tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml \
  8. From the container created on the contrail controller node in Step 7, use the db_json_exim.py script to backup data in JSON format. The db dump file will be saved in the /tmp/db-dump/ directory on this contrail controller node. Example:

    content_copy zoom_out_map
    (config-api)[user@overcloud-contrailcontroller-0 /]$ cd /usr/lib/python2.7/site-packages/cfgm_common
    (config-api)[user@overcloud-contrailcontroller-0 /usr/lib/python2.7/site-packages/cfgm_common]$ python db_json_exim.py --export-to /tmp/db-dump.json --api-conf /tmp/contrail-api.conf
    2021-06-30 19:47:27,120 INFO: Cassandra DB dumped
    2021-06-30 19:47:28,878 INFO: Zookeeper DB dumped
    2021-06-30 19:47:28,895 INFO: DB dump wrote to file /tmp/db-dump.json

    The Cassandra database instance on any contrail controller node includes the complete Cassandra database for all contrail controller nodes in the cluster. You, therefore, only need to perform step 4 through 6 from one of the contrail controller nodes.

  9. (Optional. Recommended) From the same contrail controller node, enter the cat /tmp/db-dump.json | python -m json.tool | less command to view a more readable version of the file transfer.

    content_copy zoom_out_map
    cat /tmp/db-dump.json | python -m json.tool | less
  10. From the same contrail controller node, exit out of the config api container. This will stop the container.

    content_copy zoom_out_map
    exit
  11. On each contrail controller node, start the following configuration services:

    content_copy zoom_out_map
    systemctl start tripleo_contrail_config_svc_monitor.service
    systemctl start tripleo_contrail_config_device_manager.service
    systemctl start tripleo_contrail_config_schema.service
    systemctl start tripleo_contrail_config_api.service
    systemctl start tripleo_contrail_config_nodemgr.service
    systemctl start tripleo_contrail_config_database_nodemgr.service

    This step must be performed on each individual contrail controller node.

  12. On each analytics node, start the following analytics services:

    content_copy zoom_out_map
    systemctl  start tripleo_contrail_analytics_kafka.service
    systemctl  start tripleo_contrail_analytics_snmp_nodemgr.service
    systemctl  start tripleo_contrail_analytics_alarmgen.service
    systemctl  start tripleo_contrail_analytics_alarm_nodemgr.service
    systemctl  start tripleo_contrail_analytics_topology.service
    systemctl  start tripleo_contrail_analytics_collector.service
    systemctl  start tripleo_contrail_analytics_nodemgr.service
    systemctl  start tripleo_contrail_analytics_snmp_collector.service
    systemctl  start tripleo_contrail_analytics_api.service

    This step must be performed on each individual analytics node.

  13. On each contrail controller node, enter the contrail-status command to confirm that services are in the active or running states.

    Note:

    Some command output and output fields are removed for readability. Output shown is from a single node hosting configuration and analytics services.

    content_copy zoom_out_map
    contrail-status
    Pod             Service     Original Name                 State
    analytics       api         contrail-analytics-api        running
    analytics       collector   contrail-analytics-collector  running
    analytics       nodemgr     contrail-nodemgr              running
    analytics       provisioner contrail-provisioner          running
    analytics       redis       contrail-external-redis       running
    analytics-alarm alarm-gen   contrail-analytics-alarm-gen  running
    analytics-alarm kafka       contrail-external-kafka       running
    <some output removed for readability>
    
    == Contrail control ==
    control: active
    nodemgr: active
    named: active
    dns: active
    
    == Contrail analytics-alarm ==
    nodemgr: active
    kafka: active
    alarm-gen: active
    
    == Contrail database ==
    nodemgr: active
    query-engine: active
    cassandra: active
    
    == Contrail analytics ==
    nodemgr: active
    api: active
    collector: active
    
    == Contrail config-database ==
    nodemgr: active
    zookeeper: active
    rabbitmq: active
    cassandra: active
    
    == Contrail webui ==
    web: active
    job: active
    
    == Contrail analytics-snmp ==
    snmp-collector: active
    nodemgr: active
    topology: active
    
    == Contrail config ==
    svc-monitor: active
    nodemgr: active
    device-manager: active
    api: active
    schema: active

Restore Database from the Backup in JSON Format

This procedure provides the steps to restore a system using the simple database backup JSON file that was created in Simple Database Backup in JSON Format.

To restore a system from a backup JSON file:

  1. Copy the contrail-api.conf file from the container to the host on any one of the contrail controller nodes.

    content_copy zoom_out_map
    podman cp contrail_config_api:/etc/contrail/contrail-api-0.conf /tmp/db-dump/contrail-api.conf
  2. On all of the Contrail controller nodes, stop these configuration services:

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_config_svc_monitor.service
    systemctl stop tripleo_contrail_config_device_manager.service
    systemctl stop tripleo_contrail_config_schema.service
    systemctl stop tripleo_contrail_config_api.service
    systemctl stop tripleo_contrail_config_nodemgr.service
    systemctl stop tripleo_contrail_config_database_nodemgr.service
  3. On all nodes hosting Contrail Analytics containers, stop the following services:

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_analytics_kafka.service
    systemctl stop tripleo_contrail_analytics_snmp_nodemgr.service
    systemctl stop tripleo_contrail_analytics_alarmgen.service
    systemctl stop tripleo_contrail_analytics_alarm_nodemgr.service
    systemctl stop tripleo_contrail_analytics_topology.service
    systemctl stop tripleo_contrail_analytics_collector.service
    systemctl stop tripleo_contrail_analytics_nodemgr.service
    systemctl stop tripleo_contrail_analytics_snmp_collector.service
    systemctl stop tripleo_contrail_analytics_api.service
  4. Stop the Cassandra service on all the config-db controllers.

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_config_database.service
  5. Stop the Zookeeper service on all the contrail controllers.

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_config_zookeeper.service
  6. Stop the Zookeeper service on all the contrail-analytics controllers.

    content_copy zoom_out_map
    systemctl stop tripleo_contrail_analytics_zookeeper.service
  7. Backup the Zookeeper data directory on all the controllers.

    content_copy zoom_out_map
    cd /var/lib/contrail/config_zookeeper
    cp -aR version-2/ zookper-bkp.save
  8. Delete the Zookeeper data directory contents on all the controllers.

    content_copy zoom_out_map
    rm -rf version-2
  9. Backup the Cassandra data directory on all the controllers.

    content_copy zoom_out_map
    cd /var/lib/contrail/config_cassandra
    cp -aR data/ Cassandra_data-save
  10. Delete the Cassandra data directory contents on all controllers.

    content_copy zoom_out_map
    rm -rf data/
  11. Start the Zookeeper service on all the contrail controllers.

    content_copy zoom_out_map
    systemctl start tripleo_contrail_config_zookeeper.service 
  12. Start the Zookeeper service on all the contrail-analytics controllers.

    content_copy zoom_out_map
    systemctl start tripleo_contrail_analytics_zookeeper.service
  13. Start the Cassandra service on all of the controllers.

    content_copy zoom_out_map
    systemctl start tripleo_contrail_config_database.service
  14. Use the podman images command to list the name or ID of the config api image.

    content_copy zoom_out_map
    podman image ls | grep config-api

    Example:

    content_copy zoom_out_map
    user@overcloud-contrailcontroller-0 heat-admin]# podman image ls | grep config-a
    192.168.24.1:8787/contrail/contrail-controller-config-api
    2011.L1.297   2dcd2feaeed5   1 months ago   876 MB
  15. Run a new podman container using the name or ID of the config_api image on the same contrail controller node.

    Enter the -v /etc/contrail/ssl:/etc/contrail/ssl:ro command option when cassandra_use_ssl is used as api-server configuration parameter to ensure TLS certificates are mounted to the Contrail SSL directory. This mounting ensures that this backup procedure succeeds in environments with endpoints that require TLS authentication.

    Use the registry_name and container_tag from the output of the step 14.

    content_copy zoom_out_map
    podman run --rm -it -v /tmp/db-dump/:/tmp:Z -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash <registry_name>/contrail-controller-config-api:<container tag>

    Example:

    content_copy zoom_out_map
    podman run --rm -it -v /tmp/db-dump/:/tmp:Z -v /etc/contrail/ssl:/etc/contrail/ssl:ro --network host --entrypoint=/bin/bash 192.168.24.1:8787/contrail/contrail-controller-config-api:2011.L1.297
  16. Restore the data in the new running container on the same contrail controller node.

    content_copy zoom_out_map
    cd /usr/lib/python2.7/site-packages/cfgm_common
    python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf

    Example:

    content_copy zoom_out_map
    cd /usr/lib/python2.7/site-packages/cfgm_common
    python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf
    2021-07-06 17:22:17,157 INFO: DB dump file loaded
    2021-07-06 17:23:12,227  INFO: Cassandra DB restored
    2021-07-06 17:23:14,236  INFO: Zookeeper DB restored
  17. Exit out of the config api container. This will stop the container.

    content_copy zoom_out_map
    exit
  18. Start config services on all of the controllers:

    content_copy zoom_out_map
    systemctl start tripleo_contrail_config_svc_monitor.service
    systemctl start tripleo_contrail_config_device_manager.service
    systemctl start tripleo_contrail_config_schema.service
    systemctl start tripleo_contrail_config_api.service
    systemctl start tripleo_contrail_config_nodemgr.service
    systemctl start tripleo_contrail_config_database_nodemgr.service
  19. Start services on all of the analytics nodes:

    content_copy zoom_out_map
    systemctl  start tripleo_contrail_analytics_kafka.service
    systemctl  start tripleo_contrail_analytics_snmp_nodemgr.service
    systemctl  start tripleo_contrail_analytics_alarmgen.service
    systemctl  start tripleo_contrail_analytics_alarm_nodemgr.service
    systemctl  start tripleo_contrail_analytics_topology.service
    systemctl start tripleo_contrail_analytics_collector.service
    systemctl  start tripleo_contrail_analytics_nodemgr.service
    systemctl  start tripleo_contrail_analytics_snmp_collector.service
    systemctl  start tripleo_contrail_analytics_api.service
  20. Enter the contrail-status command on each contrail controller node and, when applicable, on each analytics node to confirm that services are in the active or running states.

    Note:

    Output shown for a contrail controller node. Some command output and output fields are removed for readability.

    content_copy zoom_out_map
    contrail-status
    Pod     Service         Original Name                         State
    config  api             contrail-controller-config-api        running
    config  device-manager  contrail-controller-config-devicemgr  running
    config  dnsmasq         contrail-controller-config-dnsmasq    running
    config  nodemgr         contrail-nodemgr                      running
    config  provisioner     contrail-provisioner                  running
    config  schema          contrail-controller-config-schema     running
    config  stats           contrail-controller-config-stats      running
    <some output removed for readability>
    
    == Contrail control ==
    control: active
    nodemgr: active
    named: active
    dns: active
    
    
    == Contrail database ==
    nodemgr: active
    query-engine: active
    cassandra: active
    
    == Contrail config-database ==
    nodemgr: active
    zookeeper: active
    rabbitmq: active
    cassandra: active
    
    == Contrail webui ==
    web: active
    job: active
    
    == Contrail config ==
    svc-monitor: active
    nodemgr: active
    device-manager: active
    api: active
    schema: active
footer-navigation