Restoring Product Data from Backup
Automated Restore: Using the ncc restore
Command
To restore a Paragon Active Assurance system from a backup taken with the ncc
backup
command, first stop services:
sudo ncc services stop --with-kafka --with-zookeeper --without-timescaledb
Then run this command on the backup tarball:
ncc restore paa_backup-yyyy-mm-dd_hh-mm-ss.tar.gz
Finally, restart services:
sudo ncc services start --with-kafka --with-zookeeper --without-timescaledb
The restore command does all of the following:
- Drops, recreates, and restores the main PostgreSQL database
- Drops, recreates, and restores the plugin PostgreSQL database
- Restores the plugin tarballs and signing keys
- Removes any existing RRDs, and restores RRD files
- Restores the TimescaleDB database and TimescaleDB data
- Restores TimescaleDB configuration files
- Removes any existing OpenVPN keys, and restores OpenVPN keys
-
Restores SSL certificates and keys
- Restores Control Center configuration files
- Activates licenses
Option:
--timescaledb-in-place
: By default thencc restore
command assumes that you are doing the restore on a different host. If you want to perform an in-place restore of the TimescaleDB database, add this option to the command.
Activating a New License for the Restored System
If you are restoring the system on a new host, or on the same host but with a different UUID, the system requires a new license. Please see the Installation Guide, chapter Getting Started with Paragon Active Assurance, for details on how to obtain and activate a license.
Performing a Partial Restore
You can optionally leave out certain parts of the backup from the restore operation.
Only backup files that are not mandatory can be left out of the restore.
The following files can be omitted freely:
paa-cc-backup.log
paa_cc_rrd.tar.gz
paa_cc_configs.tar.gz
The following files can both be left out, but one is not valid without the other:
paa_cc_postgres.sql
paa_cc_plugins.sql
Proceed in the following steps (the backup version and timestamp are suppressed below for readability):
-
List the backup tarball content in order to identify parts to exclude:
tar -tf paa-cc-backup-<...>.tar.gz
-
Unpack the backup tarball, excluding files as desired. Below is an example where the RRD files are excluded:
tar -zxvf paa-cc-backup-<...>.tar.gz --exclude "paa-cc-backup-<...>/paa_cc_rrd.tar.gz"
-
Rename the backup folder to match the intended final tarball name:
mv paa-cc-backup-<...> paa-cc-backup-without-rrd-<...>
-
Create a new tarball from the remaining files:
tar -pczf paa-cc-backup-without-rrd-<...>.tar.gz paa-cc-backup-without-rrd-<...>/
Troubleshooting: Manual Restore
Should the ncc restore
command fail for some reason, you can restore the system
manually as described in this section.
In what follows, the backup files are assumed to be named in the same way as in the chapter Backing Up Product Data.
The procedure that follows requires that Control Center has been installed (so that the
netrounds
database exists) and that the backup files are from the same
version of Control Center as the installed version.
-
Stop all Paragon Active Assurance services that are accessing the database:
sudo ncc services stop --with-kafka --with-zookeeper --without-timescaledb
-
Drop the main PostgreSQL database and plugin database:
sudo -u postgres psql -c "DROP DATABASE netrounds;" sudo -u postgres psql -c 'DROP DATABASE "paa-plugins";'
-
Recreate the main PostgreSQL database and plugin database:
sudo -u postgres psql -c "CREATE DATABASE netrounds OWNER netrounds ENCODING 'UTF8' TEMPLATE 'template0';" sudo -u postgres psql -c 'CREATE DATABASE "paa-plugins" OWNER netrounds ENCODING "UTF8" TEMPLATE "template0";'
-
Restore the main PostgreSQL database and plugin database:
sudo -u postgres psql --set ON_ERROR_STOP=on netrounds < cc_postgres.sql sudo -u postgres psql --set ON_ERROR_STOP=on paa-plugins < paa_plugins.sql
Setting
ON_ERROR_STOP
to on will cause psql to exit if an error occurs. For further details, see the PostgreSQL documentation: https://www.postgresql.org/docs/9.4/backup-dump.html#BACKUP-DUMP-RESTORE.There are several options for the backup format. Please refer to the psql documentation (https://www.postgresql.org/docs/) if you are using anything other than plain .sql.
If you backed up the databases (manually) in binary format, use these commands to restore them (you will be prompted for a password which is by default "netrounds"):
pg_restore -Fc -h localhost -d netrounds -c --verbose --disable-triggers cc_postgres.binary -U netrounds --password pg_restore -Fc -h localhost -d paa-plugins -c --verbose --disable-triggers paa_plugins.binary -U netrounds --password
-
Restore the plugin tarballs and signing keys:
sudo tar -xzf paa_plugins.tar.gz -C / sudo tar -xzf paa_plugin_keys.tar.gz -C /
-
Restore the OpenVPN keys:
# Remove any existing OpenVPN keys sudo rm -rf /var/lib/netrounds/openvpn # Unpack the backed-up keys sudo tar -xzf cc_openvpn.tar.gz -C /
-
Restore the SSL certificates and keys:
sudo tar -xzf paa_cc_certs.tar.gz -C /
-
Restore RRD files, either from a tarball or from a snapshot. Compare the chapter Backing Up Product Data. In the tarball case, the procedure is as follows:
# Remove any existing RRDs sudo rm -rf /var/lib/netrounds/rrd # Unpack the backed-up RRDs sudo tar -xzf cc_rrd.tar.gz -C /
-
Restore the TimescaleDB database:
The backup directory needs to be stored under
/var/lib/netrounds/rrd/timescaledb/pgbackrest/repo/
.Before doing the restore, copy the
.conf
files in the data directory/var/lib/netrounds/rrd/timescaledb/data/
to a different location. This allows you to restore any configuration changes performed after the last full backup.-
To perform an in-place database restore, proceed as follows:
- Create a directory
/tmp/timescaledb_conf
. - Copy the
*.conf
files in/var/lib/netrounds/rrd/timescaledb/data
to/tmp/timescaledb_conf
. -
Run the command
sudo /etc/netrounds/backup-restore-timescaledb.sh --restore <backup dir>
- Copy the
*.conf
files from/tmp/timescaledb_conf
to/var/lib/netrounds/rrd/timescaledb/data
.
- Create a directory
-
To restore a database from host A on a different host B, proceed as follows:
- On host B, install and configure Control Center according to the Paragon Active Assurance Installation Guide, and enable the metrics service.
- From host A, copy the contents of
/var/lib/netrounds/rrd/timescaledb/pgbackrest
to the same directory on host B. -
On host B, run the command
sudo /etc/netrounds/backup-restore-timescaledb.sh --restore <backup dir>
- From host A, copy the
*.conf
files from/var/lib/netrounds/rrd/timescaledb/data
to the same directory on host B.
-
-
Copy the license files into their original location. (Alternatively, you may download the license files once more from the Juniper EMS Portal and apply them using the
ncc license activate
command, as explained in the Installation Guide.) -
Start Paragon Active Assurance services:
sudo ncc services start --with-kafka --with-zookeeper --without-timescaledb