Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

How to Use Command Line Tools to Administer APM

After you've installed the Address Pool Manager (APM) application, you can perform the following administration functions.

Address Pool Manager gives you two command line options for perform administrator tasks. You can either use the APM utility script (apm) or the Kubernetes Command Line tool to administer APM.

Access APM Utility Commands

Use the APM utility commands to perform administration functions.

You can use the APM utility script (apm) to administer the application and to access the CLI that you use to configure the address management functions. The Juniper APM installation places the utility script in /var/local/apm and creates a symbolic link to the script in /usr/local/bin/apm.

You can use the apm utility script (which uses the Kubernetes command line tool and Helm commands ) to do the following:

  • Create and delete objects.
  • Provide log access.
  • Conduct interactive sessions with pod containers.
  • Display the status of the APM objects.

Using the apm utility script simplifies many of your administrative duties. The script performs the tasks you need to manage APM, while masking the complexity of the kubectl command.

Table 1 lists the commands that you can invoke with the apm utility script and describes the action that occurs. Many of the individual commands have options that you can specify.

Table 1: APM Utility Script Commands
Name Action
clean Clean up unneeded releases and/or docker cache. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.2.
cli Access the CLI that you can use to configure APM features and to monitor the current status for managed BNGs.
contexts Display the available cluster contexts for control with APM
Note:

Introduced in Release 3.2.
db-info Displays current state of APM’s database microservice including the current version, stateful set pods, and their roles.
Note:

Introduced in Release 3.2.
db-switchover Forces the persistent state database (DB) primary pod to switchover to an eligible backup pod. To run this command, you need sudo privileges. Introduced in Release 3.2.
Note:

DB switchover is a service disrupting event and you only use it with the upgrade procedure.
ip Displays the IP addresses of Address Pool Manager.
Note:

Introduced in Release 3.2.
link Links a cluster to a specific software version.
Note:

Introduced in Release 3.2.
logs Display APM logs.
rename-context Rename a context. Does not effect the running Address Pool Manager on the cluster. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.2.
restart Restart one or more specified services. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.1.
rollout Upgrade or start an APM service. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.2.
save-config Saves the current configuration of the Address Pool Manager to a file outside the pod. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.2.
shell Connect to a running APM microservice. To run this command, you need sudo privileges.
setup Set up the APM application as part of the installation process. To run this command, you need sudo privileges.
start Start all APM services. To run this command, you need sudo privileges.
status Display the current status of the APM services. To run this command, you need sudo root privileges.
storage

Provides the status of the storage drivers for APM.

Note:

Introduced in Release 3.2.
stop Stop all APM services. To run this command, you need sudo privileges.
unlink Unlink components associated with the context. To run this command, you need sudo privileges.
Note:

Introduced in Release 3.2.
version Displays the version of every running microservice in the APM instance as well as the APM utility. It also lists all available APM software releases on the system.

Use the following general syntax to issue a command:

  • For a short option:

  • For a long option:

To target a command at a particular cluster context, use the context option.

To display a list of available commands with a brief description, use either the h or help option:

To display the options for a specific command:

To specify the --no-color option to disable colored-text output (used to distinguish logs from different microservices):

Upgrade APM to a New Version Using the APM installation Utility

Use this procedure to upgrade to a new version of APM which is installed on a cluster that was created by the BBE Cloudsetup utility or by Red Hat OpenShift Container Platform Console. This procedure assumes APM is running on your system.

  1. Download the APM software package from the Juniper Networks software download page to the jump host.

    APM is available as a compressed tarball image (.tgz). The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • major is the main release number of the product.
    • minor is the minor release number of the product.
    • maintenance is the revision number.
  2. Unpack the APM tarball (.tgz) file on the jump host by entering:
  3. Run the loader script after you unpack the tarball.
  4. Link to the cluster by using the link command. The link command associates the loaded APM software package to the cluster in preparation for the setup.

    • context-name is the Kubernetes context (cluster name).

    • apm-version is the software version.

  5. If you are upgrading APM on a Red Hat OpenShift Container Platform cluster, log in with the OpenShift CLI and then proceed to the next step.
    If you are installing APM on a BBE Cloudsetup created cluster, proceed to the next step.
  6. You must authenticate with the container registry in order to be able to push the APM container images. How you authenticate to the registry varies depending on if you are installing APM on a BBE Cloudsetup created cluster or on an Red Hat OpenShift Container Platform cluster (see the respective documentation for details).
  7. Run setup to complete any additional environment values.

    • context-name is the Kubernetes context (cluster names).

    • update are the prompts for only missing values (primarily used after loading a new software release).

  8. Display the running DB to see which pod is the primary pod and to determine whether to upgrade the persistent state database (DB):
  9. Display the DB version in the new package:
    Note:

    For example, the DB version 6.2.14-debian-12-r21 is later than what is running (6.2.13), so you need to upgrade the DB.
  10. Initiate a DB switchover if the jnpr-apm-redis-0 is not the primary DB.
  11. Rollout the new DB version.
  12. Upgrade the microservices with the sudo -E apm rollout –-context context-name –service service-name --version software-version command. Enter the commands in the following order:
  13. Verify that all microservices are running the new version of software:

Upgrade APM to a New Version Without Using the APM Utility

The instructions in this section describes the upgrade steps for installing APM on a preexisting Kubernetes cluster of your choice. This process is a manual process and does not use the APM utility that comes with the APM installation package.

  1. Download the APM software package from the Juniper Networks software download page to the jump host.

    APM is available as a compressed tarball image (.tgz). The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • major is the main release number of the product.
    • minor is the minor release number of the product.
    • maintainance is the revision number.
  2. Unpack the APM tarball (.tgz) file on the jump host by entering:
  3. The container images needed by APM are stores in the images subdirectory. You must push the images to the registry where the scheduled application images will be pulled from. Depending on the type of container registry being used the commands may be different. The following commands illustrate one method of pushing container images to the registry:
  4. To prepare APM for upgrade, create a new YAML configuration file for each microservice. (Table 3 describes the fields in the microservice's configuration files.)
    Note:

    You may have create a single values.yaml, during your intial installation,that contains information for all the microservices. The single values.yaml is located under the umbrella chart in the apm/apm/charts/address_pool_manager folder. The procedures in this section only describe how to upgrade if created individual YAML configuration files for each microservice.

    Create a new values.yaml file for each of the microservices, by making a copy of the file and then saving the new file. Update each file according to your Kubernetes cluster's information.
    Following are the microservices and their values.yaml file location:

    • redis microservice—Located at apm/apm/charts/redis

    • mgmt microservice—Located at apm/apm/charts/mgmt

    • addrman microservice—Located at apm/apm/charts/addrman

    • entman microservice—Located at apm/apm/charts/entman.

    • provman microservice—Located at apm/apm/charts/provman

  5. Run the dependency update command:
  6. After you have made all the desired changes to your new values.yaml files for each microservice, the microservices must be deployed with the new values.yaml files.

    Run the following commands:

  7. Verify the APM installation by running the Kubernetes Command Line Tool command kubectl get pods and verify the APM pods are running.
  8. Verify that the services are present. Run the Kubernetes Command Line Tool command kubectl get services.

Start or Stop APM Services Using the APM Utility

Use the apm utility script to start or stop all APM services. The services start in order of dependency. Essential services (db and mgmt) start first, followed by the other services. The services stop in reverse order of dependency.

  • To start all APM services:

    Note:

    We recommend that you use the sudo -E apm start ––services option to start individual services or a set of services only for troubleshooting. Use under the guidance of a Juniper Networks support representative. Use with caution as this command is like rebooting to factory-default.
    Note:

    APM starts from it's initial settings when you execute the apm setup command. Any persistent state is lost when the apm stop command is executed. The current configuration can be saved using the apm save-config command. The saved configuration is the configuration that is used the next time APM is started.

  • To stop all APM services:

Restart APM Services Using the Kubernetes Command Line Tool

Use the kubectl delete pods Kubernetes command to restart APM services. For example:

Note:

To determine the pod name, you can use the kubectl get pods -n jnpr-apm Kubernetes command (see Check the Status of APM Services Using the Kubernetes Command Line Tool).

Setup Secrets Using the APM Utility

You can setup secrets during setup or run the sudo -E apm setup --context context-name --secrets to setup secrets or update them.

Note:

If you enter a value for the secret name, you will not be asked for the key or certification files.

Display Database Information Using the APM Utility

The apm db-info command displays current state of APM’s database microservice including the current version, stateful set pods, and their roles.

apm db-info [--context context-name] [-o|--output json]

Display the Running Database Using the Kubernetes Command Line Tool

Use the kubectl exec Kubernetes command to display the running database (DB) to see which pod is the primary pod and to determine whether to upgrade the persistent state database. You should run the kubectl exec on both the primary and secondary redis server instances. For example:

Perform a Database Switchover Using the Kubernetes Command Line Tool

To force the persistent state database (DB) primary pod to switchover to an eligible backup pod, perform the following:

  1. Run the kubectl get pods Kubernetes command to determine the name of the sentinel pods.
  2. Pick any of the three sentinel pods to use with the kubectl exec command. The following example uses the jnpr-apm-redis-sentinels-0 sentinel pod. After running following commands, the roles of the redis instances are reveresed (redis-0 is the secondary and redis-1 is the primary).

Check the Status of APM Services Using the APM Utility

Use the apm status utility script to check the status of each APM service (functional component) listed in Table 2. The status shows whether a service is running, has exited, or has not started. It also displays the service name on the Kubernetes pod. You can compare uptime for the services to quickly see whether any service has been restarted.

Table 2: Services Displayed with the status Command

Microservice

Pod Prefix

addrman—Address manager

jnpr-apm-addrman

mgmt—CLI management

jnpr-apm-mgmt

redis (Remote Dictionary Server)—consists of a set of pods which provide the persistent database.

jnpr-apm-redis

entman—Entity manager

jnpr-apm-entman

provman—Provisioning manager

jnpr-apm-provman

To check the status:

  1. Display the service status.
  2. (Optional) Render the version information in JavaScript Object Notation (JSON) format, which is useful for scripting interfaces.

For example:

Check the Status of APM Services Using the Kubernetes Command Line Tool

Use the Kubernetes Command Line tool to check the status of each APM service (functional component) listed in Table 2. The status shows whether a service is running, has exited, or has not started. It also displays the service name on the Kubernetes pod. You can compare uptime for the services to quickly see whether any service has been restarted.

Table 3: Services Displayed with the status Command

Microservice

Pod Prefix

addrman—Address manager

jnpr-apm-addrman

mgmt—CLI management

jnpr-apm-mgmt

redis (Remote Dictionary Server)—consists of a set of pods which provide the persistent database.

jnpr-apm-redis

entman—Entity manager

jnpr-apm-entman

provman—Provisioning manager

jnpr-apm-provman

To check the status, run the following command:

For example:

Display APM IP Addresses Using the APM Utility

Use the apm ip utility script to display the Kubernetes objects that are necessary for the orchestration of the APM pods. For example:

Display the APM IP Addresses Using the Kubernetes Command Line Tool

Use the kubectl get services Kubernetes command to display the Kubernetes objects that are necessary for the orchestration of the APM pods. For example:

Display Logging Using the APM Utility

Use the apm logs utility script to display the logs of events that occur while APM is running. You can also use the BBE Event Collection and Visualization utility to display file-based logs collected and stored since the time APM is started. BBE Event Collection and Visualization is a cloud-based centralized utility that provides a way to capture APM logs that span the life-cycle of APM micro-services. You link to the BBE Event Collection and Visualization logging utility when you set up APM. See the Broadband Edge Event Collection and Visualization Installation Guide.

Display APM Logging

BBE Event Collection and Visualization is a cloud-based centralized utility that provides a way to capture APM logs that span the life-cycle of APM micro-services. If you have BBE Event Collection and Visualization setup, during the APM setup, you can point BBE Event Collection and Visualization to perform the logging. BBE Event Collection and Visualization has a web-based interface to OpenSearch’s capabilities for advanced searching, aggregation, viewing, and data analysis of collected syslog events.

Use the apm logs utility script to display the logs of events that occur while APM is running. The event logs include events such as those shown in the following non-exhaustive list:

  • Pool-domain registration events

  • Address allocation failures

  • Network entity connection failures

  • Startup messages

  • Network entity resynchronization events

  • Pool and partition utilization threshold and depletion warnings

By default, APM sends logs to the standard output (stdout) of the service. The output displays the circular buffer of all services or of a specified service. You can also enable logging to follow the log output of the running services. Following the log output creates an open session that continuously streams the logs to stdout.

The APM logging functions mask the underlying complexities of the kubectl log command that is collecting the log information. You can still use the kubectl log command, but that is outside the scope of this documentation.

Note:

You can use third-party applications to capture and redirect the stdout stream for the container. Refer to your third-party documentation for assistance. You can also configure Docker with different logging drivers to redirect stdout. Refer to your Docker documentation for assistance.

To display APM logs for all services:

Best Practice:

Use the apm logs ––services option only when you are troubleshooting under the guidance of a Juniper Networks support representative.

To follow the logs for all services, specify the follow option (-f):

To view logs from previous instances of the containers in a Kubernetes pod, specify the previous container option (-p):

You can use standard Ubuntu conventions to redirect the logs to a file or to the terminal and to a file. Refer to the Ubuntu documentation for more information, but you can use the following examples as a starting point:

  • Redirect all logs to only a file.

  • Redirect all logs to both the screen and to a file.

Best Practice:

Use the ––services option only for troubleshooting under the guidance of a Juniper Networks support representative.

You can specify any of the following severity levels, in order of increasing severity:

Table 4: Severity Level
Severity Level Description
debug Detailed information that is typically of interest only when you are trying to diagnose a problem. These logs are often very frequent.
info Events or non-error conditions of interest. Logs at this level provide confirmation that everything is working as expected. These logs are generally not very frequent.
warning Indicates that something unexpected happened or that some problem might occur in the near future. A simple example of the latter is the disk space low warning that indicates that you might run out of disk space soon. In either case, the software is still working as expected, but you might want to monitor it more closely. These logs are generally not very frequent.
error Indicates that a more serious problem has prevented the software from performing some function, but the software has handled the problem as gracefully as possible to continue functioning.
critical A serious error that indicates that the program itself might be unable to continue running.

You can use the --logset option to display logs either for only APM services or for only prebuilt services. If you do not use this option, then only the APM services logs are displayed.

To display logs for only APM services:

To display logs for prebuilt services:

Prebuilt services are services borrowed from other sources to provide infrastructure functions for APM. These sources include MGMT, redis, and redis-sentinel. For example, redis provides database and messaging services, MGMT provides a configuration/CLI service, and so on.

Display Logs Using the Kubernetes Command Line Tool

To display all the logs of events that occur while APM is running, run the following command:

To display logs for a specific microservice, replace the label selector (-l jnpr/logset=jnpr-apm) with the pod name. For example:

Note:

To determine the pod name, you can use the kubectl get pods -n jnpr-apm Kubernetes command (see Check the Status of APM Services Using the Kubernetes Command Line Tool).

Determine the APM Version Using the APM Utility

Use the apm version [--context <context name>] [-o|--output json] [--detail| --compare <software-version>] utility script to determine the version number of the installed APM release.

To display the release version:

To compare the specified software release versions against the current deployed release for the specified context:

Use the -j option to render the version information in JavaScript Object Notation (JSON) format.

Archive the APM Configuration Using the Kubernetes Command Line Tool

To archive a copy of the currently running APM configuration, enter the following command:

Uninstall and Remove APM Using the APM Utility

Use the apm utility script to uninstall the APM configuration. The uninstall command reverts the actions you performed when setting up APM. Use this command to return APM to the state it was in immediately after you installed the application but before you did any setup configuration.

To uninstall APM:

After you uninstall APM, we recommend that you use the Debian uninstall procedure to remove the entire package.

For the apm clean command options, see the following:

  • -h or --help—Show the help message and exit.

  • --log or -l {error | warn | info | debug}—Adjust the log level of the utility scrip.

  • --no-color—Print messages without colors.

  • --docker—Clean the local docker cache.

  • --release release-number—The release to clean (defaults to unused releases).

    --dry-run—List releases or containers that will be removed.

    --uninstall—Uninstall all software releases and remove APM from the system.

    --cluster-repos—Clean the cluster repos for the clusters that have been removed.

Uninstall and Remove APM Without Using the APM Utility

This is the uninstall procedure that you use if you installed APM without using the APM utility. You use the helm uninstall command to uninstall your APM configuration. To completely remove APM, you must run the helm uninstall command for each microservice.

To uninstall APM, run the following:

How to Access APM Configuration and Operational Commands Using the APM Utility

Access the APM CLI Using the APM Utility

To access the CLI prompt, enter the following apm utility script command:

Enter a question mark to see the available top-level CLI commands. This list of commands is a subset of the Junos OS top-level commands.

For an overview of Junos OS CLI basics, see Day One: Exploring the Junos CLI. For more detailed information, see the CLI User Guide.

Access and Use CLI Configuration Statements Using the APM Utility

  1. Use the APM utility command apm cli to access the top-level CLI prompt.
  2. Access configuration mode to configure APM and the information that APM uses to configure a managed router.
  3. Enter CLI statements to configure the APM-managed BNGs, pool domains, pools, and system attributes.
  4. Save and activate the configuration. This command succeeds only when there are no configuration syntax errors.
  5. (Optional) Exit configuration mode and return to the top-level CLI prompt.

Access and Use CLI Operational Commands Using the APM Utility

To monitor APM, view APM configuration and statistics, or run certain operations manually:

  1. Use the APM utility command apm cli to access the top-level CLI prompt.
  2. Enter specific commands.

    • Use show commands to display statistics and the relationships between partitions, BNGs, pool domains, and pools.

    • Use request commands to manually initiate certain APM operations.

How to Use the APM Command Line Tool Without Using the APM Utility

This section describes how you use the Kubernetes Command Line tool commands to perform administration functions.

You can use the Kubernetes Command Line tool to administer the application and to access the CLI that you use to configure the address management functions.

You can use the Kubernetes Command Line tool to do the following:

  • Create and delete objects.
  • Provide log access.
  • Conduct interactive sessions with pod containers.
  • Display the status of the APM objects.

Access the APM Command Line Tool Without Using the APM Utility

To access the APM Command Line tool using the Kubernetes commands, enter the following:

Enter a question mark to see the available top-level CLI commands.