Junos Space Debug Utilities Overview
Junos Space debug utilities allow you to debug issues related to Junos Space nodes and devices managed by Junos Space Network Management Platform and view details about jobs scheduled on Junos Space Network Management Platform. Junos Space debug utilities are a collection of scripts and Java applications stored at /var/log/space-debug/debug-utilities. These scripts and Java applications are organized under the following categories: deviceConnection, jobManagement, deviceImport, and HornetQ. You can save the output of the scripts at a custom location. By default, the output of the scripts is stored at the location where the scripts are stored.
The following scripts and Java applications are available for debugging:
Device-Connection Debug Scripts
The device-connection debug scripts stored at /var/log/space-debug/debug-utilities/deviceConnection/ fetch and display device-connection information from DeviceDataMatrix. DeviceDataMatrix is a memory data structure in the Junos Space Network Management Platform database that stores device-connection information. You can also view this information through JConsole or JMXTerm.
The following are the device-connection debug scripts:
getDeviceInfo.sh
getDeviceInfo.sh is a script to collect device-connection
information for a single device. The script output displays the device
ID (as stored in the Junos Space Platform database), IP address of
the device, IP address of the Junos Space node to which the device
is currently connected, status of the edit flag on the device, SSH
control channel number, number of channels opened from the device,
and details of the open channels.
DeviceDebugInfoCollector.sh
DeviceDebugInfoCollector.sh is a script to execute
frequently used Junos OS debug commands on a device. When you execute
this script, SSH connections are initiated to the device from the
Junos Space node you specified. The script output displays the list
of active management daemon (MGD) processes on the device, active
SSH daemon (SSHD) processes on the device, active SSH connections
to Junos Space Platform from the device, and all active SSH connections
from the device. You can also view additional details about each of
these processes and SSH connections.
getAllDeviceInfo.sh
getAllDeviceInfo.sh is a script to collect device-connection
information about all devices that are connected to a Junos Space
node. The script output displays the device ID (as stored in the Junos
Space Platform database), IP address of the device, IP address of
the Junos Space node to which the device is currently connected, status
of the edit flag on the device, SSH control channel number, number
of channels opened from the device, and details of the open channels
about all devices that are connected to a Junos Space node. On a multinode
setup, you can also collect this information for all Junos Space nodes.
cleanupEditChannel.sh
cleanupEditChannel.sh is a script to unlock the device
configuration on the device. Junos Space Platform sets a lock when
you deploy a configuration from Junos Space Platform or Junos Space
applications. You use this script to unlock the device configuration
if the previous deployments were erroneous and you are currently unable
to deploy the configuration from Junos Space Platform. You enter the
variable false to unlock the device configuration.
For more information about executing device-connection debug scripts, see Executing Device-Connection Debug Scripts.
Device Import Scripts and Java Applications
The device import scripts and Java applications stored at /var/log/space-debug/debug-utilities/deviceImport/ clear the device import tables and fetch device inventory information or device configuration in XML format.
The following are the device import scripts and Java applications:
cleanupDeviceImportTables.sh
cleanupDeviceImportTables.sh is a script to clean
data from device import tables. You can execute the script to fix
data errors during a device resynchronization process. You need to
manually resynchronize the device with the Junos Space Platform database
from the user interface after you execute the script.
DB-blob-reader.jar
DB-blob-reader.jar is a Java application to collect the device information XML or interface information XML. When you execute this application, the information from the XML is written to the DB-blob-reader-result.txt file. This information can be useful for debugging device resynchronization issues. You can modify the MySQL query in the DB-blob-reader.properties file and fetch information based on that MySQL query. You can specify the following in the DB-blob-reader.properties file: device ID (as stored in the Junos Space Platform database) and name of the RPC, device configuration, or interface.
For more information about executing device import scripts and Java applications, see Executing Device Import Detail Script and Java Application.
Job Management Scripts and Java Applications
The job management scripts and Java applications stored at /var/log/space-debug/debug-utilities/jobManagement/ fetch information about jobs executed from the Junos Space nodes. You can also view the output of the scripts through JConsole or JMXTerm.
The following are the job management scripts and Java applications:
SystemLoadViewer.sh
SystemLoadViewer.sh is a script to collect information
about available memory on all Junos Space nodes and the jobs triggered
on these nodes. The script output displays information such as the
memory on the nodes, number of root jobs and subjobs on each of the
nodes, type of job (root job or subjob), state of the job (running,
queued, or stopped), name of the job, queue name of the job, the time
the job was created, and the time the job was modified. The script
output also displays the top five processes that consume CPU and memory
when the script is executed.
getJobThreadSump.sh
getJobThreadSump.sh is a script to view the stack
trace of a specific job. You can also view the script output through
JConsole or JMXTerm.
JobInfoCollector.jar
JobInfoCollector.jar is a Java application to execute SQL queries and collect information about jobs. You can construct the SQL query in the JobInfoCollector.properties file. This file contains a default example query. The application can also display the hierarchy of a subjob (input as the parent job ID) and list of jobs that are currently unscheduled. You can also input a SQL query to obtain information about jobs.
For more information about executing job management scripts and Java applications, see Executing Job Management Scripts and Java Applications.
Usr/nma/bin/collectStuckJobLogFiles.pl
Usr/nma/bin/collectStuckJobLogFiles.pl is a script to collect all the troubleshooting logs and threats at the time of a job getting stuck. This Auto Gathering tool monitors and identifies the stuck job once added to crontab as required. Stuck jobs are the ones that are in pending or under progress for more than forty five minutes. Once the tool identifies such jobs, it collects all the logs and thread dump from the server, saves them in /var/tmp/stuckJobLogFiles_<timestamp>.tgz location, notifies the user via e-mail with details such as file name, file location, node, and so on.
HornetQ Scripts
The HornetQ scripts stored at /var/log/space-debug/debug-utilities/hornetQ/ display the list of all JBoss queues, of messages in a specific JBoss queue, or of jobs that are to be executed by a specific JBoss queue. You can also view the script output through JConsole or JMXTerm.
The following are the HornetQ scripts:
HornetQInfoProvider.sh
HornetQInfoProvider.sh is a script to collect details
about all HornetQ queues. The script output also lists details such
as consumer-count, message-count, and scheduled-count.
HQMessageViewer.sh
HQMessageViewer.sh is a script to view the list of
messages in a specific JBoss queue. The script output displays the
job ID and job operation name. You can view the jobs that are queued
to be executed by a specific JBoss queue.
For more information about executing HornetQ scripts, see Executing HornetQ Scripts.