Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Define Junos PyEZ Operational Tables for Parsing Unstructured Output

SUMMARY Create custom Tables that select data from CLI or vty command output to extract operational information from a Junos device.

Junos PyEZ operational (op) Tables for unstructured output extract data from the text output of a CLI command executed on a Junos device or a vty command executed on a given Flexible PIC Concentrator (FPC). The extracted data is then converted to JSON. This enables you to quickly retrieve and analyze operational state information for the device. Junos PyEZ op Tables for unstructured output are particularly useful when you need to parse command output that cannot be returned in a structured format such as XML.

This topic discusses the different components of the Table.

Summary of Parameters in Op Tables for Parsing Unstructured Output

Junos PyEZ Tables are formatted using YAML. Op Table definitions can include a number of required and optional parameters, which are summarized in Table 1.

Table 1: Parameters in Junos PyEZ Op Tables for Unstructured Output

Table Parameter Name

Table Parameter

Description

Table name

User-defined identifier for the Table.

Command

command

CLI or vty command to execute.

Command arguments

args

(Optional) When you define the command as a Jinja template, args is an associative array, or dictionary, of key-value pairs that map the variables in the command template to the default values used when the template is rendered.

Target FPC

target

(Optional) Flexible PIC Concentrator (FPC) on which to execute a vty command.

Table item

item

(Optional) String or regular expression that defines how to split the output into sections. These sections become the iterable reference for the associated View.

Specify '*' to extract and match against the whole string rather than each line.

Table item key

key

(Optional) String or list of strings that define one or more keys that uniquely identify each Table item.

Selected keys

key_items

(Optional) List of one or more Table item keys for which to return data.

Section title

title

(Optional) String that selects the section of output containing the data to parse.

Field delimiter

delimiter

(Optional) Delimiter that defines how to split the data in command output comprised of key-value pairs. The extracted data is stored as key-value pairs in a dictionary.

Eval expression

eval

(Optional) Associative array, or dictionary, of one or more key-value pairs that map a user-defined key to a string containing a mathematical expression. When you retrieve the data, the expression is evaluated using the Python eval function. The key and the calculated value are added to the final data returned by the Table and View.

Table View

view

(Optional) View that is used to extract field data from the Table items.

TextFSM template

use_textfsm

(Optional) Boolean that specifies whether a TextFSM template is used to parse the data.

TextFSM template platform identifier

platform

(Optional) When a TextFSM template is used, specify the platform for the template.

Table Name

The Table name is a user-defined identifier for the Table. The YAML file or string can contain one or more Tables. The start of the YAML document must be left justified. The following example defines a Table named ChassisFanTable:

Command

A Junos PyEZ op Table for unstructured output extracts data from the text output of a CLI or vty command. You must include the command property in the Table definition to specify the CLI command to execute on a device or the vty command to execute on a given FPC. You can define the command as a simple string or a Jinja template.

For example, the following Table executes the show chassis fan command on the device.

The following Table executes the show cmerror module brief vty command on the target FPC.

When you define the command as a Jinja template, you must also supply the args parameter with a dictionary of key-value pairs that map the variables in the template to the values used when the template is rendered. For information about defining the command as a Jinja template, see Command Arguments (args).

Command Arguments (args)

You can define the CLI or vty command for the command parameter using a Jinja template and substitute variables for the command arguments. When you use a Jinja template, you must also define the args parameter, which is a dictionary of key-value pairs that map the variable names in the Jinja template to the values used when the template is rendered. You can provide default values for the template variables in the Table definition, and you can also define the values in the Junos PyEZ application.

To define default values for the template variables, include the args parameter in the Table definition and map each template variable to its default value. The following Table defines a command using a Jinja template that has one variable, protocol. The args parameter defines the default value for protocol, which is used when you call the get() method in your script and do not provide an argument that overrides that default.

Additionally, you can define the args argument as a dictionary in the Table’s get() method to:

  • define a value for any template variable that does not have a default value defined in the Table

  • override the default value defined in the Table for one or more template variables

The following example executes the command in the previous Table using protocol 'bgp' instead of the default value of 'ospf'.

Target FPC (fpc)

Junos PyEZ op Tables can execute vty commands on a specific Flexible PIC Concentrator (FPC). When you use a vty command, the Table must include the target parameter to define the target FPC. You can set target to Null and force the user to specify the target FPC in the application, or you can set target to a default FPC, and the user can optionally override this value in the application.

The following Table executes the show memory vty command, but sets target: Null, which requires that the user supply the target FPC in the Junos PyEZ application:

The following Table executes the show memory vty command on FPC 1, unless the user overrides this value in the Junos PyEZ application.

In the Junos PyEZ application, to define the target FPC or override the default FPC defined in the Table, set the target argument in the Table’s get() method to the desired FPC, for example:

Table Item (item)

The optional Table item property is a string or regular expression that defines how to split the command output for parsing. If the output has similar, repeating sets of data, you can define item to match on and extract each iteration of the data. For example, show interfaces returns a similar set of data for many interfaces. Alternatively, you can define item: '*' when you need to extract the data as a single block of text.

Consider the following output for the show devices local vty command:

The following Table extracts each section of the output that starts with TSEC Ethernet Device Driver:. In this case, the value for key: name is derived from the capturing group in the regular expression defined in item.

You can also define item as an asterisk ('*') if you want to match against the entire section of output instead of matching each line. When you include item: '*', in most cases, you must also include the title parameter to specify the heading for the section of output to extract. Extracting the output using item: '*' is useful when you include the regex parameter in a View and want each expression to match against the entire text string. Otherwise, the regex expressions are combined and matched against each line.

The following Table extracts the text block under the heading Receive: and matches each regular expression against the entire text string:

Table Item Key (key)

The optional key property defines the output fields that are used to uniquely identify a Table item. You can identify a Table item using a single key or a list of keys. If the Table and View return iterative data, the key value must reference a variable or field name defined in the View.

Consider the following show chassis fan output:

The following Table defines the Table item key as fan-name, which maps to the value under the Item column in the output.

When you retrieve and print the data in the Junos PyEZ application, each item in the resulting dictionary uses this field’s value as its key.

You can also define key as a list to identify a Table item using a composite key. For example:

Selected Keys (key_items)

The key parameter defines the output fields that uniquely identify a Table item. When you include the key parameter in a Table, you can use the optional key_items parameter to only return data for certain key values. key_items defines the key or list of keys of the Table items for which to retrieve data. You can define the key_items parameter in the Table definition or in the Junos PyEZ application.

Consider the following show chassis fan output:

The following Table defines the Table item key as fan-name and only retrieves the data for the Table item with a key value equal to Fan 1.

In the Junos PyEZ application, to define the key_items to return or to override the key_items defined in the Table, set the key_items argument in the Table’s get() method to a list or tuple of the desired items, for example:

Section Title (title)

Tables can include the optional title parameter to define the starting point for a section in the command output from which to extract and parse the data. When the Table defines item: '*', you must include title to specify the heading for the section of output to extract

For example, consider the following command output, which is enclosed within a larger set of output:

The following Table uses the title parameter to extract and parse data from the TSEC status counters section of the output. In this case, the Table defines item as '*', which considers the data as a single text string.

Field Delimiter (delimiter)

Some operational commands return output comprised of only key-value pairs. If you want to simply retrieve the entire set of data and extract each key-value pair, you can use the delimiter parameter to define how to split each data pair instead of defining a separate View. Junos PyEZ uses the delimiter to split the data at the specified boundary and stores each key-value pair in a dictionary.

Consider the following output for the show link stats vty command.

The following Table defines the delimiter as the colon (:) character:

When you retrieve the data in the Junos PyEZ application, the Table splits each line of output at the delimiter and stores the key-value pairs in a dictionary.

Eval Expression (eval)

You can use the optional eval parameter to add or modify key-value pairs in the final data returned by the Table and View. The eval parameter maps a key name to a string containing a mathematical expression that gets evaluated by the Python eval function. You can include the eval parameter in both Tables and Views, and eval can define and calculate multiple values.

When you use eval in a Table, it references the full data dictionary for the calculation, and the key and calculated value are added as a single additional item to the dictionary. When you use eval in a View, the expression is calculated on each iteration of the data, and the calculated value is added to the data for that iteration. If the eval key name matches a key defined in the View, eval replaces the value of that key with the calculated value. If the eval key name does not match a key defined in the View, eval adds the new key and calculated value to the data.

The eval expression can reference the data dictionary returned by the View. The expression must reference data using a Jinja template variable, so that Junos PyEZ can replace the variable with the dictionary when the expression is evaluated.

The following example uses eval in the Table definition to include a single additional entry in the data dictionary. The added item’s key is cchip_errors_from_lkup_chip, and its value is the sum of the number of interrupts.

You can also define eval in the Table to calculate and add multiple key-values pairs, for example:

For information about using eval in a View, see Eval Expression (eval).

Table View (view)

The view property associates the Table definition with a particular View. A View defines how the Table output should be parsed and maps your user-defined Python variable names to output fields in the selected Table items. You can customize the View to only select certain fields from the Table items.

If the output consists of only key-value pairs, you can use the Table’s delimiter parameter to extract the data and store the key-value pairs in a dictionary. In this case, you do not need to define a separate View.

TextFSM Templates (platform and use_textfsm)

Junos PyEZ Tables can reference a TextFSM template to parse command output from Juniper Networks devices or other vendors’ devices. You must install the ntc-templates library on your Junos PyEZ server or virtual environment in order to use TextFSM templates in your Tables.

To use a TextFSM template to parse the command output, include use_textfsm: True in the Table. You can use a TextFSM template by itself or in conjunction with a Junos PyEZ View. Junos PyEZ uses the platform and command values to determine the template’s filename.

For example, the following Table uses the juniper_junos_show_arp_no-resolve.textfsm template to parse command output from a Juniper Networks Junos device:

For detailed information about using Junos PyEZ Tables with TextFSM templates, see Use Junos PyEZ Tables with TextFSM Templates.