Use Junos PyEZ Tables with TextFSM Templates
SUMMARY Junos PyEZ op Tables can reference a TextFSM template, by itself or in conjunction with a Junos PyEZ View, to parse CLI or VTY command output from any network device.
Understanding TextFSM Templates
Junos PyEZ op Tables can extract data from CLI or VTY command output. The Table can reference a View to map fields in the command output to Python objects. Starting in Junos PyEZ Release 2.4.0, Junos PyEZ op Tables can also reference a TextFSM template, by itself or in conjunction with a View, to parse the command output. Junos PyEZ op Tables can use TextFSM templates to parse command output from any network device, regardless of the vendor, network operating system, or command.
TextFSM
is a Python library that parses semi-formatted CLI
output, such as show command output, from network devices. It was
developed by Google and later released under the Apache 2.0 licence.
The module requires a template and some input text. The template uses
regular expressions to describe how to parse the data, and you can
define and apply multiple templates to the same data.
TextFSM
’s CliTable
class enables users to map a command on a given platform to the
template that parses the command output. Network to Code, a network
automation company, has developed a Python wrapper for CliTable
along with a repository of TextFSM templates
for network devices. You can install the ntc-templates
library on your Junos PyEZ server or virtual environment,
as appropriate, and then reference the NTC templates and other TextFSM
templates in your Junos PyEZ Tables.
The NTC templates parse show command output from network devices. Each NTC template defines the expected output fields for a given command, and for each item, maps the data to a header. The NTC template filename identifies the vendor, network operating system, and command (with underscores), so the system can easily determine which template to use for a given platform and command.
vendor_os_command.textfsm
For example, consider the juniper_junos_show_arp_no-resolve.textfsm template.
Value Required MAC ([A-Fa-f0-9\:]{17}) Value Required IP_ADDRESS ([A-Fa-f0-9:\.]+) Value Required INTERFACE (\S+) Value FLAGS (\S+) Start ^MAC\s+Address\s+Address\s+Interface\s+Flags\s*$$ ^${MAC}\s+${IP_ADDRESS}\s+${INTERFACE}\s+${FLAGS} -> Record ^Total.* ^\s*$$ ^{master:\d+} ^. -> Error
The template parses the show arp no-resolve
command output from Juniper Networks
Junos devices.
user@host> show arp no-resolve MAC Address Address Interface Flags 02:01:00:00:00:05 10.0.0.5 em0.0 none 30:7c:5e:48:4b:40 198.51.100.77 fxp0.0 none f8:c0:01:18:8b:67 198.51.100.254 fxp0.0 none 02:00:00:00:00:10 128.0.0.50 em0.0 none Total entries: 4
Junos PyEZ op Tables can use an NTC template or other TextFSM
template to parse unstructured command output. The Table uses a TextFSM
template by defining the following fields. Junos PyEZ uses the platform
and command
values
to determine the template’s filename.
command: command
—Command that generates the output to parse. The command must map to the command string in the filename of an NTC template or other TextFSM template.key: key
—Field defined in the TextFSM template or Junos PyEZ View that is used to uniquely identify the record item.platform: platform
—Vendor and operating system for the TextFSM template, for example,juniper_junos
. The platform value must match the platform string in the filename of an NTC template or other TextFSM template.use_textfsm: True
—Indicate that the Junos PyEZ Table should parse the command output by using the TextFSM template for the given platform and command.
How to Use TextFSM Templates to Parse Command Output
Junos PyEZ Tables can use TextFSM templates, including the predefined NTC templates, to parse show command output from Junos devices.
To use TextFSM templates in a Junos PyEZ Table:
Install the
ntc-templates
library on your Junos PyEZ server or virtual environment.user@host:~$ pip3 install ntc_templates
Create a custom Junos PyEZ Table that includes the
command
,key
,platform
, anduse_textfsm
arguments, as well as any additional arguments required for your operations.--- ArpTableTextFSM: command: show arp no-resolve platform: juniper_junos key: MAC use_textfsm: True
The Junos PyEZ application uses the
platform
andcommand
values to determine the template’s filename, which in this case is juniper_junos_show_arp_no-resolve.textfsm.Create a Junos PyEZ application that uses the Table to retrieve the data.
from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader from pprint import pprint import json import yaml import yamlordereddictloader yaml_table = """ --- ArpTableTextFSM: command: show arp no-resolve platform: juniper_junos key: MAC use_textfsm: True """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=yamlordereddictloader.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM(dev).get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python3 junos-pyez-arptable-textfsm.py {'02:00:00:00:00:10': {'FLAGS': 'none', 'INTERFACE': 'em0.0', 'IP_ADDRESS': '128.0.0.50'}, '02:01:00:00:00:05': {'FLAGS': 'none', 'INTERFACE': 'em0.0', 'IP_ADDRESS': '10.0.0.5'}, '30:7c:5e:48:4b:40': {'FLAGS': 'none', 'INTERFACE': 'fxp0.0', 'IP_ADDRESS': '198.51.100.77'}, 'f8:c0:01:18:8b:67': {'FLAGS': 'none', 'INTERFACE': 'fxp0.0', 'IP_ADDRESS': '198.51.100.254'}}
The Table uses the NTC template to extract the output fields. For each Table item, the application returns the defined key and the data for each field.
How to Use TextFSM Templates with Junos PyEZ Views to Parse Command Output
Junos PyEZ Tables can combine a TextFSM template and a Junos PyEZ View to parse command output. The TextFSM template maps the data to a header. In the View, you can map your variable names to the headings defined in the template for the fields you want to return. This is useful, for example, when you want to use different variable names than the ones defined in the template or when you want to return different fields. Junos PyEZ only returns the fields that are common to both the TextFSM template and the Junos PyEZ View.
The following example uses the juniper_junos_show_arp_no-resolve.textfsm template to parse the command output. The Junos PyEZ View maps the data to new variable names and only returns a subset of the fields. To review the template, see Understanding TextFSM Templates.
To use a TextFSM template and a View in a Junos PyEZ Table:
Create a custom Junos PyEZ Table that includes the
command
,key
,platform
,use_textfsm
, andview
arguments, as well as any additional arguments required for your operations.--- ArpTableTextFSM2: command: show arp no-resolve platform: juniper_junos key: - ip - mac use_textfsm: True view: ArpViewTextFSM2
Create the Junos PyEZ View that defines which template fields to return and the corresponding variable name for each field.
ArpViewTextFSM2: fields: mac: MAC ip: IP_ADDRESS interface: INTERFACE
In this case, the View does not map the
FLAGS
field defined in the TextFSM template, and the parsed data does not include this value.Create a Junos PyEZ application that uses the Table to retrieve the data.
from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader from pprint import pprint import json import yaml import yamlordereddictloader yaml_table = """ --- ArpTableTextFSM2: command: show arp no-resolve platform: juniper_junos key: - ip - mac use_textfsm: True view: ArpViewTextFSM2 ArpViewTextFSM2: fields: mac: MAC ip: IP_ADDRESS interface: INTERFACE """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=yamlordereddictloader.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM2(dev).get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python3 junos-pyez-arptable-textfsm2.py {"('10.0.0.5', '02:01:00:00:00:05')": {'interface': 'em0.0', 'ip': '10.0.0.5', 'mac': '02:01:00:00:00:05'}, "('128.0.0.50', '02:00:00:00:00:10')": {'interface': 'em0.0', 'ip': '128.0.0.50', 'mac': '02:00:00:00:00:10'}, "('198.51.100.254', 'f8:c0:01:18:8b:67')": {'interface': 'fxp0.0', 'ip': '198.51.100.254', 'mac': 'f8:c0:01:18:8b:67'}, "('198.51.100.77', '30:7c:5e:48:4b:40')": {'interface': 'fxp0.0', 'ip': '198.51.100.77', 'mac': '30:7c:5e:48:4b:40'}}
The Table uses the NTC template and View to extract the output fields. For each Table item, the application returns the defined key and the data for the fields mapped to the variable names defined in the View.
How to Use Custom TextFSM Templates
Junos PyEZ Tables can use the TextFSM templates that are installed
as part of the ntc-templates
package, or
they can reference custom TextFSM templates. To use custom TextFSM
templates in your Junos PyEZ application, you must stage the template
and then specify the absolute path to the template directory when
you define the Table instance.
To use a custom TextFSM template in a Junos PyEZ Table:
Create a directory for your custom templates.
user@host:~$ mkdir TextFSMTemplates
In the templates directory, create your template and name the file using the platform_command.textfsm filename convention.
user@host:~$ vi TextFSMTemplates/my_platform_show_arp_no-resolve.textfsm
Create a Junos PyEZ Table that defines the same
platform
andcommand
values as the template filename.--- ArpTableTextFSM3: command: show arp no-resolve platform: my_platform key: - MAC - IP_ADDRESS use_textfsm: True
In your Junos PyEZ application, specify the absolute path to the custom templates directory when you define the Table instance.
from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader from pprint import pprint import json import yaml import yamlordereddictloader yaml_table = """ --- ArpTableTextFSM3: command: show arp no-resolve platform: my_platform key: - MAC - IP_ADDRESS use_textfsm: True """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=yamlordereddictloader.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM3(dev, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python3 junos-pyez-arptable-textfsm3.py
How to Use Junos PyEZ Tables with TextFSM Templates to Parse Any Vendor’s Command Output
Junos PyEZ Tables can use TextFSM templates to parse command output from any vendor’s network
device. You can retrieve the output in your Python application or read the output from a
file. Then, when you create the Junos PyEZ Table instance, you can pass the command output
string to the Table’s raw
argument instead of passing in a
Device
instance.
For example, consider the following TextFSM template:
user@host:~$ cat TextFSMTemplates/cisco_xr_show_alarms_detail.textfsm Value Required description (.+?) Value Required location (\S+) Value aid (\S+) Value tag (\S+) Value module (\S+) Start # Match the timestamp at beginning of command output ^Active Alarms (Detail) for .+ ^-+ -> Continue.Record ^Description:\s+${description}\s*$$ ^Location:\s+${location} ^AID:\s+${aid} ^Tag String:\s+${tag} ^Module Name:\s+${module}
The template parses the show alarms detail
command output from the given device.
RP/0/RP0/CPU0:host#show alarms detail Wed May 5 12:17:00.187 UTC -------------------------------------------------------------------------------- Active Alarms (Detail) for 0/RP0 -------------------------------------------------------------------------------- Description: hw_optics: RX LOS LANE-0 ALARM Location: 0/RP0/CPU0 AID: XR/HW_OPTICS/5 Tag String: DEV_SFP_OPTICS_PORT_RX_LOS_LANE0 Module Name: Optics0/0/0/31 EID: CHASSIS/LCC/1:CONTAINER/CC/1:PORT/OPTICS/31 Reporting Agent ID: 196713 Pending Sync: false Severity: Major Status: Set Group: Software Set Time: 04/27/2021 09:47:16 UTC Clear Time: - Service Affecting: NotServiceAffecting Transport Direction: NotSpecified Transport Source: NotSpecified Interface: N/A Alarm Name: OPTICS RX LOS LANE-0 -------------------------------------------------------------------------------- Description: hw_optics: RX LOS LANE-1 ALARM Location: 0/RP0/CPU0 AID: XR/HW_OPTICS/6 Tag String: DEV_SFP_OPTICS_PORT_RX_LOS_LANE1 Module Name: Optics0/0/0/31 EID: CHASSIS/LCC/1:CONTAINER/CC/1:PORT/OPTICS/31 Reporting Agent ID: 196713 Pending Sync: false Severity: Major Status: Set Group: Software Set Time: 04/27/2021 09:47:16 UTC Clear Time: - Service Affecting: NotServiceAffecting Transport Direction: NotSpecified Transport Source: NotSpecified Interface: N/A Alarm Name: OPTICS RX LOS LANE-1 ...
The following example defines a Junos PyEZ Table that uses the
custom TextFSM template, cisco_xr_show_alarms_detail.textfsm, in conjunction with a View to parse the show alarms
detail
command output. The example uses the netmiko
library to retrieve the data directly from
the device. When the application creates the Table instance, the raw
argument passes in the command output, and the template_dir
argument defines the path to the directory
containing the custom template.
from netmiko import ConnectHandler from jnpr.junos.factory.factory_loader import FactoryLoader from pprint import pprint import json import yaml import yamlordereddictloader yaml_table = """ --- XRAlarmsTable: command: show alarms detail key: - description - location platform: cisco_xr use_textfsm: True view: XRAlarmsView XRAlarmsView: fields: description: description location: location aid: aid tag: tag module: module severity: severity status: status group: group time: time affect: affect """ dev_credentials = { 'device_type': 'cisco_xr', 'host': '198.51.100.101', 'username': 'admin', 'password': 'password', } net_connect = ConnectHandler(**dev_credentials) output = net_connect.send_command('show alarms detail') globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=yamlordereddictloader.Loader))) stats = XRAlarmsTable(raw=output, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(stats.to_json())) with open('show-alarms-detail.txt', 'w') as fp: fp.write(output)
When you execute the application, it retrieves the command output from the device and uses the TextFSM template in the specified directory along with the View to parse the output. Junos PyEZ only returns the fields that are common to both the TextFSM template and the Junos PyEZ View. The application also saves the command output to a file, so the output can be processed later, as shown in the next example.
user@host:~$ python3 junos-pyez-textfsm-alarms.py {"('hw_optics: RX LOL LANE-0 ALARM', '0/RP0/CPU0')": {'aid': 'XR/HW_OPTICS/29', 'description': 'hw_optics: ' 'RX LOL ' 'LANE-0 ' 'ALARM', 'location': '0/RP0/CPU0', 'module': 'Optics0/0/0/31', 'tag': 'DEV_SFP_OPTICS_PORT_RX_CDR_LOL_LANE0'}, "('hw_optics: RX LOL LANE-1 ALARM', '0/RP0/CPU0')": {'aid': 'XR/HW_OPTICS/30', 'description': 'hw_optics: ' 'RX LOL ' 'LANE-1 ' 'ALARM', 'location': '0/RP0/CPU0', 'module': 'Optics0/0/0/31', 'tag': 'DEV_SFP_OPTICS_PORT_RX_CDR_LOL_LANE1'}, ...
The following example uses the same TextFSM template and Junos PyEZ View as the previous example, but in this case, the command output is read from a file.
from jnpr.junos.factory.factory_loader import FactoryLoader from pprint import pprint import json import yaml import yamlordereddictloader yaml_table = """ --- XRAlarmsTable: command: show alarms detail key: - description - location platform: cisco_xr use_textfsm: True view: XRAlarmsView XRAlarmsView: fields: description: description location: location aid: aid tag: tag module: module severity: severity status: status group: group time: time affect: affect """ with open('show-alarms-detail.txt') as fp: output = fp.read() globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=yamlordereddictloader.Loader))) stats = XRAlarmsTable(raw=output, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(stats.to_json()))