Understanding the Junos Telemetry Interface Export Format of Collected Data
The Junos telemetry interface supports two ways of exporting data in the protocol buffers (gpb) format:
Through UDP from so-called native sensors that export data close to the source, such as the line card or network processing unit (NPU). Juniper Networks defines the data model, which is open and extensible.
Through gRPC remote procedure calls (gRPC) that export data through the Routing Engine. The data model is defined by OpenConfig, which supports the use of vendor-neutral data models to configure and manage the network. OpenConfig for Junos OS supports the YANG data models. For platforms that are running a version of Junos OS based on an upgraded FreeBSD kernel only, you must install a separate package called Network Agent that functions as a gRPC server and terminates the RPC interfaces. For all other versions of Junos OS, the Network Agent functionality is embedded in the software. You must also install the OpenConfig for Junos OS module and the YANG models.
This section describes the format of data exported from native sensors using UDP. The data is encapsulated into a UDP header, which is in turn encapsulated in the IPv4 payload. This model of the Junos telemetry interface is based a distributed architecture, through which the data generated by configured sensors is exported directly from the data plane, bypassing the control plane, and thus conserving these resources to perform other necessary functions.
The Junos telemetry interface was introduced in Junos OS Release 15.1F3, on MX Series routers with interfaces configured on MPC1 through MPC6E, and on PTX Series routers with interfaces configured on FPC3. Starting in Junos OS Release 15.1F5, Junos telemetry interface is also supported on MPC7E, MPC8E, and MPC9E on MX Series routers.
Starting with Junos OS Release 16.1R3, FPC1, FPC2, and dual Routing Engines on PTX Series routers are also supported.
Starting with Junos OS Release 17.2R1, QFX10000 and QFX5200 switches are also supported. On QFX5200 switches, only gRPC streaming is supported.
Starting with Junos OS Release 17.3R1, Junos telemetry interface is supported on the Routing Control and Board (RCB) on PTX3000 routers, QFX5110 switches, and EX4600 and EX9200 switches.
Starting with Junos OS Release 17.4R1, MX2008 routers are supported.
Understanding the Sensor Data Encapsulation Format
A native sensor exports data close to the source using UDP. Various types of telemetry data, such as physical interface statistics, firewall filter counter statistics, or statistics for label-switched paths (LSPs) can be exported. A sensor starts to emit data as soon as it is enabled.
The sensor data is represented as a single structured protocol buffers message, named
TelemetryStream. The message, or .proto file,
shown below, includes several attributes that identify the data source, such as a
line card, a Packet Forwarding Engine, or a Routing Engine. The name of the
configured sensor is also included. For more information about how to configure
sensors, see https://www.juniper.net/documentation/us/en/software/junos/interfaces-telemetry/topics/task/junos-telemetry-interface-configuring.html. For a a list of supported native sensors, see
https://www.juniper.net/documentation/us/en/software/junos/cli-reference/topics/ref/statement/sensor-edit-services-analytics.html.
You must also download the .proto files for all the sensors
supported to a streaming server or collector. From a Web browser, navigate to the
All Junos Platforms software download URL on the Juniper Networks page: https://www.juniper.net/support/downloads/. After you select the name of
the Junos OS platform and the release number, go to the Tools
section and download the Junos telemetry interface Data Model
Files package. For more information about configuring a
streaming-server, see https://www.juniper.net/documentation/us/en/software/junos/cli-reference/topics/ref/statement/streaming-server-edit-services-analytics-junos-telemetry-interface.html.
Native Protocol Buffers (protobuf) Message Structure
The information streamed to the collector is sent using the top-level message
structure of TelemetryStream (telemetry_top.proto file). The
message contains the metadata of the sensor data being streamed (for example, sensor
path, the system from where data is being sent, the node from where data is sent,
and so on). The actual sensor data is sent as an extension to the top-level message.
A separate proto file is sent for the sensor data. The telemetry_top.proto
file and the sensor proto file are used to decode the sensor data at the
collector.
Structure of telemetry_top.proto File
message TelemetryStream {
required string system_id = 1
[(telemetry_options).is_key = true];
optional uint32 component_id = 2
[(telemetry_options).is_key = true];
optional string sensor_name = 4
[(telemetry_options).is_key = true];
….
optional EnterpriseSensors enterprise = 101;
}
message EnterpriseSensors {
extensions 1 to max;
}
extend EnterpriseSensors {
// re-use IANA assigned numbers
optional JuniperNetworksSensors juniperNetworks = 2636;
}
message JuniperNetworksSensors {
extensions 1 to max; → ends with the extension message
} This file ends with the extension message.
Structure of Sensor Proto File
This file begins with the extension message.
extend JuniperNetworksSensors { → begins with the extension message
optional Port jnpr_interface_ext = 3;
}
message Port {
repeated InterfaceInfos interface_stats = 1;
}
message InterfaceInfos {
required string if_name = 1
[(telemetry_options).is_key = true];
optional uint64 init_time = 2;
…..
}The TelemetryStream message also includes optional nested structures
that carry different types of data. The nested structures can also carry privately
defined sensor data such as EnterpriseSensors. See the example
below:
//
// This file defines the top level message used for all Juniper
// Telemetry packets encoded to the protocol buffer format.
// The top level message is TelemetryStream.
//
import "google/protobuf/descriptor.proto";
extend google.protobuf.FieldOptions {
optional TelemetryFieldOptions telemetry_options = 1024;
}
message TelemetryFieldOptions {
optional bool is_key = 1;
optional bool is_timestamp = 2;
optional bool is_counter = 3;
optional bool is_gauge = 4;
}
message TelemetryStream {
// router name or export IP address
required string system_id = 1 [(telemetry_options).is_key = true];
// line card / RE (slot number)
optional uint32 component_id = 2 [(telemetry_options).is_key = true];
// PFE (if applicable)
optional uint32 sub_component_id = 3 [(telemetry_options).is_key = true];
// configured sensor name
optional string sensor_name = 4 [(telemetry_options).is_key = true];
// sequence number, monotonically increasesing for each
// system_id, component_id, sub_component_id + sensor_name.
optional uint32 sequence_number = 5;
// timestamp (milliseconds since 00:00:00 UTC 1/1/1970)
optional uint64 timestamp = 6 [(telemetry_options).is_timestamp = true];
// major version
optional uint32 version_major = 7;
// minor version
optional uint32 version_minor = 8;
optional IETFSensors ietf = 100;
optional EnterpriseSensors enterprise = 101;
}
message IETFSensors {
extensions 1 to max;
}
message EnterpriseSensors {
extensions 1 to max;
}
extend EnterpriseSensors {
// re-use IANA assigned numbers
optional JuniperNetworksSensors juniperNetworks = 2636;
}
message JuniperNetworksSensors {
extensions 1 to max;
}
Individual companies, such as Juniper Networks, define and maintain the attributes generated by enterprise sensors. Each company is assigned a unique attribute identifier. The current convention is to use IANA-assigned enterprise MIB identifiers for each attribute. For Juniper Networks, this assigned identifier is 2636.
Recommended: To verify that a particular message type has been exported
and received, check for those attributes under
TelemetryStream.enterprise.juniperNetworks in the gpb
message.
See Table 1 for descriptions of each element collected by sensor data, including semantics and corresponding schema.
|
Element Type |
Description |
|---|---|
|
Counter |
An unsigned integer that increases monotonically. When it reaches its maximum value, it starts back at zero. |
|
Gauge |
An unsigned 32-bit or 64-bit integer that can increase or decrease in value. An example of the data represented by this element is the instantaneous value of a specific resource, such as queue depth or temperature. |
|
Rate |
Rate at which a base metric changes, such as a counter or a gauge. For this element type, units of measurement are defined explicitly (such as bits per second), as well the interval over which the rate is collected. |
|
Average |
The average of several samples of a base metric. For example, an average queue depth data element would be calculated by averaging several elements of the queue depth. For this element type, we strongly recommend defining the number of measurements used to compute the average, as well as the time interval between the measurements. Otherwise, you should define explicitly the means by which this average value is calculated. |
|
Peak |
Maximum value among several samples of a base metric. For example, a peak queue depth element would be calculated by comparing several measurements of the queue depth and selecting the maximum. For this data element type, we strongly recommend that you define the number of measurements used to compute the peak value, as well as the time interval between measurements. Otherwise, define explicitly how this peak value is defined. You must also know whether this value is never cleared and thus represents the overall maximum value over all time. |
Each data element type also includes element subsets. For example, the data
elements Counter and Gauge would include
subsets for rate, average, and
peak measurements.
Change History Table
Feature support is determined by the platform and release you are using. Use Feature Explorer to determine if a feature is supported on your platform.