Network Analytics

Analytics Feature Overview

You enable network analytics by configuring queue (microburst) monitoring and high-frequency traffic statistics monitoring.

Queue (microburst) monitoring:

You use microburst monitoring to look at traffic queue conditions on the network. A microburst occurrence indicates to the Packet Forwarding Engine that a user-specified queue depth or latency threshold is reached. The queue depth is the buffer (in bytes) containing the data, and latency is the time (in nanoseconds or microseconds) the data stays in the queue.

You can configure queue monitoring based on either queue depth or latency (but not both), and configure the frequency (polling interval) at which the Packet Forwarding Engine checks for microbursts and sends the data to the Routing Engine for processing. You may configure queue monitoring globally for all physical interfaces on the system, or for a specific interface on the switch. However, the specified queue monitoring interval applies either to all interfaces, or none; you cannot configure the interval for each interface.

High-frequency traffic statistics monitoring:

You use high-frequency traffic statistics monitoring to collect traffic statistics at specified polling intervals. Similar to the queue monitoring interval, the traffic monitoring interval applies either to all interfaces, or none; you cannot configure the interval for each interface.

Both traffic and queue monitoring are disabled by default. You must configure each type of monitoring using the CLI. In each case, the configuration for an interface always takes precedence over the global configuration.

The analyticsd daemon in the Routing Engine generates local log files containing queue and traffic statistics records. You can specify the log filename and size, and the number of log files. If you do not configure a filename, the data is not saved.

You can display the local log file or specify a server to receive the streaming data containing the queue and traffic statistics.

For each port, information for the last 10 records of traffic statistics and 100 records of queue statistics is cached. You may view this information by using the show analytics commands.

To store traceoptions data, you configure the traceoptions statement at the [edit services analytics] hierarchy level.

Network Analytics Enhancements Overview

Beginning in Junos OS Release 13.2X51-D15, the network analytics feature provides the following enhancements:

• Resources—Consist of interfaces and system. The interfaces resource allows you to configure an interface name and an associated resource profile name for each interface. With the system resource, you can configure the polling intervals for queue monitoring and traffic monitoring, and an associated resource profile for the system.

• Resource profile—A template that contains the configurations for queue and traffic monitoring, such as depth threshold and latency threshold values, and whether each type of monitoring is enabled or disabled. Once a resource profile is configured, you apply it to a system or interfaces resource.

• Collector—A server for collecting queue and traffic monitoring statistics, and can be a local or remote server. You can configure a local server to store monitoring statistics in a log file, or a remote server to receive streamed statistics data.

• Export profile—You must configure an export profile if you wish to send streaming data to a remote collector. In the export profile, you define the category of streamed data (system-wide or interface-specific) to determine stream type the collector will receive. You can specify both system and interface stream categories. System data includes system information and status of queue and traffic monitoring. Interface-specific data includes interface information, queue and traffic statistics, and link, queue, and traffic status.

• Google Protocol Buffer (GBP) stream format—A new streaming format for monitoring statistics data that is sent to a remote collector in a single AnRecord message. The format of this stream which provides nine types of information is shown in Table 1.

  Table 1: Google Protocol Buffer (GBP) stream format

  Message	Description
  System information	General system information, including boot time, model information, serial number, number of ports, and so on
  System queue status	Queue status for the system in general
  System traffic status	Traffic status for the system in general
  Interface information	Includes SNMP index, slot, port, and other information
  Queue statistics for interfaces	Queue statistics for specific interfaces
  Traffic statistics for interfaces	Traffic statistics for specific interfaces
  Link status for interfaces	Includes link speed, state, and so on
  Queue status for interfaces	Queue status for specific interfaces
  Traffic status for interfaces	Traffic status for specific interfaces

• The analytics.proto file—Provides a template for the GBP stream format. This file can be used for writing your analytics server application. To download the file, go to: /documentation/en_US/junos13.2/topics/reference/proto-files/analytics-proto.txt

• Use of threshold values—The Analytics Manager (analyticsm) will generate a queue statistics record when the lower queue depth or latency threshold value is exceeded.

• User Datagram Protocol (UDP)—Additional transport protocol you can configure, in addition to Transmission Control Protocol (TCP), for the remote streaming server port.

• Single file for local logging—Replaces the separate log files for queue and traffic statistics.

• Change in latency measurement—Configuration and reporting of latency values have changed from microseconds to nanoseconds.

• Change in reporting of the collection time in UTC format—Statistics collection time is reported in microseconds instead of milliseconds.

• New operational mode command show analytics collector—Replaces the show analytics streaming-server command.

• Changes in command output format—Include the following changes:

  • Addition of unicast, multicast, and broadcast packet counters in queue and traffic statistics.

  • Reversal of the sequence of statistics information in the output. The most recent record is displayed at the beginning, and the oldest record at the end of the output.

  • Removal of traffic or queue monitoring status information from the global portion of the show analytics configuration and show analytics status command output if there is no global configuration.

  • Addition of n/a to the interface-specific portion of the show analytics configuration and show analytics status command output if a parameter is not configured (for example, depth threshold or latency threshold).

Summary of CLI Changes

Starting in Junos OS Release 13.2X51-D15, enhancements to the network analytics feature result in changes in the CLI when you configure the feature. See Table 2 for a summary of CLI changes.

traffic-statistics {
    interval interval;
}
queue-statistics {
    interval interval;
}

resource {
    system {
        polling-interval {
            queue-monitoring interval;
            traffic-monitoring interval;
        }
    }
}

traffic-statistics {
    file filename;
    size size;
    files number;
}
queue-statistics {
    file filename;
    size size;
    files number;
}

collector {
    local {
        file filename {
            files number;
            size size;
        }
    }
}

interfaces {
    all {
        queue-statistics;
        traffic-statistics;
        depth-threshold {
            high number;
            low number;
        }
    }
}

interfaces {
    interface{
        queue-statistics;
        traffic-statistics;
        latency-threshold
        high number;
        low number;
    }
}

streaming-servers {
    address ip-address {
        port number {
            stream-format format;
        }
    }
}

streaming-servers {
    address ip-address {
        port number {
            stream-type type;
            stream-type type;
        }
    }
}

collector {
    address ip-address {
        port number1 {
            transport tcp;
            transport udp;
        }
        port number2 {
            transport udp;
        }
    }
}

JavaScript Object Notation (JSON)

The JavaScript Object Notation (JSON) streaming format supports the following data:

• Queue statistics data. For example:

  content_copy zoom_out_map

  {"record-type":"queue-stats","time":1383453988263,"router-id":"qfx5100-switch",
  "port":"xe-0/0/18","latency":0,"queue-depth":208}

  See Table 3 for more information about queue statistics output fields.

• Traffic statistics. For example:

  content_copy zoom_out_map

  {"record-type":"traffic-stats","time":1383453986763,"router-id":"qfx5100-switch",
  "port":"xe-0/0/16","rxpkt":26524223621,"rxpps":8399588,"rxbyte":3395100629632,
  "rxbps":423997832,"rxdrop":0,"rxerr":0,"txpkt":795746503,"txpps":0,"txbyte":101855533467,
  "txbps":0,"txdrop":0,"txerr":0}

  See Table 4 for more information about traffic statistics output fields.

Comma-separated Values (CSV)

The Comma-separated Values (CSV) streaming format supports the following data:

• Queue statistics. For example:

  content_copy zoom_out_map

  q,1383454067604,qfx5100-switch,xe-0/0/18,0,208

  See Table 3 for more information about queue statistics output fields.

• Traffic statistics. For example:

  content_copy zoom_out_map

  t,1383454072924,qfx5100-switch,xe-0/0/19,1274299748,82950,163110341556,85603312,0,0,
  27254178291,8300088,3488534810679,600002408,27268587050,3490379142400

  See Table 4 for more information about traffic statistics output fields.

Tab-separated Values (TSV)

The Tab-separated Values (TSV) streaming format supports the following data:

• Queue statistics. For example:

  content_copy zoom_out_map

  q       585870192561703872      qfx5100-switch       xe-0/0/18       (null)  
  208     2

  See Table 3 for more information about queue statistics output fields.

• Traffic statistics. For example:

  content_copy zoom_out_map

  t       1383454139025   qfx5100-switch  xe-0/0/19       1279874033      82022 
  163823850036    84801488        0       0       27811618258     8199630 
  3559887126455   919998736       27827356915     3561901685120

  See Table 4 for more information about traffic statistics output fields.

• Queue Statistics Output for JSON, CSV, and TSV
• Traffic Statistics Output for JSON, CSV, and TSV

Google Protocol Buffer (GPB)

This streaming format provides:

• Support for nine types of messages, based on resource type (system-wide or interface-specific).

• Sends messages in a hierarchical format.

• You can generate other stream format messages (JSON, CSV, TSV) from GPB formatted messages.

• Includes a 8-byte message header. See Table 5 for more information.

Table 5 describes the GPB stream format message header.

The following GPB prototype file (analytics.proto) provides details about the streamed data:

package analytics;

// Traffic statistics related info
message TrafficStatus {
    optional uint32             status          = 1;
    optional uint32             poll_interval   = 2;
}

// Queue statistics related info
message QueueStatus {
    optional uint32             status          = 1;
    optional uint32             poll_interval   = 2;
    optional uint64             lt_high         = 3;
    optional uint64             lt_low          = 4;
    optional uint64             dt_high         = 5;
    optional uint64             dt_low          = 6;
}

message LinkStatus {
    optional uint64             speed           = 1;
    optional uint32             duplex          = 2;
    optional uint32             mtu             = 3;
    optional bool               state           = 4;
    optional bool               auto_negotiation= 5;
}

message InterfaceInfo {
    optional uint32             snmp_index      = 1;
    optional uint32             index           = 2;
    optional uint32             slot            = 3;
    optional uint32             port            = 4;
    optional uint32             media_type      = 5;
    optional uint32             capability      = 6;
    optional uint32             porttype        = 7;
}

message InterfaceStatus {
    optional LinkStatus         link            = 1;
    optional QueueStatus        queue_status    = 2;
    optional TrafficStatus      traffic_status  = 3;
}

message QueueStats {
    optional uint64             timestamp       = 1;
    optional uint64             queue_depth     = 2;
    optional uint64             latency         = 3;
}

message TrafficStats {
    optional uint64             timestamp       = 1;
    optional uint64             rxpkt           = 2;
    optional uint64             rxucpkt         = 3;
    optional uint64             rxmcpkt         = 4;
    optional uint64             rxbcpkt         = 5;
    optional uint64             rxpps           = 6;
    optional uint64             rxbyte          = 7;
    optional uint64             rxbps           = 8;
    optional uint64             rxcrcerr        = 9;
    optional uint64             rxdroppkt       = 10;
    optional uint64             txpkt           = 11;
    optional uint64             txucpkt         = 12;
    optional uint64             txmcpkt         = 13;
    optional uint64             txbcpkt         = 14;
    optional uint64             txpps           = 15;
    optional uint64             txbyte          = 16;
    optional uint64             txbps           = 17;
    optional uint64             txcrcerr        = 18;
    optional uint64             txdroppkt       = 19;
}

message InterfaceStats {
    optional TrafficStats       traffic_stats   = 1;
    optional QueueStats         queue_stats     = 2;
}

//Interface message
message Interface {
    required string             name            = 1;                
    optional bool               deleted         = 2;
    optional InterfaceInfo      information     = 3;
    optional InterfaceStats     stats           = 4;
    optional InterfaceStatus    status          = 5;
}

message SystemInfo {
    optional uint64             boot_time       = 1;
    optional string             model_info      = 2;
    optional string             serial_no       = 3; 
    optional uint32             max_ports       = 4;
    optional string             collector       = 5;
    repeated string             interface_list  = 6;
}

message SystemStatus {
    optional QueueStatus        queue_status    = 1;
    optional TrafficStatus      traffic_status  = 2;
}

//System message
message System {
    required string             name            = 1;
    optional bool               deleted         = 2;
    optional SystemInfo         information     = 3;
    optional SystemStatus       status          = 4;
}

message AnRecord {
    optional uint64             timestamp       = 1;
    optional System             system          = 2;
    repeated Interface          interface       = 3;
}