Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

NetFlow Collector Overview

NetFlow collector is a data collection tool in Paragon Pathfinder.

NetFlow collector uses the netflowd microservice in the northstar namespace to collect and report data about traffic flow in the network. Netflowd is automatically installed as part of the Pathfinder installation package. Netflowd receives the NetFlow data from the routers, decodes the records, and aggregates the data. Netflowd uses this aggregated data to create demands, which indicate the amount of traffic flow in the network. Netflowd stores the data in the Time Series Database (TSDB) and shares the data with the Path Computation Server (PCS). The aggregated data is used to generate Demand reports that are available in Paragon Pathfinder (Reports > Demand). These reports provide information on network traffic. This data is also used in Paragon Planner to generate Demand reports, plan, and model the network.

Note:
  • The PCS monitors traffic from the autonomous systems (AS) and VPNs.
  • The PCS supports both IPv4 and IPv6 traffic.

Pathfinder leverages the Junos OS implementation of flow monitoring and aggregation by using NetFlow Version 9 and Version 10 (IPFIX) flow templates. See the following Junos OS documentation for background:

  • Configuring Flow Aggregation to Use Version 9 Flow Templates.
  • Configuring Flow Aggregation to Use IPFIX Flow Templates on MX, vMX and T Series Routers, EX Series Switches and NFX250.

  • Configuring Flow Aggregation to Use IPFIX Flow Templates on PTX Series Routers.

Demand Generation

Netflowd uses four aggregation keys (obtained from the NetFlow data that netflowd collects) to generate the demands:

  • Ingress provider edge (PE) device (that is the device reporting the flow)
  • BGP next hop IP address

  • Routing table name

    • When this key is present, it is the name of the VRF for which the ingress interface is configured.
    • This key is absent if no VPN is associated with the demand. In this case, the ingress interface is configured in the default routing table.
    • This key is displayed as NONE if netflowd is not able to determine whether the ingress interface is configured in the default routing table or on a VRF. That would happen, for example, if the PCS was not able to collect the snmp-indexes for the interfaces.
  • Specification of IPv4 (displayed as IP in the Demand tab of the network information table) or IPv6

The values of the keys are indicated in the names of the demands which are displayed in the Name column of the Demand tab in the network information table. Here are some examples:

  • vmx102_10.1.0.10/32_vpn100_IP
  • vmx102_10.1.0.10/32_IP (if no VPN is associated with the demand)

  • vmx102_10.1.0.10/32_NONE_IP (if it is unknown whether the ingress interface is configured on the default routing table or on a VRF)

NetFlow Collector Requirements

To use NetFlow collector in Pathfinder, you must:

(Optional) Customize NetFlowd parameters from the CLI. See Customize Netflowd Parameters from the CLI.

Configuration on the Network Routers

To use NetFlow collector in Pathfinder, you must configure the network routers for flow monitoring (NetFlow v9 or v10) according to the router's operating system documentation.

Note:

Currently, only Juniper Networks devices and Cisco IOS-XR devices can be configured with NetFlow v9 and v10.

Here are some important considerations to keep in mind when you configure the routers:

  • The NetFlow process (netflowd) identifies the device, which is reporting the flow, through the source address (inline-jflow statement). Configure this parameter as the router’s loopback address.
  • The flow-active-timeout parameter has a default value of 60 seconds. We recommend keeping it at 60 seconds or less.

  • Configure the flow-server's IP address as the virtual IP (VIP) address that is configured for the Nginx Ingress Controller.
The following Junos OS example shows the NetFlow v9 configuration statements.

At the interface hierarchy level:

At the forwarding-options hierarchy level:

At the chassis hierarchy level:

At the services hierarchy level:

The following Junos OS example shows NetFlow v10 configuration statements.

At the interface hierarchy level:

At the forwarding-options hierarchy level:

At the chassis hierarchy level:

At the services hierarchy level:

Customize Netflowd Parameters from the CLI

The parameters related to netflowd are configured by default and you can view these parameters from the CLI. Optionally, you can customize these parameters as well. Table 1 describes the netflowd parameters that you can customize from the CLI.

Table 1: Netflowd Parameters

Parameter

Command

Command

enable-ssl set northstar analytics netflowd enable-ssl Configure this parameter to enable netflowd to establish a Secure Socket Layer (SSL) connection to the native datastore.
logging-parameters set northstar analytics netflowd logging-parameters Configure the level of information that is captured in the log file.

The default level is info. If you want more information to be included in the log file, you can set the level to debug. The log file will include all the flows which are received from each device, identified by the source IP address. You can also view, for each flow, all the fields that netflowd processes and parses.

default-sampling-interval

set northstar analytics netflowd default-sampling-interval

Configure the default sampling interval that is used if the router does not provide the interval in the Template FlowSet.

Default: 1.

publish-interval

set northstar analytics netflowd publish-interval

Configure the interval (in seconds or minutes) to publish records to both the TSDB and the PCS. Traffic is aggregated per publishing interval.

This value must be equal to or greater than the reporting time configured in the router (flow-active-timeout value) to ensure that for every publishing interval, all active flows are reported.

Default: 60s.

notify-final-bandwidth-on-inactive-flow

set northstar analytics netflowd notify-final-bandwidth-on-inactive-flow

Configure this parameter to enable netflowd to send one final update after a flow is no longer active, reporting the bandwidth as 0.

By default, this parameter is not configured. So, the bandwidth value is not reported once a flow becomes inactive; the last reported active value is the last value displayed.

aggregate-by-prefix

set northstar analytics netflowd aggregate-by-prefix

Configure this parameter to enable netflowd to aggregate all traffic from a specific ingress provider edge (PE) router to a specific destination (prefix) within the specified period. By default, NetFlow aggregates traffic by PE routers, but for some applications (such as Egress Peer Engineering and Ingress Peer Engineering), you would want the traffic to be aggregated by prefix.

stats-interval

set northstar analytics netflowd stats-interval

Configure the interval (in seconds) at which statistics are printed to the log file. By default, the interval is not configured, so the statistics are not printed to the log file.

generate-as-demands

set northstar analytics netflowd generate-as-demands

Configure this parameter to enable netflowd to generate AS demands.

By default, this parameter is not configured. So, AS demands are not displayed through REST APIs or in Demand reports in the GUI, even if valid NetFlow records are being exported.

top-prefixes set northstar analytics netflowd top-prefixes

Configure the number of prefixes (in terms of the aggregated traffic volume) to be exported.

Range: 1 through 10,000

top-prefixes-export-ticks set northstar analytics netflowd top-prefixes-export-ticks

Configure the number of intervals above which traffic is aggregated for the top N prefixes, where the export interval length is determined by the publish-interval parameter.

Example: If you set the publish-interval as 60s and top-prefixes-export-ticks as 5, the top N prefixes are exported (published) every 5 minutes (5x60s = 5m).

workers set northstar analytics netflowd workers

Configure the number of processes to be started. When set to 0, it takes the value of the number of cores in the system.

Default: 1