Apstra Flow Collector
Inputs
- EF_FLOW_SERVER_UDP_IP
- EF_FLOW_SERVER_UDP_PORT
- EF_FLOW_SERVER_UDP_READ_BUFFER_MAX_SIZE
- EF_FLOW_PACKET_STREAM_MAX_SIZE
EF_FLOW_SERVER_UDP_IP
The Apstra Flow collector receives network flow records over UDP. Use this setting to specify the interface IP address that the collector will listen on.
- Valid values:
0.0.0.0
or any valid IP address to which the UDP socket can be bound. - Default IP address:
0.0.0.0
(listens on all interfaces)
EF_FLOW_SERVER_UDP_PORT
Use this setting to specify the UDP port on which the collector creates a socket
to receive incoming packets. You can specify multiple ports, separated by a
comma. For example: 2055,6343,4739
.
Valid values: Any valid port number. Common values include:
2055
: Netflow standard port4739
: IPFIX standard port6343
: sFlow standard port9995-9998
: Commonly use port numbers
EF_FLOW_SERVER_UDP_READ_BUFFER_MAX_SIZE
The size (in bytes) of the UDP receive buffer that the UDP server requests, is
created by the operating system kernel when the socket is created. If this value
exceeds the maximum allowed buffer size (net.core.rmem_max
on
Linux), the maximum allowed size is used.
- Default:
33554432
EF_FLOW_PACKET_STREAM_MAX_SIZE
- Default:
16384
bytes
Decoder/Processor
- EF_PROCESSOR_DECODE_IPFIX_ENABLE
- EF_PROCESSOR_DECODE_NETFLOW1_ENABLE
- EF_PROCESSOR_DECODE_NETFLOW5_ENABLE
- EF_PROCESSOR_DECODE_NETFLOW6_ENABLE
- EF_PROCESSOR_DECODE_NETFLOW7_ENABLE
- EF_PROCESSOR_DECODE_NETFLOW9_ENABLE
- EF_PROCESSOR_DECODE_SFLOW5_ENABLE
- EF_PROCESSOR_DECODE_SFLOW_FLOWS_ENABLE
- EF_PROCESSOR_DECODE_SFLOW_FLOWS_KEEP_SAMPLES
- EF_PROCESSOR_DECODE_SFLOW_COUNTERS_ENABLE
- EF_PROCESSOR_DECODE_MAX_RECORDS_PER_PACKET
- EF_PROCESSOR_TRANSLATE_KEEP_IDS
- EF_PROCESSOR_ENRICH_ASN_PREF
- EF_PROCESSOR_ENRICH_JOIN_ASN
- EF_PROCESSOR_ENRICH_JOIN_GEOIP
- EF_PROCESSOR_ENRICH_JOIN_NETATTR
- EF_PROCESSOR_ENRICH_JOIN_SUBNETATTR
- EF_PROCESSOR_ENRICH_JOIN_SEC
- EF_PROCESSOR_EXPAND_CLISRV
- EF_PROCESSOR_EXPAND_CLISRV_NO_L4_PORTS
- EF_PROCESSOR_IFA_ENABLE
- EF_PROCESSOR_IFA_WORKER_SIZE
EF_PROCESSOR_DECODE_IPFIX_ENABLE
Set to true
to enable decoding of IPFIX records.
- Valid values:
true
,false
- Default:
true
EF_PROCESSOR_DECODE_NETFLOW1_ENABLE
Set to true
to enable decoding of Netflow v1 records.
- Valid values:
true
,false
- Default:
true
EF_PROCESSOR_DECODE_NETFLOW5_ENABLE
Set to true
to enable decoding of Netflow v5 records.
- Valid values:
true
,false
- Default:
true
EF_PROCESSOR_DECODE_NETFLOW6_ENABLE
Set to true
to enable decoding of Netflow v6 records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_NETFLOW7_ENABLE
Set to true
to enable decoding of Netflow v7 records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_NETFLOW9_ENABLE
Set to true
to enable decoding of Netflow v9 records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_SFLOW5_ENABLE
Set to true
to enable decoding of sFlow v5 records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_SFLOW_FLOWS_ENABLE
Set to true
to enable decoding of sFlow
flow_sample
and flow_sample_expanded
records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_SFLOW_FLOWS_KEEP_SAMPLES
When set to true
, the packet data from an sFlow
sampled_header
record is stored in
l2.section.sample
as a hex-encoded string.
- Valid values:
true
,false
- Default:
false
EF_PROCESSOR_DECODE_SFLOW_COUNTERS_ENABLE
Set to true
to enable decoding of sFlow
counters_sample
and
counters_sample_expanded
records.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_DECODE_MAX_RECORDS_PER_PACKET
Corrupt packets can cause issues decoding records. To prevent this, you can use
this setting to limit the number of records that will be decoded from a packet.
When the network between the device and collector has an MTU larger than
1500
, the default value might be exceeded by normal
packets. The EF_PROCESSOR_DECODE_MAX_RECORDS_PER_PACKET
setting
allows you to increase the threshold, when necessary.
- Default value:
64
EF_PROCESSOR_TRANSLATE_KEEP_IDS
Use this setting to specify the identifier values to be included in the final dataset.
Valid values:
none
: All identifiers are removed from the final dataset.default
: Most identifiers are removed from the final dataset. Note that some identifiers required for common use-cases (such as raw protocol port values) are included in the final dataset.all
: All identifiers are included in the final dataset.- Default value:
default
EF_PROCESSOR_ENRICH_ASN_PREF
If you enable enrichment with autonomous system (AS) attributes, and if AS is
already indicated directly in the flow record data, you can use the
EF_PROCESSOR_ENRICH_ASN_PREF
setting to specify which
source is preferred. If the preferred source is not available for a given
record, the decoder fall backs to the alternate option.
- Valid values:
lookup
: The AS is determined by lookup.flow
: The AS is indicated directly in the flow record data.
- Default value:
lookup
EF_PROCESSOR_ENRICH_JOIN_ASN
Some features require that related values from separate fields are stored as an
array in a single field. A join of AS related fields is enabled when
EF_PROCESSOR_ENRICH_JOIN_ASN
is set to
true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_JOIN_GEOIP
Some features require that related values from separate fields are stored as an
array in a single field. A join of GeoIP related fields is enabled when
EF_PROCESSOR_ENRICH_JOIN_GEOIP
is set to
true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_JOIN_NETATTR
Some features require that related values from separate fields are stored as an
array in a single field. A join of network attribute related fields is
enabled when EF_PROCESSOR_ENRICH_JOIN_NETATTR
is set to
true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_JOIN_SUBNETATTR
Some features require that related values from separate fields are stored as an
array in a single field. A join of IP subnetwork attribute related fields
is enabled when EF_PROCESSOR_ENRICH_JOIN_SUBNETATTR
is set to
true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_JOIN_SEC
Some features require that related values from separate fields are stored as an
array in a single field. A join of security attribute related fields is
enabled when EF_PROCESSOR_ENRICH_JOIN_SEC
is set to
true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_EXPAND_CLISRV
The collector infers the client/server relationship of two source/destination
endpoints. The EF_PROCESSOR_EXPAND_CLISRV
setting determines if
inference is enabled or disabled.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_EXPAND_CLISRV_NO_L4_PORTS
For flow records related to protocols that include "no layer-4 ports", the
collector infers the client/server relationship of the two source/destination
endpoints by using the order of the IP addresses. Use this
EF_PROCESSOR_EXPAND_CLISRV_NO_L4_PORTS
setting to enable or
disable inference. The default setting is true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_IFA_ENABLE
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_IFA_WORKER_SIZE
Use to specify the the number of IFA Hop record processors to start.
- Default number:
4 * the number of license units
Sampling Rates
Devices can sample packets to reduce the overall volume of traffic metered for flow accounting, The various sampling rate configuration options are described as follows:
- EF_PROCESSOR_ENRICH_SAMPLERATE_CACHE_SIZE
- EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_ENABLE
- EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_PATH
- EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_OVERRIDE
EF_PROCESSOR_ENRICH_SAMPLERATE_CACHE_SIZE
The Apstra Flow collector adjusts the calculation of bytes and packets based on
the sampling rate that is used. Usually devices inform the collector of the
sampling rate either within the flow record or as option data sent periodically
by the device. Use the
EF_PROCESSOR_ENRICH_SAMPLERATE_CACHE_SIZE
setting to
specify the size of the cache to be used to hold sample rate information learned
from option data.
- Default value:
32768
EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_ENABLE
Sometimes, a device might not transmit information about the sampling rate for
which it is configured. Use the
EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_ENABLE
setting to
statically define the sampling rate in the file provided to the collector.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_PATH
If static sample rates are configured for devices in a file, the
EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_PATH
setting specifies the
path from where that file can be loaded.
For example:
'192.0.2.1': 1024 '192.0.2.2': 512
The default path is: /etc/juniper/settings/sample_rate.yml
EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_OVERRIDE
In some use cases, you might want to use a user-defined sample rate rather than
the rate provided by the device. Set
PROCESSOR_ENRICH_SAMPLERATE_USERDEF_OVERRIDE
to
true
to check for a user-defined rate even if the device
has already provided a rate.
- Valid values:
true
,false
- Default value:
false
General Settings
EF_PROCESSOR_ENRICH_TOTALS_IF_NO_DELTAS
Most flow exporters provide byte and packet quantities as delta values. Delta values refer to the byte and packet quantities since the last flow record was reported. However, some exporters, such as the Juniper MX-Series router sending IPFIX, provide these quantities only as total values. Total values refers to the quantity over the entire lifetime of the flow.
In cases where the exporter sends only totals, you might want to use these
values to populate the flow.bytes
and
flow.packets
. When
EF_PROCESSOR_ENRICH_TOTALS_IF_NO_DELTAS
is set to
true
, the total quantities are used.
Total quantities can be problematic for many datastores. A simple sum of total values across multiple records within a time window will not produce an accurate quantity, as is it does with delta values. As a result, long-lived flows can over-report bytes and packets values if total values are used.
- Valid values:
true
,false
- Default value:
true
Applications
- EF_PROCESSOR_ENRICH_APP_ID_ENABLE
- EF_PROCESSOR_ENRICH_APP_ID_PATH
- EF_PROCESSOR_ENRICH_APP_ID_TTL
- EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE
- EF_PROCESSOR_ENRICH_APP_IPPORT_PATH
- EF_PROCESSOR_ENRICH_APP_IPPORT_TTL
- EF_PROCESSOR_ENRICH_APP_IPPORT_PRIVATE
- EF_PROCESSOR_ENRICH_APP_IPPORT_PUBLIC
- EF_PROCESSOR_ENRICH_APP_REFRESH_RATE
EF_PROCESSOR_ENRICH_APP_ID_ENABLE
- Valid values:
true
,false
- Default:
false
EF_PROCESSOR_ENRICH_APP_ID_PATH
If the vendor-defined AppID to application attribute mappings is enabled
(EF_PROCESSOR_ENRICH_APP_ID_ENABLE
is
true
) this setting specifies the path to the file.
The default path is: /etc/juniper/app/appid.yml
EF_PROCESSOR_ENRICH_APP_ID_TTL
Use this setting to specify the length of time the application attributes are cached after they are initially fetched.
Changes to the underlying files are not made (even after the files were re-loaded at the refresh interval) until the AppID has expired from the cache.
- Default value:
7200
EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE
Various flow record sources send the mapping of application IDs to applications names as option data. In cases where no application identity technology is available, you can specify applications by IP address and port number.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_APP_IPPORT_PATH
When user-defined IP/port to application mappings is enabled, the
(EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE
is
true
) setting specifies the path to this file.
For example:
192.168.1.0/24: 8090: name: "Synergy-cidr-port" category: "category-cidr-port" subcategory: "subcategory-cidr-port" metadata: ".location": "austin-cidr-port" "business.unit": "finance-cidr-port" "dev.unit": "dev-cidr-port" "app.count": 27 192.168.1.1-192.168.1.20: 8090: name: "Synergy-iprange-port" category: "category-iprange-port" subcategory: "subcategory-iprange-port" metadata: .location: "austin-iprange-port" 8090-9000: name: "Synergy-iprange-portrange" category: "category-iprange-portrange" subcategory: "subcategory-iprange-portrange" metadata: .location: "austin-iprange-portrange" business.unit: "finance-iprange-portrange" qa.unit: "qa-iprange-portrange" finace.unit: "finance-iprange-portrange" 192.168.1.1: 8090: name: "Synergy-ip-port" category: "category-ip-port" subcategory: "subcategory-ip-port" metadata: .location: "austin-ip-port" business.unit: "finance-ip-port"
- Default path:
/etc/juniper/app/ipport.yml
EF_PROCESSOR_ENRICH_APP_IPPORT_TTL
Use this setting to specify the length of time application attributes are cached after they are initially fetched.
Changes to the underlying files are not made, even after the files have been reloaded at the refresh interval, until the IP/Port has expired from the cache.
- Default value:
7200
EF_PROCESSOR_ENRICH_APP_IPPORT_PRIVATE
If user-defined application attributes are enabled
(EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE
is
true
) this setting specifies whether application names are
checked for private IP addresses.
- Valid values:
true
,false
- Default:
true
EF_PROCESSOR_ENRICH_APP_IPPORT_PUBLIC
If user-defined application attributes are enabled
(EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE
is
true
) this setting specifies whether application names are
checked for public IP addresses.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_APP_REFRESH_RATE
Files defined for application attribute enrichment can be loaded automatically to refresh values without restarting the collector. Use this setting to specifies the refresh interval, in minutes, that the file will be reloaded.
- Default value:
15
(0
value disables this setting)
IP Addresses
Name Resolution
You can configure the collector to resolve IP addresses to hostnames. The following settings allow this feature to be tuned to the needs of your environment.
- EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE
- EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_IP
- EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_TIMEOUT
- EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PRIVATE
- EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PUBLIC
- EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_PATH
- EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_REFRESH_RATE
- EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_PATH
- EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_REFRESH_RATE
EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE
Use this setting to enables DNS reverse lookups of IP addresses found in the received flow records.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_IP
The collector uses the operating system's configured name resolution to resolve IP addresses to hostnames. This is the default behavior. Optionally, you can specify a nameserver to use instead.
If configured, this setting must contain a valid IP address.
- Default:
empty
EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_TIMEOUT
If EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_IP
contains a
valid IP address, this setting contains the timeout period, in milliseconds,
for queries to the name server.
- Default:
3000
EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PRIVATE
When DNS resolution is enabled
(EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE
is
true
), this setting specifies whether private IP addresses
will be resolved to hostnames.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PUBLIC
If DNS resolution is enabled
(EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE
set to
true
), this setting specifies whether public IP
addresses will be resolved to hostnames.
- Valid values:
true
,false
- Default:
true
EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_PATH
The EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_PATH
setting
specifies the path to the file containing user-defined hostname mappings.
This feature is enabled only if a path is configured, otherwise it is
disabled.
'192.0.2.1': 'host1' '192.0.2.2': 'host2'
- Default setting:
''
- Recommended path:
/etc/juniper/hostname/user_defined.yml
EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_REFRESH_RATE
Use this setting to automatically load refresh values without restarting the collector. The value you specify indicates the refresh interval time, in minutes, that the file will take to reload.
- Default value:
15
( if set to0
, refresh values are disabled)
EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_PATH
For more control of when enrichment is applied, you can include or exclude IP
addresses from hostname enrichment by AS or CIDR. Use this setting to
specify the path to the inclu_excl.yml
file. For more
information about the include/exclude functionality, see Scoping Enrichment with Include/Exclude.
- Default setting:
''
- Recommended path:
/etc/juniper/hostname/incl_excl.yml
EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_REFRESH_RATE
Use this setting to automatically refresh values without restarting the collector. The value you specify indicates the refresh interval, in minutes, that the file will take to reload.
- Default value:
15
( if set to0
, refresh values are disabled)
Maxmind
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_ENABLE
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_PATH
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_PATH
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_VALUES
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_LANG
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_REFRESH_RATE
- EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_REFRESH_RATE
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_ENABLE
Use this setting
(EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_ENABLE
is
true
) to allow the collector to determine
attributes associated with the ASs to which a public IP address belongs.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_PATH
Use this setting to specify the path to the Maxmind database. Enrichment
with AS attributes is enabled using lookups in a Maxmind database when
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_ENABLE
is
true
.
- Default path:
/etc/juniper/maxmind/GeoLite2-ASN.mmdb
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
Set EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
to
true
to allow the collector to determine GeoIP
attributes associated with a public IP address.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_PATH
If enrichment with GeoIP attributes is enabled using lookups in a Maxmind
database
((EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
is true
), this specifies the path to the Maxmind
database.
- Default path:
/etc/juniper/maxmind/GeoLite2-City.mmdb
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_VALUES
If enrichment with GeoIP attributes is enabled using lookups in a Maxmind
database
(EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
is
true
), this setting specifies the GeoIP attributes
from the Maxmind database to be included in the resulting record.
- Valid values:
city
,continent
,continent_code
,country
,country_code
,location
,timezone
- Default values:
city,country,country_code,location,timezone
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_LANG
If enrichment with GeoIP attributes is enabled using lookups in a Maxmind
database
(EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE
is
true
), this setting to specifies the language to be
used for any language-specific values.
- Valid values
de
: Germanen
: Englishes
: Spanishfr
: Frenchja
: Japanesept-BR
: Brazilian Portugueseru
: Russianzh-CN
: Simplified Chinese
- Default value:
en
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH
For more control of when enrichment is applied, you can include or
exclude IP addresses from GeoIP enrichment by ASs or CIDRs. The
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH
setting specifies the path to the incl_excl.yml
file.
- Default setting:
''
- Recommended path:
/etc/juniper/hostname/incl_excl.yml
For more details on the include/exclude functionality see Scoping Enrichment with Include/Exclude.
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_REFRESH_RATE
The file specified in
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH
can be loaded automatically to refresh values without restarting the
collector. Use this setting to specify the refresh interval, in minutes,
the file will take to reload.
- Default value:
15
(Note: when set to0
, the refresh interval is not used).
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_REFRESH_RATE
The file specified in
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH
can be loaded automatically to refresh values without restarting the
collector. Use this setting to specify the refresh interval, in
minutes, the file will take to reload.
- Default value:
15
(Note: when this value is set to0
, the refresh interval is not used).
User-Defined Metadata
User-defined metadata adds additional information to a record for a given IP address. It can also be used to override existing fields. You can specify metadata for CIDR blocks, IP ranges or individual IP addresses.
- EF_PROCESSOR_ENRICH_IPADDR_METADATA_ENABLE
- EF_PROCESSOR_ENRICH_IPADDR_METADATA_USERDEF_PATH
- EF_PROCESSOR_ENRICH_IPADDR_METADATA_REFRESH_RATE
EF_PROCESSOR_ENRICH_IPADDR_METADATA_ENABLE
Use this setting to enable or disable user-defined metadata enrichment. The
default is true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_IPADDR_METADATA_USERDEF_PATH
If the user-defined metadata enrichment is enabled
(EF_PROCESSOR_ENRICH_IPADDR_METADATA_ENABLE
is
true
), this setting specifies the path to the metadata
file. If this value is undefined or empty, metadata enrichment is
disabled.
For more information on user-defined metadata functionality, see: User-Defined Metadata Enrichment.
- Default value:
''
- Recommended path:
/etc/juniper/metadata/ipaddrs.yml
EF_PROCESSOR_ENRICH_IPADDR_METADATA_REFRESH_RATE
The file specified in
EF_PROCESSOR_ENRICH_IPADDR_METADATA_USERDEF_PATH
can be
loaded automatically to refresh values without restarting the collector.
This value specifies the refresh interval, in minutes, that the file will be
reloaded. The value of 0
disables refreshing of the
values.
- Default value:
15
Network Interfaces
Option Records
The Apstra Flow collector will attempt to determine network interface attributes learned from NetFlow v9 or IPFIX option records.
EF_PROCESSOR_ENRICH_NETIF_FLOW_OPTIONS_ENABLE
Setting this value to false
will disable the enrichment of
records with interface attributes learned from NetFlow or IPFIX options
records.
- Valid values:
true
,false
- Default value:
true
SNMP
Flow records generally include the indexes of ingress and egress interfaces by which the network traffic traversed the exporting device. The collector will attempt to determine the names, and attributes of these interfaces, as learned by polling the exporting device using SNMP.
- EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
- EF_PROCESSOR_ENRICH_NETIF_SNMP_PORT
- EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION
- EF_PROCESSOR_ENRICH_NETIF_SNMP_COMMUNITIES
- EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_USERNAME
- EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PROTOCOL
- EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PASSPHRASE
- EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PROTOCOL
- EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PASSPHRASE
- EF_PROCESSOR_ENRICH_NETIF_SNMP_TIMEOUT
- EF_PROCESSOR_ENRICH_NETIF_SNMP_RETRIES
EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
Use this setting to specify if SNMP polls are to be used to gather the network interface attributes.
- Valid values:
true
,false
- Default value:
false
EF_PROCESSOR_ENRICH_NETIF_SNMP_PORT
If SNMP polling of attributes is enabled
(EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
is
true
), this setting specifies the UDP port that is used
for such polls.
- Default UDP port:
161
(the default SNMP port number)
EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION
If SNMP polling of attributes is enabled
(EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
is
true
), this setting specifies the SNMP version that is
used for such polls.
All network devices that are polled must support this version of SNMP.
Valid values:
1
: SNMPv12
: SNMPv2c3
: SNMPv3
EF_PROCESSOR_ENRICH_NETIF_SNMP_COMMUNITIES
If SNMP polling of attributes is enabled
(EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
is
true
), this setting specifies the SNMP community
strings that may be used for such polls. If a comma-separated list is
specified, the collector will try each community in the order specified.
Once a community returns a successful response, the collector remembers the
community for future polls of the device.
All network devices polled must be configured to all visibility of collected attributes using this community. It may be necessary to specify a view associated with this community. See the documentation for your devices for help in determining the correct configuration steps.
- Example:
public,private,whatever
- Default setting:
public
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_USERNAME
Use this setting to specify the username used to authenticate the device using SNMPv3.
- Default setting:
''
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PROTOCOL
Use this setting to specify the authentication protocol used to authenticate the username with the device using SNMPv3.
Valid values:
noauth
,md5
,sha
,sha224
,sha256
,sha384
,sha512
- Default value:
noauth
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PASSPHRASE
Use this setting to specify the authentication passphrase used to authenticate the username with the device using SNMPv3.
- Default passphrase:
''
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PROTOCOL
Use this setting to sepcify the privacy protocol used to encrypt SNMPv3 traffic between the SNMP input and the device.
Valid values:
nopriv
,des
,aes
,aes192
,aes256
,aes192c
,aes256c
- Default value:
nopriv
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PASSPHRASE
Use this setting to specify the privacy passphrase used to encrypt SNMPv3 traffic between the SNMP input and the device.
- Default passphrase:
''
EF_PROCESSOR_ENRICH_NETIF_SNMP_TIMEOUT
If SNMP polling of attributes is enabled
(EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
set
true
), this setting specifies the number of seconds to
wait for the polled device to respond.
- Default value:
2
EF_PROCESSOR_ENRICH_NETIF_SNMP_RETRIES
If SNMP polling of attributes is enabled
(EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE
is
true
), this setting specifies the number of retries to
attempt after the initial poll has timed out or otherwise fails. The timeout
period is doubled for each retry.
- Default value:
1
User-Defined Metadata
User-defined metadata allows you to add additional information to a record for a given network interface or to override existing fields.
- EF_PROCESSOR_ENRICH_NETIF_METADATA_ENABLE
- EF_PROCESSOR_ENRICH_NETIF_METADATA_USERDEF_PATH
- EF_PROCESSOR_ENRICH_NETIF_METADATA_REFRESH_RATE
EF_PROCESSOR_ENRICH_NETIF_METADATA_ENABLE
Use this setting to enable or disable user-defined metadata enrichment. The
default value is true
.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_NETIF_METADATA_USERDEF_PATH
If user-defined metadata enrichment is enabled
(EF_PROCESSOR_ENRICH_NETIF_METADATA_ENABLE
is
true
) this setting specifies the path to the metadata
file. If this value is undefined or empty, metadata enrichment is
disabled.
For more details on user-defined metadata, see User-Defined Metadata.
- Default setting:
''
- Recommended path:
/etc/juniper/metadata/netifs.yml
EF_PROCESSOR_ENRICH_NETIF_METADATA_REFRESH_RATE
The file specified in
EF_PROCESSOR_ENRICH_NETIF_METADATA_USERDEF_PATH
can be
loaded automatically to refresh values without restarting the collector.
This value specifies the refresh interval, in minutes, that the file will be
reloaded.
- Default value:
15
(The value of0
disables refreshing of the values).
Community/Conversation IDs
- EF_PROCESSOR_ENRICH_COMMUNITYID_ENABLE
- EF_PROCESSOR_ENRICH_COMMUNITYID_SEED
- EF_PROCESSOR_ENRICH_CONVERSATIONID_ENABLE
- EF_PROCESSOR_ENRICH_CONVERSATIONID_SEED
EF_PROCESSOR_ENRICH_COMMUNITYID_ENABLE
Use this setting to specify if flow records should be enriched with a Community ID value.
For more information about community IDs see the community-id-spec.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_COMMUNITYID_SEED
This setting is a 16-bit value used as the seed for determining the Community ID of a flow record.
- Default value:
0
EF_PROCESSOR_ENRICH_CONVERSATIONID_ENABLE
Use this setting to enable or disable flow records enriched with a Conversation ID value. This value is similar to a community ID, however rather than being based on the SRC/DST relationship of two endpoints, this value is based on the client/server perspective. Although multiple unique sessions (such as a unique client-side port for each session) have their own Community ID, they share the same Conversation ID. This allows for greater flexibility when exploring a complex flow dataset.
- Valid values:
true
,false
- Default value:
true
EF_PROCESSOR_ENRICH_CONVERSATIONID_SEED
This setting is a 16-bit value used as the seed for determining the conversation ID of a flow record.
- Default value:
0