User-Defined Metadata
User-Defined Metadata Enrichment
IP Address Enrichment Module
The IP address enrichment module provides supplemental information for IP addresses, such as hostname, autonomous system, geolocation, reputation and user-defined metadata. This information is cached for improved performance and flow record throughput. For more control of when enrichment is applied, you can include or exclude IP addresses from various enrichers by CIDR, IP range or individual IP address.
For example:
# Specify whether the IP/CIDR/Range is considered to be "internal". 192.0.2.0/24: internal: true # Additional options are name, vlan, tags and metadata. 192.0.2.192/26: name: atlanta_guest_wifi vlan: 1001 tags: - wifi - dhcp metadata: dhcp.pool.name: atlanta_guest_wifi .site.id: atlanta # Metadata fields beginning with a . will be organized under the object containing the IP address. 192.0.2.194-192.0.2.198: metadata: .site.bldg.id: hq .site.floor.id: 2 .site.rack.id: 1 # An individual IP address. 192.0.2.194: metadata: device.type.name: wifi_ap
Metadata Types (IP Addresses)
The user-defined metadata enricher supports a combination of predefined metadata types and enables you to provide custom data as key-value pairs. Table 1 describes the metadata types you can use for IP addresses.
Attribute | Data Type | Field Populated | Description |
---|---|---|---|
internal |
boolean | <object>.isInternal |
Specifies whether or not the IP belongs to a network considered to be internal. |
name |
string | <object>.ip.subnet.name |
Name given to this subnet. |
vlan |
number (0-4094) | <object>.vlan.tag.id |
A VLAN ID. |
tags |
array of strings | <object>.ip.subnet.tags |
Tags that describe attributes of the subnet or IP address. |
metadata |
sequence of attributes | <object><attribute> or
<attribute> |
Key-value pairs added at the IP object or record levels. |
Detailed Attribute Descriptions
-
internal
: Boolean attribute used to specify whether the CIDR, range or IP address is internal or external. This differs from whether the IP address is within a private or public IP range.Some private IP addresses are considered external, such as IPs used within a DMZ. Similarly some public IPs are still considered internal if the IPs are assigned to resources operated by the organization and to which access is generally restricted.
-
name
: string attribute used to provide a user-friendly name to a subnet relevant to the user or organization.Note:Only a single
name
value is returned for a given IP address. Make sure that there are no conflicting names among overlapping CIDRs, ranges and IP addresses. If you must assign multiple values, add these values to thetags
attribute. -
vlan
: Enables you to specify a VLAN tag for a CIDR, range or IP address. This tag is typically assigned to source and destination and client and server related fields.-
This tag does not conflict with VLAN tags provided in the flow records from network devices.
-
Devices report on the VLAN tags observed on their own interfaces, not the flow endpoints.
-
The VLAN tags reported by devices are typically assigned to the in and out related fields.
-
-
tags
: Array of string values for attributes that further describe the CIDR, range or IP address. -
metadata
: List of key-value pairs added as fields to the record. These fields can be custom fields specific or existing fields from the Apstra Flow CODEX schema. When you specify CODEX fields, the configured metadata value overrides any values that exist in the record.
You can specify key names with or without a leading ".
"
- If specified with a leading "
.
" the field is placed within the parent object containing the network interface. - If specified without a leading "
.
" the field is placed at the root of the record.
Consider the IP address from flow.src.ip.addr
:
- If the metadata key is defined as
.site.name
, the value is assigned toflow.src.site.name
. - If the metadata key is defined as
site.name
, the value is assigned directly tosite.name
.
Merging Values from Multiple Definitions
You can merge attribute values for an IP address that matches multiple CIDR, range or IP address entries into a single result set.
For example:
192.168.0.0/16: metadata: .geo.loc.coord: 48.167106,11.486918 .geo.city.name: Munich .geo.country.code: DE .geo.country.name: Germany .geo.tz.name: Europe/Berlin 192.168.1.0/24: name: munich_hq tags: - campus metadata: sec.zone.name: campus 192.168.1.151-192.168.1.200: tags: - guest_wifi - dhcp metadata: .host.name: guest_wifi .ip.addr: 192.168.1.0
The above example includes:
- A Class C private network
192.168.0.0/16
that includes location metadata. - A
192.168.0.0/24
subnet tagged as the campus network and the firewall zone to which it belongs. - A range of IP address that belong to the guest WiFi and are provided by DHCP.
Because the value flow.src.ip.addr
(192.168.1.152
), matches all three entries in the above
configuration, the resulting enrichment fields added to the record will be:
flow.src.ip.subnet.name: munich_hq flow.src.ip.subnet.tags: [campus guest_wifi dhcp] flow.src.geo.loc.coord: 48.167106,11.486918 flow.src.geo.city.name: Munich flow.src.geo.country.code: DE flow.src.geo.country.name: Germany flow.src.geo.tz.name: Europe/Berlin sec.zone.name: campus flow.src.host.name: guest_wifi flow.src.ip.addr: 192.168.1.0
In the above use case, the host.name
and
ip.addr
were overridden to generic static values
anonymizing the individual guest WiFi users. This enables the traffic to be
collected and analyzed without tracking each guest individually. This also
allows network or security operations to investigate suspect traffic they
might want to block, while preserving individual guests' privacy.
Scoping Enrichment with Include/Exclude
You can include or exclude the hostname, DNS, and Maxmind GeoIP enrichment features
to a subset of IP addresses by specifying ASs or CIDRs. You can specify the
Include/exclude definitions in the provided YAML files to update and refresh without
the need to restart the Apstra Flow collector. See
/etc/juniper/hostname/incl_excl.yml
and
/etc/juniper/hostname/user_defined.yml
.
The following output shows an example of include/exclude definitions:
include: asn: - 14168 cidr: - 10.0.0.0/8 - 192.168.0.0/16 exclude: #asn: # - cidr: - 192.168.100.0/24
Evaluation of Include/Exclude Definitions
It is important to understand how include/exclude definitions are evaluated to ensure your configuration provides the desired outcome.
The following rules apply:
-
If no specific include values are defined, everything is included.
-
Exclude values are evaluated within the scope of included values.
Examples of Include/Exclude Definitions
While the following examples use only CIDRs, the same logic applies to ASN values.
- no include/exclude definitions
- only include is defined
- only exclude is defined
- both include/exclude are defined
no include/exclude definitions
# no path provided or an empty file
If no include/excludes are defined, everything is included.
IP Address |
Included |
---|---|
192.168.0.1 |
yes |
10.0.0.1 |
yes |
10.111.0.1 |
yes |
only include is defined
include: cidr: - 10.0.0.0/8
Only those IP addresses within a defined AS or CIDR are included. In this
example, only IP addresses within the CIDR 10.0.0.0/8
are
included.
IP Address |
Included |
---|---|
192.168.0.1 |
no |
10.0.0.1 |
yes |
10.111.0.1 |
yes |
only exclude is defined
exclude: cidr: - 10.111.0.0/16
All IP address not specifically excluded by the defined AS or CIDR are
included. In this example, all IP addresses except those within the CIDR
10.111.0.0/16
are included.
IP Address |
Included |
---|---|
192.168.0.1 |
yes |
10.0.0.1 |
yes |
10.111.0.1 |
no |
both include/exclude are defined
include: cidr: - 10.0.0.0/8 exclude: cidr: - 10.111.0.0/16
Only those IP addresses within a specified AS or CIDR are included, except those within an excluded AS or CIDR.
IP Address |
Included |
---|---|
192.168.0.1 |
no |
10.0.0.1 |
yes |
10.111.0.1 |
no |
192.168.0.1
is not included because it is not within an included AS or CIDR.10.0.0.1
is included because it is within an included AS or CIDR.10.111.0.1
is not included.