Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

User-Defined Metadata

User-Defined Metadata Enrichment

IP Address Enrichment Module

The IP address enrichment module provides supplemental information for IP addresses, such as hostname, autonomous system, geolocation, reputation and user-defined metadata. This information is cached for improved performance and flow record throughput. For more control of when enrichment is applied, you can include or exclude IP addresses from various enrichers by CIDR, IP range or individual IP address.

For example:

Metadata Types (IP Addresses)

The user-defined metadata enricher supports a combination of predefined metadata types and enables you to provide custom data as key-value pairs. Table 1 describes the metadata types you can use for IP addresses.

Table 1: IP Address Metadata Types
Attribute Data Type Field Populated Description
internal boolean <object>.isInternal Specifies whether or not the IP belongs to a network considered to be internal.
name string <object>.ip.subnet.name Name given to this subnet.
vlan number (0-4094) <object>.vlan.tag.id A VLAN ID.
tags array of strings <object>.ip.subnet.tags Tags that describe attributes of the subnet or IP address.
metadata sequence of attributes <object><attribute> or <attribute> Key-value pairs added at the IP object or record levels.

Detailed Attribute Descriptions

  • internal: Boolean attribute used to specify whether the CIDR, range or IP address is internal or external. This differs from whether the IP address is within a private or public IP range.

    Some private IP addresses are considered external, such as IPs used within a DMZ. Similarly some public IPs are still considered internal if the IPs are assigned to resources operated by the organization and to which access is generally restricted.

  • name: string attribute used to provide a user-friendly name to a subnet relevant to the user or organization.

    Note:

    Only a single name value is returned for a given IP address. Make sure that there are no conflicting names among overlapping CIDRs, ranges and IP addresses. If you must assign multiple values, add these values to the tags attribute.

  • vlan: Enables you to specify a VLAN tag for a CIDR, range or IP address. This tag is typically assigned to source and destination and client and server related fields.

    • This tag does not conflict with VLAN tags provided in the flow records from network devices.

    • Devices report on the VLAN tags observed on their own interfaces, not the flow endpoints.

    • The VLAN tags reported by devices are typically assigned to the in and out related fields.

  • tags: Array of string values for attributes that further describe the CIDR, range or IP address.

  • metadata: List of key-value pairs added as fields to the record. These fields can be custom fields specific or existing fields from the Apstra Flow CODEX schema. When you specify CODEX fields, the configured metadata value overrides any values that exist in the record.

You can specify key names with or without a leading "."

  • If specified with a leading "." the field is placed within the parent object containing the network interface.
  • If specified without a leading "." the field is placed at the root of the record.

Consider the IP address from flow.src.ip.addr:

  • If the metadata key is defined as .site.name, the value is assigned to flow.src.site.name.
  • If the metadata key is defined as site.name, the value is assigned directly to site.name.

Merging Values from Multiple Definitions

You can merge attribute values for an IP address that matches multiple CIDR, range or IP address entries into a single result set.

For example:

The above example includes:

  • A Class C private network 192.168.0.0/16 that includes location metadata.
  • A 192.168.0.0/24 subnet tagged as the campus network and the firewall zone to which it belongs.
  • A range of IP address that belong to the guest WiFi and are provided by DHCP.

Because the value flow.src.ip.addr (192.168.1.152), matches all three entries in the above configuration, the resulting enrichment fields added to the record will be:

Note:

In the above use case, the host.name and ip.addr were overridden to generic static values anonymizing the individual guest WiFi users. This enables the traffic to be collected and analyzed without tracking each guest individually. This also allows network or security operations to investigate suspect traffic they might want to block, while preserving individual guests' privacy.

Scoping Enrichment with Include/Exclude

You can include or exclude the hostname, DNS, and Maxmind GeoIP enrichment features to a subset of IP addresses by specifying ASs or CIDRs. You can specify the Include/exclude definitions in the provided YAML files to update and refresh without the need to restart the Apstra Flow collector. See /etc/juniper/hostname/incl_excl.yml and /etc/juniper/hostname/user_defined.yml.

The following output shows an example of include/exclude definitions:

Evaluation of Include/Exclude Definitions

It is important to understand how include/exclude definitions are evaluated to ensure your configuration provides the desired outcome.

The following rules apply:

  • If no specific include values are defined, everything is included.

  • Exclude values are evaluated within the scope of included values.

Examples of Include/Exclude Definitions

Note:

While the following examples use only CIDRs, the same logic applies to ASN values.

no include/exclude definitions

If no include/excludes are defined, everything is included.

Table 2: no include/exclude IP Addresses

IP Address

Included

192.168.0.1

yes

10.0.0.1

yes

10.111.0.1

yes

only include is defined

Only those IP addresses within a defined AS or CIDR are included. In this example, only IP addresses within the CIDR 10.0.0.0/8 are included.

Table 3: only include IP Addresses

IP Address

Included

192.168.0.1

no

10.0.0.1

yes

10.111.0.1

yes

only exclude is defined

All IP address not specifically excluded by the defined AS or CIDR are included. In this example, all IP addresses except those within the CIDR 10.111.0.0/16 are included.

Table 4: only exclude IP Addresses

IP Address

Included

192.168.0.1

yes

10.0.0.1

yes

10.111.0.1

no

both include/exclude are defined

Only those IP addresses within a specified AS or CIDR are included, except those within an excluded AS or CIDR.

Table 5: include/exclude IP Addresses

IP Address

Included

192.168.0.1

no

10.0.0.1

yes

10.111.0.1

no

  • 192.168.0.1 is not included because it is not within an included AS or CIDR.
  • 10.0.0.1 is included because it is within an included AS or CIDR.
  • 10.111.0.1 is not included.