Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Prevention Of Asset Growth Deviations

After you confirm that the reported asset growth is legitimate, there are several ways to prevent JSA from triggering growth deviation messages for that asset.

Use the following list to help you decide how to prevent asset growth deviations:

Stale Asset Data

Stale asset data can be problematic when the rate at which new asset records are created exceeds the rate at which stale asset data is removed. Controlling and managing asset retention thresholds is the key to addressing asset growth deviations that are caused by stale asset data.

Stale asset data is historical asset data that is not actively or passively observed within a specific time. Stale asset data is deleted when it exceeds the configured retention period.

The historical records become active again if they are observed by JSA passively, through events and flows, or actively, through port and vulnerability scanners.

Preventing asset growth deviations requires finding the right balance between the number of IP addresses allowed for a single asset and the length of time that JSA retains the asset data. You must consider the performance and manageability trade-offs before you configure JSA to accommodate high levels of asset data retention. While longer retention periods and higher per-asset thresholds might appear desirable all the time, a better approach is to determine a baseline configuration that is acceptable for your environment and test that configuration. Then, you can increase the retention thresholds in small increments until the right balance is achieved.

Asset Blocklists and Allowlists

JSA uses a group of asset reconciliation rules to determine if asset data is trustworthy. When asset data is questionable, JSA uses asset blocklists and allowlists to determine whether to update the asset profiles with the asset data.

An asset blocklist is a collection of data that JSA considers untrustworthy. Data in the asset blocklist is likely to contribute to asset growth deviations and JSA prevents the data from being added to the asset database.

An asset allowlist is a collection of asset data that overrides the asset reconciliation engine logic about which data is added to an asset blocklist. When the system identifies a blocklist match, it checks the allowlist to see whether the value exists. If the asset update matches data that is on the allowlist, the change is reconciled and the asset is updated. Allowlisted asset data is applied globally for all domains.

The asset blocklists and allowlists are reference sets. You can view and modify the asset blocklist and allowlist data using the Reference Set Management tool in the JSA console. For more information about working with reference sets, see Reference Sets Overview.

Alternatively, you can use the command line interface (CLI) or the RestFUL API endpoint to update the content of the asset blocklists and allowlists.

Asset Blocklists

An asset blocklist is a collection of data that JSA considers untrustworthy based on the asset reconciliation exclusion rules. Data in the asset blocklist is likely to contribute to asset growth deviations and JSA prevents the data from being added to the asset database.

Every asset update in JSA is compared to the asset blocklists. Blocklisted asset data is applied globally for all domains. If the asset update contains identity information (MAC address, NetBIOS host name, DNS host name, or IP address) that is found on a blocklist, the incoming update is discarded and the asset database is not updated.

The following table shows the reference collection name and type for each type of identity asset data.

Table 1: Reference Collection Names for Asset Blocklist Data

Type of identity data

Reference collection name

Reference collection type

IP addresses (v4)

Asset Reconciliation IPv4 Blacklist

Reference Set [Set Type: IP]

DNS host names

Asset Reconciliation DNS Blacklist

Reference Set [Set Type: ALNIC*]

NetBIOS host names

Asset Reconciliation NetBIOS Blacklist

Reference Set [Set Type: ALNIC*]

MAC Addresses

Asset Reconciliation MAC Blacklist

Reference Set [Set Type: ALNIC*]

ALNIC is an alphanumeric type that can accommodate both host name and MAC address values.

You can use the Reference Set Management tool to edit the blocklist entries. For information about working with reference sets, see Juniper Secure Analytics Administration Guide.

Your JSA administrator can modify the blocklist entries to ensure that new asset data is handled correctly.

Asset Allowlists

You can use asset allowlists to keep JSA asset data from inadvertently reappearing in the asset blocklists.

An asset allowlists is a collection of asset data that overrides the asset reconciliation engine logic about which data is added to an asset blacklist. When the system identifies a blocklist match, it checks the allowlists to see whether the value exists. If the asset update matches data that is on the allowlists, the change is reconciled and the asset is updated. allowlisted asset data is applied globally for all domains.

You can use the Reference Set Management tool to edit the allowlist entries. For information about working with reference sets, see Juniper Secure Analytics Administration Guide.

Your JSA administrator can modify the allowlist entries to ensure that new asset data is handled correctly.

Example of an Allowlist Use Case

The allowlist is helpful if you have asset data that continues to show up in the blocklists when it is a valid asset update. For example, you might have a round robin DNS load balancer that is configured to rotate across a set of five IP addresses. The Asset Reconciliation Exclusion rules might determine that the multiple IP addresses associated with the same DNS host name are indicative of an asset growth deviation, and the system might add the DNS load balancer to the blocklist. To resolve this problem, you can add the DNS host name to the Asset Reconciliation DNS Whitelist.

Mass Entries to the Asset Allowlist

An accurate asset database makes it easier to connect offenses that are triggered in your system to physical or virtual assets in your network. Ignoring asset deviations by adding mass entries to the asset allowlist is not helpful in building an accurate asset database. Instead of adding mass allowlist entries, review the asset blocklist to determine what is contributing to the deviating asset growth and then determine how to fix it.

Types Of Asset Allowlists

Each type of identity data is kept in a separate allowlist. The following table shows the reference collection name and type for each type of identity asset data.

Table 2: Reference Collection Name for Asset Allowlist Data

Type of data

Reference collection name

Reference collection type

IP addresses

Asset Reconciliation IPv4 Whitelist

Reference Set [Set Type: IP]

DNS host names

Asset Reconciliation DNS Whitelist

Reference Set [Set Type: ALNIC*]

NetBIOS host names

Asset Reconciliation NetBIOS Whitelist

Reference Set [Set Type: ALNIC*]

MAC addresses

Asset Reconciliation MAC Whitelist

Reference Set [Set Type: ALNIC*]

* ALNIC is an alphanumeric type that can accommodate host name and MAC address values.

Updating the Asset Blocklists and Allowlists by Using Reference Set Utility

You can use the JSA reference set utility to add or modify the entries that are on the asset blocklists or allowlists.

To manage your reference sets, run the ReferenceDataUtil.sh utility from /opt/qradar/bin on the JSA console.

The commands to add new values to each list are described in the following table. The parameter values must exactly match the asset update values that are provided by the originating asset data source.

Table 3: Command Syntax to Modify Asset Blocklist and Allowlist Data

Name

Command syntax

Asset Reconciliation IPv4 Blacklist

ReferenceDataUtil.sh add "Asset Reconciliation IPv4 Blacklist" IP

For example, this command adds IP address 192.168.3.56 to the blocklist:

ReferenceDataUtil.sh add "Asset Reconciliation IPv4 Blacklist" 192.168.3.56

Asset Reconciliation DNS Blacklist

ReferenceDataUtil.sh add "Asset Reconciliation DNS Blacklist" DNS

For example, this command adds domain name 'misbehaving.asset.company.com' to the blocklist:

ReferenceDataUtil.sh add "Asset Reconciliation DNS Blacklist" "misbehaving.asset.company.com"

Asset Reconciliation NetBIOS Blacklist

ReferenceDataUtil.sh add "Asset Reconciliation NetBIOS Blacklist" NETBIOS

For example, this command removes NetBIOS host name 'deviantGrowthAsset-156384' from the blocklist:

ReferenceDataUtil.sh delete "Asset Reconciliation NetBIOS Blacklist" "deviantGrowthAsset-156384"

Asset Reconciliation MAC Blacklist

ReferenceDataUtil.sh add "Asset Reconciliation MAC Blacklist" MACADDR

For example, this command adds MAC address '00:a0:1a:2b:3c:4d' to the blocklist:

ReferenceDataUtil.sh add "Asset Reconciliation MAC Blacklist" "00:a0:1a:2b:3c:4d"

Asset Reconciliation IPv4 Whitelist

ReferenceDataUtil.sh add "Asset Reconciliation IPv4 Whitelist" IP

For example, this command deletes IP address 192.0.2.0 from the allowlist:

ReferenceDataUtil.sh delete "Asset Reconciliation IPv4 Whitelist" 192.0.2.0

Asset Reconciliation DNS Whitelist

ReferenceDataUtil.sh add "Asset Reconciliation DNS Whitelist" DNS

For example, this command adds domain name 'loadbalancer.company.com' to the allowlist:

ReferenceDataUtil.sh add "Asset Reconciliation DNS Whitelist" "loadbalancer.company.com"

Asset Reconciliation NetBIOS Whitelist

ReferenceDataUtil.sh add "Asset Reconciliation NetBIOS Whitelist" NETBIOS

For example, this command adds NetBIOS name 'assetName-156384' to the allowlist:

ReferenceDataUtil.sh add "Asset Reconciliation NetBIOS Whitelist" "assetName-156384"

Asset Reconciliation MAC Whitelist

ReferenceDataUtil.sh add "Asset Reconciliation MAC Whitelist" MACADDR

For example, this command adds MAC address '00:a0:1a:2b:3c:4d' to the allowlist :

ReferenceDataUtil.sh add "Asset Reconciliation MAC Whitelist" "00:a0:1a:2b:3c:4d"

Updating the Blocklists and Allowlists Using the RESTful API

You can use the JSA RESTful API to customize the content of the asset blocklists and allowlists.

You must specify the exact name of the reference set that you want to view or update.

  • Asset Reconciliation IPv4 Blacklist

  • Asset Reconciliation DNS Blacklist

  • Asset Reconciliation NetBIOS Blacklist

  • Asset Reconciliation MAC Blacklist

  • Asset Reconciliation IPv4 Whitelist

  • Asset Reconciliation DNS Whitelist

  • Asset Reconciliation NetBIOS Whitelist

  • Asset Reconciliation MAC Whitelist

  1. Type the following URL in your web browser to access the RESTful API interface:

    https://ConsoleIPaddress/api_doc

  2. In the navigation pane on the left, find 4.0>/reference_data >/sets > /{name}.

  3. To view the contents of an asset blocklist or allowlist, follow these steps:

    1. Click the GET tab and scroll down to the Parameters section.

    2. In the Value field for the Name parameter, type the name of the asset blocklist or allowlist that you want to view.

    3. Click Try It Out and view the results at the bottom of the screen.

  4. To add a value to an asset blocklist or allowlist, follow these steps:

    1. Click the POST tab and scroll down to the Parameters section.

    2. Type in the values for the following parameters:

      Table 4: Parameters That Are Required to Add New Asset Data

      Parameter name

      Parameter description

      name

      Represents the name of the reference collection that you want to update.

      value

      Represents the data item that you want to add to the asset blocklist or allowlist. Must exactly match the asset update values that are provided by the originating asset data source.

    3. Click Try It Out to add the new value to the asset allowlist or asset blocklist.

For more information about using the RESTful API to change the reference sets, see the Juniper Secure Analytics API Guide.

Tuning the Asset Profiler Retention Settings

JSA uses the asset retention settings to manage the size of the asset profiles.

The default retention period for most asset data is 120 days after the last time it was either passively or actively observed in JSA. User names are retained for 30 days.

Asset data that is added manually by JSA users does not usually contribute to asset growth deviations. By default, this data is retained forever. For all other types of asset data, the Retain Forever flag is suggested only for static environments.

You can adjust the retention time based on the type of asset identity data that is in the event. For example, if multiple IP addresses are merging under one asset, you can change the Asset IP Retention period from 120 days to a lower value.

When you change the asset retention period for a specific type of asset data, the new retention period is applied to all asset data in JSA. Existing asset data that already exceeds the new threshold is removed when the deployment is complete. To ensure that you can always identify named hosts even when the asset data is beyond the retention period, the asset retention cleanup process does not remove the last known host name value for an asset.

Before you determine how many days that you want to retain the asset data, understand the following characteristics about longer retention periods:

  • provides a better historical view of your assets.

  • creates larger data volumes per asset in the asset database.

  • increases the probability that stale data will contribute to asset growth deviation messages.

  1. On the navigation menu (), click Admin.

  2. In the System Configuration section, click Asset Profiler Configuration.

  3. Click Asset Profiler Retention Configuration.

  4. Adjust the retention values and click Save.

  5. Deploy the changes into your environment for the updates to take effect.

Tuning the Number Of IP Addresses Allowed for a Single Asset

JSA monitors the number of IP addresses that a single asset accumulates over time.

By default, JSA generates a system message when a single asset accumulates more than 75 IP addresses. If you expect assets to accumulate more than 75 IP addresses, you can tune the Number of IPs Allowed for a Single Asset value to avoid future system messages.

Setting the limit for the number of IP addresses too high prevents JSA from detecting asset growth deviations before they have a negative impact on the rest of the deployment. Setting the limit too low increases the number of asset growth deviations that are reported.

You can use the following guideline when you tune the Number of IPs Allowed for a Single Asset setting for the first time.

Number of IP addresses that are allowed for a single asset = (<retention time (days)> x <estimated IP addresses per day>) + <buffer number of IP addresses>

Where

  • <estimated IP addresses per day> is the number of IP addresses that a single asset might accumulate in one day under normal conditions

  • <retention time (days)> is the preferred amount of time to retain the asset IP addresses

  1. On the navigation menu (), click Admin.

  2. In the Assets section, click Asset Profiler Retention Configuration.

  3. Click Asset Profiler Retention Configuration.

  4. Adjust the configuration values and click Save.

  5. Deploy the changes into your environment for the updates to take effect.

Tuning the Number of MAC Addresses Allowed for a Single Asset

JSA monitors the number of MAC addresses that a single asset accumulates over time.

By default, JSA generates a system message when a single asset accumulates more than ten IP addresses. If you expect assets to accumulate more than ten MAC addresses, you can tune the Number of MAC Addresses Allowed for a Single Asset value to avoid future system messages.

Setting the limit for the number of MAC addresses too high prevents JSA from detecting asset growth deviations before they have a negative impact on the rest of the deployment. Setting the limit too low increases the number of asset growth deviations that are reported.

You can use the following guideline when you tune the Number of MAC Addresses Allowed for a Single Asset setting for the first time.

Number of MAC addresses that are allowed for a single asset = (<retention time (days)> x <estimated MAC addresses per day>) + <buffer number of MAC addresses>

Where

  • <estimated MAC addresses per day> is the number of MAC addresses that a single asset might accumulate in one day under normal conditions

  • <retention time (days)>is the preferred amount of time to retain the asset MAC addresses

  1. On the navigation menu, click Admin.

  2. In the Assets section, click Asset Profiler Configuration.

  3. Click Asset Profiler Configuration.

  4. Adjust the Number of MAC Addresses Allowed for a Single Asset value and click Save.

  5. Deploy the changes into your environment for the updates to take effect.

Identity Exclusion Searches

Identity exclusion searches can be used to manage single assets that accumulate large volumes of similar identity information for known, valid reasons.

For example, log sources can provide large volumes of asset identity information to the asset database. They provide JSA with near real-time changes to asset information and they can keep your asset database current. But log sources are most often the source of asset growth deviations and other asset-related anomalies.

When a log source sends incorrect asset data to JSA, try to fix the log source so that the data it sends is usable by the asset database. If the log source cannot be fixed, you can build an identity exclusion search that blocks the asset information from entering the asset database.

You can also use an identity exclusion search where Identity_Username+Is Any Of + Anonymous Logon to ensure that you are not updating assets that are related to service accounts or automated services.

Differences Between Identity Exclusion Searches and Blacklists

While identity exclusion searches appear to have similar functionality to asset blacklists, there are significant differences.

Blacklists can specify only raw asset data, such as MAC addresses and host names, that is to be excluded. Identity exclusion searches filter out asset data based on search fields like log source, category, and event name.

Blacklists do not account for the type of data source that is providing the data, whereas identity exclusion searches can be applied to events only. Identity exclusion searches can block asset updates based on common event search fields, such as event type, event name, category, and log source.

Creating Identity Exclusion Searches

To exclude certain events from providing asset data to the asset database, you can create a JSA identity exclusion search.

The filters that you create for the search must match events that you want to exclude, not the events that you want to keep.

You might find it helpful to run the search against events that are already in the system. However, when you save the search, you must select Real Time (streaming) in the Timespan options. If you do not choose this setting, the search will not match any results when it runs against the live stream of events that are coming into JSA.

When you update the saved identity exclusion search without changing the name, the identity exclusion list that is used by the Asset Profiler is updated. For example, you might edit the search to add more filtering of the asset data that you want to exclude. The new values are included and the asset exclusion starts immediately after the search is saved.

  1. Create a search to identify the events that do not provide asset data to the asset database.

    1. On the Log Activity tab, click Search >New Search.

    2. Create the search by adding search criteria and filters to match the events that you want to exclude from asset updates.

    3. In the Time Range box, select Real Time (streaming) and then click Filter to run the search.

    4. On the search results screen, click Save Criteria and provide the information for the saved search.

      Note:

      You can assign the saved search to a search group. An Identity Exclusion search group exists in the Authentication, Identity and User Activity folder.

    5. Click OK to save the search.

  2. Identify the search that you created as an identity exclusion search.

    1. On the navigation menu (), click Admin.

    2. In the System Configuration section, click Asset Profiler Configuration.

    3. Click Manage Identity Exclusion at the bottom of the screen.

    4. Select the identity exclusion search that you created from the list of searches on the left and click the add icon (>).

      Tip:

      If you can't find the search, type the first few letters into the filter at the top of the list.

    5. Click Save.

  3. On the Admin tab, click Deploy changes for the updates to take effect.

Advanced Tuning Of Asset Reconciliation Exclusion Rules

You can tune the Asset Reconciliation Exclusion rules to refine the definition of deviating asset growth in one or more of the rules.

For example, consider this normalized template from an Asset Reconciliation Exclusion rule.

Apply AssetExclusion: Exclude DNS Name By IP on events which are detected by the Local system and NOT when any of Identity Host Name are contained in any of Asset Reconciliation DNS Whitelist - AlphaNumeric (Ignore Case), Asset Reconciliation DNS Blacklist - AlphaNumeric (Ignore Case) and when at least N1 events are seen with the same Identity Host Name and different Identity IP in N2

This table lists the variables in the rule template that can be tuned and the result of the change. Avoid changing other variables in the template.

Table 5: Options for Tuning the Asset Reconciliation Rules

Variable

Default value

Tuning result

N1

3

Tuning this variable to a lower value results in more data being added to the blacklist because fewer events with conflicting data are needed for the rule to fire.

Tuning this variable to a higher value results in less data being added to the blacklist because more events with conflicting data are needed for the rule to fire.

N2

2 hours

Tuning this variable to a lower value reduces the window of time in which N1 events must be seen for the rule to fire. The time required to observe matching data is decreased, which results in less data being added to the blacklist.

Tuning this variable to a higher value increases the time in which N1 events must be seen for the rule to fire. The time to observe matching data is increased, which results in more data being added to the blacklist.

Increasing the time period might impact system memory resources as data is tracked over longer periods of time.

The Asset Reconciliation Exclusion rules are system-wide rules. Changes to the rules affect the way that the rule behaves throughout the entire system.

Applying Different Tuning for Rules

It might be necessary to apply different tuning for rules in different parts of the system. To apply different tuning for rules, you must duplicate the Asset Reconciliation Exclusion rules that you want to tune and add one or more tests to constrain the rules so that you test only certain parts of the system. For example, you might want to create rules that test only networks, log sources, or event types.

Always be cautious when you are adding new rules to the system because as some tasks and CRE rules might impact system performance. It might be beneficial to add the new rules to the top of each test stack to allow the system to bypass the remainder of the test logic whenever an asset update matches the criteria for the new rule.

  1. Duplicate the rule.

    1. On the Offenses tab, click Rules and select the rule that you want to copy.

    2. Click Actions >Duplicate.

      It can be helpful if the name of the new rule is indicative of the reason for duplicating it.

  2. Add a test to the rule.

    Determine a filter that you want to use to apply the rule only to a subset of system data. For example, you can add a test that matches only events from a specific log source.

  3. Tune the variables of the rule to achieve the wanted behavior.

  4. Update the original rule.

    1. Add the same test that you added to the duplicate rule to the original rule, but this time invert the rules AND and AND NOT operators.

      Inverting the operators prevents events from being triggered in both rules.

Example: Asset Exclusion Rules That Are Tuned to Exclude IP Addresses from the Blacklist

You can exclude IP addresses from being blacklisted by tuning the asset exclusion rules.

As the Network security administrator, you manage a corporate network that includes a public wifi network segment where IP address leases are typically short and frequent. The assets on this segment of the network tend to be transient, primarily notebooks and hand-held devices that log in and out of the public wifi frequently. Commonly, a single IP address is used multiple times by different devices over a short time.

In the rest of your deployment, you have a carefully managed network that consists only of inventoried, well-named company devices. IP address leases are much longer in this part of the network, and IP addresses are accessed by authentication only. On this network segment, you want to know immediately when there are any asset growth deviations and you want to keep the default settings for the asset reconciliation exclusion rules.

Blacklisting IP Addresses

In this environment, the default asset reconciliation exclusion rules inadvertently blacklist the entire network in a short time.

Your security team finds the asset-related notifications that are generated by the wifi segment are a nuisance. You want to prevent the wifi from triggering any more deviating asset growth notifications.

Tuning Asset Reconciliation Rules to Ignore Some Asset Updates

You review the Asset deviation by log source report in the last system notification. You determine that the blacklisted data is coming from the DHCP server on your wifi.

The values in the Event Count column, Flow Count column and the Offenses column for the row corresponding to the AssetExclusion: Exclude IP By MAC Address rule indicate that your wifi DHCP server is triggering this rule.

You add a test to the existing asset reconciliation exclusion rules to stop rules from adding wifi data to the blacklist.

Apply AssetExclusion:Exclude IP by MAC address on events which are detected by the Local system and NOT when the event(s) were detected by one or more of MicrosoftDHCP @ microsoft.dhcp.test.com and NOT when any of Domain is the key and any of Identity IP is the value in any of Asset Reconciliation Domain IPv4 Whitelist - IP Asset Reconciliation Domain IPv4 Blacklist - IP and when at least 3 events are seen with the same Identity IP and different Identity MAC in 2 hours.

The updated rule tests only the events from the log sources that are not on your wifi DHCP server. To prevent wifi DHCP events from undergoing more expensive reference set and behavior analysis tests, you also moved this test to the top of the test stack.

Related Documentation