Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Data Retention

Retention buckets define how long event and flow data is retained in JSA.

As JSA receives events and flows, each one is compared against the retention bucket filter criteria. When an event or flow matches a retention bucket filter, it is stored in that retention bucket until the deletion policy time period is reached. The default retention period is 30 days; then, the data is immediately deleted.

Retention buckets are sequenced in priority order from the top row to the bottom row. A record is stored in the bucket that matches the filter criteria with highest priority. If the record does not match any of your configured retention buckets, the record is stored in the default retention bucket, which is always located below the list of configurable retention buckets.

Tenant Data

You can configure up to 10 retention buckets for shared data, and up to 10 retention buckets for each tenant.

When data comes into the system, the data is assessed to determine whether it is shared data or whether the data belongs to a tenant. Tenant-specific data is compared to the retention bucket filters that are defined for that tenant. When the data matches a retention bucket filter, the data is stored in that retention bucket until the retention policy time period is reached.

If you don't configure retention buckets for the tenant, the data is automatically placed in the default retention bucket for the tenant. The default retention period is 30 days, unless you configure a tenant-specific retention bucket.

For more information about tenant data retention, see Retention Policies for Tenants.

Configuring Retention Buckets

Configure retention policies to define how long JSA is required to keep event and flow data, and what to do when that data reaches a certain age.

Changes to the retention bucket filters are applied immediately to incoming data only. For example, if you configured a retention bucket to retain all data from source IP address 10.0.0.0/8 for 1 day, and you later edit the filter to retain data from source IP 192.168.0.1, the change is not retroactive. Immediately upon changing the filter, the retention bucket has 24 hours of 10.0.0.0/8 data, and all data that is collected after the filter change is 192.168.0.1 data.

The retention policy on the bucket is applied to all data in the bucket, regardless of the filters criteria. Using the previous example, if you changed the retention policy from 1 day to 7 days, both the 10.0.0.0/8 data and the 192.168.0.1 data in the bucket is retained for 7 days.

The Distribution of a retention bucket indicates the retention bucket usage as a percentage of total data retention in all your retention buckets. The distribution is calculated on a per-tenant basis.

  1. On the navigation menu (), click Admin.

  2. In the Data sources section, click Event Retention or Flow Retention.

  3. If you configured tenants, in the Tenant list, select the tenant that you want the retention bucket to apply to.

    Note:

    To manage retention policies for shared data in a multi-tenant configuration, choose N/A in the Tenant list.

  4. To configure a new retention bucket, follow these steps:

    1. Double-click the first empty row in the table to open the Retention Properties window.

    2. Configure the retention bucket parameters.

      Learn more about retention bucket parameters:

      Properties

      Description

      Name

      Type a unique name for the retention bucket.

      Keep data placed in this bucket for

      The retention period that specifies how long the data is to be kept. When the retention period is reached, data is deleted according to the Delete data in this bucket parameter. JSA does not delete data within the retention period.

      Delete data in this bucket

      Select Immediately after the retention period has expired to delete data immediately on matching the Keep data placed in this bucket for parameter. The data is deleted at the next scheduled disk maintenance process, regardless of disk storage requirements.

      Deletions that are based on storage space begin when the free disk space drops to 15% or less, and the deletions continue until the free disk space is 18% or the policy time frame that is set in the Keep data placed in this bucket for field runs out. For example, if the used disk space reaches 85% for records, data is deleted until the used percentage drops to 82%. When storage is required, only data that matches the Keep data placed in this bucket for field is deleted.

      If the bucket is set to Delete data in this bucket: immediately after the retention period has expired, no disk space checks are done and the deletion task immediately removes any data that is past the retention.

      Description

      Type a description for the retention bucket.

      Current Filters

      Configure the filter criteria that each piece of data is to be compared against.

    3. Click Add Filter after you specify each set of filter criteria.

    4. Click Save.

  5. To edit an existing retention bucket, select the row from the table and click Edit.

    Refer to Step 4 for information about the retention policy properties.

  6. To delete a retention bucket, select the row from the table and click Delete.

  7. Click Save.

    Incoming data that matches the retention policy properties is immediately stored in the retention bucket.

Managing Retention Bucket Sequence

You can change the order of the retention buckets to ensure that data is being matched against the retention buckets in the order that matches your requirements.

Retention buckets are sequenced in priority order from the top row to the bottom row on the Event Retention and Flow Retention windows. A record is stored in the first retention bucket that matches the record parameters.

You cannot move the default retention bucket. It always resides at the bottom of the list.

  1. On the navigation menu (), click Admin.

  2. In the Data sources section, click Event Retention or Flow Retention.

  3. If you configured tenants, in the Tenant list, select the tenant for the retention buckets that you want to reorder.

    Note:

    To manage retention policies for shared data in a multi-tenant configuration, choose N/A in the Tenant list.

  4. Select the row that corresponds to the retention bucket that you want to move, and click Up or Down to move it to the correct location.

  5. Click Save.

Enabling and Disabling a Retention Bucket

When you configure and save a retention bucket, it is enabled by default. You can disable a bucket to tune your event or flow retention.

When you disable a bucket, any new events or flows that match the requirements for the disabled bucket are stored in the next bucket that matches the event or flow properties.

  1. On the navigation menu (), click Admin.

  2. In the Data sources section, click Event Retention or Flow Retention.

  3. If you configured tenants, in the Tenant list, select the tenant for the retention bucket that you want to change.

    Note:

    To manage retention policies for shared data in a multi-tenant configuration, choose N/A in the Tenant list.

  4. Select the retention bucket you want to disable, and then click Enable/Disable.

Deleting a Retention Bucket

When you delete a retention bucket, only the criteria that defines the bucket is deleted. The events or flows that were stored in the bucket are collected by the default retention bucket. The default retention period is 30 days; then, the data is immediately deleted.

  1. On the navigation menu (), click Admin.

  2. In the Data sources section, click Event Retention or Flow Retention.

  3. If you configured tenants, in the Tenant list, select the tenant for the retention bucket that you want to delete.

    Note:

    To manage retention policies for shared data in a multi-tenant configuration, choose N/A in the Tenant list.

  4. Select the retention bucket you want to delete, and then click Delete.

Related Documentation