Event and Flow Notifications for JSA Appliances

Explanation

If too many indexes are enabled or the system is overburdened, the system might drop the event or flow from the index portion.

User Response

Select one of the following options:

• If the dropped index interval occurs with SAR sentinel notifications, the issue is likely due to system load or low disk space.

• To temporarily disable some indexes to reduce the system load, on the Admin tab, click the Index Management icon.

Explanation

A TCP-based protocol dropped an established connection to the system.

The number of connections that can be established by TCP-based protocols is limited to ensure that connections are established and events are forwarded. The event collection service (ECS) allows a maximum of 15,000 file handles and each TCP connection uses three file handles.

TCP protocols that provide drop connection notifications include the following protocols:

• TCP syslog protocol

• TLS syslog protocol

• TCP multi-line protocol

User Response

Review the following options:

• Distribute events to more appliances. Connections to other event and flow processors distribute the work load from the console.

• Configure low priority TCP log source events to use the UDP network protocol.

• Tune the system to reduce the volume of events and flows that enter the event pipeline.

Explanation

If there is an issue with the event pipeline or you exceed your license limits, an event or flow might be dropped.

Dropped events and flows cannot be recovered.

User Response

Review the following options:

• Verify the incoming event and flow rates on your system. If the license is exceeded and the event pipeline is dropping events, expand your license to handle more data.

• Review the recent changes to rules or custom properties. Rule or custom property changes can cause changes to your event or flow rates and might affect system performance.

• Determine whether the issue is related to SAR notifications. SAR notifications might indicate that queued events and flows are in the event pipeline. The system usually routes events to storage, instead of dropping the events.

• Tune the system to reduce the volume of events and flows that enter the event pipeline.

Explanation

To prevent queues from filling, and to prevent the system from dropping events, the event collection system (ECS) routes data to storage. Incoming events and flows are not categorized. However, raw event and flow data is collected and searchable.

User Response

Review the following options:

• Verify the incoming event and flow rates. If the event pipeline is queuing events, expand your license to hold more data. To determine how close you are to your EPS/FPM license limit, monitor the Event Rate (Events Per Second Raw) graph on the System Monitoring dashboard. The graph shows you the current data rate. Compare the data rate to the per-appliance license configuration in your deployment.

• Review recent changes to rules or custom properties. Rule or custom property changes might cause sudden changes to your event or flow rates. Changes might affect performance or cause the system to route events to storage.

• DSM parsing issues can cause the event data to route to storage. To verify whether the log source is officially supported, see the Configuring DSMs Guide.

• SAR notifications might indicate that queued events and flows are in the event pipeline.

• Tune the system to reduce the volume of events and flows that enter the event pipeline. Events must be tuned at the source, not in the product. You can set coalescing on and configure your retention buckets to limit the number of stored events. License throttling monitors the number of incoming events to the system to manage input queues and licensing. For more information about retention buckets, see the Juniper Secure Analytics Administration Guide.

Explanation

During normal processing, custom event and custom flow properties that are marked as optimized are extracted in the pipeline during processing. The values are used in the custom rules engine (CRE) and search indexes.

Regex statements, which are improperly formed regular expressions, can cause events to be incorrectly routed directly to storage.

User Response

Select one of the following options:

• Disable any custom property that was recently installed.

• Review the payload of the notification. If possible, improve the regex statements that are associated with the custom property.

  For example, the following payload reports the regex pattern:

• Modify the custom property definition to narrow the scope of categories that the property tries to match.

• Specify a single event name in the custom property definition to prevent unnecessary attempts to parse the event.

• Order your log source parsers from the log sources with the most sent events to the least and disable unused parsers.

Explanation

The JSA Flow Processor process contains an advanced function for configuring a server IP address for time synchronization. In most cases, do not configure a value. If configured, the Flow process attempts to synchronize the time every hour with the IP address time server.

User Response

In the deployment actions, select the Flow process. Click Actions >Configure and click Advanced. In the Time Synchronization Server IP Address field, clear the value and click Save.

Explanation

Each appliance is allocated a specific volume of event and flow data from the license pool. In the last hour, the appliance exceeded the allocated EPS or FPM.

If the appliance continues to exceed the allocated capacity, the system might queue events and flows, or possibly drop the data when the backup queue fills.

User Response

• Adjust the license pool allocations to increase the EPS and FPM capacity for the appliance.

• Tune the system to reduce the volume of events and flows that enter the event pipeline.