Log Source Identifier Pattern
Use the log source identifier pattern to customize the identifier that is sent with payloads when an event is posted. You can choose identifiers for your event formats and event types.
To use the full capabilities of the log source identifier pattern, you must have a basic understanding of regular expressions (regex) and how to use them.
The log source identifier pattern accepts a list of key-value pairs. A key is an identifier that is represented as a regex. The value is a regex that matches to event data. When the regex matches to the data within the event, then it posts the event with the identifier that is associated with that value.
Basic example
Key1 = value1
Key2 = value2
Key3 = value3
In this example, if an event contains a value of “value1”, the identifier for that event is key1. If an event contains a value of “value2”, the identifier is Key2. If an event contains both “value1” and “value2”, the first identifier, Key1, is matched.
In this example, you must manually create three Syslog log sources, one for each of the identifier options.
Regex example
Key1 = value1
Key2 = \d\d\d (where \d\d\d is a regex that matches to three consecutive digits.)
\1= \d(\d) (where \d(\d) is a regex that matches to two consecutive digits and captures the second digit. \1 references the first captured value in the value field.)
In this example, the following statements are true:
-
\d represents a "digit".
-
If an event contains a value of “value1”, the identifier for that event is Key1.
-
If an event contains “111” or “652” or any three consecutive digits, the identifier would be Key2.
-
If an event contains two consecutive digits such as “11’, “73”, and “24”, the identifier is \1. The regex saves the second value (“1”, “3” and “4” in the examples) for future use with the parentheses. The value that is saved by the regex is used later as the identifier. If you set the key to “\1”, the key matches to the first saved value in the regex. In this case, the identifier is not a hardcoded value, but instead it can be ten values (0 - 9.) Three identifiers are in the sample events (“1”, “3” and “4”.)
You must create a log source with the identifiers Key1 and Key2, and a log source for each possible Key1 value. In this example, for the events to go to the correct log source, you must create three Syslog log sources. For the log sources, one has the identifier set to “1”, one has the identifier set to “3” and one has the identifier set to “4”. To capture all of the possible identifiers, you need ten Syslog log sources. Each log source corresponds to a single digit.
Tips for using the Log Source Identifier Pattern
It is important to know what type of data you are receiving and how granular you need your log sources to be. Each JSA environment is unique. The following tips can help you to configure the log source identifier pattern.
Keep the data separated at the source
Most gateway supported services, such as Microsoft Azure Event Hubs and Google Pub Sub, offer ways to separate the data at the source. Keep your data in separate sources to reduce the complexity on the JSA side. For example, if you want to collect Windows Logs, Linux Logs and Audit Logs, use three separate gateway log sources to simplify the configuration. If you collect all of those logs in one source, JSA must identify the events and associate them with the correct log source.
Hardcode the regex if possible
If you hardcode the key, all of the events that match the value’s regex are collected by a single log source. This action requires less effort to create and maintain log sources, and they are easier to monitor.
Use online regex testers
Before you save and enable your log source, use online tools to test the regex.
Use the Event Retriever to determine the identifier
Use the Event Retriever to display the assigned log source identifier in JSA. You can also use it to test that your regex and identifier are matching correctly.