Creating Reference Data Collections by Using the Command Line

Use the command line to manage reference data collections that cannot be managed in JSA, such as reference maps, map of sets, map of maps, and tables. Although it's easier to manage reference sets using JSA, use the command line when you want to schedule management tasks.

Use the ReferenceDataUtil.sh script to manage reference sets and other types of reference data collections.

When you use an external file to populate the reference data collection, the first non-comment line in the file identifies the column names in the reference data collection. Each line after that is a data record that gets added to the collection. While the data type for the reference collection values is specified when the collection is created, each key is an alphanumeric string.

The following table shows examples of how to format data in an external file that is to be used for populating reference maps.

You can also create reference data collections by using the /reference_data endpoint in the JSA RESTful API.

1. Using SSH, log in to JSA as the root user.

2. Go to the /opt/qradar/bin directory.

3. To create the reference data collection, type the following command:

   ./ReferenceDataUtil.sh create name [SET | MAP | MAPOFSETS | MAPOFMAPS | REFTABLE] [ALN | NUM | IP | PORT | ALNIC | DATE] [-timeoutType=[FIRST_SEEN | LAST_SEEN]] [-timeToLive=]

4. To populate the map with data from an external file, type the following command:

   ./ReferenceDataUtil.sh load name filename [-encoding=...] [-sdf=" ... "]

Here are some examples of how to use the command line to create different types of reference data collections:

• Create an alphanumeric map:

  ./ReferenceDataUtil.sh create testALN MAP ALN

• Create a map of sets that contains port values that will age out 3 hours after they were last seen:

  ./ReferenceDataUtil.sh create testPORT MAPOFSETS PORT -timeoutType=LAST_SEEN -timeToLive='3 hours'

• Create a map of maps that contains numeric values that will age out 3 hours 15 minutes after they were first seen:

  ./ReferenceDataUtil.sh create testNUM MAPOFMAPS NUM -timeoutType=FIRST_SEEN -timeToLive='3 hours 15 minutes'

• Create a reference table where the default format is alphanumeric:

  ./ReferenceDataUtil.sh create testTable REFTABLE ALN -keyType=ipKey:IP,portKey:PORT,numKey:NUM,dateKey:DATE

Log in to JSA to create rules that add data to your reference data collections. You can also create rule tests that detect activity from elements that are in your reference data collection.