Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Event and Flow Searches

You can perform searches on the Log Activity, Network Activity, and Offenses tabs.

Use search and index options in JSA that improve search performance and return quicker results. To find specific criteria, advanced searches use AQL search strings.

You can specify filter criteria to search for events, flows, and offenses. After you perform a search, you can save the search criteria and the search results.

If your JSA administrator configured resource restrictions to set time or data limitations on event and flow searches, the resource restriction icon appears next to the search criteria.

Creating a Customized Search

You can search for data that match your criteria by using more specific search options. For example, you can specify columns for your search, which you can group and reorder to more efficiently browse your search results.

The duration of your search varies depending on the size of your database.

You can add new search options to filter through search results to find a specific event or flow that you are looking for.

The following table describes the search options that you can use to search event and flow data:

Table 1: Search Options

Options

Description

Group

Select an event search group or flow search group to view in the Available Saved Searches list.

Select an event search group to view in the Available Saved Searches list.

Type Saved Search or Select from List

Type the name of a saved search or a keyword to filter the Available Saved Searches list.

Available Saved Searches

This list displays all available searches, unless you use Group or Type Saved Search or Select from List options to apply a filter to the list. You can select a saved search on this list to display or edit.

Search

The Search icon is available in multiple panes on the search page. You can click Search when you are finished configuring the search and want to view the results.

Include in my Quick Searches

Select this check box to include this search in your Quick Search menu.

Include in my Dashboard

Select this check box to include the data from your saved search on the Dashboard tab. For more information about the Dashboard tab, see Dashboard Management.

Note:

This parameter is only displayed if the search is grouped.

Set as Default

Select this check box to set this search as your default search.

Share with Everyone

Select this check box to share this search with all other users.

Real Time (streaming)

Displays results in streaming mode.

Note:

When Real Time (streaming) is enabled, you are unable to group your search results. If you select any grouping option in the Column Definition pane, an error message opens.

Last Interval (auto refresh)

The Log Activity and Network Activity tabs are refreshed at one-minute intervals to display the most recent information.

Recent

After you select this option, you must select a time range option from the list.

Note:

The results from the last minute might not be available. Select the <Specific Interval> option if you want to see all results.

Specific Interval

After you select this option, you must select the date and time range from the Start Time and End Time calendars.

Data Accumulation

Displayed when you load a saved search.

If no data is accumulating for this saved search, the following information message is displayed: Data is not being accumulated for this search.

If data is accumulating for this saved search, the following options are displayed:

When you click or hover your mouse over the column link, a list of the columns that are accumulating data opens.

Use the Enable Unique Counts/Disable Unique Counts link to display unique event and flow counts instead of average counts over time. After you click the Enable Unique Counts link, a dialog box opens and indicates which saved searches and reports share the accumulated data.

Current Filters

Displays the filters that are applied to this search.

Save results when the search is complete

Saves the search results.

Display

Species a predefined column that is set to display in the search results.

Name

The name of your custom column layout.

Save Column Layout

Saves a custom column layout that you modified.

Delete Column Layout

Deletes a saved custom column layout.

Type Column or Select from List

Filter the columns that are listed in the Available Columns list.

For example, type Device to display a list of columns that include Device in the column name.

Available Columns

Columns that are currently in use for this saved search are highlighted and displayed in the Columns list.

Add and remove column arrows (top set)

Use the top set of arrows to customize the Group By list.

  • To add a column, select one or more columns from the Available Columns list and click the right arrow.

  • To remove a column, select one or more columns from the Group By list and click the left arrow.

Add and remove column arrows (bottom set)

Use the bottom set of arrows to customize the Columns list.

  • To add a column, select one or more columns from the Available Columns list and click the right arrow.

  • To remove a column, select one or more columns from the Columns list and click the left arrow.

Group By

Specifies the columns on which the saved search groups the results.

  • To move a column up the priority list, select a column and click the up arrow. You can also drag the column up the list.

  • To move a column down the priority list, select a column and click the down arrow. You can also drag the column down the list.

The priority list specifies in which order the results are grouped. The search results are grouped by the first column in the Group By list and then grouped by the next column on the list.

Columns

Specifies columns that are chosen for the search. You can select more columns from the Available Columns list. You can further customize the Columns list by using the following options:

  • To move a column up the priority list, select a column and click the up arrow. You can also drag the column up the list.

  • To move a column down the priority list, select a column and click the down arrow. You can also drag the column down the list.

If the column type is numeric or time-based and an entry is in the Group By list, then the column includes a list. Use the list to choose how you want to group the column.

If the column type is group, the column includes a list to choose how many levels that you want to include for the group.

Move columns between the Group By list and the Columns list

Move columns between the Group By list and the Columns list by selecting a column in one list and dragging it to the other.

Order By

From the first list, select the column by which you want to sort the search results. Then, from the second list, select the order that you want to display for the search results.

Results Limit

Specifies the number of rows that a search returns on the Edit Search window. The Results Limit field also appears on the Results window.

  • For a saved search, the limit is stored in the saved search and re-applied when search is loaded.

  • When you are sorting a column in the search result that has a row limit, sorting is done within the limited rows, which are shown in the data grid.

  • For a grouped by search where time series chart is turned on, the row limit applies only to the data grid. The Top N list in the time series chart controls how many time series are drawn in the chart.

  1. Choose a search option:

    • To search events, click the Log Activity tab.

    • To search flows, click the Network Activity tab.

  2. From the Search list, select New Search.

  3. Select a previously saved search.

  4. To create a search, in the Time Range pane, select the options for the time range that you want to capture for this search.

    Note:

    The time range that you select might impact performance, when the time range is large.

  5. Enable unique counts in the Data Accumulation pane.

    Note:

    Enabling unique counts on accumulated data, which is shared with many other saved searches and reports might decrease system performance.

  6. In the Search Parameters pane, define your search criteria.

    1. From the first list, select a parameter that you want to search for.

    2. From the second list, select the modifier that you want to use for the search.

      Note:

      To search for an event or flow whose custom property does not have a value, use the is N/A operator. To search for an event or flow whose custom property has a value, use the is not N/A operator.

    3. From the entry field, type specific information that is related to your search parameter.

    4. Click Add Filter.

    5. Repeat these steps for each filter that you are adding to the search criteria.

  7. To automatically save the search results when the search is complete, select the Save results when search is complete check box, and then type a name for the saved search.

  8. In the Column Definition pane, define the columns and column layout that you want to use to view the results:

    1. From the Display list, select the preconfigured column that is set to associate with this search.

    2. Click the arrow next to Advanced View Definition to display advanced search parameters.

    3. Customize the columns to display in the search results.

    4. In the Results Limit field, type the number of rows that you want the search to return.

    Tip:

    If you configure a log source that belongs to multiple log source groups but has only one event that matches your search criteria, the search generates results for each log source group (including the parent group) that the event belongs to. This is expected behavior.

  9. Click Filter.

Creating a Custom Column Layout

Create a custom column layout by adding or removing columns in an existing layout.

  1. On the Log Activity or the Network Activity tab, click Search >Edit Search.

  2. In the Column Definition pane, select an existing column layout in the Display list.

    When you modify the layout, the name in the Display list is automatically changed to Custom.

  3. Modify your search grouping.

    1. To add a column to your search group, select a column from the Available Columns list and click the right arrow to move the column to the Group By list.

    2. To move a column from the Columns list to your search group, select a column from the Columns list and drag it to the Group By list.

    3. To remove a column from your search group, select the column from the Group By list and click the left arrow.

    4. To change the order of your column groupings, use the up and down arrows or drag the columns into place.

  4. Modify your column layout.

    1. To add a column to your custom layout, select a column from the Available Columns list and click the right arrow to move the column to the Columns list.

    2. To move a column from the Group By list to your custom layout, select a column from the Group By list and drag it to the Columns list.

    3. To remove a column from your custom layout, select the column from the Columns list and click the left arrow.

    4. To change the order of your columns, use the up and down arrows or drag the columns into place.

  5. In the Name field, enter the name of your custom column layout.

  6. Click Save Column Layout.

Deleting a Custom Column Layout

You can delete an existing user-created column layout.

  1. On the Log Activity or the Network Activity tab, click Search >Edit Search.

  2. In the Column Definition pane, select an existing user-created column layout in the Display list.

  3. Click Delete Column Layout.

Querying with Dynamic Search

Use the dynamic search API to search for data that involves aggregated functions, such as COUNT, SUM, MAX, and AVG. For example, you can count the number of asset IDs per asset hostname by using the COUNT_PER function.

You can build your query on the following data sources:

  • Assets

  • Offenses

  • Vulninstances

You can add a field without a function as a simple field, or you can add a field with a function as a complex field to build columns. You can also add conditions to filter your data.

  1. Click the Admin tab.

  2. In the Dynamic Search section, click Dynamic Search.

  3. Select a Data Source.

  4. Complete the Available Columns and Available Filters sections.

  5. To add a name, description, range of the search, retention period, or search type to your query, enable one or more Extra Search Properties.

  6. To copy your JSON script, click Generate JSON.

    Your results appear in the JSON generated by your query section. Click Copy to Clipboard to copy your JSON script.

  7. To reset your selections, click Reset.

  8. Click Run Query.

The results of your query are listed in plain text or link format. For example, if you chose to query the ASSET_ID field, you can click the results to view the Asset Summary window for each asset ID.

Saving Search Criteria

You can save configured search criteria so that you can reuse the criteria and use the saved search criteria in other components, such as reports. Saved search criteria does not expire.

If you specify a time range for your search, then your search name is appended with the specified time range. For example, a saved search named Exploits by Source with a time range of Last 5 minutes becomes Exploits by Source - Last 5 minutes.

If you change a column set in a previously saved search, and then save the search criteria using the same name, previous accumulations for time series charts are lost.

  1. Choose one of the following options:

    • Click the Log Activity tab.

    • Click the Network Activity tab.

  2. Click the Log Activity tab.

  3. Perform a search.

  4. Click Save Criteria.

  5. Enter values for the parameters:

    Option

    Description

    Parameter

    Description

    Search Name

    Type the unique name that you want to assign to this search criteria.

    Assign Search to Group(s)

    Select the check box for the group you want to assign this saved search. If you do not select a group, this saved search is assigned to the Other group by default. For more information, see Managing Search Groups.

    Manage Groups

    Click Manage Groups to manage search groups. For more information, see Managing Search Groups.

    Timespan options:

    Choose one of the following options:

    • Real Time (streaming) - Select this option to filter your search results while in streaming mode.

    • Last Interval (auto refresh) Select this option to filter your search results while in auto-refresh mode. The Log Activity and Network Activity tabs refreshes at one-minute intervals to display the most recent information.

    • Last Interval (auto refresh) Select this option to filter your search results while in auto-refresh mode. The Log Activity and Network Activity tabs refreshes at one-minute intervals to display the most recent information.

    • Recent Select this option and, from this list box, select the time range that you want to filter for.

    • Specific Interval- Select this option and, from the calendar, select the date and time range you want to filter for.

    Include in my Quick Searches

    Select this check box to include this search in your Quick Search list box on the toolbar.

    Include in my Dashboard

    Select this check box to include the data from your saved search on the Dashboard tab. For more information about the Dashboard tab, see Dashboard Management.

    Note:

    This parameter is only displayed if the search is grouped.

    Set as Default

    Select this check box to set this search as your default search.

    Share with Everyone

    Select this check box to share these search requirements with all users.

  6. Click OK.

Scheduled Search

Use the Scheduled search option to schedule a search and view the results.

You can schedule a search that runs at a specific time of day or night.

If you schedule a search to run in the night, you can investigate in the morning. Unlike reports, you have the option of grouping the search results and investigating further. You can search on number of failed logins in your network group. If the result is typically 10 and the result of the search is 100, you can group the search results for easier investigating. To see which user has the most failed logins, you can group by user name. You can continue to investigate further.

You can schedule a search on events or flows from the Reports tab. You must select a previously saved set of search criteria for scheduling.

  1. Create a report

    Specify the following information in the Report Wizard window:

    • The chart type is Events/Logs or Flows.

    • The report is based on a saved search.

    • Generate an offense.

      You can choose the create an individual offense option or the add result to an existing offense option.

      You can also generate a manual search.

  2. View search results

You can view the results of your scheduled search from the Offenses tab.

  • Scheduled search offenses are identified by the Offense Type column.

    If you create an individual offense, an offense is generated each time that the report is run. If you add the saved search result to an existing offense, an offense is created the first time that the report runs. Subsequent report runs append to this offense. If no results are returned, the system does not append or create an offense.

  • To view the most recent search result in the Offense Summary window, double-click a scheduled search offense in the offense list. To view the list of all scheduled search runs, click Search Results in the Last 5 Search Results pane.

You can assign a Scheduled search offense to a user.

Quick Filter Search Options

Search event and flow payloads by typing a text search string that uses simple words or phrases.

Quick filter is one of the fastest methods that you use to search for event or flow payloads for specific data. For example, you can use quick filter to find these types of information:

  • Every firewall device that is assigned to a specific address range in the past week

  • A series of PDF files that were sent by a Gmail account in the past five days

  • All records in a two-month period that exactly match a hyphenated user name

  • A list of website addresses that end in .ca

You can filter your searches from these locations:

  • Log Activity toolbar and Network Activity toolbars--Select Quick Filter from the list box on the Search toolbar to type a text search string. Click the Quick Filter icon to apply your Quick Filter to the list of events or flows.

  • Add Filter Dialog box--Click the Add Filter icon on the Log Activity or Network Activity tab.Select Quick Filter as your filter parameter and type a text search string.

  • Flow search pages --Add a quick filter to your list of filters.

Note:

Note: Quick Filter searches that use a time frame outside of the Payload Index Retention setting can trigger slow and resource-intensive system responses. For example, if the payload index retention is set for 1 day, and you use a time frame for the last 30 hours in the search.

When you view flows in real-time (streaming) or last interval mode, you can type only simple words or phrases in the Quick Filter field. When you view events or flows in a time-range, follow these syntax guidelines:

Table 2: Quick Filter Syntax Guidelines

Description

Example

Include any plain text that you expect to find in the payload.

Firewall

Search for exact phrases by including multiple terms in double quotation marks.

“Firewall deny"

Include single and multiple character wildcards. The search term cannot start with a wildcard.

F?rewall or F??ew*

Group terms with logical expressions, such as AND, OR, and NOT. To be recognized as logical expressions and not as search terms, the syntax and operators must be uppercase.

(%PIX* AND ("Accessed URL" OR "Deny udp src") AND 10.100.100.*)

When you create search criteria that includes the NOT logical expression, you must include at least one other logical expression type, otherwise, no results are returned.

(%PIX* AND ("Accessed URL" OR "Deny udp src") NOT 10.100.100.*)

Precede the following characters by a backslash to indicate that the character is part of your search term: + - && || ! () {} [] ^ " ~ * ? : \.

"%PIX\-5\-304001"

Limitations

Quick filter searches operate on raw event or flow log data and don't distinguish between the fields. For example, quick filter searches return matches for both source IP address and destination IP address, unless you include terms that can narrow the results.

Search terms are matched in sequence from the first character in the payload word or phrase. The search term user matches user_1 and user_2, but does not match the following phrases: ruser, myuser, or anyuser.

Quick filter searches use the English locale. Locale is a setting that identifies language or geography and determines formatting conventions such as collation, case conversion, character classification, the language of messages, date and time representation, and numeric representation.

The locale is set by your operating system. You can configure JSA to override the operating system locale setting. For example, you can set the locale to English and the JSA console can be set to Italiano (Italian).

If you use Unicode characters in your quick filter search query, unexpected search results might be returned.

If you choose a locale that is not English, you can use the Advanced search option in JSA for searching event and payload data.

How Does Quick Filter Search and Payload Tokens Work?

Text that is in the payload is split into words, phrases, symbols, or other elements. These tokens are delimited by space and punctuation. The tokens don't always match user-specified search terms, which cause some search terms not to be found when they don't match the generated token. The delimiter characters are discarded but exceptions exist such as the following exceptions:

  • Periods that are not followed by white space are included as part of the token.

    For example, 1.2.3.4:56 is tokenized as host token 1.2.3.4 and port token 56.

  • Words are split at hyphens, unless the word contains a number, in which case, the token is not split and the numbers and hyphens are retained as one token.

  • Internet domain names and email addresses are preserved as a single token.

    1.2.3.4/home/www is tokenized as one token and the URL is not separated.

    1.2.3.7:/calling1/www2/scp4/path5/fff is tokenized as host 1.2.3.7 and the remainder is one token /calling1/www2/scp4/path5/fff

File names and URL names that contain more than one underscore are split before a period (.).

Example of multiple underscores in a file name:

If you use hurricane_katrina_ladm118.jpg as a search term, it is split into the following tokens:

  • hurricane

  • katrina_ladm118.jpg

Search the payload for the full search term by placing double quotation marks around the search term: "hurricane_katrina_ladm118.jpg"

Example of multiple underscores in a relative file path:

The thumb.ladm1180830/thumb.ladm11808301806.hurricane_katrina_ladm118.jpg is split into the following tokens:

  • thumb.ladm1180830/thumb.ladm11808301806.hurricane

  • katrina_ladm118.jpg

To search for hurricane_katrina_ladm118.jpg, which consists of one partial and one full token, place an asterisk in front of the query term, *hurricane_katrina_ladm118.jpg

Advanced Search Options

Use the Advanced Search field to enter an Ariel Query Language (AQL) that specifies the fields that you want and how you want to group them to run a query.

Note:

When you type an AQL query, use single quotation marks for a string comparison, and use double quotation marks for a property value comparison.

The Advanced Search field has auto completion and syntax highlighting.

Use auto completion and syntax highlighting to help create queries. For information about supported web browsers, see Supported Web Browsers

Note:

If you use a quick filter on the Log Activity tab, you must refresh your browser window before you run an advanced search.

Accessing Advanced Search

Access the Advanced Search option from the Search toolbar that is on the Network Activity and Log Activity tabs to type an AQL query.

Access the Advanced Search option from the Search toolbar that is on the Log Activity tab to type an AQL query.

Select Advanced Search from the list box on the Search toolbar.

Expand the Advanced Search field by following these steps:

  1. Drag the expand icon that is at the right of the field.

  2. Press Shift + Enter to go to the next line.

  3. Press Enter.

You can right-click any value in the search result and filter on that value.

Double-click any row in the search result to see more detail.

All searches, including AQL searches, are included in the audit log.

AQL Search String Examples

The following table provides examples of AQL search strings.

Table 3: Examples Of AQL Search Strings

Description

Example

Select default columns from events.

Select default columns from flows.

SELECT * FROM events

SELECT * FROM flows

Select default columns from events.

SELECT * FROM events

Select specific columns.

SELECT sourceip, destinationip FROM events

Select specific columns and order the results.

SELECT sourceip, destinationip FROM events ORDER BY destinationip

Run an aggregated search query.

SELECT sourceip, SUM(magnitude) AS magsum FROM events GROUP BY sourceip

Run a function call in a SELECT clause.

SELECT CATEGORYNAME(category) AS namedCategory FROM events

Filter the search results by using a WHERE clause.

SELECT CATEGORYNAME(category) AS namedCategory, magnitude FROM events WHERE magnitude > 1

Search for events that triggered a specific rule, which is based on the rule name or partial text in the rule name.

SELECT LOGSOURCENAME(logsourceid), * from events where RULENAME(creeventlist) ILIKE '%suspicious%'

Reference field names that contain special characters, such as arithmetic characters or spaces, by enclosing the field name in double quotation marks.

SELECT sourceip, destinationip, "+field/name+" FROM events WHERE "+field/name+" LIKE '%test%'

The following table provides examples of AQL search strings for X-Force.

Table 4: Examples Of AQL Search Strings for X-Force

Description

Example

Check an IP address against an X-Force category with a confidence value.

select * from events where XFORCE_IP_CONFIDENCE('Spam',sourceip)>3

Search for X-Force URL categories associated with a URL.

select url, XFORCE_URL_CATEGORY(url) as myCategories from events where XFORCE_URL_CATEGORY(url) IS NOT NULL

Retrieve X-Force IP categories that are associated with an IP.

select sourceip, XFORCE_IP_CATEGORY(sourceip) as IPcategories from events where XFORCE_IP_CATEGORY(sourceip) IS NOT NULL

For more information about functions, search fields and operators, see the Juniper Secure Analytics Ariel Query Language guide.

AQL Search String Examples

Use the Ariel Query Language (AQL) to retrieve specific fields from the events, flows, and simarc tables in the Ariel database.

Note:

When you build an AQL query, if you copy text that contains single quotation marks from any document and paste the text into JSA, your query will not parse. As a workaround, you can paste the text into JSA and retype the single quotation marks.

Reporting Account Usage

Different user communities can have different threat and usage indicators.

Use reference data to report on several user properties, for example, department, location, or manager. You can use external reference data.

The following query returns metadata information about the user from their login events.

Insight Across Multiple Account Identifiers

In this example, individual users have multiple accounts across the network. The organization requires a single view of a users activity.

Use reference data to map local user IDs to a global ID.

The following query returns the user accounts that are used by a global ID on events that are flagged as suspicious.

The following query shows the activities that are completed by a global ID.

Identify Suspicious Long-term Beaconing

Many threats use command and control to communicate periodically over days, weeks, and months.

Advanced searches can identify connection patterns over time. For example, you can query consistent, short, low volume, number of connections per day/week/month between IP addresses, or an IP address and geographical location.

The following query detects potential instances of hourly beaconing.

Tip:

You can modify this query to work on proxy logs and other event types.

The following query detects potential instances of daily beaconing.

The following query detects daily beaconing between a source IP and a destination IP. The beaconing times are not at the same time each day. The time lapse between beacons is short.

The following query detects daily beaconing to a domain by using proxy log events. The beaconing times are not at the same time each day. The time lapse between beacons is short.

The url_domain property is a custom property from proxy logs.

External Threat Intelligence

Usage and security data that is correlated with external threat intelligence data can provide important threat indicators.

Advanced searches can cross-reference external threat intelligence indicators with other security events and usage data.

This query shows how you can profile external threat data over many days, weeks, or months to identify and prioritize the risk level of assets and accounts.

Asset Intelligence and Configuration

Threat and usage indicators vary by asset type, operating system, vulnerability posture, server type, classification, and other parameters.

In this query, advanced searches and the asset model provide operational insight into a location.

The Assetproperty function retrieves property values from assets, which enables you to include asset data in the results.

The following query shows how you can use advanced searches and user identity tracking in the asset model.

The AssetUser function retrieves the user name from the asset database.

Network LOOKUP Function

You can use the Network LOOKUP function to retrieve the network name that is associated with an IP address.

Rule LOOKUP Function

You can use the Rule LOOKUP function to retrieve the name of a rule by its ID.

The following query returns events that triggered a specific rule name.

Full TEXT SEARCH

You can use the TEXT SEARCH operator to do full text searches by using the Advanced search option.

In this example, there are a number of events that contain the word "firewall" in the payload. You can search for these events by using the Quick filter option and the Advanced search option on the Log Activity tab.

  • To use the Quick filter option, type the following text in the Quick filter box: 'firewall'

  • To use the Advanced search option, type the following query in the Advanced search box:

Custom Property

You can access custom properties for events and flows when you use the Advanced search option.

The following query uses the custom property "MyWebsiteUrl" to sort events by a particular web URL:

SELECT "MyWebsiteUrl", * FROM events ORDER BY "MyWebsiteUrl"

Converting a Saved Search to an AQL String

Convert a saved search to an AQL string and modify it to create your own searches to quickly find the data you want. Now you can create searches faster than by typing the search criteria. You can also save the search for future use.

  1. Click the Log Activity or Network Activity tab.

  2. From the Search list, select New Search or Edit Search.

  3. Select a previously saved search.

  4. Click Show AQL.

  5. From the AQL window, click Copy to Clipboard.

  6. In the Search Mode section, click Advanced Search.

  7. Paste the AQL string text into the Advanced Search text box.

  8. Modify the string to include the data you want to find.

  9. Click Search to display the results.

Related Documentation