Configure Data Summarization

Creating a Raw Data Summarization Profile

To create a raw data summarization profile that can be applied to a device group:

1. Click Settings > Summarization Profiles link in the left-nav bar.

2. Select Raw Data from the Summarization Profiles list.

   The Raw Data Summarization Profiles page is displayed.

3. Click (+) icon to add a summarization profile.

   The Add Raw Data Summarization Profile page is displayed.

4. In the Name text box, enter the name of the profile.

5. Click Add Type Aggregate to add an aggregate type.

   The Name and Function drop-down lists are displayed.

   Follow these steps to select a name data type and associate it with a data summarization algorithm.

   The algorithm configured for a specific sensor path name overrides the algorithm configured for the corresponding data type.

   1. Select a name data type from the Name drop-down list.

      The available name data types to choose from are string, integer, boolean, float, and unsigned integer.

      Starting in Paragon Insights Release 4.2.0, you can also select unsigned integer as a name data type. An unsigned integer is a data type that can contain values from 0 through 4,294,967,295.

   2. After you have selected a name data type, you associate it with a data summarization function.

      To associate a name data type with a data summarization function, select a function from the Functions drop-down list.

      The available functions to choose from are latest, count, mean, min, max, on-charge, stddev, and sum.

   3. (Optional) To add another aggregate type, click Add Type Aggregate, and repeat step 5.a and step 5.b.

6. Click Add Path Aggregate to add an aggregate path.

   The Name and Function drop-down lists are displayed.

   To assign a sensor path name and associate it with a data summarization algorithm:

   Note:

   The algorithm configured for a specific sensor path name overrides the algorithm configured for the corresponding data type.

   1. Enter a sensor path name in the Name text box.

      You can enter a path name for a sensor that is not supported by Paragon Insights. For sensors supported by Paragon Insights, the path name must be entered in the following format:

      Sensor	Path Name Format	Example
      Open Config	sensor-path	/components/component/name
      Native GPB	sensor-name:sensor-path	jnpr_qmon_ext:queue_monitor_element_info.percentage
      iAgent	yaml-table-name:sensor-path	REutilizationTable:15_min_cpu_idle
      SNMP	snmp-table-name:sensor-path	.1.3.6.1.2.1.2.2:jnxLED1Index ospfNbrTable:ospfNbrIpAddr
      Syslog	pattern-set: sensor-path	interface_link_down:operational-status
      Flow (NetFlow)	template-name:sensor-path	hb-ipfix-ipv4-template:sourceIPv4Address

   2. After you have entered a sensor path name, you associate it with a data summarization function.

      To associate a sensor path name with a data summarization function, select a function from the Functions drop-down list.

      The available functions to choose from are latest, count, mean, min, max, on-charge, stddev, and sum.

   3. (Optional) To add another aggregate path, click Add Path Aggregate, and repeat step 6.a and step 6.b.

7. Click Save to only save the configuration.

   Click Save & Deploy to save and immediately deploy the configuration.

8. You can now apply the raw data summarization profile that you created to a specific device group. For more information, see Applying Data Summarization Profiles to a Device Group.

Creating a Data Rollup Summarization Profile

Paragon Insights Release 4.0.0 supports data rollup summarization. You can create a rollup summarization profile to summarize processed data that is stored in fields in the TSDB. Field data is processed data that provides information on network devices and its components, and is stored in fields in the TSDB. A field is a single piece of information that forms a record in a database. In TSDB, multiple fields of processed data make a record. Data rollup summarization enables efficient data storage and also ensures retaining of data for a longer duration.

You can create a data rollup summarization profile to apply to a device group from the:

• Paragon Insights graphical user interface (GUI)

• Command line interface (CLI)

Create a Data Rollup Summarization Profile by using Paragon Insights UI

To create a data rollup summarization profile:

1. Click Settings > Summarization Profiles link in the left-nav bar.

2. Select Data Rollup from the Summarization Profiles list.

   The Data Rollup Summarization Profiles page is displayed.

3. Click (+) icon to add a summarization profile.

   The Add Data Rollup Summarization Profile page is displayed.

4. Enter the name of the profile in the Name text box.

   The maximum length is 64 characters.

   Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*”

5. Click Add Rule to add an existing Paragon Insights rule for which rollup summarization must be applied.

   The Name and Apply on Existing Data drop-down lists are displayed.

   Follow these steps to select a rule, and to apply the rule to the profile. You can also apply the rule to existing data.

   1. Select an existing Paragon Insights from the Name drop-down list.

   2. To apply the rule that you selected to existing data, select True from the Apply on Existing Data drop-down list.

      The default value is False.

   3. Click Add Field to define fields of the rule configured for which data rollup summarization must be applied.

      To associate a field to an aggregate function:

      1. Select a field from the Name drop-down list for which data must be aggregated.

      2. Select one or more aggregate functions from the Aggregate Function drop-down list that you want to apply to a field.

   4. (Optional) To add another rule to the profile, click Add Rule, and repeat step 5.a through step 5.c.

6. Click Add Data Rollup Order to define the frequency at which rollup summarization should occur.

   The Name and Retention Policy drop-down lists, and the Rollup Interval text box are displayed.

   To define a data rollup order:

   1. Enter a name to identify the data rollup order in the Name text box.

      The maximum length is 64 characters.

      Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*”

   2. Enter a value in the Rollup Interval text box to define the interval in which data is summarized.

      Regex pattern: “[1-9][0-9]*[mhdw]”, where m is minutes, h is hours, d is days, and w is weeks.

      Minimum value is 30m. Maximum value is 52w.

   3. Select the retention policy for the rollup order from the Retention Policy drop-down list. A retention policy defines how long you want to retain the rolled-up data.

      Selecting a retention policy is optional. If you do not select a retention policy, the device group retention policy is considered by default.

   4. (Optional) To define another data rollup order, click Add Data Rollup Order, and repeat step 6.a through step 6.c.

7. Click Save to only save the configuration.

   Click Save and Deploy to save and immediately deploy the configuration.

8. You can now apply the data rollup summarization profile that you created to a specific device group. For more information, see Applying Data Summarization Profiles to a Device Group.

Create a Data Rollup Summarization Profile by using CLI

Figure 1 is an example configuration of how you can configure a data rollup summarization profile from the CLI.

Figure 1: Example CLI Configuration of Creating a Data Rollup Summarization Profile

Applying Data Summarization Profiles to a Device Group

After you create a data summarization profile, you can apply the profile to a specific device group to start summarizing TSDB data:

1. Click Configuration > Device Group in the left-nav bar.

   The Device Group Configuration page is displayed.

2. Select the check box next to the name of the device group to which you want to apply the data summarization profile.

3. Click the Edit Device Group icon to edit the device group.

   The Edit <device-group-name> page is displayed.

4. Apply a raw data summarization profile.

   To apply a raw data summarization profile to a device group:

   1. Click Summarization.

      The Time Span and Data Summarization text boxes are displayed.

   2. Enter the Time Span in seconds (s), minutes (m), hours (h), days (d), weeks (w), or years (y).

   3. Choose the data summarization profiles from the drop-down list to apply the ingest data. To edit or view details about saved data summarization profiles, go to the Data Summarization page and click the Settings menu option in the left-nav bar.

      If you select two or more profiles, the following guidelines apply:

      • If the same data type or sensor path name is configured in two or more profiles, the associated algorithms will be combined.

      • The table that stores the summarization output includes columns of summarized data for each algorithm associated with each data field collected by Paragon Insights. The naming convention for each column is as follows:

        Number of algorithms associated with a data field	Column name for the summarized output
        1	field-name Example: 5_sec_cpu_idle
        2	field-name_first-algorithm-name, field-name_ second-algorithm-name Example: 5_sec_cpu_idle_MIN, 5_sec_cpu_idle_MAX
        3	field-name_first-algorithm-name, field-name_ second-algorithm-name, field-name_ third-algorithm-name...

   Apply a data rollup summarization profile

   Points to remember before you apply a data rollup summarization profile to a device group:

   • Ensure that the rules present in the rollup profile are already associated with the device group.

   • You can add one or more than one rollup summarization profile to a device group.

   • Rules configured across all the profiles associated with the device group must be unique.

   • While associating a rollup profile with a device group, the interval of the first data rollup order must be less then the device group retention policy to avoid data overflow. The device group retention policy is set to 7 days by default.

   • When you want to remove a rule that is associated to a device group, you must first remove the data rollup summarization profile.

   To apply a data rollup summarization profile to a device group:

   1. Click Rollup Summarization.

      The Rollup Summarization Profiles drop-down list is displayed.

   2. Select the rollup summarization profiles you want to associate to this device group from the Rollup Summarization Profiles drop-down list.

   3. (Optional) You can also deploy rollup configuration at the device group-level by using the CLI.

      See Figure 2 for an example CLI configuration.

      Figure 2: Example CLI Configuration

5. Click Save to only save the configuration.

   Click Save and Deploy to save and immediately deploy the configuration.