Manage Time Series Database Settings
You can use the Paragon Automation GUI to configure the time series database (TSDB) settings.
To configure TSDB settings:
Selecting, deleting, or dedicating TSDB nodes must be done during a maintenance window because some services will be restarted and the Paragon Automation GUI will likely be unresponsive.
Adjust Memory Allocation for TSDB Nodes
By default, all InfluxDB pods are capped at 12-GB memory. You can adjust the memory
allocation on InfluxDB during the installation of the Paragon Automation cluster, or
in the healthbot
namespace, or while adding TSDB nodes.
Choose one of the following options to increase the memory limit.
-
During Installation
During installation of your Paragon Automation cluster, manually edit the config.yml file to increase the memory limit, before you run the
deploy
command to deploy the cluster. Edit thememory_default_max
parameter to add the memory limit. For example to cap the memory to 16-GB, edit the config.yml to includememory_default_max: 16Gi
. Note that editing the config.yml will affect all pods. -
In the
healthbot
namespaceEdit the default limit on the
healthbot
namespace.-
root@ns1:~# kubectl edit limitranges -n healthbot memory-limit
Restart the InfluxDB pod for the limit to take effect.
Note that, the new memory limit will take effect on all pods (under the
healthbot
namespace), only when the pods are restarted. -
-
While adding a TSDB node
Edit the InfluxDB deployment specifications and explicitly add the resource limit, before or after adding a TSDB node.
Determine the InfluxDB pod and deployment name.
root@ns1:~# kubectl get deploy -n healthbot | grep influx influxdb-ns4 1/1 1 1 3d23h root@ns1:~# kubectl get pod -A | grep influx healthbot influxdb-ns4-678c9b9b47-zpcwz 1/1 Running 0 61s
There might be more than one InfluxDB deployments if multiple TSDB nodes are present. If there are multiple InfluxDB deployments, we must perform these steps on each deployment.
Check the current memory limit.
root@ns1:~# kubectl describe pod -n healthbot influxdb-ns4-678c9b9b47-zpcwz ... ... Containers: influxdb: Container ID: containerd://bf47b5e7cf1cf70c1dba76aa1ccd66c689fab616b130b330feb9f8e20bf4dd51 Image: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb:23.2.0-dv Image ID: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb@sha256:614750fc042d16ef2fccedc83062248e66770754f397e5284a45d1af09fd81d4 Ports: 8086/TCP, 8088/TCP Host Ports: 0/TCP, 0/TCP State: Running Started: Tue, 24 Oct 2023 16:49:51 +0000 Ready: True Restart Count: 0 Limits: cpu: 6 memory: 12Gi Requests: cpu: 20m memory: 50Mi
Edit the limit on the pod. For example, change the limit to 16-GB.
root@ns1:~# kubectl edit deploy -n healthbot influxdb-ns4 image: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb:23.2.0-dv imagePullPolicy: IfNotPresent name: influxdb resources: limits: cpu: "6" memory: 16Gi requests: cpu: 20m memory: 50Mi
Before changing the limit, the original values under the
Resources
parameter are empty. This implies that the limit will default to what is defined at thehealthbot
namespace level.... image: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb:23.2.0-dv imagePullPolicy: IfNotPresent name: influxdb ports: - containerPort: 8086 name: http protocol: TCP - containerPort: 8088 name: rpc protocol: TCP resources: {} ...
Save and exit the InfluxDB pod. The pod is automatically restarted.
Determine the new InfluxDB pod and deployment name.
root@ns1:~# kubectl get deploy -n healthbot | grep influx
root@ns1:~# kubectl get pod -A | grep influx
Verify that the edited limit is reflected in the pod.
root@ns1:~# kubectl describe pod -n healthbot influxdb-pod-name ... ... Containers: influxdb: Container ID: containerd://3b364b69021324fe423322a6e999940925e88abe5c1c2230d8ae6a4236352303 Image: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb:23.2.0-dv Image ID: paragon-registry.local/abc.example.net/healthbot-registry/ci/healthbot_influxdb@sha256:614750fc042d16ef2fccedc83062248e66770754f397e5284a45d1af09fd81d4 Ports: 8086/TCP, 8088/TCP Host Ports: 0/TCP, 0/TCP State: Running Started: Tue, 24 Oct 2023 16:52:10 +0000 Ready: True Restart Count: 0 Limits: cpu: 6 memory: 16Gi Requests: cpu: 20m memory: 50Mi