The default monitoring stack is included with OpenShift Container Platform. All components of the default monitoring stack are installed in the openshift-monitoring project and provide monitoring features for core platform components.

Modifying any existing resource and creating additional ServiceMonitor, PodMonitor, or PrometheusRule resources in the openshift-monitoring project is not supported. The monitoring stack resets modified resources to ensure that its resources always remain in the expected state.

The monitoring stack deploys the following components in the environment for monitoring the infrastructure, receiving alerts, and consulting performance graphs.

You can enable monitoring for user-defined projects. You can collect application-specific custom metrics and also create alerts by using custom metrics with this monitoring stack. The monitoring components for user-defined projects are installed in the openshift-user-workload-monitoring project.

The monitoring stack for user-defined projects deploys the following components:

For more details about the OpenShift monitoring stack for user-defined projects, see the References section.

You access cluster alerts from the OpenShift web console at Observe → Alerting. For each alert, the Alerting page displays a brief description, the state, and the severity. You can view alert details by clicking the name of the alert. The Alert details page also displays a time-series graphic.

Figure 6.2: Cluster monitoring alerting

For more details about forwarding alerts to other systems, see the next section.

OpenShift integrates Prometheus metrics at Observe → Metrics.

From the Metrics page, enter an expression, such as a metric name, and then click Run Queries to retrieve the most recent sample for the metric.

The following example displays the instance:node_cpu_utilisation:rate1m metric over time.

Figure 6.3: Monitoring metrics

The metric contains data for each node instance in the cluster.

The OpenShift has three monitoring stack components to gather the metrics from the Kubernetes API: the kube-state-metrics, openshift-state-metrics, and node-exporter agents.

The dashboards in OpenShift cluster monitoring combine metrics from the three agents.

See the References section to learn about the complete list of exposed metrics.

Prometheus provides a query language, PromQL, to select and aggregate time-series data.

You can filter a metric to include only certain key/value pairs. For example, you can modify the previous query to show only metrics for the worker02 node by using the following expression:

instance:node_cpu_utilisation:rate1m{instance="worker02"}

Prometheus Query Language provides several operators to compute new time-series metrics. PromQL contains arithmetic operators, including addition, subtraction, multiplication, and division operators. PromQL contains comparison operators, including equality, greater-than, and less-than operators.

PromQL contains built-in functions, including the following ones, that you can include in PromQL expressions:

The following examples of Prometheus Query Language expressions use one metric from the node-exporter agent, and another metric from the kube-state-metrics agent:

The Red Hat add-on operators can define extra metrics and alerts. For example, the compliance operator exposes additional metrics to Prometheus. You can get a list of exposed metrics by using the following query:

{name=~"compliance.*"}

The OpenShift console integrates dashboards based on the gathered metrics at Observe → Dashboards. These dashboards refresh periodically to display current summary metrics and graphs.

In the graphs, which are interactive, you can further explore data features and characteristics that you observe.

The cluster monitoring dashboards serve as a good starting point for near real-time observability of cluster metrics and health.

After receiving an alert, an administrator might use the dashboards to investigate the problem. This investigation might include determining whether a specific node or project has a problem. Additionally, cluster monitoring dashboards can help identify whether a problem was temporary or appears to be persistent.

OpenShift cluster monitoring includes several default dashboards.

Figure 6.4: Monitoring dashboards

Some of the default monitoring default dashboards are as follows:

Figure 6.5: Kubernetes / Compute Resources / Cluster dashboard

You can see metrics by clicking Inspect for each parameter. Clicking Inspect shows the Metrics page where you can see metrics and a related graph.

For example, clicking Inspect for CPU Utilisation shows a graph and values for the following metrics:

cluster:node_cpu:ratio_rate5m{cluster=""}

Figure 6.6: Metrics CPU utilisation

The Kubernetes / Compute Resources / Cluster dashboard page also shows graphs for CPU, memory, and network, such as CPU Usage, CPU Quota, Memory Usage, and Memory Quota.

These graphs are common to some dashboard pages. The only difference is data filtration. For example, the Kubernetes / Compute Resources / Namespace (Workloads) dashboard filters resource usage, first by namespace and then by workload type, such as by deployment, daemon set, and stateful set.