Bookmark this page

Monitoring 3Scale Infrastructure

Objectives

After completing this section, you should be able to configure monitoring and metrics collection for the 3scale API Management infrastructure.

Outlining the Monitoring Stack

The 3scale API Management monitoring stack consists of Prometheus, Grafana, and a set of configuration objects to connect all three.

Prometheus is responsible for collecting and managing the monitoring and metrics data. Grafana provides an interface and widgets for creating dashboards with the metrics.

Both the Prometheus and Grafana operators provide Custom Resource Definitions (CRDs). With monitoring enabled, the 3scale API Management operator creates resources based on these CRDs to automatically configure Prometheus and Grafana.

Enabling Monitoring

To enable the monitoring functionality within 3scale API Management, you must do so within the APImanager definition.

For example, in the following APImanager definition, the highlighted monitoring attribute instructs the 3scale API Management operator to create applicable Prometheus and Grafana resources.

apiVersion: apps.3scale.net/v1alpha1
kind: APIManager
metadata:
  name: apimanager-sample
  namespace: 3scale
spec:
  wildcardDomain: apps.ocp4.example.com
  monitoring:
    enabled: true

Collecting Data With Prometheus

Prometheus serves the role of collecting and storing metrics data, as well as providing a query language, PromQL, for the data.

There are multiple ways to run Prometheus within Red Hat OpenShift Container Platform (RHOCP). However, this course only focuses on using the Prometheus operator.

The 3scale API Management operator is designed to integrate with the Prometheus operator. Specifically, the 3scale API Management operator configures a standard set of Prometheus alerting rules and pod monitors.

Alerting rules define certain conditions that, when met, should trigger actions. For example, you might use an alert rule to send an email notification whenever CPU usage exceeds a certain threshold. Pod monitors gather metrics data from pods within the cluster.

In addition to installing the operator, you must also create a Prometheus instance by creating a resource of type Prometheus. The following is an excerpt of an example Prometheus definition:

kind: Prometheus
...output omitted...
spec:
  serviceAccountName: prometheus-k8s 1
  podMonitorSelector:
   matchExpressions:
   - key: app 2
     operator: In
     values:
     - my-app

1

The name of the Red Hat OpenShift service account that the Prometheus instance will use. This service account should have permissions to access all selected resources.

2

This expression configures Prometheus to look for pod monitors with the app=my-app label. Alert rules and service monitors are selected similarly by using the ruleSelector and podMonitorSelector attributes, respectively.

Displaying Data With Grafana

Although Prometheus can be directly queried, its display functionality is limited. Grafana can consume data directly from Prometheus and display easy-to-read and customizable graphs.

Similar to Prometheus, this course focuses on installing Grafana by using an operator. You must also create a Grafana instance by creating a resource of type Grafana. The following is an excerpt of an example Grafana definition:

kind: Grafana 1
...output omitted...
spec:
  config: 2
    ...output omitted...
  dashboardLabelSelector:
  - matchExpressions:
    - key: app 3
      operator: In
      values:
      - my-app

1

The resource type must be Grafana.

2

You must provide a configuration object which is passed to the Grafana instance.

3

This expression configures Grafana to look for dashboard resources with the app=my-app label.

For the Grafana instance to retrieve data from the Prometheus instance, you also need a resource of type GrafanaDataSource, which provides connection information. The following is an excerpt of an example Grafana data source definition:

kind: GrafanaDataSource 1
...output omitted...
spec:
  name: middleware
  datasources:
    - name: Prometheus
      url: http://prometheus-operated:9090 2
      ...output omitted...

1

The resource type must be GrafanaDataSource.

2

You must provide a connection string that resolves to the Prometheus instance.

Available Dashboards

Once you have monitoring set up, you can view the default metrics and dashboards.

The 3scale API Management operator creates seven Grafana dashboards. Each of these uses the data that Prometheus gathers from the various components within 3scale API Management.

The following are some of the default dashboards and the metrics that they display:

  • Apicast: Shows APIcast instance metrics, such as the number of error requests in the last hour. Use the environment selection menu to choose between the staging and production APIcast instances.

  • System: Shows metrics about 3scale API Management itself, such as tenants requests per second.

  • Zync: Shows metrics from the Zync component and its RHOCP pods, such as the number of Zync requests per second and CPU usage.

 

References

For more information, refer to the Monitoring 3scale chapter in the Red Hat 3scale API Management 2.11 Operating 3scale guide at https://access.redhat.com/documentation/en-us/red_hat_3scale_api_management/2.11/html/operating_3scale/monitoring-threescale

Revision: do240-2.11-40390f6