Bookmark this page

Per-Project Resource Constraints: Limit Ranges

Objectives

  • Configure default and maximum compute resource requirements for pods per project.

Managing Namespace Resources

Cluster administrators can set resource quotas on namespaces. Namespace quotas limit the resources that workloads in a namespace use. Quotas address resource management at the cluster level.

Kubernetes users might have further resource management needs within a namespace.

  • Users might accidentally create workloads that consume too much of the namespace quota. These unwanted workloads might prevent other workloads from running.

  • Users might forget to set workload limits and requests, or might find it time-consuming to configure limits and requests. When a namespace has a quota, creating workloads fails if the workload does not define values for the limits or requests in the quota.

Kubernetes introduces limit ranges to help with these issues. Limit ranges are namespaced objects that define limits for workloads within the namespace.

Limit Ranges

The following YAML file shows an example limit range:

apiVersion: v1
kind: LimitRange
metadata:
  name: mem-limit-range
  namespace: default
spec:
  limits:
    - default:
        memory: 512Mi
      defaultRequest:
        memory: 256Mi
      type: Container

Limit ranges can specify the following limit types:

Default limit

Use the default key to specify default limits for workloads.

Default request

Use the defaultRequest key to specify default requests for workloads.

Maximum

Use the max key to specify the maximum value of both requests and limits.

Minimum

Use the min key to specify the minimum value of both requests and limits.

Limit-to-request ratio

The maxLimitRequestRatio key controls the relationship between limits and requests. If you set a ratio of two, then the resource limit cannot be more than twice the request.

This course does not cover limit-to-request ratios in detail.

Limit ranges can apply to containers, pods, images, image streams, and persistent volume claims.

Setting Maximum and Minimum Limit Ranges

When you set the max key, users cannot create workloads that declare limits or that make resource requests over the maximum.

Use maximums to prevent accidentally high resource requests and limits. These situations can exhaust quotas and cause other issues.

Consider allowing users who create workloads to edit maximum limit ranges. Although maximum limit ranges act as a convenient safeguard, excessively low limits can prevent users from creating legitimate workloads.

Minimum limit ranges are useful to ensure that users create workloads with enough requests and limits. If users create such workloads often, then consider adding minimums.

Setting Defaults

Defaults are convenient in namespaces with quotas, and eliminate a need to declare limits explicitly in each workload. When a quota is present, all workloads must specify the corresponding limits and requests. When you set the default and defaultRequest keys, workloads use the requests and limits from the limit range by default.

Defaults are especially convenient in scenarios where many workloads are created dynamically. For example, continuous integration tools might run tests for each change to a source code repository. Each test can create multiple workloads. Because many tests can run concurrently, the resource usage of testing workloads can be significant. Setting quotas for testing workloads is often needed to limit resource usage. If you set CPU and RAM quotas for requests and limits, then the continuous integration tool must set the corresponding limits in every testing workload. Setting defaults can save time with configuring limits. However, determining appropriate defaults might be complex for namespaces with varied workloads.

Creating Limit Ranges

Consider a namespace with the following quota:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: example
  namespace: example
spec:
  hard:
    limits.cpu: "8"
    limits.memory: 8Gi
    requests.cpu: "4"
    requests.memory: 4Gi

The following command creates a deployment:

[user@host ~]$ oc create deployment example --image=image
deployment.apps/example created

The quota prevents the deployment from creating pods:

[user@host ~]$ oc get event --sort-by .metadata.creationTimestamp
LAST SEEN   TYPE      REASON              OBJECT                          MESSAGE
...output omitted...
13s         Warning   FailedCreate        replicaset/example-74c57c8dff   Error creating: pods "example-74c57c8dff-rzl7w" is forbidden: failed quota: example: must specify limits.cpu for: hello-world-nginx; limits.memory for: hello-world-nginx; requests.cpu for: hello-world-nginx; requests.memory for: hello-world-nginx
...output omitted...

The following limit range includes all types of limits:

apiVersion: v1
kind: LimitRange
metadata:
  name: example
  namespace: example
spec:
  limits:
  - default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 250m
      memory: 256Mi
    max:
      cpu: "1"
      memory: 1Gi
    min:
      cpu: 125m
      memory: 128Mi
    type: Container

Limit ranges do not affect existing pods. If you delete the deployment and run the oc create command again, then the deployment creates a pod with the applied limit range.

[user@host ~]$ oc describe pod
...output omitted...
Containers:
  hello-world-nginx:
    Limits:
      cpu:     500m
      memory:  512Mi
    Requests:
      cpu:        250m
      memory:     256Mi
...output omitted...

The values correspond to the default and defaultRequest keys in the limit range.

The deployment does not contain any limits in the specification. The Kubernetes API server includes an admission controller that enforces limit ranges. The controller affects pod definitions, but not deployments, stateful sets, or other workloads.

You can replace the CPU limit, or add other resource specifications, by using the oc set resources command:

[user@host ~]$ oc set resources deployment example --limits=cpu=new-cpu-limit

You can experiment with different CPU limits.

If you request CPU values outside the range that the min and max keys define, then Kubernetes does not create the pods, and it logs warnings.

[user@host ~]$ oc get event --sort-by .metadata.creationTimestamp
LAST SEEN   TYPE      REASON              OBJECT                          MESSAGE
...output omitted...
5m43s       Warning   FailedCreate        replicaset/example-7c4dfc5fb8   Error creating: pods "example-7c4dfc5fb8-q7x94" is forbidden: maximum cpu usage per Container is 1, but limit is 1200m
...output omitted...
5m26s       Warning   FailedCreate        replicaset/example-798d65c854   Error creating: pods "example-798d65c854-b94k8" is forbidden: minimum cpu usage per Container is 125m, but request is 100m
...output omitted...

Note

When you experiment with deployments and resource quotas, consider what happens when you modify a deployment. Modifications create a replacement replica set, and the existing replica set also continues to run until the rollout completes.

The pods of both replica sets count towards the resource quota.

If the new replica set satisfies the quota, but the combined replica sets exceed the quota, then the rollout cannot complete.

When creating a limit range, you can specify any combination of the default, defaultRequest, min, and max keys. However, if you do not specify the default or defaultRequest keys, then Kubernetes modifies the limit range to add these keys. These keys are copied from the min or max keys. For more predictable behavior, always specify the default and defaultRequest keys if you specify the min or max keys.

Also, the values for CPU or memory keys must follow these rules:

  • The max value must be higher than or equal to the default value.

  • The default value must be higher than or equal to the defaultRequest value.

  • The defaultRequest value must be higher than or equal to the min value.

Do not create conflicting limit ranges in a namespace. For example, if two default CPU values are specified, then it would be unclear which one is applied.

References

For more information, refer to the Restrict Resource Consumption with Limit Ranges section in the Working with Clusters chapter in the Red Hat OpenShift Container Platform 4.14 Nodes documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/nodes/index#nodes-cluster-limit-ranges

Limit Ranges

Revision: do280-4.14-08d11e1