Cluster administrators can set resource quotas on namespaces. Namespace quotas limit the resources that workloads in a namespace use. Quotas address resource management at the cluster level.
Kubernetes users might have further resource management needs within a namespace.
Users might accidentally create workloads that consume too much of the namespace quota. These unwanted workloads might prevent other workloads from running.
Users might forget to set workload limits and requests, or might find it time-consuming to configure limits and requests. When a namespace has a quota, creating workloads fails if the workload does not define values for the limits or requests in the quota.
Kubernetes introduces limit ranges to help with these issues. Limit ranges are namespaced objects that define limits for workloads within the namespace.
The following YAML file shows an example limit range:
apiVersion: v1
kind: LimitRange
metadata:
name: mem-limit-range
namespace: default
spec:
limits:
- default:
memory: 512Mi
defaultRequest:
memory: 256Mi
type: ContainerLimit ranges can specify the following limit types:
Use the default key to specify default limits for workloads.
Use the defaultRequest key to specify default requests for workloads.
Use the max key to specify the maximum value of both requests and limits.
Use the min key to specify the minimum value of both requests and limits.
The maxLimitRequestRatio key controls the relationship between limits and requests.
If you set a ratio of two, then the resource limit cannot be more than twice the request.
This course does not cover limit-to-request ratios in detail.
Limit ranges can apply to containers, pods, images, image streams, and persistent volume claims.
When you set the max key, users cannot create workloads that declare limits or that make resource requests over the maximum.
Use maximums to prevent accidentally high resource requests and limits. These situations can exhaust quotas and cause other issues.
Consider allowing users who create workloads to edit maximum limit ranges. Although maximum limit ranges act as a convenient safeguard, excessively low limits can prevent users from creating legitimate workloads.
Minimum limit ranges are useful to ensure that users create workloads with enough requests and limits. If users create such workloads often, then consider adding minimums.
Defaults are convenient in namespaces with quotas, and eliminate a need to declare limits explicitly in each workload.
When a quota is present, all workloads must specify the corresponding limits and requests.
When you set the default and defaultRequest keys, workloads use the requests and limits from the limit range by default.
Defaults are especially convenient in scenarios where many workloads are created dynamically. For example, continuous integration tools might run tests for each change to a source code repository. Each test can create multiple workloads. Because many tests can run concurrently, the resource usage of testing workloads can be significant. Setting quotas for testing workloads is often needed to limit resource usage. If you set CPU and RAM quotas for requests and limits, then the continuous integration tool must set the corresponding limits in every testing workload. Setting defaults can save time with configuring limits. However, determining appropriate defaults might be complex for namespaces with varied workloads.
Consider a namespace with the following quota:
apiVersion: v1
kind: ResourceQuota
metadata:
name: example
namespace: example
spec:
hard:
limits.cpu: "8"
limits.memory: 8Gi
requests.cpu: "4"
requests.memory: 4GiThe following command creates a deployment:
[user@host ~]$ oc create deployment example --image=image
deployment.apps/example createdThe quota prevents the deployment from creating pods:
[user@host ~]$ oc get event --sort-by .metadata.creationTimestamp
LAST SEEN TYPE REASON OBJECT MESSAGE
...output omitted...
13s Warning FailedCreate replicaset/example-74c57c8dff Error creating: pods "example-74c57c8dff-rzl7w" is forbidden: failed quota: example: must specify limits.cpu for: hello-world-nginx; limits.memory for: hello-world-nginx; requests.cpu for: hello-world-nginx; requests.memory for: hello-world-nginx
...output omitted...The following limit range includes all types of limits:
apiVersion: v1
kind: LimitRange
metadata:
name: example
namespace: example
spec:
limits:
- default:
cpu: 500m
memory: 512Mi
defaultRequest:
cpu: 250m
memory: 256Mi
max:
cpu: "1"
memory: 1Gi
min:
cpu: 125m
memory: 128Mi
type: ContainerLimit ranges do not affect existing pods.
If you delete the deployment and run the oc create command again, then the deployment creates a pod with the applied limit range.
[user@host ~]$ oc describe pod
...output omitted...
Containers:
hello-world-nginx:
Limits:
cpu: 500m
memory: 512Mi
Requests:
cpu: 250m
memory: 256Mi
...output omitted...The values correspond to the default and defaultRequest keys in the limit range.
The deployment does not contain any limits in the specification. The Kubernetes API server includes an admission controller that enforces limit ranges. The controller affects pod definitions, but not deployments, stateful sets, or other workloads.
You can replace the CPU limit, or add other resource specifications, by using the oc set resources command:
[user@host ~]$ oc set resources deployment example --limits=cpu=new-cpu-limitYou can experiment with different CPU limits.
If you request CPU values outside the range that the min and max keys define, then Kubernetes does not create the pods, and it logs warnings.
[user@host ~]$ oc get event --sort-by .metadata.creationTimestamp
LAST SEEN TYPE REASON OBJECT MESSAGE
...output omitted...
5m43s Warning FailedCreate replicaset/example-7c4dfc5fb8 Error creating: pods "example-7c4dfc5fb8-q7x94" is forbidden: maximum cpu usage per Container is 1, but limit is 1200m
...output omitted...
5m26s Warning FailedCreate replicaset/example-798d65c854 Error creating: pods "example-798d65c854-b94k8" is forbidden: minimum cpu usage per Container is 125m, but request is 100m
...output omitted...When you experiment with deployments and resource quotas, consider what happens when you modify a deployment. Modifications create a replacement replica set, and the existing replica set also continues to run until the rollout completes.
The pods of both replica sets count towards the resource quota.
If the new replica set satisfies the quota, but the combined replica sets exceed the quota, then the rollout cannot complete.
When creating a limit range, you can specify any combination of the default, defaultRequest, min, and max keys.
However, if you do not specify the default or defaultRequest keys, then Kubernetes modifies the limit range to add these keys.
These keys are copied from the min or max keys.
For more predictable behavior, always specify the default and defaultRequest keys if you specify the min or max keys.
Also, the values for CPU or memory keys must follow these rules:
The max value must be higher than or equal to the default value.
The default value must be higher than or equal to the defaultRequest value.
The defaultRequest value must be higher than or equal to the min value.
Do not create conflicting limit ranges in a namespace. For example, if two default CPU values are specified, then it would be unclear which one is applied.
For more information, refer to the Restrict Resource Consumption with Limit Ranges section in the Working with Clusters chapter in the Red Hat OpenShift Container Platform 4.14 Nodes documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/nodes/index#nodes-cluster-limit-ranges