Red Hat OpenShift Administration II: Configuring a Production Cluster
Kubernetes manages a pod network and a service network. The pod network provides network interfaces to each pod, and by default, provides network communication between all pods. The service network provides stable addressing for services that run on pods. Furthermore, other facilities provide mechanisms to expose services outside the cluster.
However, in some cases, connecting some pods to a different network can provide benefits or help to address requirements.
For example, using a dedicated network with dedicated resources can improve the performance of specific traffic. Additionally, a dedicated network can have different security properties from the default network and help to achieve security requirements.
In addition to these advantages, using extra interfaces can also simplify some tasks, such as controlling outgoing traffic from pods.
The Multus CNI (container network interface) plug-in helps to attach pods to custom networks.
These custom networks can be either existing networks outside the cluster, or custom networks that are internal to the cluster.
To use existing custom networks, first you must make available the network on cluster nodes.
You can use operators, such as the Kubernetes NMState operator or the SR-IOV (Single Root I/O Virtualization) network operator, to customize node network configuration. With these operators, you define custom resources to describe the intended network configuration, and the operator applies the configuration.
The SR-IOV network operator configures SR-IOV network devices for improved bandwidth and latency on certain platforms and devices.
To configure secondary networks, create a NetworkAttachmentDefinition resource.
Alternatively, update the configuration of the cluster network operator to add a secondary network.
Some network attachment definitions create and manage virtual network devices, including virtual bridges.
The virtual network devices attach to existing networks that are configured and managed outside OpenShift.
Other network attachment definitions use existing network interfaces on the cluster nodes.
Network attachment definitions can also perform additional network configuration.
Network attachment resources are namespaced, and are available only to pods in their namespace.
When the cluster has additional networks, you can add the k8s.v1.cni.cncf.io/networks annotation to the pod's template to use one of the additional networks.
The value of the annotation is the name of the network attachment definition to use, or a list of maps with additional configuration options.
Besides network attachments, you can also add pods to networks that the SR-IOV network operator configures.
For example, the following deployment uses the example network:
apiVersion: apps/v1
kind: Deployment
metadata:
name: example
namespace: example
spec:
selector:
matchLabels:
app: example
name: example
template:
metadata:
annotations:
k8s.v1.cni.cncf.io/networks: example
labels:
app: example
name: example
spec:
...output omitted...Multus updates the k8s.v1.cni.cncf.io/networks-status annotation with the status of the additional networks.
[user@host ~]$oc get pod example \ -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/networks-status}'[{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.8.0.59" ], "mac": "0a:58:0a:08:00:3b", "default": true, "dns": {} },{"name": "non-http-multus/example",
"interface": "net1","ips": [ "1.2.3.4" ], "mac": "52:54:00:01:33:0a", "dns": {} }]
| The |
| To access the custom network, Multus creates a network interface in the pod.
Multus uses the |
Note
The period is the JSONPath field access operator.
Normally, you use the period to access parts of the resource, such as in the .metadata.annotations JSONPath expression.
To access fields that contain periods with JSONPath, you must escape the periods with a backslash (\).
You can create network attachment definitions of the following types:
- Host device
Attaches a network interface to a single pod.
- Bridge
Uses an existing bridge interface on the node, or configures a new bridge interface. The pods that are attached to this network can communicate with each other through the bridge, and to any other networks that are attached to the bridge.
- IPVLAN
Creates an IPVLAN-based network that is attached to a network interface.
- MACVLAN
Creates an MACVLAN-based network that is attached to a network interface.
Bridges are network interfaces that can forward packets between different network interfaces that are attached to the bridge. Virtualization environments often use bridges to connect the network interfaces of virtual machines to the network.
IPVLAN and MACVLAN are Linux network drivers that are designed for container environments. Container environments often use these network drivers to connect pods to the network.
Although bridge interfaces, IPVLAN, and MACVLAN have similar purposes, they have different characteristics, such as different usage of MAC addresses, filtering capabilities, and other features. For example, you might need to use IPVLAN instead of MACVLAN in networks with a limit of MAC addresses, because IPVLAN uses fewer MAC addresses.
The following resource definition shows a NetworkAttachmentDefinition resource for a host device.
apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: example"device": "ens4", "ipam": {
"type": "dhcp" } }
| The network name |
| The network type |
| Additional network configuration |
You can also create the same network attachment by editing the cluster network operator configuration:
apiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: ...output omitted... additionalNetworks: - name: example"device": "ens4", "ipam": {
"type": "dhcp" } } type: Raw
| The network name |
| The namespace |
| The network type |
| Additional network configuration |
The IP Address Management (IPAM) CNI plug-in provides IP addresses for other CNI plug-ins.
In the previous examples, the ipam key contains a network configuration that uses DHCP.
You can provide more complex network configurations in the ipam key.
For example, the following configuration uses a static address.
"ipam": {
"type": "static",
"addresses": [
{"address": "192.168.X.X/24"}
]
}Although all the pods in the cluster still use the cluster-wide default network to maintain connectivity across the cluster, you can define more than one additional network for your cluster. The added networks give you flexibility when you configure pods that deliver network functions.
The network isolation that an additional network provides is useful for enhanced performance or for security, depending on your needs.
References
For more information, refer to the Multiple Networks chapter in the Red Hat OpenShift Container Platform 4.14 Networking documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/networking/index#multiple-networks
For more information about the SR-IOV network operator, including supported platforms and devices, refer to the About Single Root I/O Virtualization (SR-IOV) Hardware Networks section in the Hardware Networks chapter in the Red Hat OpenShift Container Platform 4.14 Networking documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/networking/index#about-sriov
For more information, refer to the About the Kubernetes NMState Operator section in the About Networking chapter in the Red Hat OpenShift Container Platform 4.14 Networking documentation at https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html-single/networking/index#kubernetes-nmstate
