Kubernetes Network Policy by using Cilium – Beginner Level

In my blog post series about Kubernetes networking, I wrote here that by default, all pods can communicate together in a “vanilla” Kubernetes. Whatever they belong to the same namespace or not. To provide network separation, we need to implement Kubernetes Network Policy.

If your Kubernetes cluster is multi-tenant this is going to be a requirement. Multi-tenant means the same Kubernetes cluster host several applications. Each of them is in their own namespace. However the product owners are different for each application. In this case you’ll need to isolate each application from each other. It would be very bad if from a pod of your application you could reach and connect to a pod of another application and the other way around. To provide this networking isolation we need to configure and deploy Network Policy. They are often the pet peeve of many Kubernetes Administrator that are not familiar with networking. If it is your case, this blog post is for you. I’ll give you all the basics to understand it completely.

Ingress and Egress

Kubernetes Network Policy is like a police officer in front of your pods. It allows or denies traffic entering or exiting the pod. This is the first concept to grasp, the traffic can flow in 2 directions: entering or exiting. In traditional networking we call it incoming and outgoing (sometimes it is inbound and outbound) traffic. In Kubernetes we are talking about ingress and egress but they are the same thing.

Ingress or egress traffic depends on the point of reference. We basically apply a Network Policy to a pod. Cilium calls it an endpoint but it is the same. For this pod, the direction of traffic is egress when it comes from the pod to the outside. An ingress traffic is for traffic coming from outside and entering into this pod. The drawing below illustrates that concept:

On the left, the police officer looks to the pod so she will stop and filter any traffic coming from that pod before it goes outside. On the right, he looks to the outside with his back to the pod. He will then filter and stop any traffic from outside before it enters that pod.

Labels

As you may know, pods get their IP Address from Cilium when they start. A pod by design can be deleted and recreated which means its IP Address will change. In traditional networking, we filter traffic with firewalls or Access Control List (ACL). This layer 3 filtering is based on the IP Address. In Kubernetes we then have to use another way than IP Addresses and so we use labels instead. We set a label to each pod and if pods use the same label then they belong to the same group. It is the case for the replicas of a front end or a back end pod that will share the same label. We apply a Network Policy to a group of pods based on this label. Let’s look at this on a drawing:

In our example we have 2 applications in 2 separate namespaces. Each application has 2 Front End (FE) and 2 Back End (BE). In the namespace app1, the FE have the same label “app: frontend1″ and the BE have the label “app:backend1″. We apply the same to the namespace app2.

Cilium Network Policy

Kubernetes provides an object called NetworkPolicy that is implemented by the Container Network Interface (CNI). Cilium is using its own object called CiliumNetworkPolicy that extends the capabilities of that standard Network Policy object. You can learn more about it from this Isovalent blog post.

A Network Policy is a yaml file where we configure to which pod labels it applies as well as the ingress and/or egress rules we apply to them. When we apply that yaml file in our cluster, we set the police officer in front of the pods door.

Kubernetes Network Policy ingress rule

Let’s look at an example and start with an ingress rule. We will use the simple CiliumNetworkPolicy below:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "ingress-backend-rule"
  namespace: app1
spec:
  endpointSelector:
    matchLabels:
      app: backend1
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend1

This CiliumNetworkPolicy applies to any pod in the namespace app1 that has the label “app: backend1“. Only one ingress rule is applied here where traffic from all pods with the label “app: frontend1” is allowed. Any other ingress traffic will be discarded. Below is the drawing showing it in a visual way:

When any traffic in this Kubernetes cluster reach the pods BE 1 or BE 2 in the namespace app1, the ingress rules of our Network Policy applies. Only traffic coming from the pods FE 1 or FE 2 in the namespace app1 will be allowed to enter because they have the authorized label.

Kubernetes Network Policy egress rule

Let’s continue with a simple egress rule. We will use the CiliumNetworkPolicy below:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "egress-frontend-rule"
  namespace: app1
spec:
  endpointSelector:
    matchLabels:
      app: frontend1
  egress:
  - toEndpoints:
    - matchLabels:
        app: backend1

This CiliumNetworkPolicy applies to any pod in the namespace app1 that has the label “app: frontend1“. Only one egress rule is applied here where traffic to pods with the label “app: backend1” is allowed. Any other egress traffic from there pods will be discarded. Below is the drawing showing it in a visual way:

When traffic from the pods FE 1 or FE 2 in the namespace app1 is going out, the egress rule of our Network Policy applies. Only traffic to pods BE 1 and BE 2 will be allowed as they have the authorized label.

Wrap up

We’ve covered the basics of Kubernetes Network Policy by explaining what are its key components: Ingress, egress, labels and the CiliumNetworkPolicy object. Then we’ve seen a simple example of ingress and egress rules and illustrated them with a drawing. These basics should get you started in the world of Kubernetes Network Policy.

With the simple ingress and egress rules above did we completely isolate the traffic flow between FE and BE in app1? The answer is no because from BE 1 or BE 2 I can still reach any pods in the cluster. There is only an ingress rule but no egress rule. The same, any traffic can reach FE 1 or FE 2 because there is only an egress rule but no ingress rule. I would then need to configure an ingress rule and an egress rule (the same as above) into the same CiliumNetworkPolicy. Congratulations, you’ve just entered into the complex world of Kubernetes Network Policy!