Traefik

Setup instructions

Stated here just for reference, so don’t execute any of these by yourselves. Besides, so far you haven’t been granted the permissions to do so anyways … :-)

Sources:

[Install Traefik][install-traefik]
[Traefik & Kubernetes: Kubernetes Ingress Controller][traefik-kubernetes-ingress]
[Traefik & Kubernetes: Kubernetes Ingress Controller, The Custom Resource Way][traefik-kubernetes-crd]

and thus:

kubectl create namespace traefik-v2
# let's hardcode the ingress class to avoid potential mixups with other Ingress Controllers, cf. https://docs.traefik.io/providers/kubernetes-ingress/#ingressclass
helm upgrade --install --namespace=traefik-v2 traefik traefik --version 10.19.4 \
    --repo https://helm.traefik.io/traefik \
    --set=ingressClass.enabled=true --set=providers.kubernetesIngress.publishedService.enabled=true \
    --set='additionalArguments={--providers.kubernetesingress.ingressclass=traefik,--global.sendanonymoususage=false,--log.level=DEBUG}'

Traefik is an open-source Edge Router for publishing services. It receives requests on behalf of your system and finds out which components are responsible for handling them.

Traefik is natively compliant with every major cluster technology, such as Kubernetes, Docker, Docker Swarm, AWS, Mesos, Marathon, and others; and can handle many at the same time. (It even works for legacy software running on bare metal.) Containous, the company that sponsors Traefik’s development, can provide commercial support and develops an Enterprise Edition of Traefik.

With Kubernetes, Traefik supports Ingress resources as we had encountered before, as well as Traefik-specific IngressRoutes that allow to unleash the full potential of what Traefik provides.

Traefik has already been set up in our test cluster beforehand (via helm, by the way, see Setup instructions above) so we can jump right at making use of it, cf. kubectl get all --all-namespaces -l app.kubernetes.io/name=traefik, e.g.:

NAMESPACE    NAME                           READY   STATUS    RESTARTS   AGE
traefik-v2   pod/traefik-69f6b585fc-4xfhr   1/1     Running   0          2d18h

NAMESPACE    NAME              TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                      AGE
traefik-v2   service/traefik   LoadBalancer   10.0.147.225     51.145.138.238   80:31785/TCP,443:30850/TCP   2d20h

NAMESPACE    NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
traefik-v2   deployment.apps/traefik   1/1     1            1           2d20h

NAMESPACE    NAME                                 DESIRED   CURRENT   READY   AGE
traefik-v2   replicaset.apps/traefik-69f6b585fc   1         1         1       2d18h

But for now we will just use what the Ingress Controller provides without diving into too much detail on how this will be achieved.

Note

Still, you can see already that Traefik gets exposed as a LoadBalancer service, and this service’s external IP address is what we will need to access during our exercises, and we won’t see any addresses listed in our future Ingress resources, so let’s take note of this address now via export INGRESS_IP_TRAEFIK=<our_Ingress_IP>.

export INGRESS_IP_TRAEFIK=$(kubectl get service --namespace traefik-v2 traefik -o jsonpath='{.status.loadBalancer.ingress[].ip}')

should do the trick, which you can then verify via

echo $INGRESS_IP_TRAEFIK.

This variable will not persist over a logout nor will it spread to other separate sessions, so remember to set it again whenever you (re)connect to your user VM.

Namespace

With the Ingress Controller still being a pre-defined cluster-wide central solution, we again need to take care to not step on each others’ toes in the following exercises. For that we will prefix some parts of the following resources with a user-specific namespace.

Note

As we will use this personal namespace several times as a resource prefix, make it now available for easy consumption via a variable:

export NAMESPACE=$(kubectl config view --minify --output 'jsonpath={..namespace}'); echo $NAMESPACE

This variable will not persist over a logout nor will it spread to other separate sessions, so remember to set it again whenever you (re)connect to your user VM.

Exercise - Expose our sample application using Ingress via Traefik

Let’s start with our little sample application again, just like we did in the chapter concerning Ingress Controller - Nginx . In fact, we will just pick up several things that have been created in the course of that chapter, so make sure to finish that before proceeding here.

Create sampleapp-ingress-traefik.yaml by executing the following (yes, execute it all at once), utilizing your personal namespace as a host component, and our Ingress IP by way of nip.io wildcard DNS as a suffix, and of course we have to give it a new name in order to not just overwrite our previous Ingress:

cat <<.EOF > sampleapp-ingress-traefik.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sampleapp-traefik
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
    traefik.ingress.kubernetes.io/router.middlewares: $NAMESPACE-stripprefix-subpath@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
    - host: hello.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: sampleapp
                port:
                  number: 8080
          - path: /subpath/
            pathType: Prefix
            backend:
              service:
                name: sampleapp-subpath
                port:
                  number: 8080
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: stripprefix-subpath
spec:
  stripPrefix:
    prefixes:
      - /subpath
.EOF

Compared to our previous Ingress resource for ingress-nginx we see quite some similarities as well as some differences:

the basic structure is standardized, and each Ingress Controller is supposed to understand it: a spec with rules, each specifying a host and which paths are supposed to be redirected to which backend
annotations steer which Ingress Controller shall be used, and also implementation-specific details (we advise K8s here to use the Traefik Ingress Controller)
instead of pattern matching and rewriting applied to a subpath a Traefik-specific Middleware is used
- as Traefik has its own concept of namespaces for accessing several backend providers (e.g. Kubernetes, Mesos, Consul, …) the Kubernetes namespace will be prepended

With Traefik, Middleware is what does all tweaking of requests or responses. We will cover some examples further down.

Now, create the resources by running the following command:

kubectl apply -f sampleapp-ingress-traefik.yaml

Milestone: K8S/INGRESS/TRAEFIK-SAMPLEAPP-INGRESS

Verify that the Ingress is configured as intended:

kubectl describe ingress sampleapp-traefik

Name:             sampleapp-traefik
Labels:           <none>
Namespace:        <your_namespace>
Address:          <your_Ingress_IP>
Default backend:  <default>
Rules:
  Host                                             Path        Backends
  ----                                             ----        --------
  hello.<your_namespace>.<your_Ingress_IP>.nip.io
                                                   /           sampleapp:8080 (172.17.0.4:8080)
                                                   /subpath/   sampleapp-subpath:8080 (172.17.0.3:8080)
Annotations:                                       traefik.ingress.kubernetes.io/router.entrypoints: web
                                                   traefik.ingress.kubernetes.io/router.middlewares: <your_namespace>-stripprefix-subpath@kubernetescrd
Events:                              <none>

(Please note that even though the Ingress resource references a service name that will also be listed for each path, the IP addresses listed there will be those of the corresponding Pods.)

Then access our application, both normally and the subpath, via the Traefik LoadBalancer service external IP:

curl http://hello.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/hello; echo
curl http://hello.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/subpath/hello; echo

which should yield

Hello World (from sampleapp-65779bd948-6gcm7) to somebody
Hello World (from sampleapp-subpath-7d466c4ccb-jlrcb) to everyone via Ingress from a subpath

So all in all this now provides the same results as we had achieved using ingress-nginx. Let’s do the same for our ToDo application, just like we did in the previous chapter.

Exercise - Ingress’ify our ToDo application via Traefik

We will need a redirect, some Ingress backends, basic auth settings, and a certificate for TLS termination. Parts of that we had already created in the previous chapter, first locally and then put into Kubernetes, so let’s plug it all together.

TLS certificate

Self-signed will suffice for now:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
    -keyout todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io.key \
    -out todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io.crt \
    -subj "/CN=todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io"

which will output some line(s) with symbols indicating its generating data.

Verify the certificate’s CN:

openssl x509 -in todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io.crt -noout -subject

subject=CN = todo.<your_namespace>.<your_Ingress_IP>.nip.io

Load this as a Kubernetes secret:

kubectl create secret tls todo-traefik-tls-secret --key todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io.key --cert todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io.crt

secret/todo-traefik-tls-secret created

And verify it is present:

kubectl describe secrets todo-traefik-tls-secret

Name:         todo-traefik-tls-secret
Namespace:    <your_namespace>
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1180 bytes
tls.key:  1704 bytes

Remember from Exercise - create a Secret how to verify the contents?

Solution

kubectl get secrets todo-traefik-tls-secret -o jsonpath='{.data.tls\.crt}' | base64 --decode

will do the trick for the certificate, and the key can be checked likewise.

Milestone: K8S/INGRESS/TRAEFIK-TODOAPP-CERT

Basic auth

We are just going to reuse the Secret and the credentials embedded therein that we have created in the Nginx chapter, so make sure you have actually finished that.

Ingress

Thus, now it is time to plug it all together. Create a file todoapp-ingress-traefik.yaml by executing the following (yes, execute it all at once), again utilizing your personal namespace as a prefix:

cat <<.EOF > todoapp-ingress-traefik.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: redirect-to-https
spec:
  redirectScheme:
    scheme: https
    permanent: true
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: todo-traefik-redirect
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
    traefik.ingress.kubernetes.io/router.middlewares: $NAMESPACE-redirect-to-https@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
    - host: todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: todoui
                port:
                  number: 8090
---
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: minversion
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_RSA_WITH_AES_256_GCM_SHA384
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: todo-traefik
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
    traefik.ingress.kubernetes.io/router.tls.options: $NAMESPACE-minversion@kubernetescrd
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      secretName: todo-traefik-tls-secret
  rules:
    - host: todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: todoui
                port:
                  number: 8090
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: stripprefixregex-backend
spec:
  stripPrefixRegex:
    regex:
      - "/backend/v[0-9]+"
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: basic-auth-backend
spec:
  basicAuth:
    secret: todo-backend-basic-auth
    realm: "Authentication Required - ToDo App Backend"
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: todo-traefik-backend-basic-auth
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.middlewares: $NAMESPACE-stripprefixregex-backend@kubernetescrd,$NAMESPACE-basic-auth-backend@kubernetescrd
    traefik.ingress.kubernetes.io/router.tls: "true"
    traefik.ingress.kubernetes.io/router.tls.options: $NAMESPACE-minversion@kubernetescrd
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      secretName: todo-traefik-tls-secret
  rules:
    - host: todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io
      http:
        paths:
          - path: /backend/v1/
            pathType: Prefix
            backend:
              service:
                name: todobackend-v1
                port:
                  number: 8080
          - path: /backend/v2/
            pathType: Prefix
            backend:
              service:
                name: todobackend
                port:
                  number: 8080
.EOF

Apply it with kubectl apply -f todoapp-ingress-traefik.yaml and verify what has been created:

kubectl describe ingress todo-traefik-redirect todo-traefik todo-traefik-backend-basic-auth

[...]
Rules:
  Host                                            Path  Backends
  ----                                            ----  --------
  todo.<your_namespace>.<your_Ingress_IP>.nip.io
                                                  /     todoui:8090 (172.17.0.5:8090)
Annotations:                                      traefik.ingress.kubernetes.io/router.entrypoints: web
                                                  traefik.ingress.kubernetes.io/router.middlewares: <your_namespace>-redirect-to-https@kubernetescrd
[...]
Rules:
  Host                                            Path  Backends
  ----                                            ----  --------
  todo.<your_namespace>.<your_Ingress_IP>.nip.io
                                                  /     todoui:8090 (172.17.0.5:8090)
Annotations:                                      traefik.ingress.kubernetes.io/router.entrypoints: websecure
                                                  traefik.ingress.kubernetes.io/router.tls: true
                                                  traefik.ingress.kubernetes.io/router.tls.options: <your_namespace>-minversion@kubernetescrd
[...]
Rules:
  Host                                            Path           Backends
  ----                                            ----           --------
  todo.<your_namespace>.<your_Ingress_IP>.nip.io
                                                  /backend/v1/   todobackend-v1:8080 (172.17.0.9:8080)
                                                  /backend/v2/   todobackend:8080 (172.17.0.7:8080)
Annotations:                                      traefik.ingress.kubernetes.io/router.entrypoints: websecure
                                                  traefik.ingress.kubernetes.io/router.middlewares: <your_namespace>-stripprefixregex-backend@kubernetescrd,<your_namespace>-basic-auth-backend@kubernetescrd
                                                  traefik.ingress.kubernetes.io/router.tls: true
                                                  traefik.ingress.kubernetes.io/router.tls.options: <your_namespace>-minversion@kubernetescrd
[...]

Milestone: K8S/INGRESS/TRAEFIK-TODOAPP-INGRESS

Verification

Well, but does it actually work as intended? Let’s find out by first verifying TLS termination:

curl --verbose --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/

[...]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=todo.<your_namespace>.<your_Ingress_IP>.nip.io
*  start date: Dec  1 13:01:46 2023 GMT
*  expire date: Nov 30 13:01:46 2024 GMT
*  issuer: CN=todo.<your_namespace>.<your_Ingress_IP>.nip.io
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
[...]
<!DOCTYPE HTML>
<html>
<head>
    <title>Schönste aller Todo Listen</title>
[...]

Then let’s verify the redirect HTTP->HTTPS is present:

curl --verbose http://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/

[...]
< HTTP/1.1 301 Moved Permanently
< Location: https://todo.<your_namespace>.<your_Ingress_IP>.nip.io/
[...]

And confirm the contents on redirect:

curl --silent --location --insecure http://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/ | head -n 4

<!DOCTYPE HTML>
<html>
<head>
    <title>Schönste aller Todo Listen</title>

So, TLS termination, including a redirect HTTP->HTTPS, seems to work just fine.

Tip

More details / checks wanted? Run

docker run --rm -it drwetter/testssl.sh:latest https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/

and enjoy.

Now let’s check basic auth on backend:

curl --verbose --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v2/todos/; echo

[...]
< HTTP/2 401
< content-type: text/plain
< www-authenticate: Basic realm="Authentication Required - ToDo App Backend"
[...]

OK, we indeed need to authenticate, so let’s try this while inserting sample data (if none already present):

curl --request POST --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v2/todos/traefiktest --user backenddebugger:password; echo

added traefiktest

And query data from backend:

curl --silent --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v2/todos/ --user backenddebugger:password; echo

["testabc", "traefiktest"]

Also test the v1 backend:

curl --silent --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v1/todos/ --user backenddebugger:password; echo

["testabc", "traefiktest"]

Test the two different backend versions. As they only differ in handling calls to /fail let’s do just that and check their Pods afterwards:

curl --silent --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v2/fail --user backenddebugger:password; echo

fixed!

So, no crash, meaning we really reached the v2 backend Pod. Let’s compare this to

curl --silent --insecure https://todo.$NAMESPACE.$INGRESS_IP_TRAEFIK.nip.io/backend/v1/fail --user backenddebugger:password

Bad Gateway

And after a few moments we see the Pod has indeed been restarted:

kubectl get pods todobackend-v1-<ReplicaSetId>-<PodId>

NAME                              READY   STATUS    RESTARTS   AGE
todobackend-v1-84bc4474fd-bjpzd   1/1     Running   2          25m37s

So, the Pod has crashed, meaning we had really reached the v1 backend Pod.

All in all, it seems this Traefik-based Ingress is now working just like the ingress-nginx-based one we had created in the previous chapter. But what else can Traefik do?

Traefik extras

Going beyond the standardized Ingress interface Traefik is able to provide further services, similar to what ingress-nginx already allows, and then some extra:

canary deployments via weighted round-robin (WRR) services defined in an IngressRoute
circuit breakers via a fallback mechanism in case a predefined service is not reacting correctly, as e.g. measured via network error ratio, status code ratio, and/or latency
request retrying to cover temporarily unavailable services
rate-limiting to ensure some quality of service
traffic mirroring for debugging / developing

Just for reference, some more options will be used when dealing with Service Meshes : Opt-In Canary .

Checking all these is beyond the scope of these exercises, so see the Traefik documentation for details. But we are going to check out one more feature as follows.

Web interface

While ingress-nginx allows us to check the generated configuration file, Traefik provides a web interface to investigate what has been configured and what might not be working.

Provide access via port-forwarding

kubectl port-forward -n traefik-v2 $(kubectl get pods -n traefik-v2 --selector "app.kubernetes.io/name=traefik" --output name) 9000:9000

Now you can follow the instructions for the port forwarding within your SSH client, cf. the instructions for Linux / MacOS or Windows . Add a mapping from your local port 9000 to localhost:9000, and then you will be able to access this through your local browser at http://localhost:9000/dashboard/ , using dual port-forwarding (first via ssh client, then via kubectl), and you should see the application like in the following pictures:

Feel free to introduce errors into your resources, e.g. via kubectl delete middleware stripprefixregex-backend and check how this will be indicated in the web interface.