Kubernetes cluster on AWS EKS, Part 3: Setup NGINX Ingress Controller with Cert-Manager on AWS EKS

This is a series on setting up Kubernetes clusters in Amazon EKS.

In this post, we will setup Ingress Nginx Controller on AWS EKS Cluster.

AWS recommends using ALB Controller for EKS Cluster but in our experience we found NGINX to be more useful for the following reasons:

  1. Ingress Nginx Controller is much simpler to setup then compared to ALB Controller
  2. One Network LoadBalancer can be shared between multiple applications
  3. Let’s encrypt SSLs using Cert-Manager are possible. For ALB you have to use Route53.

1. Add HELM Repository

$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update

2. Install the Nginx Controller

helm upgrade --install ingress-nginx ingress-nginx \
             --repo https://kubernetes.github.io/ingress-nginx \
             --namespace ingress-nginx --create-namespace

Verify the controller is installed

kubectl get pods -n ingress-nginx

The output is similar to

NAME                                        READY   STATUS      RESTARTS    AGE
ingress-nginx-controller-89758f7c6-swwpf    1/1     Running     0           1m

Verify, the LoadBalancer is set up.

kubectl get services -n ingress-nginx

The output is similar to,

NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.100.20.84     a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com   80:31105/TCP,443:31746/TCP   3d18h

Note down the External-IP, this will be used to map the domain name to the LoadBalancer. In our case a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com

3. Setup 2048 Games as a Demo App

2048 is a Simple Game https://github.com/gabrielecirulli/2048 we can use to test the Ingress Setup.

To install the Game as a deployment, create a deployment.yaml file.

apiVersion: apps/v1
kind: Deployment

metadata:
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 5
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      containers:
      - image: alexwhen/docker-2048
        imagePullPolicy: Always
        name: app-2048
        ports:
        - containerPort: 80

Install the deployment on the Cluster.

kubectl apply -f https://raw.githubusercontent.com/skynet86/hello-world-k8s/master/hello-world.yaml

Create a Service, this will create an endpoint that we can connect to Nginx Ingress

apiVersion: v1
kind: Service
metadata:
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: app-2048

4. Map the Domain 2048.example.com

First, we need to Update the DNS record for 2048.example.com and point it to the load balancer address we noted down in step 2.

CNAME     2048.example.com      a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com

The Domain name has to be updated using the interface provided by your Domain Provider. Domain names, take about 10 minutes - 24 Hours to propagate.

To test the status of domain propagation use the dig command.

dig 2048.example.com

On opening 2048.example.com, you will be presented with an Nginx Page with a 502 error code, which means the domain is successfully pointing to our LoadBalancer.

5. Create an Ingress

Create an Ingress file called ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:

name: nginx-2048-ingress
annotations:
   kubernetes.io/ingress.class: nginx

spec:
  rules:
  - host: 2048.example.com
    http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: app-2048
             port:
               number: 80

Apply the Ingress Controller to the cluster

kubectl apply -f ingress.yaml

On opening http://2048.example.com you will be presented with the 2048 Game.

6. Install Cert-Manager

Cert Manager is used to provision SSL Certificates to Kubernetes Clusters (https://github.com/cert-manager/cert-manager)

To install cert-manager simply run -

helm install \
     cert-manager jetstack/cert-manager \
         --namespace cert-manager \
         --create-namespace \
         --version v1.11.0 \
         --set installCRDs=true

This takes a few minutes to set up. To verify the installation run

kubectl get pods -n cert-manager

The output is similar to:

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-85945b75d4-xxxx               1/1     Running   0          9m
cert-manager-cainjector-7f694c4c58-xxxx    1/1     Running   0          9m
cert-manager-webhook-7cd8c769bb-x  xxx     1/1     Running   0          9m

In the next step, we configure a Cluster Issuer LetsEncrypt in our case to issue SSLs on the Fly. Create a file called cluster-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
  metadata:
name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
           ingress:
              class: nginx

7. Update the ingress to use SSL

Finally, we add the SSL to our Ingress Controller

Update the Ingress we created in Step 5, like so

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:

name: nginx-2048-ingress
annotations:
   kubernetes.io/ingress.class: nginx
   cert-manager.io/cluster-issuer: letsencrypt-prod

spec:
  rules:
  - host: 2048.example.com
    http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: app-2048
             port:
               number: 80
  tls:
    - hosts:
        - 2048.example.com
      secretName: tls-2048-kibana

Apply the updated, ingress

kubectl apply -f ingress.yaml

SSL takes a few minutes to issue. The issued Certificate is stored in the secret tls-2048-kibana. On opening https://2048.example.com you will be presented with the 2048 Game.

Need help on your Ruby on Rails or React project?

Join Our Newsletter