This is a series on setting up Kubernetes clusters in Amazon EKS.
In this post, we will setup Ingress Nginx Controller on AWS EKS Cluster.
AWS recommends using ALB Controller for EKS Cluster but in our experience we found NGINX to be more useful for the following reasons:
- Ingress Nginx Controller is much simpler to setup then compared to ALB Controller
- One Network LoadBalancer can be shared between multiple applications
- Let’s encrypt SSLs using Cert-Manager are possible. For ALB you have to use Route53.
1. Add HELM Repository
$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
2. Install the Nginx Controller
helm upgrade --install ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx --create-namespace
Verify the controller is installed
kubectl get pods -n ingress-nginx
The output is similar to
NAME READY STATUS RESTARTS AGE
ingress-nginx-controller-89758f7c6-swwpf 1/1 Running 0 1m
Verify, the LoadBalancer is set up.
kubectl get services -n ingress-nginx
The output is similar to,
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.100.20.84 a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com 80:31105/TCP,443:31746/TCP 3d18h
Note down the External-IP, this will be used to map the domain name to the LoadBalancer. In our case a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com
3. Setup 2048 Games as a Demo App
2048 is a Simple Game https://github.com/gabrielecirulli/2048 we can use to test the Ingress Setup.
To install the Game as a deployment, create a deployment.yaml file.
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-2048
spec:
selector:
matchLabels:
app.kubernetes.io/name: app-2048
replicas: 5
template:
metadata:
labels:
app.kubernetes.io/name: app-2048
spec:
containers:
- image: alexwhen/docker-2048
imagePullPolicy: Always
name: app-2048
ports:
- containerPort: 80
Install the deployment on the Cluster.
kubectl apply -f https://raw.githubusercontent.com/skynet86/hello-world-k8s/master/hello-world.yaml
Create a Service, this will create an endpoint that we can connect to Nginx Ingress
apiVersion: v1
kind: Service
metadata:
name: service-2048
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
type: NodePort
selector:
app.kubernetes.io/name: app-2048
4. Map the Domain 2048.example.com
First, we need to Update the DNS record for 2048.example.com and point it to the load balancer address we noted down in step 2.
CNAME 2048.example.com a4217761afdb3457683821e38a3d3de7-XXXXXXXXX.us-east-2.elb.amazonaws.com
The Domain name has to be updated using the interface provided by your Domain Provider. Domain names, take about 10 minutes - 24 Hours to propagate.
To test the status of domain propagation use the dig command.
dig 2048.example.com
On opening 2048.example.com, you will be presented with an Nginx Page with a 502 error code, which means the domain is successfully pointing to our LoadBalancer.
5. Create an Ingress
Create an Ingress file called ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-2048-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: 2048.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app-2048
port:
number: 80
Apply the Ingress Controller to the cluster
kubectl apply -f ingress.yaml
On opening http://2048.example.com you will be presented with the 2048 Game.
6. Install Cert-Manager
Cert Manager is used to provision SSL Certificates to Kubernetes Clusters (https://github.com/cert-manager/cert-manager)
To install cert-manager simply run -
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0 \
--set installCRDs=true
This takes a few minutes to set up. To verify the installation run
kubectl get pods -n cert-manager
The output is similar to:
NAME READY STATUS RESTARTS AGE
cert-manager-85945b75d4-xxxx 1/1 Running 0 9m
cert-manager-cainjector-7f694c4c58-xxxx 1/1 Running 0 9m
cert-manager-webhook-7cd8c769bb-x xxx 1/1 Running 0 9m
In the next step, we configure a Cluster Issuer LetsEncrypt in our case to issue SSLs on the Fly. Create a file called cluster-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: [email protected]
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
7. Update the ingress to use SSL
Finally, we add the SSL to our Ingress Controller
Update the Ingress we created in Step 5, like so
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-2048-ingress
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
rules:
- host: 2048.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app-2048
port:
number: 80
tls:
- hosts:
- 2048.example.com
secretName: tls-2048-kibana
Apply the updated, ingress
kubectl apply -f ingress.yaml
SSL takes a few minutes to issue. The issued Certificate is stored in the secret tls-2048-kibana.
On opening
https://2048.example.com
you will be presented with the 2048 Game.
