Deployment of a Static Website with Nginx on Kubernetes using Kubeadm, Hosted on AWS

The goal of this project is to set up a static website using Nginx on a Kubernetes cluster. This cluster will be established and managed using Kubeadm and will be hosted on AWS EC2 instances. The entire infrastructure setup will be automated through Terraform.

• AWS Account
• Basic knowledge of AWS, Kubernetes, Nginx, and Terraform
• Terraform installed on your local machine

The project involves setting up a Kubernetes cluster using Kubeadm on AWS EC2 instances. A Network Load Balancer will distribute the traffic across Kubernetes nodes. The DNS configuration will be managed by AWS Route 53, and Nginx Ingress will control the external access to the static website. TLS certificates will be managed by Cert-Manager for secure connections.

• Automate the setup of AWS EC2 instances and other necessary infrastructure components using Terraform scripts.

Provided is the Terraform scripts used to automate the setup of 3 AWS EC2 instances and other necessary infrastructure components for the Kubernetes cluster.

• ec2_instances.tf

• gateway.tf

• output.tf

• provider.tf

• security_group.tf

• vpc.tf

Initialize Terraform

Open your terminal. Navigate to the directory containing the Terraform files.

Run

terraform init

This command initializes Terraform, installs the AWS provider, and prepares the environment for deployment. Plan Infrastructure Deployment

terraform plan

Review the actions Terraform will perform before actually making changes to your AWS infrastructure. Apply Configuration

terraform apply

Confirm the action by typing yes when prompted. Terraform will now provision the AWS infrastructure as defined in your configuration files. Access Outputs

After successful deployment, use terraform output to view important outputs like public IP addresses of your EC2 instances. Clean Up Resources

To remove the infrastructure and avoid unnecessary AWS charges, run terraform destroy. Confirm the action when prompted, and Terraform will delete the resources it created.

• Deploy and configure a Kubernetes cluster on the provisioned AWS EC2 instances using Kubeadm for cluster creation, Containerd as container runtime and weave for Weave Net for Network Policy.

Preparation of EC2 Instances

• Ensure all EC2 instances are up and running as per the Terraform deployment.
• SSH into each EC2 instance.
• Update the package lists and install any pending updates.
• (Optional) To change hostname run

sudo hostnamectl set-hostname <hostname>

Reboot instances for hostname change to take effect

Run the following script on all nodes

Run bash chmod +x setup-kubernetes.sh to make the script executable.

Execute the script with

sudo ./setup-kubernetes.sh

You should see

kubelet set on hold.
kubeadm set on hold.
kubectl set on hold.

This script automates the setup process for a Kubernetes node. It starts by disabling swap and setting up necessary kernel modules like 'overlay' and 'br_netfilter'. Then, it adjusts sysctl parameters to ensure proper network forwarding and bridge filtering. The script proceeds to install containerd, a container runtime, and configures it to use systemd as the cgroup driver. It also handles the addition of Docker's and Kubernetes' official repositories and GPG keys to the system's package manager. Finally, the script installs Kubernetes components including kubelet, kubeadm, and kubectl, and ensures their versions are held to prevent unintended upgrades.

On the master node, you will initialize the cluster

sudo kubeadm init

To start using your cluster, you need to run the following as a regular user

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

export KUBECONFIG=/etc/kubernetes/admin.conf

Deploying the Weave CNI To allow your pods to communicate, you need to deploy a CNI. I've chosen Weave:

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Joining the Worker Nodes On each worker node, you'll need to join them to the cluster using the token generated by kubeadm init. You will see an output like this:

kubeadm join [your-master-ip]:6443 --token [your-token] --discovery-token-ca-cert-hash sha256:[your-hash]

Run code as root

Verifying the Cluster Finally, you can check if your nodes are part of the cluster:

kubectl get nodes

You shoud see

NAME            STATUS   ROLES           AGE     VERSION
control-plane   Ready    control-plane   8m15s   v1.28.4
node1           Ready    <none>          54s     v1.28.4
node2           Ready    <none>          43s     v1.28.4

• Install NGINX Ingress Controller

Add the NGINX Ingress repository to Helm

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Run bash helm repo ls to see installed chats

NAME            URL
ingress-nginx   https://kubernetes.github.io/ingress-nginx

Install the NGINX Ingress Controller with the release name nginx-ingress

helm install nginx-ingress ingress-nginx/ingress-nginx --namespace ingress --set controller.replicaCount=2 --create-namespace

Install helm using the official helm Installation guide

Check if the NGINX Ingress Controller pods are running:

kubectl get pods -n ingress

You get

NAME                                                      READY   STATUS    RESTARTS   AGE
nginx-ingress-ingress-nginx-controller-5c46774b45-bnxgx   1/1     Running   0          111s
nginx-ingress-ingress-nginx-controller-5c46774b45-vdlpc   1/1     Running   0          111s

Get the nginx-ingress controller loadbalancer port number for the http and https traffic

kubectl get svc -n ingress

NAME                                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
nginx-ingress-ingress-nginx-controller             LoadBalancer   10.107.207.241   <pending>     80:**30495**/TCP,443:**30293**/TCP   2m23s
nginx-ingress-ingress-nginx-controller-admission   ClusterIP      10.98.12.216     <none>        443/TCP                      2m23s

• Implement an AWS Network Load Balancer to distribute incoming traffic efficiently across the Kubernetes nodes.

  • Create 2 target groups one for http traffic and the other for https

    • Log in to AWS Management Console and navigate to the EC2 Dashboard.
    • Under Load Balancing, select Target Groups and click Create Target Group
    • Choose the Target Type as instances
    • Set the Name (eg http-tg, https-tg )
    • For Protocol select tcp and set the port number of the http target group to the http port number of the ingress-controller. Do same for the https target group.
    • Select the VPC created
    • Configure Health Check protocol as tcp
    • Register the two worker nodes ec2s
    • Review and click Create

  • Create a Network Load Balancer

    • Under Load Balancing, select Load Balancers and click Create Load Balancer
    • Choose Network Load Balancer, click create and configure
      • Name: Assign a unique name
      • Scheme: Choose internet-facing
      • IP address type: Select IPv4
      • Choose a VPC.
      • Select at least two subnets in different Availability Zones.
      • Choose the default security group.(Make sure it allows traffic on ports 80 and 443)
      • Define 2 listeners: For the first one TCP, port 80, and choose the HTTP target group and for the second one TCP, port 443, and choose the HTTPS target group.
      • Click create Load Balancer

• Use AWS Route 53 to manage the DNS for effective domain name resolution and routing to the static website.
  • Navigate to the Route 53 Dashboard
    • Click Create Hosted Zone.
    • Enter your Domain Name.
    • Choose Public Hosted Zone and click Create.
  • Create an Alias Record:
    • In the Hosted Zone, click on the domain name and click Create Record.
    • Select Alias
    • set Record Name (e.g., www) or leave empty to use root domain
    • Record type, choose A – Routes traffic to an IPv4 address and some AWS resources
    • For Route Traffic To, Alias to Network Load Balancer
    • Select your Region and Load Balancer.
    • Routing policy, choose simple routing
    • Click Create Record.

• Configure Nginx Ingress in the Kubernetes cluster to manage and control external access to the website.

  • Create an nginx.yaml file in the home directory containing the deployment and service to run nginx. To create the nginx deployment and service, run

    kubectl apply -f nginx.yaml

  • Create an ingress-resource using code from ingress-resource.yaml and run

    kubectl apply -f ingress-resource.yaml

  Access the domain name to see the Welcome to nginx! page

• Integrate Cert-Manager in the Kubernetes cluster for handling TLS certificate issuance and management.

  • Install cert-manager using the following command

  kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.2/cert-manager.yaml

  • Check if cert-manager pods are running

  kubectl get po -n cert-manager

  You should get this

  NAME                                      READY   STATUS    RESTARTS   AGE
  cert-manager-7d75f47cc5-j6jtf             1/1     Running   0          53s
  cert-manager-cainjector-c778d44d8-mg2dd   1/1     Running   0          53s
  cert-manager-webhook-55d76f97bb-r6zr4     1/1     Running   0          53s
  

• Create a letsencrypt cluster issuer using code from letsencrypt-issuer.yaml and run

  kubectl apply -f letsencrypt-issuer.yaml

  You should get this

  clusterissuer.cert-manager.io/letsencrypt-prod created
  

• Now upgrade the ingress resource to use cert-manager and create a tls certificate

  • Use ingress-resource-tls.yaml to update the ingress resource. Run

  kubectl apply -f ingress-resource.tls

You site is now secured

This README.md is part of the project on deploying a static website using Kubernetes, Nginx, Kubeadm, and AWS, managed with Terraform.