Journey to Kubernetes - Part 1 of 7 - The Recipe

I want my own Kubernetes cluster!

Finally I've finished studying and testing my personal Kubernetes cluster, where I'm going to move all the services I manage, with the goal of being able to add new services in a unified, scalable and highly available way.

In detail:

• A dozen websites (wordpress, phpbb, ghost...)
• Mail server
• ownCloud server
• Other minor web services

Now let's list the key players of this study.

---

The recipe

To get a stable, distributed and scalable cluster I used the components I'll list below. The road wasn't easy: I had to run a huge number of tests, since some systems didn't prove up to the job of a stable setup. By the end of the journey I managed to reach a solution that performs well and is fault tolerant.

---

Tools used

Cloud web services

• Hetzner Cloud as the VM provider https://www.hetzner.com/cloud
• Amazon route 53 for managing DNS records https://aws.amazon.com/it/route53/

Infrastructure

• Terraform for creating and provisioning the base infrastructure https://www.terraform.io/
• Kubespray for creating the Kubernetes cluster https://github.com/kubernetes-incubator/kubespray

File-system

• GlusterFS - Ansible playbook for managing distributed file-systems across the nodes https://www.jeffgeerling.com/blog/simple-glusterfs-setup-ansible
• Kubernetes Local provisioner - Ansible playbook for automatically creating Kubernetes PVs https://github.com/nutellinoit/local-storage-playbook

Managing distributed Mysql databases

• Mysql operator - Oracle's operator for self-healing MySQL clusters https://github.com/oracle/mysql-operator

Automatic restore and migration of workloads from backups

• Beekup suite for restoring website backups and for migrating mailboxes via imap https://github.com/beeckup

Security

• Let’s encrypt for issuing SSL certificates using the open source project https://github.com/Neilpang/acme.sh
• Wireguard internal mesh vpn https://www.wireguard.com/
• Iptables

Mail server

• Haproxy as the entrypoint for the mail server
• Poste.io as the mail server

In this first article I won't go into the details of each of the parts that make up the cluster, but I'll write 6 more articles covering the following phases:

• Creating and preparing the virtual machines
  • Terraform
  • Aws route 53
• Creating the Kubernetes cluster
  • Kubespray
• Creating the file-system
  • GlusterFS
  • Local Provisioner
• Creating the workloads
  • Scalable, HA Wordpress
  • Mailserver
• Resilience tests
  • Checking multiple nodes going down in an 8-node cluster, checking a full restart (simulating a forced power off of all the machines)

Stay tuned!