From Docker to Kubernetes using Minikube remotely

To follow along, you will need to:

I will be using DigitalOcean for several reasons:

You should pick a minimum of 2vCPU and 4GB.

With the credit from this link, you can run a 4vCPU with 8GB of RAM 24/7 for 2 months with extra block storage entirely for free.

Let’s get a few of the basic steps behind us:

First create the droplet and provide your ssh key. This is straight forward. Take note of your public IP. If you need some help with that, check out this article. The article also show how to create an ssh key if you don’t have one yet.

While your droplet is starting (it only takes a few seconds), you can install kubectl on our local machine, we will need it later to access and control our Kubernetes cluster and resources:

If you are not using a Mac, you may check out this documenation.

kubectl is the tool you will be using to query your cluster, create resources, monitor, etc…​

We will now login on this server. I will call it minikube. We can add an entry in ~/.ssh/config to make it more convenient:

We log in first with ssh root@minikube.

Let’s first store your name in a variable so all the scripts work with your name without you having to fix all the commands:

We then update and install a few utilities:

We will now install Docker, follow the guide and run Step 1 and Step 2 from Install Docker and make sure to run the optional Step 2 so you can run docker without having to prepend the sudo keyword. In this step, replace ${USER} by ${ME}.

Now copy your ssh public key for the new user:

Logff and log back in as non-root, this time, we can simply run:

To make sure everything worked so far, lets test our docker installation:

This command will show Unable to find image 'nginx:latest' locally. This is normal. It will then download the nginx image. It then returns immediately after showing a long hash you can ignore. We now have a web server (nginx) running and locally serving http requests on port 80. Let’s check:

You should see:

At that point, if you don’t know what is nginx, don’t worry about it. The point here was just to test 'some' Docker container and make sure everything is working fine.

We no longer need this container, let’s remove it:

If you do some research, you will find many Kubernetes supervisors. Here are a few:

I am sure you can find some more. All of those do (almost) the same thing: they expose a standard API and allow supervising a cluster of nodes on which you will run your containers.

There are however big differences with the solutions linked above:

We will be using minikube, which is considered as a local development solution.

If you want a robust Production Kubernetes cluster, it will have to run on multi nodes cluster to ensure redundancy. The control plane managing your cluster will also need to be redundant.

That being said, there is nothing wrong with going from a non-redundant setup where few containers run unsupervised to a still-non-redundant setup where you start supervising your containers.

minikube is a mini-kubernetes cluster typically used for local development. It runs on a single machine and this perfect for our start. Let’s install it on our server.

Let’s store the public IP of our server:

Now we can start minikube:

At that point, we have a Kubernetes cluster up and running and by the way, it is running into a single docker container:

While we don’t really need kubectl on the server (we will later only use it from our local machine), it can help to troubleshoot and ensure everything is working fine locally.

kubectl is the command you will need to control your cluster. It connects to the API Server started by minikube on port :8443.

For now, let’s use kubectl from the server and run a simple query:

It should return something like:

While this is all good, usually one does not ssh into the cluster to control it. There are a few issues we need to solve in order to reach the minikube api server remotely. The first issue is that the firewall of our VPS let only traffic go through on port :22. This is a default safe practise.

Traffic on any other port will be blocked as shown below.

So let’s now see the options we have to connect and control our Kubernetes remotely:

Using a ssh tunnel is both convenient and secure. It allows accessing network resources (an API, a web server, a database, etc…​) available on a remote server, as if it would be running locally on our machine.

All we need is the ability to connect to the server via ssh.

We will need to establish the tunnel everytime we want to use it. We will see some options to do that.

Using a proxy is much more comfortable to access your Kubernetes cluster since you can "just use it"^TM without having to establish a ssh tunnel as we just saw. Comfort does not come at the cost of security here. Your remote minikuke created some certificates (like a VPN would do for instance) and only users with valid certificates will be able to access the remote minikube.

It requires however a few preparation steps related to the networking. For now, as we saw previously, we cannot reach ANY port but :22 on the remote server. Even if we open our cloud firewall for :8443, we will NOT reach the minikube API server.

You may have spotted the reason earlier, let’s have a look at minikube, which runs in a docker container btw:

The problem lays in this 127.0.0.1:49189→8443/tcp: inside the container, the api server runs on port :8443. This port is exposed but only available on the local interface 127.0.0.1.

We need a solution to expose the API server to the outside world and a simple option is to use an additionnal container: Minikube Proxy Docker Image.

This container will allow the following:

You can visit the Repo on Gtilab to learn more but here is the tldr;, a command to run on the remote server :

This container plays the spy by joining the minikube network from the inside!

If we strip what is not critical in this command, we are left with:

You may check that the container called minikube-proxy is now running with docker ps.

At that point, make sure you added a Firewall rule to allow traffic on port :18443.

We installed kubectl on our local machine already, we will now need to tell it how to connect to the remote minikube and we will need the following:

We will now be working on our local machine. Open a new shell there.

Since we need to fetch a few files from our server, let’s define a location for that and create the folder:

Let’s make sure you have all you need: running env | grep MY_ should show 3 variables:

Now let’s fetch the CA, the client certificate and key:

You should now have the following files:

We should now have everything we need locally. Let’s create a context that will allow kubectl to know where and how to connect to our server. Before we start, you may want to see your current config. It is located in ~/.kube/config but you don’t need to remember that, kubectl can show it to you:

If you installed docker desktop, you likely see the following. If you did not install Docker Desktop on your machine, the list will be empty, that’s fine:

At that point, if we would not have paid attention earlier, we would have to ignore tls handshake with --insecure-skip-tls-verify but we can do better.

Make sure you did define all the ENV we need with env | grep MY_. If you see your 3 variables, you can move on.

You may also want to check that you did not create a context with the name we will use. The following should return nothing:

Let’s tell kubectl about our server:

Out cluster is now known:

We now need a user and minikube created it one for us, called minikube:

We are almost done, we now need to put that alltogether as a context:

We now tell kubectl to USE that new context:

Your new context should now show upt with:

Let’s now finally use that cluster a little bit!

Let’s first see how to create and start a pod. A pod may run one or more containers. All the containers in a pod can communicate using localhost. This is convenient if you have container tighly related. For our example, we create a pod with one nginx container:

The Kubernetes (close) equivalent to:

looks like:

You may wonder about that --restart=Never and may remember using --restart=Always when using Docker. Kubernetes can manage the life cycle of your pods and containers. Unless you provide this --restart=Never, Kubernetes will created additionnal resources to take care of restarting and scaling as required.

After sending this command, you will want to see your pod. You may spam some kubectl get pods or:

Let’s delete this pod now:

Before we create more on our cluster, let’s create a namespace to keep things organized and tiddy. You can see that as a folder and all the Kubernetes you will created will be located in this namespace. Later you can view only the resources in your namespace without be polluted by the view of other resources from your colleagues for instance.

You can then query the list of namespaces with:

or save some typing with:

or even:

Now everything we create will be located under the demo namespace.

What we called resources so far is named a Workload and Kubernetes defines several types of workloads.

While creating them using the command line, it becomes quickly cumbersome. Instead, we can tell kubectl to apply what is described in a file. It can use yaml or json.

Let’s have a look at our previous created demo namespace:

Here is an example to create (again!) our web pod running one nginx container:

Tip: You will likely find a plugin for your favorite editor that will help you compose those spec files.

You can copy the content shown above in a pod.yaml file anywhere on your local machine. Now let’s apply those specs:

So far, we deployed neither something fancy nor vers Kubernety, let’s do better !

As before, you can create this deployment.yaml anywhere and run:

Asking for the list of pods with kubectl get pods --watch, we can see that we have now 3 pods running a pretented api:

Now run the following from another terminal and come back:

You will see that one pod gets deleted but since we asked the replicat-set to keep 3 instances up, a new pod will immediately be spawned and replace the one that died.

Your Kubernetes cluster is running but would that not be great to see what is going on using the Kubernetes Dashboard?

Would that not be greater to see fancy charts and metrics?