Would you be interested in nodeless Kubernetes API-as-a-service?
I'm writing Kubernetes controllers at work. For a controller, all I need is the API server - not any nodes to schedule pods/containers. I've used minikube and moved to kind recently to test them locally. However, both of them eat a lot of CPU and memory since they need to run apiserver, controller-manager, etcd, kube-scheduler, kubelet components that make up a Kubernetes cluster. Also I'm on macOS, so Docker creates a VM for those containers to work.
Last week, I set up a cluster using one RPi so that I can run my controller in my laptop and talk to that cluster. All kubernetes-related stuff is running on RPi.
So I'm wondering whether a Kubernetes cluster without any nodes would be useful for folks. I feel like a lot of dev/test use cases are built on running many pods on the cluster (which you'd need at least one node) so I might be alone in this use case. But in total, it saves you around 1-2 CPU core and 512-1024MB of RAM.
Another question I have is that would you be interested in this if you were able to join your laptop as a node. Essentially, you'd be offloading all Kubernetes-related containers to another service and it'd schedule only your app's containers to your laptop.
Interested people, please sign up here so that you get notified when I do a benchmark on this including the cloud choices, pricr and performance comparison. See you there!
https://sendfox.com/lp/mpe92y
An unrelated idea I had a while ago was hosting the apiserver but it’d act just as a data store with auth; no nodes, no containers etc. Just like key value stores but with all the goodness of Kubernetes API. Would that be useful?
I think you could achieve this using AWS EKS or Azure AKS. They both provide free control planes so you could spin up a cluster with zero nodes and still talk to the control plane.
I just decided to do a benchmark against different options for kube developer testing scenario including how much they cost. Just created this list to notify people when the benchmark is done. See you there and marry christmas!
https://sendfox.com/lp/mpe92y
Yes it’s possible though not free. I think GKE offered a master node for free but started charging for it recently.
What do you think about the latter use case where you run only what you want in your laptop? Or maybe run some in cloud some in your laptop?
I am a Kuberrnetes user but I don't see how this would be useful, at least for what I do. Have you also tried k3d for dev stuff? Merry Xmas btw :)
Merry christmas to you, too!🎄
Is k3d any different than kind? My main concern has been the high cpu/memory usage of kubernetes components of these tools.
Let’s say you run “kind create cluster” and it creates an apiserver then you add your laptop as worker node and deploy the app to the cluster. Would this be useful? I guess I’d need to run some benchmark to show how much apiserver/etcd/cm/scheduler is actually stealing from your cpu.
k3s uses k3s in Docker, and k3s is very light.
Yes but from component perspective the only big difference with k8s is etcd/sqlite and in k3s you get a few additional things like helm-controller as far as i know. SQlite is faster but if you use single multitenant etcd for N clusters, you essentially get rid of db from the picture.
Actually all my ideas about the gain here feel like very rough guesses. So I’ll go ahead and do a benchmark between different options. I created this email list just now, subscribe if you want to get notified when I post the benchmark in my blog :) Thank you for the feedback!