This is going to be a multi part blog series about setting up a Kubernetes cluster at home, on a budget. In this first post we’ll talk about the why of this, and speccing out the hardware. I plan for this to be roughly equivalent to a production k8s deployment, using TLS/SSL, enforcing policies, and connecting it to my NAS via NFS for persistent storage where needed.

Why do you want a K8s cluster at home?!

We’ll I’ve always been more of a hardware nut, I find the cloud and everything it brings to be very awesome, and much prefer using it professionally. But when it comes to doing things at home there is something about being able to go and look at the physical hardware that tickles me. On top of that running at five node cluster in the cloud on the cheap is still something like $50+ a month. There are some upsides and downsides to this.

Pros:

• One time cost for hardware
• You get to pick exactly what hardware is backing the cluster
• You can run local (home network) things without a VPN
• You can point it out to your friends who don’t understand why you waste your personal time on such weird things and think you might need a life

Cons:

• You only have your home bandwidth if you are serving public facing stuff
• It takes up space and power
• You have to maintain the hardware

The biggest con in my opinion is the bandwidth. I want to move some of the websites I run to k8s but with a measly 10mbps upload speed it isn’t really feasible. Aside from that I want to move some of my internal things over to k8s to get some more experience using it and hosting it locally lets me tear it down and do weird things with it and not have to worry about the cost of it sitting idle.

Oh you want to know why I want to run a personal k8s cluster at all? Well it is a product that fascinates me greatly, and I like doing overly complicated projects at home.

What kind of hardware to run it on

Well if you read the title you probably already know the answer, but let’s stroll down the path that led me here so you can decide if you prefer another route.

I initially had planned to run the cluster on four second hand SuperMicro servers, I had been trolling around Ebay for some time keeping an eye out for someone unloading a bunch of cheap hardware. I came close a few times to pulling the trigger, but I couldn’t justify spending ~$1000 for something that ultimately would sit idle most of the time. This would also mean that the servers sitting idle are still draining a fair amount of power. I’m ok with my server I use as a hosting point for NAS, Plex, Home Assistant, and a few other things, but multiplying that seemed way too much overkill.

I had also thought about just running the cluster inside my server. It is a pretty beefy box with 32GB of ECC memory, and 24 available cores (two six core Xeons with HyperThreading) but it is old hardware, and I frequently dick around with it and didn’t want the extra VMs hogging resources. It also just wasn’t as appealing as getting new hardware for some reason, probably because buying new stuff and putting it together is always exciting to me.

Finally I came around to the idea of running a k8s cluster on Raspberry Pis. I always shied away from the idea as they are only carrying 1GB of memory, but after hearing of some folks running it and being happy with it I started digging a bit deeper. I had gone as far on this as actually gathering a list of things to get, and the total came out to a meager $300 for a five node cluster, for everything except the cases which I planned to 3D print. Here is what I had for reference.

RPi BoM:

So for just a tad over $300, you get a five node k8s cluster! I was pretty hot on this and ready to pull the trigger. Before I did though I started researching RPi clones to see if any were offering something comparable. I came across ODROID, they have several models to offer, varying in price range, but I ultimately set my eye on the HC2 (Home Cloud 2). This was a trimmed down version of the XU4, it packs an 8 core big.LITTLE Samsung cpu (4x2.0ghz and 4x1.2ghz) which is an older SoC that was pretty commonly found in phones. It is pretty bare bones, with no GPU, and only one USB, ethernet, and a power connection on the “front”, and a SATA3 port on the back, yes, a full SATA3 port! It also comes strapped to a case that acts as a big heatsink, which is stackable, so bonus there. It is a bit more expensive, and a bit more work to get it running the way I decided to go, but here is the list for the ODROID.

ODROID BoM:

Now, the catch here is that I’ll need to wire up the plugs to the power supply, but being no stranger to these things it doesn’t present a problem for me, but keep it in mind! You can always buy a bunch of their power bricks for $7 instead.

So for a difference of $63 I am getting double the cores, faster cores, double the memory, and a full SATA port I can add storage to later. The pick seemed obvious to me, so I pulled the trigger and am anxiously awaiting my cluster to arrive.

I should note as well that I don’t bundle a switch, or anything outside getting the cluster powered up because I already have all of the infrastructure in place. So don’t forget to include that if you don’t have it.

Bonus round

With this setup, I had to ditch my idea of a tower of Pi which was going to have a fan as well to get some active cooling in there. With the HC2s having basically a giant heatsink wrapping them, and they stack, I don’t need to print a tower out, but I do still want to add a fan or two for active cooling. So once the parts come in I plan to design a “case” for the tower and supply to stay together as one unit, I’ll link to it when I’ve got it up.

This also presents a small challenge of installing Ubuntu, thankfully you can flash a pre baked image onto the SD card, and just wait for it to come online. From there I’ll assign it an IP in my router so we can leave it DHCP but it will always get the same IP, and have a .local name. I’ll post more details about this when we get there.

Link to Part 2