I just got myself some mini PC’s and want to set up a k8s lab to strengthen my skills, I was laid off 8mo ago and haven’t been able to find a new job.
I want to automate everything I can while building this and was just looking for input on how people might recommend I do it.
I was thinking of using pxeboot to install Debian using preseed to automate the installation.
Then maybe install k8s on VMs on the servers, but I’m not sure if I should do that or just keep it on the hosts themselves. They’re decently specced. AMD Ryzen 3 2200 CPUs with 24gb ram and an SSD/NVME combo.
So, any advice for where I should start or tips/tricks?
You must log in or register to comment.
I’ve been setting up a baremetal Talos-based cluster on 4 SFF machines and using Flux to declaratively deploy things to the cluster. Once you wrap your head around how Flux operates, deploying things becomes fairly straightforward.
For me, the most difficult process was deploying the cluster properly, but the recent-ish video posted on the Tailscale YT channel helped a LOT.
Check out this repo: https://github.com/khuedoan/homelab (I’m just a fan)!
I run k3s on my bare metal cluster and for a lab I think it is excellent. Gets the cluster up and running easily so you can learn about actual Kubernetes stuff.
Personally I run it on fedora coreos, but I would not recommend that for everyone. I would just use whatever distri you like. All the usual suspects will work perfectly.
I’d highly recommend Talos for this if you haven’t already decided. It’s entirely API driven and immutable. It can be bootstrapped from PXE and truly feels like a local cloud environment.
Talos is amazing and if you want to start from a fully automated setup (GitOps, Renovate), I highly recommend using https://github.com/onedr0p/cluster-template
This is the way. They have a image generator too
In the real world, you’re more likely to see multiple vms on each server and each vm running a k8s node. So I’d go with that setup.
Once you have that going, play with clustering services within k8s: Kafka, redis, etcd, postgres.
PXE is unnecessary unless you’re going to be creating a reusable boot image. Just faster to use LiveUSB.
What did you getaid off, and what are you trying to apply to? Maybe help to understand on what you’re trying to learn.
Just for your own sanity, just install Talos on the 3 machines, understand how to join them to a cluster, then deploy some stuff around the cluster. Get a feel for the basics before you get into the mess of trying to do it all in VMs.
I’d also check some comparisons on the various flavors of different lube stacks: k3s, microk8s, kubedge…etc. Theres so many now it’s hard to track.
I think I’d install proxmox on all machines because there is a proxmox provider for Terraform. Then, manually create the VMs, and to learn the barebones, use kubeadm to set everything up, and kubectl to manage it. Once comfortable and knowledgeable with that, start messing around with Terraform and Ansible.
