[{"content":"By the time TC-003 came around I\u0026rsquo;d already proven my Ansible playbooks could stand up a working Kubernetes cluster. One machine, one run, clean output. Figured that was the hard part done.\nTC-003 — my third test case for validating this automation across different hardware — was supposed to be the easy step before going public with the repo. Lenovo M920Q as the control plane, a Dell laptop joined as the worker, same playbooks that already worked once. Run it, confirm it still works, ship it.\nTen bugs later I understood why nobody trusts automation after testing it exactly once.\nWhy the second machine wrecked everything the first one didn\u0026rsquo;t My first successful run worked because everything happened to line up. The hostname in my inventory matched what the playbooks assumed. The interface name matched the hardcoded pattern. My own user already had config files sitting where they needed to be from earlier manual setup.\nNone of that was the playbooks working. That was the playbooks getting lucky.\nA genuinely different machine doesn\u0026rsquo;t get that same luck. The M920Q\u0026rsquo;s ethernet interface was eno2, not eno1 like every other ThinkCentre I\u0026rsquo;d touched. The Dell laptop connected through a USB ethernet adapter, so its interface name came out as enx00051bded752 — its MAC address wearing a trench coat. My test inventory called the nodes m920q and testnode, not master. Every one of those small differences turned out to be load-bearing, and I only found out once they all gave way at once.\nThe bugs, in the order I found them Bug 1 — checking the wrong machine. One playbook checked whether containerd\u0026rsquo;s config needed a fix by reading a file — except it read that file off the Ansible control node, not the actual target. My control node already had the fix from earlier testing, so the check always came back \u0026ldquo;already fixed\u0026rdquo; and skipped the node that actually needed it. Fix was deleting the check entirely. The underlying sed command was already idempotent — already fixed, it does nothing; not fixed, it fixes it. The condition wasn\u0026rsquo;t protecting anything, it was just lying to me.\nBug 2 — a hostname that only worked by accident. Six playbooks used hosts: master — run this only on a host literally named master. My test inventory named the control plane m920q. Every one of those plays silently skipped. Swapped them all to hosts: masters, the inventory group instead of a literal name, so it matches whatever\u0026rsquo;s in that group no matter what I called it. One spot also needed hostvars['master'] swapped to dynamically grab whoever\u0026rsquo;s first in the group. First real sign my automation wasn\u0026rsquo;t hardware-agnostic at all — it was name-agnostic in my head and hardcoded everywhere it mattered.\nBug 3 — root has no idea where the cluster lives. Several kubectl tasks run with elevated privileges, so they execute as root. Root has no kubeconfig sitting around unless you put one there — only my own user did, from setup I\u0026rsquo;d done by hand. Without it, kubectl tries localhost:8080, gets refused, dies. Pointed those tasks at /etc/kubernetes/admin.conf instead, since that one always exists after kubeadm init.\nBug 4 — right command, wrong namespace. A Calico fix targeted calico-system. On a kubeadm cluster, Calico lives in kube-system. Nothing wrong with the command, it was just knocking on the wrong door.\nBug 5 — a backslash that had no business being there. kubeadm init prints its join command across two lines, joined by a line-continuation backslash. My playbook stitched those two lines together with a space, but the literal backslash and extra whitespace came along uninvited, so kubeadm join read it as two broken arguments instead of one. Fixed with a regex that strips the backslash and cleans up the whitespace before the command runs. The kind of bug you only find once you actually parse real output instead of assuming it\u0026rsquo;ll look the way you imagined.\nBug 6 — assuming a tool exists because it existed last time. A handful of tasks install things with Helm. Helm itself was never installed by the automation — it was just sitting on my first machine because I\u0026rsquo;d put it there manually months ago and forgot that wasn\u0026rsquo;t part of the script. Added tasks to download and install it first, with creates: /usr/local/bin/helm so it skips itself once it\u0026rsquo;s already there.\nBug 7 — same root problem, different tool. Identical story to Bug 3, just for every Helm task instead of every kubectl task. Same fix.\nBug 8 — a chart update I never asked for. MetalLB picked up a new subchart dependency (frr-k8s) sometime between my last test and this one. It expected Prometheus monitoring values that didn\u0026rsquo;t exist, and without them it died with a nil pointer error buried in its template logic. Disabled the serviceMonitor option it wanted. Not really my bug — more a reminder that \u0026ldquo;worked last time\u0026rdquo; has an expiration date the second you\u0026rsquo;re pulling from an upstream chart repo that ships changes whether you asked for them or not.\nBug 9 — the one that took down both nodes at once. This was the real one. Calico needs to figure out which interface to use, and I\u0026rsquo;d told it exactly which names to look for — eno1 and enp0s31f6. The M920Q was eno2. The Dell laptop was that USB-adapter name. Neither matched. Calico crashed on both machines with the same error — couldn\u0026rsquo;t auto-detect an interface.\nFirst instinct was widening the pattern to catch more eno-style names. Didn\u0026rsquo;t help — enx is a different naming scheme entirely, no amount of widening an eno regex gets you there. Considered just letting Calico grab whatever interface it finds first, then talked myself out of it — any node running Tailscale has a tailscale0 virtual interface sitting right there, and \u0026ldquo;first found\u0026rdquo; gives zero guarantee it picks the real ethernet connection over that.\nWhat actually worked:\nIP_AUTODETECTION_METHOD=skip-interface=tailscale.*\nStop guessing what the right interface is called. Exclude the one you already know is wrong. Calico finds whatever real ethernet interface exists — Lenovo\u0026rsquo;s name for it, Dell\u0026rsquo;s name for it, whatever the next machine calls it — because the only thing it\u0026rsquo;s told to avoid is Tailscale\u0026rsquo;s virtual interface.\nBug 10 — a typo. The ArgoCD chart reference was argocd/argocd. The real chart name is argocd/argo-cd. One character off, fails every time, no partial credit.\nWhere it landed m920q : ok=61 changed=18 unreachable=0 failed=0 skipped=2 rescued=0 ignored=1 testnode: ok=25 changed=3 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0\nBoth nodes Ready. Calico, MetalLB, Nginx Ingress, cert-manager, Sealed Secrets, ArgoCD, Prometheus, Grafana, Loki, Promtail all came up clean. TC-003 passed.\nWhat ten bugs in one sitting actually tells you None of these are exotic. Wrong namespace. A hostname pattern that only matched one specific word. A missing kubeconfig path. A tool that was never actually installed. A typo in a chart name. On their own, every one of these is kind of boring.\nWhat\u0026rsquo;s not boring is the pattern. Every single one was invisible on the first machine because the first machine happened to agree with whatever assumption I\u0026rsquo;d buried in the code without realizing it. The hostname matched. The interface matched. Helm was already sitting there from manual setup I\u0026rsquo;d half-forgotten about. None of that was the automation working — it was the automation never being asked a question it couldn\u0026rsquo;t answer, until a second, genuinely different machine asked it ten in a row.\nThat\u0026rsquo;s the actual point of a test like TC-003. Not proving the happy path works — I already knew it did. Proving the thing survives contact with hardware it\u0026rsquo;s never met. Ten bugs found and killed in one night is ten fewer ways this breaks on whatever shows up next.\n","permalink":"https://beyondthecert.dev/posts/tc-003-ten-bugs/","summary":"\u003cp\u003eBy the time TC-003 came around I\u0026rsquo;d already proven my Ansible playbooks could stand up a working Kubernetes cluster. One machine, one run, clean output. Figured that was the hard part done.\u003c/p\u003e\n\u003cp\u003eTC-003 — my third test case for validating this automation across different hardware — was supposed to be the easy step before going public with the repo. Lenovo M920Q as the control plane, a Dell laptop joined as the worker, same playbooks that already worked once. Run it, confirm it still works, ship it.\u003c/p\u003e","title":"TC-003: Ten Bugs Stood Between My Playbooks and a Public Repo"},{"content":"Last weekend I moved apartments. Five Kubernetes nodes, a NAS, and a switch came with me. By the time I was done unpacking boxes, the cluster was already back online — and I barely had to touch the network configuration to make it happen.\nThat wasn\u0026rsquo;t luck. Months ago, when I first built this cluster, I already knew a move was on the table at some point — nothing concrete, just the normal background awareness that apartments don\u0026rsquo;t last forever. So I made a deliberate call: route the whole cluster over Tailscale IPs instead of local network addresses. I figured if the day ever came, I didn\u0026rsquo;t want to be reconfiguring DHCP reservations and static routes while also carrying boxes down three flights of stairs.\nI just never got to actually test that decision until the day it mattered.\nThe setup Genesis1 is my homelab — five bare metal Kubernetes nodes, three control planes and two workers, plus a Synology NAS handling storage and backups. It runs a self-hosted photo library, a password manager, and an automation pipeline that pulls my Oura Ring and Apple Health data every night and dumps it into a time-series database for tracking with my coach.\nEvery node talks to every other node over Tailscale. Not as a remote-access VPN bolted on the side — as the actual network the cluster lives on. etcd traffic, API server communication, kubelet checking in with the control plane, all of it rides on 100.x.x.x Tailscale addresses instead of the usual 192.168.x.x you\u0026rsquo;d get from your router.\nLocal IPs are tied to whatever network you happen to be plugged into. Tailscale IPs follow the device. Plug the same machine into a different router and the Tailscale address doesn\u0026rsquo;t care. That was the whole bet.\nShutdown day Before any of it went into a truck, the cluster had to come down clean. Workers drained first, control planes after, NAS last. A couple of stateful workloads had PodDisruptionBudgets that pushed back on the drain — InfluxDB and n8n both refused to evict because losing their single replica would technically violate the budget. For a one-time full shutdown that\u0026rsquo;s fine to override:\nkubectl drain worker1 \u0026ndash;ignore-daemonsets \u0026ndash;delete-emptydir-data \u0026ndash;disable-eviction\nThat flag bypasses the PDB entirely instead of waiting around for an eviction that\u0026rsquo;s never going to be allowed. Repeated that across both workers, then the three control planes in reverse order of how much I\u0026rsquo;d miss them if something went wrong — third control plane first, then the second, then the main one last. SSH\u0026rsquo;d into the NAS and shut it down properly so the RAID array wouldn\u0026rsquo;t get cranky about it.\nThen it was just furniture-moving for the rest of the day.\nPower-on at the new place NAS first, give it a couple minutes to fully boot, then the control planes, then the workers — same order as shutdown, reversed. Within about five minutes of flipping everything on, every node had quietly reconnected to Tailscale on its own. No re-keying, no manual reauth, nothing.\nkubectl get nodes\nNAME STATUS ROLES AGE VERSION master Ready,SchedulingDisabled control-plane 67d v1.32.13 master2 Ready,SchedulingDisabled control-plane 67d v1.32.13 master3 Ready,SchedulingDisabled control-plane 7d9h v1.32.13 worker1 Ready,SchedulingDisabled worker 67d v1.32.13 worker2 Ready,SchedulingDisabled worker 67d v1.32.13\nAll five Ready, just sitting cordoned from the earlier drain. Uncordoned everything and moved on:\nThat part went exactly the way I\u0026rsquo;d hoped. The cluster genuinely did not seem to notice it had changed buildings.\nThe one thing that didn\u0026rsquo;t survive the move Not everything turned out to be network-agnostic. One of my control planes — a small desktop I\u0026rsquo;d added a few weeks earlier specifically to get a third etcd member — picked up a different local IP from the new apartment\u0026rsquo;s DHCP than it had at the old place. 192.168.1.169 became 192.168.1.168. One digit.\n{\u0026ldquo;level\u0026rdquo;:\u0026ldquo;error\u0026rdquo;,\u0026ldquo;msg\u0026rdquo;:\u0026ldquo;creating peer listener failed\u0026rdquo;,\u0026ldquo;error\u0026rdquo;:\u0026ldquo;listen tcp 192.168.1.169:2380: bind: cannot assign requested address\u0026rdquo;}\nNo etcd, no API server on that node. One of my three control planes was effectively dead.\nThe fix was a few steps, none of them hard, just things I had to work through one at a time. First, point the static pod manifests at the new IP:\nsudo sed -i \u0026rsquo;s/192.168.1.169/192.168.1.168/g\u0026rsquo; /etc/kubernetes/manifests/etcd.yaml sudo sed -i \u0026rsquo;s/192.168.1.169/192.168.1.168/g\u0026rsquo; /etc/kubernetes/manifests/kube-apiserver.yaml\nThen etcd kept getting rejected with tls: bad certificate — because even though the manifests now pointed at the right IP, the actual peer certificate\u0026rsquo;s Subject Alternative Name still said 192.168.1.169. Renewing the cert with kubeadm\u0026rsquo;s built-in renew command didn\u0026rsquo;t pick up the new IP either; it just reissued the same cert with the same stale SAN. Had to delete it and regenerate from scratch:\nsudo rm /etc/kubernetes/pki/etcd/peer.crt /etc/kubernetes/pki/etcd/peer.key sudo kubeadm init phase certs etcd-peer\nThat generated a fresh cert with the correct IP in it. Last step, tell the rest of the etcd cluster about the node\u0026rsquo;s new peer address so the other two members would stop rejecting its handshake:\nkubectl exec -n kube-system etcd-master \u0026ndash; etcdctl \u0026ndash;endpoints=https://127.0.0.1:2379 \u0026ndash;cacert=/etc/kubernetes/pki/etcd/ca.crt \u0026ndash;cert=/etc/kubernetes/pki/etcd/server.crt \u0026ndash;key=/etc/kubernetes/pki/etcd/server.key member update 23130d76a17b1dd1 \u0026ndash;peer-urls=https://192.168.1.168:2380\nDeleted the etcd pod on that node to force a clean restart with the new cert and updated peer URL, and within a couple of minutes it rejoined and the cluster had full quorum again.\nThe whole thing took about half an hour, mostly spent reading logs to figure out which layer was actually broken — first the manifest\u0026rsquo;s IP, then the cert\u0026rsquo;s SAN, then the membership record. Three different things all pointing at the same underlying problem, which is a good way to learn that \u0026ldquo;TLS handshake failed\u0026rdquo; rarely tells you exactly which of those three it is on the first try.\nWhat actually held up Everything else came back without me touching it:\nAll five nodes reconnected to Tailscale on their own, no manual steps ArgoCD picked up right where it left off and reconciled every application Ingress, internal DNS, service-to-service traffic — all fine, none of it ever depended on local IPs to begin with The health data pipeline ran its nightly sync that same night like nothing had happened The only thing that needed hands-on attention was the one piece whose security model — not its networking — was still tied to a local address.\nThe actual lesson I didn\u0026rsquo;t design this cluster to be portable as some kind of clever flex. I designed it that way because I had a feeling I\u0026rsquo;d be moving eventually and didn\u0026rsquo;t want network config to be one more thing to deal with on top of everything else moving day brings. Most of the stack honored that decision without me having to think about it again. One layer — a certificate issued weeks earlier when I added that third control plane — quietly didn\u0026rsquo;t, because at the time, certs and local IPs were just a mechanical detail of joining a node, not something I cross-checked against the bigger architectural bet I\u0026rsquo;d already made.\nLesson noted for next time: when a node joins this cluster, double check what its certs are actually bound to before assuming \u0026ldquo;it\u0026rsquo;s all on Tailscale anyway\u0026rdquo; covers it. It mostly does. It\u0026rsquo;s the part that doesn\u0026rsquo;t that finds you on moving day.\n","permalink":"https://beyondthecert.dev/posts/i-moved-my-homelab-tailscale/","summary":"\u003cp\u003eLast weekend I moved apartments. Five Kubernetes nodes, a NAS, and a switch came with me. By the time I was done unpacking boxes, the cluster was already back online — and I barely had to touch the network configuration to make it happen.\u003c/p\u003e\n\u003cp\u003eThat wasn\u0026rsquo;t luck. Months ago, when I first built this cluster, I already knew a move was on the table at some point — nothing concrete, just the normal background awareness that apartments don\u0026rsquo;t last forever. So I made a deliberate call: route the whole cluster over Tailscale IPs instead of local network addresses. I figured if the day ever came, I didn\u0026rsquo;t want to be reconfiguring DHCP reservations and static routes while also carrying boxes down three flights of stairs.\u003c/p\u003e","title":"I Moved My Homelab Across Town and Let Tailscale Handle the Network"},{"content":"When people ask me what made me decide to automate my homelab cluster, they expect a story about frustration — some late night where everything broke and I snapped. That\u0026rsquo;s not what happened.\nThe first time I built a bare metal Kubernetes cluster manually, it took me a week and countless hours. Manual steps, configuration files, troubleshooting, and learning. And when it was done, I looked at it and thought: the next evolution of this is automation. Not because I was tired of doing it manually — but because I wanted to be able to reproduce it at a moment\u0026rsquo;s notice.\nThat idea became Kubernetes The Homelab Way.\nWhat It Is Kubernetes The Homelab Way is a public repo that takes you from bare metal hardware to a fully running Kubernetes cluster with a push of a button. We\u0026rsquo;re talking:\nOS preparation and kernel configuration containerd runtime installation and configuration kubeadm, kubelet, kubectl installation — always fetching the latest stable version dynamically Cluster initialization with Calico CNI Worker node join and multi-master HA support Full stack deployment: MetalLB, Nginx Ingress, cert-manager, Sealed Secrets, ArgoCD Observability: Prometheus, Grafana, Loki, Promtail What took me a week now takes about 30 minutes. Conservatively.\nAnd here\u0026rsquo;s the part I\u0026rsquo;m proud of: it always fetches the latest version of both Kubernetes and Ubuntu. You could run this today or a year from now and you will always be working with current, supported software. No hardcoded versions going stale. Every component is deployed via Helm charts fetching the latest stable releases — you\u0026rsquo;re never working with outdated configurations.\nHow It Works If you handed me a brand new x86_64 machine still in the box, here\u0026rsquo;s exactly what happens:\nClone the repo Update your inventory file — hostname, IP address, network interface. That\u0026rsquo;s it. Run ansible-playbook site.yml The automation handles everything from there. It preps the OS, installs containerd so pods can communicate, installs the Kubernetes tooling, initializes the control plane, joins your worker nodes and additional control plane nodes for HA, and deploys the full stack. By the time it\u0026rsquo;s done you have a production-grade cluster with GitOps, load balancing, TLS certificate management, and full observability — running on hardware you own.\nThe Gap Between CKA and Production There\u0026rsquo;s a conversation that doesn\u0026rsquo;t happen enough in the Kubernetes community: the gap between passing the CKA and actually operating production Kubernetes.\nThe CKA teaches you Kubernetes conceptually. It teaches you kubectl commands. It gives you a mental model of how the pieces fit together. That\u0026rsquo;s valuable — I passed it and it shaped how I think.\nBut operating production Kubernetes is different. How it\u0026rsquo;s deployed depends on the company. What\u0026rsquo;s running on it depends on the team. The tooling around it — GitOps, service mesh, observability, secrets management — none of that is on the exam.\nThis repo bridges that gap. It exposes you to the full operational picture: not just the cluster but everything that makes a cluster production-worthy. You see how ArgoCD manages deployments. You see how cert-manager issues TLS certificates automatically. You see how Prometheus and Grafana give you visibility into what\u0026rsquo;s happening. That operational intuition is what separates someone who passed a cert from someone who can run infrastructure.\nThe Hardest Bug There were a lot of bugs. I ran test after test — at one point the same validation failed 15 times in a single night before it finally passed. Each failure revealed another edge case in how components interact.\nBut the one that took the longest was the Calico/Tailscale conflict.\nHere\u0026rsquo;s the thing about Kubernetes: every component in your cluster needs to know the IP addresses of every other component. Those IPs need to be stable. If you don\u0026rsquo;t assign static IPs, addresses can change and break the cluster.\nI use Tailscale — a private VPN service that assigns permanent IPs to all my nodes regardless of what network they\u0026rsquo;re on. It\u0026rsquo;s what lets my cluster survive moving apartments. But Calico, the CNI plugin that handles pod networking, was auto-detecting Tailscale\u0026rsquo;s virtual interface instead of the actual ethernet interface. That caused a conflict at the networking layer that took multiple iterations to solve correctly.\nThe fix once we understood the root cause:\nIP_AUTODETECTION_METHOD=skip-interface=tailscale.*\nTell Calico to use any interface except Tailscale\u0026rsquo;s. Hardware agnostic. Works on any x86_64 machine regardless of what the ethernet interface is named. Not everyone will use Tailscale. But for my use case it was essential, and now it\u0026rsquo;s baked in as a conditional fix that only runs when Tailscale is detected.\nWhy Bare Metal? There\u0026rsquo;s a question I get asked: why not just use EKS or GKE?\nManaged services make sense for production businesses — you\u0026rsquo;re not paying engineers to babysit control planes. But for someone filling in foundational gaps, managed services abstract away exactly what I needed to learn.\nWhen EKS provisions a cluster, you don\u0026rsquo;t see how the control plane comes together. You don\u0026rsquo;t see CNI plugin behavior. The abstraction that makes managed services powerful is the same abstraction that hides the learning.\nBoth have their place. I just know where I am in my journey and what I needed to get to the next level.\nHow to Use It All you need is a spare x86_64 machine — any spare machine. Old laptop, mini PC, whatever\u0026rsquo;s catching dust. The automation installs everything. There\u0026rsquo;s documentation. Honestly just bring patience, maybe some coffee, and a couple of spare machines and you\u0026rsquo;ll have a working cluster.\nThe one thing you\u0026rsquo;ll need to customize is your inventory file: hostname, IP address, network interface name. That\u0026rsquo;s the only machine-specific configuration. Everything else is handled.\nWhy I\u0026rsquo;m Sharing It A year ago I didn\u0026rsquo;t know any of this. I was figuring out Kubernetes one error message at a time. I have connections in the industry. I know people who are exactly where I was 12 months ago.\nThis is a letter to them. And to anyone else who wants to build something real instead of clicking through a managed service.\nKelsey Hightower shared Kubernetes The Hard Way for free. A lot of people in this community share their work for free. That spirit is why I\u0026rsquo;m here. Someone contributed something that helped me — this is my way of contributing back.\nIf this saves someone a week of manual work, it was worth building.\nWhat Surprised Me I thought I\u0026rsquo;d say the technical complexity surprised me. Or the number of bugs. Or how long debugging took.\nBut what actually surprised me most is that I did it. I followed through. There\u0026rsquo;s a particular kind of beauty in seeing something that lived only in your mind become something that\u0026rsquo;s alive — running on hardware in your apartment, deploying workloads, accepting traffic. It\u0026rsquo;s there. It\u0026rsquo;s doing its job.\nAnd the other thing that surprised me is the person I\u0026rsquo;m becoming. A year ago my Kubernetes skills were light. I was learning what a pod was. Now I\u0026rsquo;m the person building this, documenting it, and sharing it with the community. That\u0026rsquo;s not the same person. The journey isn\u0026rsquo;t done — but that transformation is real.\nWhat also surprised me is how much fun I\u0026rsquo;m having. This doesn\u0026rsquo;t feel like work. And I say that knowing it can blur the line between my actual job and my hobby — because this cluster is going to need maintenance, upgrades, and attention on a regular basis. That\u0026rsquo;s a real commitment. But somehow that doesn\u0026rsquo;t feel like a burden. It feels like exactly where I want to be spending my time.\nRepo: github.com/BeyondTheCert/Kubernetes-The-Homelab-Way\nBlog: beyondthecert.dev\nIf you build something with it, let me know. If you find a bug, open an issue. If this helped you, share it with someone who needs it.\n","permalink":"https://beyondthecert.dev/posts/from-10-days-to-30-minutes/","summary":"\u003cp\u003eWhen people ask me what made me decide to automate my homelab cluster, they expect a story about frustration — some late night where everything broke and I snapped. That\u0026rsquo;s not what happened.\u003c/p\u003e\n\u003cp\u003eThe first time I built a bare metal Kubernetes cluster manually, it took me a week and countless hours. Manual steps, configuration files, troubleshooting, and learning. And when it was done, I looked at it and thought: the next evolution of this is automation. Not because I was tired of doing it manually — but because I wanted to be able to reproduce it at a moment\u0026rsquo;s notice.\u003c/p\u003e","title":"From 10 Days to 30 Minutes: Why I Automated My Bare Metal Kubernetes Cluster"},{"content":"I wasn\u0026rsquo;t doing anything dramatic. No major deployment. No risky configuration change.\nI was trying to add a NAS to my homelab. New hardware means more devices, more devices means more outlets, more outlets means buying a power strip. What nobody tells you when you start this journey is that cable management becomes its own project the moment hardware starts multiplying. I ran out of space on my existing surge protector so I needed to plug the new one in somewhere close to the router, close to the switch, close to everything.\nSo I unplugged the router to free up an outlet, plugged the power strip in, then plugged the router back into the strip. Simple enough.\nRouter went down. Internet went down. Expected. Thirty seconds later everything came back — except my cluster.\nThe First Sign FreeLens couldn\u0026rsquo;t connect. The error was a dial TCP timeout trying to reach port 6443 on master. That\u0026rsquo;s the Kubernetes API server port. If you can\u0026rsquo;t reach 6443 you can\u0026rsquo;t talk to the cluster at all.\nI SSH\u0026rsquo;d into master directly since Tailscale was still up. The node was reachable but kubectl wasn\u0026rsquo;t working. The API server wasn\u0026rsquo;t responding.\nThe Swap Problem First thing I checked was kubelet. It was crash looping. The logs told me exactly why: failed to run Kubelet: running with swap on is not supported, please disable swap\nWhen master rebooted after the power interruption swap came back on. Kubernetes requires swap to be disabled — it can\u0026rsquo;t manage memory properly when the OS is swapping to disk. Kubelet refuses to start with swap enabled.\nThe fix is three commands:\nsudo swapoff -a sudo sed -i \u0026#39;/ swap / s/^\\(.*\\)$/#\\1/g\u0026#39; /etc/fstab sudo rm -f /swap.img The first turns swap off immediately. The second comments it out of fstab so it doesn\u0026rsquo;t come back on reboot. The third deletes the swap file entirely — the permanent fix.\nI ran those on master. Kubelet came back. But the cluster still wasn\u0026rsquo;t healthy.\nThat\u0026rsquo;s because swap had come back on every node that rebooted — not just master. I had to SSH into master2, worker1, and worker2 and run the same three commands on each one.\nThe etcd Problem Even after fixing swap on all four nodes the cluster still wouldn\u0026rsquo;t fully recover. The API server kept crashing with an authentication error trying to reach etcd.\nHere\u0026rsquo;s what happened: master2 went down abruptly when the router dropped. etcd is the database that stores everything about your cluster — every pod, every config, every secret. It\u0026rsquo;s designed so that multiple copies run across your masters and they stay in sync by agreeing with each other before making any changes.\nThe key word is agreeing. With two masters and one suddenly offline the remaining copy couldn\u0026rsquo;t make decisions on its own — it needed the other one to confirm. So it froze. Nothing could be written. The API server couldn\u0026rsquo;t function.\nWhen master2 came back it had to recover from an unclean shutdown. The two etcd instances hadn\u0026rsquo;t reestablished their connection yet. I had to stop etcd on both nodes and let kubelet restart them so they could find each other and sync back up.\nOnce both were talking again the API server came back, the cluster recovered, and everything went back to normal.\nThe Real Lesson Three things I\u0026rsquo;d do differently.\n1. Get a UPS.\nA UPS would have kept the router powered during the whole outlet swap. No internet interruption, no reboot trigger, no incident. It doesn\u0026rsquo;t prevent you from physically unplugging things but it protects against accidental power loss while you\u0026rsquo;re working around your gear. It\u0026rsquo;s on my list.\n2. Bring the cluster down gracefully before touching anything.\nThis is the real one. Before you touch any network gear, any power infrastructure, anything physical — drain and shut down the cluster first.\nkubectl drain worker1 --ignore-daemonsets --delete-emptydir-data kubectl drain worker2 --ignore-daemonsets --delete-emptydir-data sudo shutdown -h now Do your physical work. Power everything back on. Let the cluster come up clean. No swap surprise, no etcd quorum loss, no debugging session.\nSame discipline you\u0026rsquo;d use in a data center. Apply it at home too.\n3. Run three masters, not two.\nTwo masters sounds like redundancy. It isn\u0026rsquo;t.\nWith two copies of etcd you need both of them alive and talking. Lose one and the cluster freezes. You have no high availability — you have a single point of failure split across two machines.\nWith three masters you only need two of them to agree. Lose one and the other two keep the cluster running. You have time to fix the dead node without the whole thing going down.\nI\u0026rsquo;m adding a third master. A small form factor machine — 8GB RAM is enough since masters only run control plane components. The cost is low. The peace of mind is worth it.\nWhat to Do Right Now If you just built a bare metal Kubernetes cluster:\nPermanently disable swap on every node — now, not after the next reboot surprises you:\nsudo swapoff -a sudo sed -i \u0026#39;/ swap / s/^\\(.*\\)$/#\\1/g\u0026#39; /etc/fstab sudo rm -f /swap.img Always drain and shut down gracefully before touching physical infrastructure.\nPlan for a third master.\nOne unplugged cable took my cluster down for two hours. These three things would have prevented all of it. Hopefully this saves you the same two hours.\n","permalink":"https://beyondthecert.dev/posts/one-unplugged-cable/","summary":"\u003cp\u003eI wasn\u0026rsquo;t doing anything dramatic. No major deployment. No risky configuration change.\u003c/p\u003e\n\u003cp\u003eI was trying to add a NAS to my homelab. New hardware means more devices, more devices means more outlets, more outlets means buying a power strip. What nobody tells you when you start this journey is that cable management becomes its own project the moment hardware starts multiplying. I ran out of space on my existing surge protector so I needed to plug the new one in somewhere close to the router, close to the switch, close to everything.\u003c/p\u003e","title":"One Unplugged Cable. Two Hours of Debugging. Here's What Broke My Bare Metal Kubernetes Cluster."},{"content":"If you\u0026rsquo;re building a bare metal Kubernetes cluster with Calico and you\u0026rsquo;ve also installed Tailscale for remote access — read this before you spend hours debugging pod networking.\nI didn\u0026rsquo;t. I spent the hours. Here\u0026rsquo;s what I learned.\nThe Setup Four nodes. Two ThinkPad T480s as control plane. Two ThinkCentre M720q desktops as workers. I installed Tailscale on every node so the cluster stays reachable across apartment moves — Tailscale IPs don\u0026rsquo;t change even when your home network does.\nStack: kubeadm, Calico v3.29, Tailscale, Ubuntu Server 24.04.\nWhat Broke Workers joined the cluster fine. Then things got weird.\nPods on master could reach the API server. Pods on workers couldn\u0026rsquo;t. Every connection to 10.96.0.1:443 — the ClusterIP for the API server — timed out. MetalLB kept crashing. Same error, over and over.\nThe workloads weren\u0026rsquo;t the problem. The nodes were.\nBGP — What It Is and Why It Matters Here Calico uses BGP internally to tell nodes where pods live. Each node manages a range of pod IPs and advertises that range to every other node. That\u0026rsquo;s how a pod on master knows to route traffic through worker1 to reach a pod living there.\nFor BGP to work, each node registers a peering address — the IP that other nodes send routing updates to. Calico picks this automatically by scanning your network interfaces.\nThat auto-detection is where things went wrong.\nThe Conflict When you install Tailscale on a Linux machine it creates a virtual interface called tailscale0. From the OS perspective it looks like any other network interface — same as your ethernet port, just virtual.\nCalico scanned my interfaces, saw tailscale0, and chose it over the actual ethernet interfaces. Every node ended up registering a Tailscale IP as its BGP peering address instead of its real ethernet IP.\nThe problem: BGP peering over Tailscale doesn\u0026rsquo;t work. Tailscale is an encrypted overlay VPN — it doesn\u0026rsquo;t carry routing protocols. The BGP sessions never came up. Pod routes were never exchanged. Workers had no idea where anything lived.\nThat\u0026rsquo;s the whole story. MetalLB runs as a pod on a worker. Worker pods can\u0026rsquo;t reach the API server. MetalLB crashes. Repeat.\nThe Fix Two commands. Both matter.\nStep 1 — Tell Calico which interfaces to use:\nkubectl set env daemonset/calico-node -n kube-system \\ IP_AUTODETECTION_METHOD=interface=eno1,enp0s31f6 eno1 is the NIC name on the ThinkCentres. enp0s31f6 is the NIC name on the ThinkPads. The pipe-separated list tells Calico to use whichever one exists on the current node. This fixes future restarts.\nStep 2 — Fix the wrong addresses already in Calico\u0026rsquo;s datastore:\ncalicoctl patch node worker1 \\ --patch=\u0026#39;{\u0026#34;spec\u0026#34;:{\u0026#34;bgp\u0026#34;:{\u0026#34;ipv4Address\u0026#34;:\u0026#34;\u0026lt;worker1-ethernet-ip\u0026gt;/24\u0026#34;}}}\u0026#39; calicoctl patch node worker2 \\ --patch=\u0026#39;{\u0026#34;spec\u0026#34;:{\u0026#34;bgp\u0026#34;:{\u0026#34;ipv4Address\u0026#34;:\u0026#34;\u0026lt;worker2-ethernet-ip\u0026gt;/24\u0026#34;}}}\u0026#39; Replace \u0026lt;worker1-ethernet-ip\u0026gt; and \u0026lt;worker2-ethernet-ip\u0026gt; with the actual ethernet IPs of your worker nodes. Run ip addr show on each node if you\u0026rsquo;re not sure.\nStep 1 alone isn\u0026rsquo;t enough. The wrong addresses are already registered. You have to correct them in the datastore or Calico keeps peering over Tailscale on the next restart.\nAfter both steps Calico re-established BGP over ethernet. Pod routes came back. MetalLB stopped crashing.\nWhat to Remember If you\u0026rsquo;re running Calico and Tailscale on the same nodes — set IP_AUTODETECTION_METHOD before you join your workers. Don\u0026rsquo;t wait for things to break.\nCalico sees tailscale0 as a valid interface. Without explicit guidance it may choose it. On some machines it will. On others it won\u0026rsquo;t. It depends on interface ordering and your specific hardware.\nDon\u0026rsquo;t leave it to chance.\n","permalink":"https://beyondthecert.dev/posts/calico-tailscale-bgp-conflict/","summary":"\u003cp\u003eIf you\u0026rsquo;re building a bare metal Kubernetes cluster with Calico and you\u0026rsquo;ve also installed Tailscale for remote access — read this before you spend hours debugging pod networking.\u003c/p\u003e\n\u003cp\u003eI didn\u0026rsquo;t. I spent the hours. Here\u0026rsquo;s what I learned.\u003c/p\u003e\n\u003ch2 id=\"the-setup\"\u003eThe Setup\u003c/h2\u003e\n\u003cp\u003eFour nodes. Two ThinkPad T480s as control plane. Two ThinkCentre M720q desktops as workers. I installed Tailscale on every node so the cluster stays reachable across apartment moves — Tailscale IPs don\u0026rsquo;t change even when your home network does.\u003c/p\u003e","title":"Calico and Tailscale Have a BGP Conflict. Here's Exactly What Breaks and How to Fix It."},{"content":"Project Genesis1: I Built a 4-Node Kubernetes Cluster on Bare Metal. Here\u0026rsquo;s What I Learned. At work, we provision managed Kubernetes clusters. Click a button, get a cluster. Everything abstracted. Everything handled. At home? I built every piece myself. Two control planes. Two workers. GitOps pipeline. Monitoring stack. The blog you\u0026rsquo;re reading right now? It\u0026rsquo;s running on this infrastructure. Why build it? To prove to myself I understand what\u0026rsquo;s underneath the abstraction. To own the gaps instead of hiding them. To build the kind of operational depth that doesn\u0026rsquo;t come from any exam. This is Project Genesis1. And you\u0026rsquo;re standing on it.\nThe Abstraction Problem When you use a managed service or cloud provider, everything is abstracted away. You interact with the cluster, but you don\u0026rsquo;t know what\u0026rsquo;s going on underneath. You get the whole picture without knowing what it took to get there. Going bare metal forces you to deal with constraints. You bootstrap every component yourself — the API server, the scheduler, the controller manager, etcd. You configure networking. You handle storage. You see how the pieces actually fit together. It\u0026rsquo;s like putting a puzzle together piece by piece versus getting the completed puzzle handed to you. After passing CKA in January, I wanted to go deeper. The exam taught me how to USE Kubernetes. It didn\u0026rsquo;t teach me how to BUILD it. How to troubleshoot when BGP peering fails. How to debug redirect loops between Cloudflare and ArgoCD. How to make a cluster survive when you move apartments. I needed operational depth. The kind you only get by breaking things and fixing them yourself.\nThe Hardware\nThe idea evolved over a year. It started with Mischa van den Burg\u0026rsquo;s YouTube videos about building clusters from old hardware. Then it became two ThinkPads as control planes and Raspberry Pis as workers. Then Dell minis. Then, when budget constraints hit, Lenovo ThinkCentre M720q tiny desktops. The final setup: Two Lenovo ThinkPad T480s as control planes for high availability. Two ThinkCentre M720q tiny desktops as workers — 16GB RAM each, 32GB total across workers. All connected via Tailscale mesh network. Tailscale solved a critical problem. I\u0026rsquo;m moving apartments in June — different location, different router, different IP addresses. If I hardcoded home network IPs into the cluster configuration, the move would break everything. I\u0026rsquo;d have to rebuild from scratch. Tailscale assigns static IPs that never change. The cluster uses Tailscale IPs for all internal communication. When I move, the home network IPs change, but the Tailscale IPs stay the same. The cluster doesn\u0026rsquo;t even notice.\nThe Build\nAll 4 nodes Ready. master and master2 as control planes, worker1 and worker2 as workers. v1.32.13.\nI ran kubeadm init on the first control plane. Joined the second control plane for HA. Joined both workers. Installed Calico for pod networking. Then things broke. The Calico/Tailscale conflict: Pods on worker nodes couldn\u0026rsquo;t reach the Kubernetes API server. MetalLB kept crashing with dial tcp 10.96.0.1:443: i/o timeout errors. The problem? Calico uses BGP to advertise pod network routes between nodes. When Calico auto-detected IP addresses, it picked up the Tailscale interface instead of the ethernet interface. Tailscale doesn\u0026rsquo;t support BGP peering. The pod network routes were never established. I diagnosed it by checking the Calico logs: Using autodetected IPv4 address on interface tailscale0. The fix required two steps. First, tell Calico which interface to use: bashkubectl set env daemonset/calico-node -n kube-system IP_AUTODETECTION_METHOD=interface=eno1,enp0s31f6 Second, patch each worker node\u0026rsquo;s BGP address using calicoctl to replace the Tailscale IPs with the home network IPs. Both steps required — one without the other is incomplete. This is the kind of issue you never encounter in managed Kubernetes. And it\u0026rsquo;s exactly why I built this. The ArgoCD redirect loop: After deploying ArgoCD and exposing it through Cloudflare Tunnel, visiting argocd.beyondthecert.dev threw ERR_TOO_MANY_REDIRECTS. ArgoCD\u0026rsquo;s server forces HTTPS by default — any HTTP request gets a 301 redirect to HTTPS. Cloudflare was set to Full SSL mode, which expects HTTPS from the origin server and also redirects HTTP to HTTPS. Infinite loop. The browser kept bouncing between Cloudflare and ArgoCD, each one redirecting to the other. The fix: disable HTTPS enforcement on the ArgoCD server by patching the ConfigMap with server.insecure: \u0026ldquo;true\u0026rdquo;. Then change Cloudflare\u0026rsquo;s SSL mode from Full to Flexible. Flexible means Cloudflare handles HTTPS with visitors but connects to the origin over HTTP. Loop broken. After five days of building and debugging, everything clicked. ArgoCD watching GitHub. Prometheus scraping metrics. Grafana dashboards showing real-time cluster state. Loki collecting logs from every pod on every node. This blog deployed via GitOps. Then I cordoned all four nodes and unplugged the network switch to organize cables. Everything came back. That\u0026rsquo;s when I stopped being scared of my own cluster. The final stack: ComponentPurposeKubernetesOrchestrationCalicoPod networkingMetalLBLoad balancingNginx IngressTraffic routingcert-managerTLS automationArgoCDGitOpsPrometheus + GrafanaMonitoringLokiCentralized loggingSealed SecretsEncrypted secretsCloudflare TunnelPublic accessTailscaleRemote SSHHugo + GitHub ActionsBlog + CI/CD\nArgoCD watching Genesis1-GitOps on GitHub. Push markdown, blog updates automatically.\nWrite a markdown file. Git push. GitHub Actions builds a Docker image. ArgoCD syncs the new image. The blog updates automatically.\nWhat It Teaches\nReal-time cluster metrics. CPU, memory, pod counts across all namespaces.\nCKA taught me kubectl commands and how to troubleshoot pods. It didn\u0026rsquo;t teach me how Calico\u0026rsquo;s BGP peering works. It didn\u0026rsquo;t teach me Cloudflare\u0026rsquo;s SSL modes. It didn\u0026rsquo;t teach me that Tailscale and Calico conflict. You learn those things by running workloads on infrastructure you built. By debugging when something breaks at midnight. By reading logs until you find the one line that reveals the issue. That\u0026rsquo;s the operational depth that comes after the cert. And it\u0026rsquo;s not in any exam. I expected the build to take weeks, maybe a month or two. It took five days. kubeadm automates what Kubernetes the Hard Way made me do manually — certificate generation, component configuration, static pod manifests. But the debugging? That took longer than the build itself.\nA Note On This Site This blog isn\u0026rsquo;t hosted on Vercel or Netlify or GitHub Pages. It\u0026rsquo;s a pod running on worker nodes in my apartment. When you visit beyondthecert.dev, you\u0026rsquo;re hitting nginx pods on my workers, routed through Cloudflare Tunnel, deployed by ArgoCD watching GitHub. Within the first 24 hours of going live, Loki captured a real visitor from Slovakia browsing the blog.\nWithin the first 24 hours of going live, Loki — the logging stack running on the cluster — captured a visitor from Slovakia browsing the blog. Real logs. Real visitor. Real cluster. The blog is a portfolio piece. The infrastructure is a portfolio piece. Both prove capability at the same time.\nWhat\u0026rsquo;s Next I\u0026rsquo;m treating this cluster like production. The NAS is arriving soon with 8TB of storage for persistent volumes. Immich will run on this cluster to replace Google Photos. This is real infrastructure running real workloads. That\u0026rsquo;s why I\u0026rsquo;m already planning Genesis2 — a sandbox cluster where I can break things without consequences. A place to test new CNI plugins. To practice chaos engineering. To prepare for CKS. To experiment with features I won\u0026rsquo;t touch on Genesis1. The cert gets you in the door. This is what\u0026rsquo;s behind it. Do it. Be patient. I\u0026rsquo;m still learning. The work isn\u0026rsquo;t done.\n","permalink":"https://beyondthecert.dev/posts/project-genesis1/","summary":"\u003cp\u003eProject Genesis1: I Built a 4-Node Kubernetes Cluster on Bare Metal. Here\u0026rsquo;s What I Learned.\nAt work, we provision managed Kubernetes clusters. Click a button, get a cluster. Everything abstracted. Everything handled.\nAt home? I built every piece myself. Two control planes. Two workers. GitOps pipeline. Monitoring stack. The blog you\u0026rsquo;re reading right now? It\u0026rsquo;s running on this infrastructure.\nWhy build it? To prove to myself I understand what\u0026rsquo;s underneath the abstraction. To own the gaps instead of hiding them. To build the kind of operational depth that doesn\u0026rsquo;t come from any exam.\nThis is Project Genesis1. And you\u0026rsquo;re standing on it.\u003c/p\u003e","title":"Project Genesis1: I Built a 4-Node Kubernetes Cluster on Bare Metal. Here's What I Learned."},{"content":"If you found your way here from Medium, you already know the backstory. The layoff. The relocation. The certifications. The grind.\nIf you haven\u0026rsquo;t read it, start there — it sets the context for everything here.\nThis site is the next chapter.\nWhat This Is Beyond The Cert is where I document what happens after you pass the exam. Not the study guides. Not the practice tests. The part that comes after — when you realize the certification got you in the door, but what\u0026rsquo;s behind it is a whole different conversation.\nI\u0026rsquo;m a platform engineer at a Fortune 500 company. My world is Kubernetes — provisioning clusters, patching them, operating them at scale. Eight months into the role, I know exactly where my niche is.\nAnd I can tell you that the CKA, as valuable as it was, only scratched the surface.\nReal understanding comes from building. From breaking things at 11pm and figuring out why. From doing something manually until you understand it deeply enough to automate it.\nThat\u0026rsquo;s what this site is about. And honestly — it\u0026rsquo;s also how I fight imposter syndrome. Writing it down, building it in public, owning the gaps. That\u0026rsquo;s the whole point.\nWhat You\u0026rsquo;ll Find Here Technical deep dives written the way I wish someone had written them for me. Not just the commands — the nuances. The things that don\u0026rsquo;t show up in documentation. The questions I asked that took hours to answer.\nThe connections between what you learn in a homelab and what you see in production.\nYou\u0026rsquo;ll find the Kubernetes The Homelab Way series — a step-by-step guide to building a bare-metal Kubernetes cluster from scratch, written by someone who just did it. Every issue encountered. Every fix. Every nuance documented.\nYou\u0026rsquo;ll find posts about platform engineering, GitOps, observability, and the operational depth that keeps me up at night — in a good way.\nA Note On This Site You\u0026rsquo;re reading this on infrastructure I built.\nThis blog runs on a bare-metal Kubernetes cluster — physical machines on my desk, configured by hand, deployed through GitOps. Every post I publish is a git push that ArgoCD turns into a deployment.\nThat\u0026rsquo;s not a flex. That\u0026rsquo;s the point. I\u0026rsquo;m still learning. This whole thing is proof of that.\nThe best way to understand something is to run it yourself.\nCome Along If you\u0026rsquo;re grinding through certs wondering what comes next — this is for you.\nIf you\u0026rsquo;re early in your platform engineering career trying to build real depth — this is for you.\nIf you just want to see what a bare-metal Kubernetes homelab looks like end to end — this is definitely for you.\nThe cert gets you in the door. This is what\u0026rsquo;s behind it.\nI\u0026rsquo;ll be here consistently this time.\nWelcome to Beyond The Cert.\nLet\u0026rsquo;s build.\n","permalink":"https://beyondthecert.dev/posts/allow-me-to-reintroduce-myself/","summary":"\u003cp\u003eIf you found your way here from Medium, you already know the backstory. The layoff. The relocation. The certifications. The grind.\u003c/p\u003e\n\u003cp\u003eIf you haven\u0026rsquo;t read it, \u003ca href=\"https://cromyhector.medium.com/3-5-years-of-silence-heres-what-happened\"\u003estart there\u003c/a\u003e — it sets the context for everything here.\u003c/p\u003e\n\u003cp\u003eThis site is the next chapter.\u003c/p\u003e\n\u003ch2 id=\"what-this-is\"\u003eWhat This Is\u003c/h2\u003e\n\u003cp\u003eBeyond The Cert is where I document what happens after you pass the exam. Not the study guides. Not the practice tests. The part that comes after — when you realize the certification got you in the door, but what\u0026rsquo;s behind it is a whole different conversation.\u003c/p\u003e","title":"Allow Me To Reintroduce Myself"},{"content":"At work, I click a button and get a Kubernetes cluster.\nClick another button, add a node.\nWhen you deal with that much automation, you start to forget — or you just don\u0026rsquo;t wonder — how things actually work underneath.\nSo I built Kubernetes from scratch. No buttons. No managed services. Just me and the components.\nThe Abstraction Thing Look, abstraction is good. It\u0026rsquo;s how we ship fast and scale without losing our minds.\nBut here\u0026rsquo;s the thing: when everything \u0026ldquo;just works,\u0026rdquo; you stop asking how.\nI\u0026rsquo;ve been working with Kubernetes for about 8 months now. Everything was already set up when I started. Clusters were running. I learned to use it — deploy pods, troubleshoot issues, work with the networking.\nBut I never really questioned what was happening under the hood.\nAfter I passed CKA in January, I wanted to go deeper. Not breadth — I got that from the cert. I wanted depth.\nMy mentor (platform engineer, teaches Linux at Yellowtail) mentioned Kubernetes the Hard Way back in October, right after I passed RHCSA. I looked at it that night, bookmarked it, and said \u0026ldquo;I\u0026rsquo;ll do this after CKA.\u0026rdquo;\nFour weeks ago, I started.\nLast night, I finished.\nIt Felt Like Running Commands At First Honestly — at first, I felt like I was just running commands.\nGenerate a certificate. Copy a config file. Start a service. Repeat.\nI thought I could knock this out in a weekend. Took three weeks (not because it\u0026rsquo;s hard — because I wasn\u0026rsquo;t doing it every night. Monday, Tuesday, Wednesday. Sometimes life came up and I\u0026rsquo;d skip a session).\nBut as I compiled notes and kept going, things started clicking.\nIt was like a puzzle. Each piece you add, you start seeing how the whole thing fits together.\nThe Thing That Clicked For Me Here\u0026rsquo;s what made the whole thing worth it:\nComponents don\u0026rsquo;t just talk to each other — they authenticate each other.\nIn CKA, you learn that Kubernetes uses TLS certificates. Cool. You accept it and move on.\nIn KTHW, you actually provision those certificates yourself. For etcd. For the API server. For the controller manager. For the scheduler.\nYou generate them. You sign them. You configure each component to trust specific certificate authorities.\nAnd that\u0026rsquo;s when it hits you.\nThe API server doesn\u0026rsquo;t just \u0026ldquo;trust\u0026rdquo; etcd. It verifies etcd\u0026rsquo;s certificate before accepting a connection.\nEtcd doesn\u0026rsquo;t just \u0026ldquo;accept\u0026rdquo; requests from the controller manager. It checks the controller\u0026rsquo;s client certificate first.\nThe scheduler presents its own certificate when talking to the API server. Mutual authentication.\nEvery component authenticates every other component — before any data moves.\nWhy This Actually Matters This actually matters.\nIf someone gets access to your cluster network, they still can\u0026rsquo;t do anything. They can\u0026rsquo;t impersonate the API server — they don\u0026rsquo;t have its private key. They can\u0026rsquo;t read secrets from etcd — the connection requires mutual TLS.\nEach component only trusts specific other components, and that trust is cryptographically proven with every connection.\nAnd when you\u0026rsquo;re troubleshooting production at 2am and you see \u0026ldquo;kubelet can\u0026rsquo;t join the cluster,\u0026rdquo; you need to know: Is it the kubelet\u0026rsquo;s client cert? The API server\u0026rsquo;s cert? The CA chain? Did something expire?\nCKA taught me certificates exist.\nKTHW taught me why they exist.\nThe Other Thing That Clicked Here\u0026rsquo;s the other realization: etcd doesn\u0026rsquo;t know anything about Kubernetes.\nIt\u0026rsquo;s just a key-value store. It stores something like /registry/pods/default/nginx as a chunk of JSON data. It doesn\u0026rsquo;t care that it\u0026rsquo;s a pod. It doesn\u0026rsquo;t understand \u0026ldquo;desired state\u0026rdquo; or \u0026ldquo;deployments\u0026rdquo; or \u0026ldquo;services.\u0026rdquo;\nThat\u0026rsquo;s the API server\u0026rsquo;s job.\nThe API server is the translator. You run kubectl create pod nginx, and the API server turns that into \u0026ldquo;write this specific data to etcd at this specific path.\u0026rdquo;\nWhen the controller manager needs to know \u0026ldquo;what pods exist?\u0026rdquo;, it asks the API server. Not etcd.\nWhen the scheduler needs to assign a pod to a node, it talks to the API server. Not etcd.\nWhen the kubelet needs to know \u0026ldquo;what should I be running?\u0026rdquo;, it watches the API server. Not etcd.\nOnly the API server talks to etcd. Everything else goes through the API server.\nThat\u0026rsquo;s why when you build it manually, the startup order matters:\netcd starts first (stores everything) API server starts next (talks to etcd, serves everyone else) Controller, scheduler, kubelet start last (all talk to API server) The API server isn\u0026rsquo;t just \u0026ldquo;an API.\u0026rdquo; It\u0026rsquo;s the central nervous system of the cluster. Every component depends on it.\nYou don\u0026rsquo;t see this dependency chain when everything\u0026rsquo;s already running. You see it when you start services one by one and watch them fail until the API server is up.\nWhat The Buttons Hide When you click a button to spin up a Kubernetes cluster, all of this happens in the background:\nCertificate authorities get generated Component certificates get signed You tell each component exactly who it should trust Mutual TLS gets established Components start in the right order Dependencies get satisfied one by one You never see it. You never think about it.\nUntil it breaks.\nHonestly? It Wasn\u0026rsquo;t Hard I thought this was going to be some brutal grind.\nIt wasn\u0026rsquo;t.\nEverything was pretty straightforward. Yeah, there were moments where something didn\u0026rsquo;t work and I had to troubleshoot it. But nothing was actually difficult.\nIt was just… satisfying. Like putting together a puzzle where each piece shows you how the system actually works.\nWhy I Did This (Real Talk) I didn\u0026rsquo;t do this thinking \u0026ldquo;this is going to boost my career\u0026rdquo; or \u0026ldquo;this is going to get me promoted.\u0026rdquo;\nI did it because I wanted to understand how Kubernetes works under the hood.\nAfter seven months of cert grinding in a new city, I\u0026rsquo;m taking a more balanced approach now. Health. Building a tribe. Networking with people. But I\u0026rsquo;m not going to just stop learning.\nWill KTHW help my career? Probably. But that\u0026rsquo;s a byproduct. I did it to learn, to get better. The career stuff follows from that.\nShould You Do KTHW? Hell yes — if you want to understand how Kubernetes actually works.\nBut it depends on your goal.\nIf certifications are your focus, CKA covers the operations side well. If you want to understand the architecture — why components exist, how they fit together — KTHW fills that gap.\nFor me? I wanted to know why things work, not just what they do. I wanted to see the pieces fit together.\nIf that\u0026rsquo;s you too, do it.\nOne thing will click for you. Won\u0026rsquo;t be the same thing that clicked for me. But it\u0026rsquo;ll be worth it.\nWhat\u0026rsquo;s Next Six months ago, I didn\u0026rsquo;t know Kubernetes the Hard Way existed.\nFour weeks ago, I started.\nLast night, I finished.\nProud. Accomplished. Ready for what\u0026rsquo;s next.\nIf you\u0026rsquo;re thinking about doing KTHW — stop thinking, start building.\nHave you done Kubernetes the Hard Way? What clicked for you?\n","permalink":"https://beyondthecert.dev/posts/i-built-kubernetes-from-scratch/","summary":"\u003cp\u003eAt work, I click a button and get a Kubernetes cluster.\u003c/p\u003e\n\u003cp\u003eClick another button, add a node.\u003c/p\u003e\n\u003cp\u003eWhen you deal with that much automation, you start to forget — or you just don\u0026rsquo;t wonder — how things actually work underneath.\u003c/p\u003e\n\u003cp\u003eSo I built Kubernetes from scratch. No buttons. No managed services. Just me and the components.\u003c/p\u003e\n\u003ch2 id=\"the-abstraction-thing\"\u003eThe Abstraction Thing\u003c/h2\u003e\n\u003cp\u003eLook, abstraction is good. It\u0026rsquo;s how we ship fast and scale without losing our minds.\u003c/p\u003e","title":"I Built Kubernetes From Scratch. Here's What Clicked"},{"content":"I passed the CKA in January after four attempts. It taught me how to use Kubernetes. But understanding how the pieces actually fit together — that comes from building it yourself.\nThe CKA taught me how to deploy pods, manage services, troubleshoot containers. Mumshad\u0026rsquo;s KodeKloud course covered the architecture — the API server, etcd, scheduler, controller-manager, kubelet. But KTHW makes you build those connections yourself: generating the certificates that secure communication between components, creating the kubeconfig files that authenticate them to each other, manually starting each service and seeing how they register.\nThat\u0026rsquo;s why I\u0026rsquo;m doing Kubernetes the Hard Way.\nIf you\u0026rsquo;ve been in the Kubernetes space long enough, you\u0026rsquo;ve heard of it. Created by Kelsey Hightower, it\u0026rsquo;s a guide to building a Kubernetes cluster from scratch — no managed services, no automation, no shortcuts. You provision VMs, generate TLS certificates for every component, configure SSH access, bootstrap etcd, and manually wire together the control plane and worker nodes.\nTwo weeks in, I\u0026rsquo;m 7 of 12 sections complete. I\u0026rsquo;ve provisioned four VMs on Google Cloud, set up a jumpbox, distributed SSH keys across machines, generated certificates for the API server, kubelet, kube-proxy, and every other component, created kubeconfig files for authentication, configured data encryption at rest, and bootstrapped etcd.\nAnd honestly? It felt like I was just running commands from a tutorial.\nCopy a command. Paste it. Press enter. Repeat. I wasn\u0026rsquo;t inventing solutions or designing architecture — I was following a script. And that felt hollow. Like I wasn\u0026rsquo;t really learning, just executing someone else\u0026rsquo;s instructions.\nBut the tutorial isn\u0026rsquo;t the teacher. The bugs are.\nSix Problems Section 03 was supposed to be straightforward. It wasn\u0026rsquo;t.\nNot because the commands were complex, but because several things broke in ways the tutorial didn\u0026rsquo;t account for. Every problem forced me to dig into SSH configurations, file permissions, and cloud provider defaults.\nProblem 1: ssh-copy-id failed for root\nGCP disables root login by default, and root has no password, so the command failed silently. Fix: SSH as my GCP user, then use sudo to manually write the jumpbox\u0026rsquo;s public key into /root/.ssh/authorized_keys on each machine.\nThis taught me that cloud providers lock down root access for security. Understanding why root login is disabled matters more than just knowing the workaround.\nProblem 2: GCP private key not on jumpbox\nGCP\u0026rsquo;s private key only existed on my Windows machine, not the jumpbox. Fix: Copy the key from Windows to the jumpbox using gcloud compute scp, then set correct permissions with chmod 600.\nSSH keys are local to the machine that generated them. If you need to SSH from Machine A to Machine B, Machine A needs the private key.\nProblem 3: PermitRootLogin still set to no\nI ran the sed command to change PermitRootLogin no to yes. The command returned successfully. Root SSH still didn\u0026rsquo;t work. GCP\u0026rsquo;s Debian image has a different sshd_config format than the tutorial expected.\nLesson: Config files vary by OS and cloud provider. Don\u0026rsquo;t assume tutorial commands work verbatim everywhere. Always verify after every change.\nProblem 4: authorized_keys missing on node-0 and node-1\nThe SSH key distribution worked for server but failed silently for node-0 and node-1. Those nodes were missing the /root/.ssh/ directory entirely. I had to create it manually with mkdir -p before copying the key.\nSilent failures are the hardest to debug. Check the basics: Does the target directory exist? Do file permissions allow writes?\nProblem 5: hostname \u0026ndash;fqdn returned the wrong hostname\nGCP automatically adds its own hostname entries to /etc/hosts with a comment \u0026ldquo;Added by Google.\u0026rdquo; These entries took priority over mine. I had to remove GCP\u0026rsquo;s entries and add my own with the correct FQDN format.\nCloud providers inject their own configuration into VMs. You have to identify and override those defaults. This is the kind of thing managed Kubernetes services handle invisibly — but when you\u0026rsquo;re building from scratch, you see every layer.\nProblem 6: Kubeconfigs missing from server after scp\nI ran the scp command to copy kubeconfig files from the jumpbox to the server machine. The command completed without errors. The files weren\u0026rsquo;t there. Ran it again. They showed up.\nSometimes commands fail silently over SSH. Network hiccups, permission issues, timing problems. Always verify after every step. Trust, but verify.\nWhat I Actually Learned The tutorial gives you the commands. The bugs teach you why they matter.\nEach problem taught me something different — SSH authentication, file permissions, config files, cloud provider quirks, and why you need to verify everything.\nThese are operations fundamentals that managed Kubernetes services hide from you. You never think about SSH keys or hostname resolution because the platform handles it. That\u0026rsquo;s great for productivity. But it means you never learn what\u0026rsquo;s actually happening under the hood.\nKTHW teaches you this by making you fix it yourself. No GUI. No automated error recovery. No \u0026ldquo;undo\u0026rdquo; button. If something breaks, you dig into logs, check file permissions, read config files, and figure it out.\nThe Groundwork Right now, I\u0026rsquo;m in the groundwork phase. Sections 1–7 focus on infrastructure setup — provisioning VMs, configuring SSH, generating certificates, bootstrapping etcd. It\u0026rsquo;s easy to feel like you\u0026rsquo;re just running commands without understanding why.\nBut this groundwork is what managed services hide from you.\nWhen you spin up a GKE cluster, Google handles:\nCertificate generation for every component Kubeconfig creation and distribution etcd bootstrapping and high availability SSH key management Hostname resolution and DNS You never see it. You just get a working cluster.\nKTHW makes you do all of that manually, so when something breaks — and it will — you know where to look.\nWhat\u0026rsquo;s Next The infrastructure is in place — certificates, kubeconfig files, etcd running. Next up: the control plane, then worker nodes, then networking.\nBecause passing the CKA taught me how to use Kubernetes. KTHW is teaching me how it works.\nThat\u0026rsquo;s the progression: certification → internals → operations.\nIf You\u0026rsquo;re Doing KTHW If you\u0026rsquo;re working through Kubernetes the Hard Way and feeling like you\u0026rsquo;re \u0026ldquo;just running commands,\u0026rdquo; you\u0026rsquo;re not alone. That\u0026rsquo;s the point.\nThe tutorial is the script. The bugs are the teacher.\nStick with it. The groundwork pays off when you start configuring actual Kubernetes components and see how all these certificates, kubeconfig files, and hostnames actually get used.\n","permalink":"https://beyondthecert.dev/posts/building-kubernetes-the-hard-way/","summary":"\u003cp\u003eI passed the CKA in January after four attempts. It taught me how to use Kubernetes. But understanding how the pieces actually fit together — that comes from building it yourself.\u003c/p\u003e\n\u003cp\u003eThe CKA taught me how to deploy pods, manage services, troubleshoot containers. Mumshad\u0026rsquo;s KodeKloud course covered the architecture — the API server, etcd, scheduler, controller-manager, kubelet. But KTHW makes you build those connections yourself: generating the certificates that secure communication between components, creating the kubeconfig files that authenticate them to each other, manually starting each service and seeing how they register.\u003c/p\u003e","title":"Building Kubernetes the Hard Way: What \"Just Running Commands\" Actually Teaches You"},{"content":"You see the announcement: \u0026ldquo;I passed the CKA!\u0026rdquo; Here\u0026rsquo;s what you don\u0026rsquo;t see:\nMy story? 41 → 47 → 65 → 81.\nFour attempts. Three failures. One certification.\nHere\u0026rsquo;s what actually happened.\nThe Beginning: September to December 2025 I\u0026rsquo;d just passed the RHCSA in September 2025. Working in a heavy Kubernetes shop, I figured the CKA would give me the baseline I needed. I knew what K8s was, but I\u0026rsquo;d never worked with it in a production environment. My company hired me with light K8s skills — now it was time to formalize them.\nFrom September to early December, I worked through Mumshad\u0026rsquo;s KodeKloud course — solid foundation. I did the practice tests. I felt ready.\nI scheduled Exam 1 for December 21st.\nExam 1: December 21st — Score: 41/66 I was confident going in. But practice tests and the real exam? Two completely different experiences.\nThe check-in process threw me off. Once I got into the PSI browser environment, I wasted 5–7 minutes just figuring out navigation. Turns out the terminal was hiding behind the browser.\nI\u0026rsquo;d built what I called my \u0026ldquo;Road to 66%\u0026rdquo; — my strategic question sequence based on topics I knew would get me to passing: CNI, CRDs, HPA, storage classes, ingress, Helm, sidecars, troubleshooting.\nBut nerves hit. The environment felt laggy. Before I knew it, an hour had passed.\nTwo problems killed me:\nIngress: Ran into YAML syntax errors when I tried k apply. Wasted too much time. Helm: Hit an annotation error. More wasted time. Final result: 11 of 16 questions attempted. Score: 41/66.\nI wasn\u0026rsquo;t surprised. I knew where I went wrong. I just needed to get back to the drawing board.\nThe Break: December 22nd — January 5th I didn\u0026rsquo;t touch Kubernetes for two weeks over the holidays. But I thought about it constantly. I replayed scenarios in my head, analyzing what went wrong.\nJanuary 6th, I reintroduced myself to K8s. Same practice routine. Same \u0026ldquo;Road to 66%\u0026rdquo; sequence.\nExam 2 was scheduled for January 18th.\nExam 2: January 18th — Score: 47/66 (The Shock) This time, I felt confident. I\u0026rsquo;d seen the exam before. I knew the terminal was hiding behind the browser. I knew the check-in quirks.\nI went in ready to dominate.\nI attempted 13 questions (up from 11).\nProgress on some issues:\nHelm: Fixed the annotation error from Exam 1. Ingress: No YAML syntax errors this time. But when I curled to verify, no 200 response. I panicked. Deleted the LoadBalancer service, created a NodePort service. Still didn\u0026rsquo;t work. Wasted 15 minutes. And then there was sidecar hell.\nThe sidecar question was my kryptonite. I\u0026rsquo;d do k edit deployment, add the sidecar container perfectly, try to save with :wq\u0026hellip; and instead of returning to the terminal, I\u0026rsquo;d stay stuck in the YAML editor. There was a syntax error somewhere, but I could never find it in time.\nBut I walked out feeling good. I\u0026rsquo;d attempted 13 questions instead of 11. Even if I barely passed — 67, 70 — a pass is a pass.\nThe next day: 47/66.\nI was shocked. Devastated. Not much improvement from Exam 1. Worse? I\u0026rsquo;d exhausted my free retake. I\u0026rsquo;d have to buy a new voucher at full price.\nIt affected me for days.\nThe Analysis: January 19th — 23rd After two days, I shifted my approach: debug what went wrong instead of dwelling on it.\nI asked AI: \u0026ldquo;Which topics get me to 66%?\u0026rdquo; I analyzed every mistake. I identified the pattern.\nThe breakthrough: I was too declarative.\nExams 1 and 2, I was writing YAML from scratch using the Kubernetes docs. I\u0026rsquo;d reference the examples, modify them for the question, then apply. When it didn\u0026rsquo;t work, finding the syntax error in the YAML burned too much time.\nThe shift: Go imperative. From January 20th to 23rd, I practiced nothing but imperative commands.\nI bought a new voucher on January 22nd. Scheduled Exam 3 for Saturday, January 24th.\nExam 3: January 24th — Score: 65/66 (One Point Away) This time, I went full imperative where I could.\nIngress: Created it imperatively. Curled. Got a 200 response. Done. Helm: Fixed my chart name issue. The result of going imperative? I had way more time. Attempted 14 questions (up from 13).\nBut sidecar was still my nemesis. k edit deployment, add sidecar, try :wq\u0026hellip; stuck in YAML hell again.\nSunday, January 25th, results came in: 65/66.\nOne point shy.\nBut I wasn\u0026rsquo;t disappointed. I was close. I knew what to fix. I just needed to nail sidecar.\nAn 18-point jump in 6 days. From 47 to 65. The imperative strategy worked. I just needed to execute cleaner.\nThe Final Push: January 24th — 28th Sidecars aren\u0026rsquo;t conceptually difficult — it\u0026rsquo;s the YAML formatting under pressure that kills you. I asked AI to generate messy YAML files. I practiced editing sidecars until I could do it in my sleep.\nScheduled for Wednesday, January 28th.\nExam 4: January 28th — Score: 81/66 (PASSED) Fourth time. Same sequence. Same strategy. Attempted 15 questions (up from 14).\nIngress: Imperative. Returned 200. Done. Helm: Fixed. Done. HPA, storage classes, network policies: Imperative where possible. Done. And then… the sidecar question. My final boss.\nI saved it for second-to-last. Knocked out everything else first.\nFinally, I opened the sidecar question. k edit deployment. Made my changes. Held my breath.\n:wq\nThe terminal prompt came back.\nI knew right then. I passed.\nThe next day: 81/66.\nWhat Actually Worked Resources:\nKodeKloud (Mumshad\u0026rsquo;s course, practice exams, lightning labs) Tech with Piyush (40 Days of Kubernetes playlist) AI for weak spot analysis and sidecar YAML practice Strategy:\nImperative over declarative (where possible) Time management: Skip questions eating too much time, come back later Each exam = fix one mistake + attempt one more question Not \u0026ldquo;study harder\u0026rdquo; but \u0026ldquo;debug smarter\u0026rdquo; PSI Browser Tips:\nTerminal is hiding behind the browser (click to bring it forward) Environment can be laggy — imperative commands save time Aliases: Just k for kubectl. Kept it simple.\nHow I Feel Now When I started this journey in January 2022, I feared containers. We often fear what we don\u0026rsquo;t understand.\nNow I\u0026rsquo;m CKA certified.\nBut the job\u0026rsquo;s not done. The CKA gave me the baseline I needed, but there\u0026rsquo;s still work to do. I\u0026rsquo;m actively building depth in the areas where I feel shallow.\nYou see the 81. You don\u0026rsquo;t see the 41, the 47, the 65. You don\u0026rsquo;t see the sidecar hell, the ingress pain, the Helm mistakes.\nBut that\u0026rsquo;s the journey. That\u0026rsquo;s what it takes.\nIf you\u0026rsquo;re stuck where I was — failing exams, close but not passing — stop studying harder. Start debugging smarter.\nEach exam is a bug report. Fix one bug. Attempt one more question. That\u0026rsquo;s how you go from 41 to 81.\n","permalink":"https://beyondthecert.dev/posts/i-didnt-pass-the-cka-by-studying-harder/","summary":"\u003cp\u003eYou see the announcement: \u0026ldquo;I passed the CKA!\u0026rdquo; Here\u0026rsquo;s what you don\u0026rsquo;t see:\u003c/p\u003e\n\u003cp\u003eMy story? 41 → 47 → 65 → 81.\u003c/p\u003e\n\u003cp\u003eFour attempts. Three failures. One certification.\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s what actually happened.\u003c/p\u003e\n\u003ch2 id=\"the-beginning-september-to-december-2025\"\u003eThe Beginning: September to December 2025\u003c/h2\u003e\n\u003cp\u003eI\u0026rsquo;d just passed the RHCSA in September 2025. Working in a heavy Kubernetes shop, I figured the CKA would give me the baseline I needed. I knew what K8s was, but I\u0026rsquo;d never worked with it in a production environment. My company hired me with light K8s skills — now it was time to formalize them.\u003c/p\u003e","title":"I Didn't Pass the CKA by Studying Harder. I Passed by Debugging Smarter."},{"content":"I haven\u0026rsquo;t posted on Medium since August of 2022. A lot changed and has happened since then.\nWhen I last wrote, I was four weeks into my first tech role. Six months of grinding through a DevOps bootcamp had finally paid off. From accountant to engineer — the switch was real.\nSo started to enjoy the fruits of my labor, lived a little. As a result I stopped learning outside of work. I got comfortable.\nThree and a half years went by.\nThe Wake-Up Call In July 2024, my company was acquired. I took it as a sign to start sharpening my skills again.\nI picked up Imran Afzal\u0026rsquo;s Complete Linux Course on Udemy and purchased Asghar Ghori\u0026rsquo;s RHCSA Red Hat Enterprise Linux 9 Study Guide.\nProgress was slow though. I needed more structure.\nThen in February 2025, the mass layoff came. I was out.\nThe Rebuild I used the layoff productively. I enrolled in a Linux/sysadmin bootcamp to get the structure I needed. Took a database class — gave me some understanding but wasn\u0026rsquo;t for me. Also stumbled upon Mischa van den Burg\u0026rsquo;s KubeCraft community during this period, which planted a seed: eventually build a Kubernetes homelab. Not for a cert. For real depth.\nWhile grinding through the bootcamp, I went through countless interviews. Kubernetes kept coming up throughout the process. I knew I wanted to tackle K8s eventually, but I also knew Linux was the foundation. Walk before you run.\nEventually landed a platform engineer role at a Fortune 500 company — serendipitous considering the steps I had already been taking toward Kubernetes. It required relocation. I took it.\nThe First Seven Months The first seven months were a blur — new city, new role, bootcamp classes four nights a week, two certifications, all while ramping up in a completely new environment. No familiar faces, no established routine. Just figuring it out.\nPassed RHCSA in September on my second attempt — only two weeks into cert prep because my voucher was expiring.\nThen went after the CKA.\nFailed three times. Third attempt I scored 65 out of 66. One point away.\nFourth attempt: 81. Passed.\nWhere I Am Now I\u0026rsquo;m a platform engineer at a Fortune 500 managing Kubernetes cluster lifecycle at scale. RHCSA and CKA certified. Seven months in and finally able to breathe.\nBut passing certs isn\u0026rsquo;t the same as understanding deeply. I can execute commands and troubleshoot issues. I want more than surface-level knowledge.\nSo I\u0026rsquo;m going back to basics. Building real depth.\nNo more three and a half year gaps. No more coasting.\nThe Lesson The moment you stop learning in tech, you start falling behind. I learned that the hard way.\nAs Kodak Black said: \u0026ldquo;Can\u0026rsquo;t run \u0026lsquo;round here acting like I made it.\u0026rdquo;\nIf you\u0026rsquo;re in a similar spot — coasting, comfortable, or coming back from a setback — start again. It doesn\u0026rsquo;t have to be perfect. Just start.\nI\u0026rsquo;ll be here consistently this time.\nWelcome to Beyond the Cert.\nLet\u0026rsquo;s build.\n","permalink":"https://beyondthecert.dev/posts/3-5-years-of-silence/","summary":"\u003cp\u003eI haven\u0026rsquo;t posted on Medium since August of 2022. A lot changed and has happened since then.\u003c/p\u003e\n\u003cp\u003eWhen I last wrote, I was four weeks into my first tech role. Six months of grinding through a DevOps bootcamp had finally paid off. From accountant to engineer — the switch was real.\u003c/p\u003e\n\u003cp\u003eSo started to enjoy the fruits of my labor, lived a little. As a result I stopped learning outside of work. I got comfortable.\u003c/p\u003e","title":"3.5 Years of Silence. Here's What Happened."},{"content":"I\u0026rsquo;m Claude R. Hector — platform engineer, CKA, RHCSA.\nBy day I\u0026rsquo;m a platform engineer at a Fortune 500 company managing Kubernetes cluster lifecycle at scale. Eight months into the role and I\u0026rsquo;ve found my niche. This is exactly where I want to be.\nThe path here wasn\u0026rsquo;t straight.\nI spent years as an accountant before making the switch to tech. Six months in a DevOps bootcamp, a few years in my first role, a layoff, another bootcamp, a relocation, two certifications, and here we are. The full story is in the 3.5 Years of Silence post if you want the backstory.\nWhat I\u0026rsquo;m working with At work: Enterprise-scale Kubernetes — both on-premises and cloud. Cluster lifecycle management at scale. Provisioning, patching, operations.\nAt home: A bare-metal Kubernetes cluster — two ThinkPad masters, two ThinkCentre workers, a NAS incoming. The site you\u0026rsquo;re reading right now runs on it. Deployed via ArgoCD from a GitHub repository.\nCertifications CKA — Certified Kubernetes Administrator (January 2026, 4th attempt) RHCSA — Red Hat Certified System Administrator (September 2025) CKS — in progress Why this blog exists I wanted to go deeper. The cert taught me how to use Kubernetes. This blog is about understanding how it actually works — building it, breaking it, documenting what I learn along the way.\nWriting in public also keeps me accountable. And if something I write saves someone else three hours of debugging, that\u0026rsquo;s a win.\nOutside of work I train seriously — bodybuilding is my thing outside of tech. Based in Charlotte, NC. My tribe is in Fort Lauderdale. I love to travel when I can.\nGet in touch Email: claude@beyondthecert.dev GitHub: github.com/BeyondTheCert Medium: cromyhector.medium.com LinkedIn: linkedin.com/in/claudehector ","permalink":"https://beyondthecert.dev/about/","summary":"about","title":"About"}]