Posts tagged: kubernetes Archive

💭 kubernetes is beautiful r/kubernetes

2026-03-23T21:29:40Z

This is a fantastic progression through kuberentes concepts. From running a pod, to making it resiliant, holding secrets, accepting traffic, and autoscaling.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

dot dockerenv

2026-03-18T10:07:16Z

Today I learned that docker creates an empty /.dockerenv file to indicate that you are running in a docker container. Other runtimes like podman commonly use /run/.containerenv. kubernetes uses neither of these, the most common way to detect if you are running in kubernetes is to check for the presence of the KUBERNETES_SERVICE_HOST environment variable. There will also be a directory at /var/run/secrets/kubernetes.io/serviceaccount that contains the service account credentials if you are running in kubernetes.

diff kubernetes manifest with cluster

2026-02-05T09:37:39Z

Like a dufus this morning I did a hard reset on a git repo for getting I was working on a manifest for. You see I generally use argo, but occasionally I have no idea what I am doing or want yet and I start raw doggin it, fully aware that I’m going to just nuke this namespace before getting it into a proper argocd.

I was overjoyed when I found out that you can diff your manifests with live production using the kubectl diff command. It uses standard diff so you can bring all your fancy diff viewers you like.

# regular manifest
kubectl diff -f k8s/shots -n shot
# kustomize
kubectl diff -k k8s -n go-waylonwalker-com
# using a fancy diff viewer
kubectl diff -f k8s/shots -n shot | delta
# using an even fancier diff viewer
# pinkies out for this one
kubectl diff -f k8s/shots -n shot | delta --diff-so-fancy

Now I can get those changes back that I thought I lost, and apply updates with confidence knowing what is about to change.

💭 AI, DevOps, and Kubernetes: Kelsey Hightower on What’s Next - ...

2025-12-13T04:07:15Z

Kelsey has a really good lightbulb moment here about platform engineering.

“if you had to do all the deployments for the entire company what questions would you ask of the development team?”

That’s your api, your platform, this is your product as a platform engineer. It’s not images, docker, terraform, hcl, yaml, kubernetes, It’s building out the right api for your company to deploy its products effectively.

https://www.youtube.com/watch?v=HdUbTyvrfKo&t=429s

timestamped

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

The Right Reasons To Run Kubernetes In Your Homelab

2025-12-10T09:48:14Z

Running kubernetes in your homelab is a fantastic way to learn, explore, express yourself, and run services that you use.

The Right Reasons To Run Kubernetes In Your Homelab #

There are not many

You want to learn kubernetes
You like kubernetes
You want to learn to scale

There are also The Wrong Reasons To Run Kubernetes In Your Homelab

You want to learn kubernetes #

Homelabbing is a such a great way to learn new skills, deploy real apps that you use. Create new custom apps for your specific use cases that no one else has. You should absolutely run kubernetes in your homelab if you want to learn it.

I would recommend to start locally, pull up kind, minikube, or k3d and start from your local machine before putting it on a server.

When you decide you are ready for a server, you probably don’t need any crazy hardware. You can probably run on some old retired Dell Optiplex or an old desktop someone is throwing out as it no longer runs windows.

You like kubernetes #

Hell Yeah Brother, 100% no better reason to run kubernetes at home than because you enjoy it. I’m with you. There’s nothing quite like having git ops kick in and deploy new services, updates, watching deployments rollover with zero downtime. Watching your cluster heal itself when a node goes down. Never ssh-ing in to do deployments. Still owning your entire hardware.

You want to learn to scale #

This is probably a stretch reason, maybe not a good one, there are probably better ways, but here we go.

Don’t claim that you need scale in your homelab, you don’t. But it sure is fun to run a cluster of nodes, and load balancing services that run across them. Solving these hard problems to scale across machines is hard. There’s no way around it, there’s a lot to think about. Doing so in a low stakes environment that you have skin in the game is a great way to learn.

I run kubernetes in my homelab #

I run it and I really like it

What flavor of autism did you guys get, I got the kind where I run kubernetes in my basement.

Here are some things I really like about it, and Yes I know you can achieve most of these without kubernetes.

I don’t have to ssh, hardly ever.
I can see everything I’m running, and its defined in a manifest
k9s is amazing, and I use it all the time.
- shell into running pods
- restart deployments
- scale deployments
- trigger cronjobs
- watch logs
- I can scale our minecraft server to 0 in seconds if we are in a different season of life
ArgoCD is amazing
- I ❤️ gitops
Ingress just works
Longhorn
- makes snapshots and backups easy
- makes multi node easy
zero-downtime deployments
self healing health checks

I mostly do very simple things, deployments with a container, a volume and ingress. Probably things that you could easily run on fly.io. Theres nothing really fancy. I just like how easy this setup works for me.

minecraft server memory

2025-12-10T08:46:36Z

I learned to today that setting MEMORY on your minecraft server causes the JVM to egregiously allocate all of that memory. Not setting it causes slow downs and potential crashes, but setting INIT_MEMORY and MAX_MEMORY gives us the best of both worlds. It is allowed to use more, but does not gobble it all up on startup.

In this economy we need to save all the memory we can!

Here is a non-working snippet for a minecraft server deployment in kubernetes.

      containers:
        - name: dungeon
          image: itzg/minecraft-server
          env:
            - name: EULA
              value: "true"
            - name: INIT_MEMORY
              value: "512M"
            - name: MAX_MEMORY
              value: "3G"

and in docker compose

  dungeon:
    image: itzg/minecraft-server
    environment:
      EULA: "true"
      INIT_MEMORY: "512M"
      MAX_MEMORY: "3G"

The Wrong Reasons To Run Kubernetes In Your Homelab

2025-12-06T09:46:47Z

Running kubernetes in your homelab is complex, time consuming, there are almost no docs to help you (homelab focused docs for things you want to install), and nothing is copy paste. You have to make everything happen yourself.

The Wrong Reasons To Run Kubernetes In Your Homelab #

I run compose and think kubernetes is the next logical step
Techno Tim runs it
I heard it’s what cool kids do
Kubernetes BTW
Talos Linux looks cool
I found a cool helm chart on GitHub
I need scale

There are also The Right Reasons To Run Kubernetes In Your Homelab.

I run compose and think kubernetes is the next logical step #

No it’s not. It’s much different than running docker, compose, swarm. It’s meant for scale, it’s complex, it’s made for enterprise, not your local development or your homelab. It can do these things, it can do them quite well, but it’s not the target audience.

Techno Tim runs it #

I heard it’s what cool kids do

You need to rethink who the cool kids are, touch some grass. Tim also does it for his job, he likes it, he knows it, he wants to lean on it and learn more.

Kubernetes BTW #

Kubernetes does not make you look cool, it makes you look like you are trying to over optimize and over engineer your life. It’s not worth it, in fact nothing in life is worth worrying about what everyone else thinks of you.

Talos Linux looks cool #

Talos is an S tier OS wherever you deploy it. It is a secure, minimal, kubernetes first OS. They also have some really great people working there putting Talos in some really cool places like backpack or Apple Power Mac

I found a cool helm chart on GitHub #

No you didn’t. Everything in homelab is compose first. A few things have a k8s option, but almost nothing is k8s first.

I need scale #

No. You’re homelab does not need scale. If you think it does, you have some real shit hardware, some bad optimizations, or somehow you have a startup you need to launch cause you got more users than most.

k3s system-upgrade minor by minor

2025-12-05T09:25:39Z

The k3s system-upgrade controller is a fantastic tool for upgrading k3s automatically. It has done a fantastic job for me every time I’ve used it. Today I ran it on a cluster that needed to upgrade several minors and I learned that the controller does not pick up on changes to the channel url if you change from minor to minor.

The solution I came up with was to name the plan with the version it supports. Then on each patch upgrade, change both the plan name and the channel. I use gitops with argocd, it automcatically cleaned up old plans, created new plans, and the system-upgrade-controller picked up the plan and started applying immediately.

# Server plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: server-plan-v1.33 # <- This is important if you want to change the channel name
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: node-role.kubernetes.io/control-plane
      operator: In
      values:
      - "true"
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/v1.33
---
# Agent plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: agent-plan-v1.33 # <- This is important if you want to change the channel name
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: node-role.kubernetes.io/control-plane
      operator: DoesNotExist
  prepare:
    args:
    - prepare
    - server-plan
    image: rancher/k3s-upgrade
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/v1.33

I’d love to see a better way if you have a way to upgrade through minors, or manually control the minor of your cluster let me know.

💭 PETaflop cluster - Justin Garrison

2025-11-12T17:44:41Z

PETaflop cluster

AI is a pain in the back.

Justin Garrison · justingarrison.com

Justin makes the coolest kubernetes clusters wishing I could see it in the flesh at Kubecon.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

Should I kubernetes My Homelab

2025-08-11T09:28:16Z

Yes

Ok we should probably dive deeper into this, but good chance if you are here and have made it this far you it would probably be a fine choice. The choice is quite time and skill dependant.

Time #

First thing up, if you like copy pasting thing into your homelab, changing a few config options, but mostly running it as the docs instructed, kubernetes is not for you. The homelab/self hosting space is heavily reliant on docker compose, 90% of the things you want to run will likely have a docker command, and likely a docker compose example that you can copy paste and get running right away. Maybe 5% of projects have something for kubernetes, you Will have to do it yourself.

Kubernetes is very DIY in the self hosting space, and not very plug and play.

Skill #

💭 Meet Gor | Just Fucking use kubernetes

2025-08-02T14:53:45Z

External Link

meetgor.com

Sometimes, all you need is a mindset shift, a blocker in your mind that holds you back from doing certain things. And for me, I have consumed enough tutorials and posts about Kubernetes, that I need to put to use and create. I have been stuck in the learning cycle, lets push to prod with kubernetes.

This hurts. I know others with this learning style that need to see the full picture before actually doing something with new tech. The way I first got into kubernetes I was looking for the easy route and somehow k8s came up several times as a suggested route Looking for a Heroku replacement, What I found was shocking!, So I dove in head first with k3s and kompose. What I found was that it was not all that hard once you start to see how the pieces fit together, no amount of reading tutorials would have gotten me there.

Does anyone care if you use simple yet fragile bash scripts or heavy weight Kubernetes cluster for just clicking buttons and creating and updating rows in a database? No!

You know what, let’s fucking use Kubernetes.

Let’s Gooo. Use what is right for you and stop parroting kubernets is hard, heavy, for big companies, maybe actually try it first.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 Kubernetes Isn't for You

2025-08-02T14:40:42Z

Kubernetes Isn

Kubernetes isn

sliplane.io

This post feels like it was written by someone who has never tried kubernetes, someone who reads twitter, listens to t3.gg and thePrimeagen (who cant even container let alone kubernetes). If you cant run linux, use bash, build your own docker images, run docker comfortably. If infra is not your thing kubernetes is probably not for you.

Kubernetes Was Built for Google

Just like how react was built for facebook to solve facebook problems with many teams contributing effectively to the same interactive interfaces. Turns out that react is actually a pretty good product if you have a highly interactive page, and if this is your bread and butter, you can make overly heavy static sites with too much build very effectively. It works and runs much of the internet now.

We are getting serious. We need serious tools. Big companies use Kubernetes. We should too. It feels more professional. It sounds like we know what we are doing.

If anyone uses these reasons to pitch kubernetes to me they don’t belong in a position to make any sort of decision. The first one could be a heading with maybe something under it.

But Kubernetes should not be your first infrastructure step. It should be a response to real pain, not an emotional milestone

As with anything, it depends! Keeping with the react example, if you have a team with these skills its a solid choice, maybe its overkill, but you got this skills to start here then go for it.

If you have never given something a real shot then don’t be writing articles shitting on the tech. Actually if you do not have a deep expertise in it you probably should not be writing articles shitting on other tech. If you are the CEO of an alternative, you definitely should not be writing articles shitting on your competition. just build the biggest fucking building in town .

If you are kube curios give kind and kompose a try, you will be surprised at how quickly you can get something up and running in kubernetes. You might be surprised at how easy it is to remotely manage, add gitops workflows with argocd. Give k9s a try and you can see all of your nodes, services, ingress, pvcs, EVERYTHING you have deployed and its status in one easy to use TUI.

I avoided kubernetes for a long time because articles like this told me to and I never gave it a fair try.

Check out just fucking use kubernetes for a satirical opposite take.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

just fucking use kubernetes

2025-07-01T12:46:33Z

You want to run containers?

JUST FUCKING USE KUBERNETES.

Obvious satire

If you don't like harsh language this is not the post for you.  Obviously

ripping off motherfuckingwebsite.

ThIs is AI SLoP

If you don't like if you can fuck off to the next post, I'm having fun here,

but satire is not my strong suit and needed some help.

Seealso

“But it’s complicated!” #

Shut up. Close twitter and fucking do something. Life is complicated. You know what else is complicated? Email. DNS. Life. Kubernetes is the least painful way to orchestrate containers at scale. Docker Compose is for your laptop.

Swarm is dead.
Nomad is just sad.
Systemd units? Get out of here.

“But my app is small!” #

SO IS YOUR AMBITION. You could write a bunch of bash scripts and hope they work on prod. You could SSH into servers and handcraft your infra like it’s 2011. Or you could just fucking use Kubernetes and sleep at night.

I can just throw my script in crontab #

Tell that to your boss when your cronjob failed 16 times in the last week without anyone noticing. kubernete makes it fucking simple, want retry ask for it. Hanging script, activeDeadlineSeconds that bitch. Connecting to six other services in your shitty ass infrastructure this shit retries automatically.

“I don’t need autoscaling!” #

Cool. Tell that to your boss when the CEO tweets your link and the site goes down harder than your last date.

“But YAML is ugly!” #

So is your Terraform, your Ansible, your Prometheus config, your custom CI/CD scripts written in Bash, and the spaghetti you called a monolith before you went microservices and made it worse.

“Kubernetes is too heavy!” #

Compared to what?

Your handcrafted, artisanal, single-node LXC setup running on an Intel NUC from 2014?

Heard of k3s? k0s? No. These fuckers will have you running kubernetes running on your grandpas goddamn gateway 2000 right next to AOL messenger without splooging out the the ashes of his Marlboro Reds.

SSH? #

You don’t need no goddam ssh to install your 200lb gorilla editor so you can hand edit your init files and carefully contruct your init system. This is kubernetes, you use the fucking api, all you need is a connection and a kubeconfig. This motherfucker runs containers so you can keep your bitch-ass editor where it belongs, off the fucking host machine!

“What if it’s overkill?” #

What if YOU are underkill?

“How do I do zero-downtime deploys?” #

Probes my dude, you fucking probe your shit. Rolling out a new deployment kubernets won’t cut over to your broke ass release if that shit don’t pass. No more writing janky scripts that SSH into prod and run git pull while praying to the CI/CD gods.

“What if I still fuck it up - How do I roll back?” #

k9s is your best friend, pop that shit open find your broke ass deployment, jump owner to the replicaset and roll that bitch back to the working shit.

I need to scale #

This shit is built in, add a goddamn replica or 6 for fuck sake, need autoscaling use the HPA. This aint your granpas hand fucking crafted pet server, its fucking cattle. Load balancing just fucking happens, don’t think about it just use it, and it will work for your six goddamn friends that actually use your shit.

I want gitops #

Let me introduce you to argocd, this fucker uses helm so one fucking command and your ass is on the beach while your dev team deploys their own shit.

USE KUBERNETES #

It fucking works.
Everyone else is using it.
There are like 500 open-source projects built just to make it easier.
It runs on your laptop, your server, the cloud, and inside your dreams.
It will make your resume better.

Not convinced? #

Here’s your alternative stack:

A bash script that restarts Docker when it dies.
A Makefile that deploys via SCP.
A cron job that prays to the log gods.
A wiki page explaining how to debug your hand-rolled bullshit.
You. Crying.

So yeah #

Save yourself. Save your team. Save your infrastructure.

JUST #

FUCKING #

USE #

KUBERNETES #

(or don’t, and become a DevOps cautionary tale)

Seealso

kubernetes node labels

2025-05-28T18:49:19Z

If you need to target a specific k8s node in the cluster, you can use labels. You want to treat your nodes as much like cattle as you can, but sometimes budgets get in the way. You might be like me and just run any free hardware you can get in your cluster, or you might have some large storage or gpu needs that you can’t afford to put on every node in the cluster.

kubectl get nodes --show-labels

# add the bigpool label
kubectl label node k8s-1 bigpool=true
kubectl get nodes --show-labels

# remove the bigpool label
kubectl label node k8s-1 bigpool-

To use the label in a pod set spec.nodeSelector to the label that you applied.

apiVersion: v1
kind: Pod
metadata:
  name: busybox
spec:
  containers:
  - name: busybox
    image: busybox
  nodeSelector:
    bigpool: "true"

💭 k8s-monitoring-helm/charts/k8s-monitoring/docs/examples/privat...

2025-05-23T19:58:59Z

k8s-monitoring-helm/charts/k8s-monitoring/docs/examples/private-image-registries/globally/values.yaml at main · grafana/k8s-monitoring-helm

Contribute to grafana/k8s-monitoring-helm development by creating an account on GitHub.

GitHub · github.com

k8s-monitoring requires setting imageregistry and pullsecrets twice

global:
  image:
    registry: my.registry.com
    pullSecrets:
      - name: my-registry-creds
  imageRegistry: my.registry.com
  imagePullSecrets:
    - name: my-registry-creds

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 Why is Kubernetes everywhere? | Kelsey Hightower - YouTube

2025-01-29T16:28:21Z

Kelsey says several times in this interview, you don’t need kubernetes. If you are running one node you don’t need kubernetes. My question though is, would you use kubernetes? Ya I get it if you are a web developer, data scientist, backend dev, but if you are looking to bee a whole ass engineer, or infrastructure engineer, you know kubernetes, Should you use kubernetes on single node?

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 GitHub - ngalaiko/tree-sitter-go-template: Golang template gra...

2025-01-06T18:49:13Z

GitHub - ngalaiko/tree-sitter-go-template: Golang template grammar for tree-sitter

Golang template grammar for tree-sitter. Contribute to ngalaiko/tree-sitter-go-template development by creating an account on GitHub.

GitHub · github.com

This setup fixed my nvim syntax highlighting in helm templates.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

k3s config after first install

2024-09-15T16:57:41Z

After first setting up a new k3s instance your kubeconfig file will be located in /etc/rancher/k3s/k3s.yaml.

You cans use it from here by setting $KUBECONFIG to that file.

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Or you can copy it to ~/.kube/config

cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

If you have installed k3s on a remote server and need the config on your local machine then you will need to modify the server address to reflect the remote server.

scp user@<server-ip>:/etc/rancher/k3s/k3s.yaml ~/.kube/config

Warning

only do this if you don’t already have a ~/.kube/config file, otherwise copy it to a new file and set your $KUBECONFIG env variable to use it.

Now you will need to open that file and change the server address, making sure to keep the port number.

apiVersion: v1
clusters:
  - cluster:
      certificate-authority-data: ****
      server: https://<server-ip>:6443
    name: default

k8s kustomize diff

2024-07-06T09:42:42Z

I’ve started leaning in on kubernetes kustomize to customize my manifests per deployment per environment. Today I learned that it comes with a diff command.

kubectl diff -k k8s/overlays/local

You can enable color diffs by using an external diff provider like colordiff.

export KUBECTL_EXTERNAL_DIFF="colordiff -N -u"

You might need to install colordiff if you don’t already have it.

sudo pacman -S colordiff

sudo apt install colordiff

Now I can try out kustomize changes and see the change with kustomize diff.

kubectl dash k

2024-07-05T20:15:11Z

Kubernetes ships with a feature called kustomize that allows you to customize your manifests in a declarative way. It’s a bit like helm, but easier to use. I found this useful.

kustomization.yaml #

In order to use kustomize you need to have a kustomization.yaml, kustomization.yml, or Kustomization file in the directory you are applying.

kubectl apply -k .
error: unable to find one of 'kustomization.yaml', 'kustomization.yml' or 'Kustomization' in directory '/tank/git/kustomize-playground'

The kustomization.yaml file cannot be empty.

❯ kubectl apply -k pod
error: kustomization.yaml is empty

lets make an mvp #

mkdir pod
touch pod/pod.yaml
touch pod/kustomization.yaml

# pod.yaml

# kustomization.yaml

Overlays #

Overlays must point to yaml or a directory with a kustomization.yaml file, and cannot reside inside the same directory causing a circular reference.

Layout #

k8s
├── base
│   ├── deployment.yaml
│   └── kustomization.yaml
├── overlays
│   ├── local
│   │   └── kustomization.yaml
│   ├── dev
│   │   └── kustomization.yaml
│   └── prod
│       ├── special-prod.yaml
│       └── kustomization.yaml

base #

Place all of the common k8s manifests here in the base directory along with kustomization.yaml. You can split them up into separate files or just one file, I’m keeping it to one for simiplicity.

base - kustomization.yaml #

In the base kustomization.yaml file add all of the manifests that you want to use as a resource.

resources:
  - deployment.yaml

overlays #

Now for each separate environment that you want to have create a directory in overlays with a kustomization.yaml file, and any special manifests that only apply to a particular environment.

overlays - kustomization.yaml #

resoures #

resources:
  - ../../base
# any other specific resources, maybe special ones for prod here
  - sealed-dotenv.yaml

images #

images:
  - name: my-image
    newName: docker.io/myrepo/myimage
    newTag: 1.0.0

💭 Kustomize: The Best Way to Manage Your Kubernetes Configs - Yo...

2024-07-03T13:31:47Z

Great intro into kustomize. This helped me get started with kustomize.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

kind cluster

2024-07-02T08:01:20Z

kind{.hoverlink} is a very useful tool to quickly standup and teardown kubernetes clusters. I use it to run clusters locally. Generally they are short lived clusters for trying, testing, and learning about kubernetes.

Kind is Kubernetes in Docker, its very fast to get a new cluster up and running. Other than checking a box in docker desktop it is the easiest way currently to get a cluster up and running. I’ve used docker desktop for k8s before I really developed on k8s and it was buggy at the time and sometimes started and sometimes didn’t, when it didnt I had no idea how to fix it. I’d suggest kind as the best option to get a cluster up and running locally.

Not Production #

If you are looking for a production ready cluster this is not it. I really like k3s{.hoverlink}. At the time that I chose k3s it was the most lightweight option that easily supported multi-node clusters.

Starting a kind cluster #

The first step, and maybe only one that you need is to create a cluster and give it a name. This command will edit your $KUBECONFIG file, and set the kind cluster as your default cluster to interact with.

kind create cluster --name <CLUSTER_NAME>

Using podman as a backend #

I use podman as my docker engine, kind works with docker and podman, but docker by default, in order to switch to podman you need to set an environment variable.

export KIND_EXPERIMENTAL_PROVIDER=podman

This will tell kind to use podman as the backend provider instead of docker.

Loading images #

If your images are not publically available from a registry, you can load them in kind using the kind load docker-image command.

kind load docker-image $REPOSITORY:$TAG --name <CLUSTER_NAME>

Note

 the CLUSTER_NAME is the name that you gave kind when you started the kind cluster.

Argocd #

Argocd is a great way to setup gitops workflows in kubernetes. compared to just hand-rolling kubectl apply, argo holds the state and is able to not only apply new and change, but cleanup removed things.

setting up a kind cluster with argocd installed

You can stand up argocd in kind for learning argo or getting a nice visual. But often when I use kind its overkill. The cluster is not long lived, I don’t care if things are not cleaned up, and I want to quickly apply changes without a commit and push to a git repo.

Install sealed-secreats via manifest

2024-07-02T07:54:01Z

Yesterday I realized that I have overlooked the default installation method of the sealed secrets controller for kubernetes kubeseal this whole time an jumped straight to the helm section. I spun up a quick kind cluster and had it up quickly. I can’t say this is any better or worse than helm as I have never needed to customize the install. According to the docs you can customize it with [[ kustomize ]] or helm.

# option if you don't have a cluster try with kind

kind create cluster

curl -L https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.27.0/controller.yaml > controller.yaml

kubectl apply -f controller.yaml

💭 Releases · stackrox/kube-linter

2024-06-24T17:09:37Z

GitHub - stackrox/kube-linter: KubeLinter is a static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices.

KubeLinter is a static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices. - stackrox/kube-linter

GitHub · github.com

A linter for linting kubernetes manifests and help charts.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 argoproj/argo-events: Event-driven Automation Framework for Ku...

2024-06-09T14:30:26Z

GitHub - argoproj/argo-events: Event-driven Automation Framework for Kubernetes

Event-driven Automation Framework for Kubernetes. Contribute to argoproj/argo-events development by creating an account on GitHub.

GitHub · github.com

Argo events is an event driven automation framework for kubernetes that can create kubernetes objects among other things based on events. I’ve been using native kubernetes cronjobs to kick off jobs based on a cron trigger.

For instance I am running reader.waylonwalker.com every hour, to rebuild the site and re-deploy it. It takes about two minutes to fetch every rss feed, so this is a nice application of a job compared to a web server fetching the feeds live. Now my posts may be up to an hour stale but they load fast.

Argo events takes event drien architecture to the next level allowing to be triggered by many more things, and do many more things than creating a cron job. I’m definitely thinking about dropping this in my homelab.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 eraser-dev/eraser: 🧹 Cleaning up images from Kubernetes nodes

2024-05-25T01:26:09Z

GitHub - eraser-dev/eraser: 🧹 Cleaning up images from Kubernetes nodes

🧹 Cleaning up images from Kubernetes nodes. Contribute to eraser-dev/eraser development by creating an account on GitHub.

GitHub · github.com

This is kinda sick, its a tool to clean up container images in a k8s cluster.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 How to Restart All Pods in a Kubernetes Namespace | Boot.dev

2024-04-25T21:59:56Z

https://boot.dev/blog/devops/how-to-restart-all-pods-in-a-kubernetes-namespace/

blog.boot.dev

As of kubernetes 1.15 there is an easy way to restart all pods in a deployment.

kubectl -n {NAMESPACE} rollout restart deploy

Thanks Lane give him a follow @wagslane

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 argocd automated sync

2024-04-19T19:36:47Z

![[none]]

---

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kanboard
  namespace: argocd
spec:
  project: default
  destination:
    namespace: kanboard
    server: 'https://kubernetes.default.svc'
  source:
    path: kanboard
    repoURL: 'https://github.com/waylonwalker/homelab-argo'
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

💭 Manual Upgrades | K3s

2024-04-19T12:51:03Z

Manual Upgrades | K3s

You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version.

docs.k3s.io

You can give k3s an install channel to install stable, latest, or specific versions like 1.26. This is handy to make sure that you install the same version on all of your workers.

curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest <EXISTING_K3S_ENV> sh -s - <EXISTING_K3S_ARGS>

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

scheduling cron jobs in kubernetes

2024-04-08T16:56:46Z

For my reader app I am using cronjobs to schedule my a new build and upload to cloudflare pages every hour. In this example I have built a docker image docker.io/waylonwalker/reader-waylonwalker-com and pushed it to dockerhub. It uses a CLOUDFLARE_API_TOKEN secret to access cloudflare, and the entrypoint itself does the build and upload.

apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: reader
  namespace: reader

---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: reader-cronjob
  namespace: reader
spec:
  schedule: "0 * * * *"
  successfulJobsHistoryLimit: 6
  failedJobsHistoryLimit: 6
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: reader-container
              image: docker.io/waylonwalker/reader-waylonwalker-com:latest
              env:
                - name: CLOUDFLARE_API_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: cloudflare-secret
                      key: cloudflare-secret
          restartPolicy: OnFailure

💭 Sealed Secrets

2024-03-28T01:07:06Z

External Link

sealed-secrets.netlify.app

kubeseal is a pretty simple to get started with way to manage secrets such that they can be stored in a git repo and be picked up by your continuous delivery service.

Sealed Secrets provides declarative Kubernetes Secret Management in a secure way. Since the Sealed Secrets are encrypted, they can be safely stored in a code repository. This enables an easy to implement GitOps flow that is very popular among the OSS community.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts

kubernetes kubeseal

2024-03-27T20:02:57Z

In my homelab kubernetes cluster I am using kubeseal to encrypt secrets. I have been using it successfully for a few months now wtih great success. It allows me to commit all of my secrets manifests to git with out risk of leaking secrets.

You see kubeseal encrypts your secrets with a private key only stored in your cluster, so only the cluster itself can decrypt them using the kubeseal controller.

KubeSeal #

https://sealed-secrets.netlify.app/

installation #

Installation happens in two steps. You need the kubernetes controller and the client side cli to create a sealed secret.

For a more complete instruction see the [docs#installation](https://github.com/bitnami-labs/sealed-secrets?tab=readme-ov-file#installation]

installation - controller #

Warning

 **context**

Make sure that you are in the right context before running any kubectl commands.

kubectl config current-context

sealed-secrets is installed using the helm package manager. To install sealed-secrets run the following command.

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets -n kube-system --set-string fullnameOverride=sealed-secrets-controller sealed-secrets/sealed-secrets

installation - client #

For the client you can check your OS package manager, brew, or the github-releases. For me I found it in the main arch repos.

paru -S kubeseal
# or
sudo pacman -S kubeseal
# or
brew install kubeseal

Note

 You will need to install kubeseal on every device that you will want to

create sealed secrets on.

Example #

Most of these commands come straight from the docs. From my experience I have always specified the namespace, my projects go per namespace and I don’t have any reason that other namepsaces should see the secret, and if they do I deploy another secret in that namespace.

# Create a json/yaml-encoded Secret somehow:
# (note use of `--dry-run` - this is just a local file!)
echo -n bar | kubectl create secret generic mysecret --dry-run=client --from-file=foo=/dev/stdin -o yaml -n thenamespace > mysecret.yaml

note that the key of the secret is foo and the value is bar

results

apiVersion: v1
data:
  foo: YmFy
kind: Secret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: thenamespace

Note

 The data is base64 encoded.

echo -n bar | base64
# YmFy

# This is the important bit:
kubeseal -f mysecret.yaml -w mysealedsecret.yaml

# At this point mysealedsecret.json is safe to upload to Github,
# post on Twitter, etc.

# Eventually:
kubectl create -f mysealedsecret.yaml -n thenamespace
# sealedsecret.bitnami.com/mysecret created

# Profit!
kubectl get secret mysecret
kubectl get secret mysecret -n thenamespace
# NAME       TYPE     DATA   AGE
# mysecret   Opaque   1      27s

cat mysealedsecret.yaml | kubeseal --validate

echo -n bar | kubectl create secret generic mysecret --dry-run=client --from-file=foo=/dev/stdin -o yaml \
  | kubeseal -o yaml -n thenamespace > mysealedsecret.yaml
echo -n baz | kubectl create secret generic mysecret --dry-run=client --from-file=bar=/dev/stdin -o yaml \
  | kubeseal -o yaml -n thenamespace --merge-into mysealedsecret.yaml

Results

---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: thenamespace
spec:
  encryptedData:
    bar: AgBLkamltcLfH1dC1JxQ3qd8lJ8aBZF2ARoq3uo055hnzXOy8g2T5liTx5UPvyPV8yyWqABU8eOnwjNhDtzSATvYeBB3fGkucdOZziWEoiNsWTR9ZtFEkod7Ya6uGkzZOJwi3IkrHFVIT9oWZQUxxJZ6vFhPiFcx9Dorr8TNSzG4KOug25+PhWPPiHDgSup5N3CkWCZaYOF7dbZRVSA4nGP1fZxjFByHP4AsdjLCHptyVbkpLRKeiXTkLxfLX4K+JLZGM41S1On5bSP56mCfv1daTJx619kDXkRLw9l21Ot283/L0NMNAiw781AefYMVoO3aHmYgcT6wAtsQAKje9fyL7DQRHt8a5NZOWukp/P6XjdXRz/nfQasQlbSTrRkDpplKIM5/WdPcBoKi+yyoOL0rZ8x1X7YzUI3BggZmzWyEPD01BK1YAHGZnYIZbbCy1JSm8JCBvP+xWMg+i0Z/DCD8nclAhH1GX2Q7/NrNHF//589AJfuriymd2+mk7uaLA4RRsY0l5QeZD6HVAqSv5jWsVQQtSftWmI9vn9oL/Pno7sEUjSDpXPfF4nnsULhxsPEe2DFAMm1kZAjdF06ueF4/x2Fdy80ZQNyycaDx2CWm4z3b14A75WGyOXl2wJZQqxrFCz8el4hD2nH3zQFEzd6AIh49myqVAGuu2qGlYP4p94LJghVa+mQjztLD/2ZUZjY+anQ=
    foo: AgAducXW6iUCY900cPDdmRfuj7tKnh2hY4C1+2hFoAtjyvjepsKNWsiPJ81t8anaMfFPat4ta060l5VtTrceFE8oS/rViz1tvNWGPBjL+GwL/QjkGl6H8ju87vKERKQn5qw7B/V5j24VM8AxOd+/vJNt/IeRIHLubvFft4hyMq4b0xmIxaemBSTxchQX/5364T3VJH2kHaqpqd+JJgQnTbiTQe/XnyZokDX8GSxw4rAbJUJSRUtY9DB9ZDu2zC5VngX+GJjbwHGbv9EKs8LShJIPrD8xHqrDmlSXGkkP01D4A6268Qoi3x5S0H5aqDgtrgBiWsNkzdKwjfrTNx7pKecOi41lyFdffHOGUew4aPPMqjzWR2TEms9WNNQXwnBdDHKMkFsisocF52BolEkcjF8g/u5h2Af92abMu6k16VybJrB7TV1set5A9W7rqG1iXI4+1W6XQfFnpja8xL/zJBvZcyHgeYMNaxa3C3s6PANhPzAUVaXV9eedAkptGJLN13IZj4LujpoAxRKo6bEdydv/5P23R3fx5PgTOpVI7riECAOIg2PThFsEoVCUwStmKCvIx1I2+YixIlv/OiaUWo4lrI/3ve5WGp4ZnuiJPk34JoYAlRbR6+sX14d8Ek6viq/pJUUIfVpNIkNMboUL4u+KpT47eyQ/mWih/KFduQyX9II/vQ+/IJGzEEHIipxAhdmV+K4=
  template:
    metadata:
      creationTimestamp: null
      name: mysecret
      namespace: thenamespace

backing up your sealing key #

kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml >main.key

converting .env files to a secret #

Working with web applications .env is a common way to store credentials. Let’s look at how we can convert these to secrets.

kubectl create secret generic mysecret --from-env-file=.env -n thenamespace --dry-run=client -o yaml > mysecret.yaml

Now you have a secret that looks like this.

apiVersion: v1
data:
  foo: YmFy
kind: Secret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: thenamespace

Seal it up just like before.

kubeseal -f mysecret.yaml -w mysealedsecret.yaml

Using the secrets #

I typically use the secrets in the container spec.

containers:
  - name: myapp
    envFrom:
      - secretRef:
          name: mysecret
    # You can still have other env vars
    env:
      - name: foo
        value: bar

Sometimes I want to mount the secret as a volume.

containers:
  - name: myapp
    volumeMounts:
      - name: mysecret
        mountPath: /mysecret
volumes:
  - name: mysecret
    secret:
      secretName: mysecret

Image Pull Secrets #

I also need to use imagePullSecrets. Let’s walk through the whole process. Starting with the secret.

kubectl create secret docker-registry regcred --docker-server=myprivateregistry.example.com --docker-username=foo --docker-password=bar --dry-run=client -o yaml

Generates the following secret.

apiVersion: v1
data:
  .dockerconfigjson: eyJhdXRocyI6eyJteXByaXZhdGVyZWdpc3RyeS5leGFtcGxlLmNvbSI6eyJ1c2VybmFtZSI6ImZvbyIsInBhc3N3b3JkIjoiYmFyIiwiYXV0aCI6IlptOXZPbUpoY2c9PSJ9fX0=
kind: Secret
metadata:
  creationTimestamp: null
  name: regcred
type: kubernetes.io/dockerconfigjson

---

the secret

Now we we can seal that secret.

kubeseal -f regcred.yaml -w regcred-sealed.yaml

And that gives us the following sealed secret that we can deploy into our cluster.

---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: regcred
  namespace: default
spec:
  encryptedData:
    .dockerconfigjson: AgATYVEywkyYEaoErQbJo6xEZfOnRn1ydNTLkO3Jt/NF/UH+0o9lHpDecRpN0XnVu8xJUcdjkgD9q2XkwP8e6qQDS2mMPTiTNIN+8gbJx97WrD1YQDT0lXBuoyi9I/iwlXxx6MgH/6GY6CeGTz5SRlvoU0Xhlt4d11s7/xapdE8QMLsAReqPEv8oZHEyAxDRrjXX0V+tO8dV+G+GXjUDMBBceLael9rvGzSKIwDVXACVqQhLkB6FoP98M+yyBE46RBNnSnS0ShQM5PprL24HKpRZ43x4RM53KBrQ7R/MxeshafY+B6vUvrolmVox4sud8xngMOcjTO28LLOrck5V8ZiDabhN7ajHEf03IESr1o/ADGf5k9988Vv1txJtsZW0K2mpRu0D7/BLVL9KzbZ5ywULqIoD/Ur2GIGnZqMAKOq4laGp/GJtMKLrhmEvekT397wC/Gf/xdDKVhHf2p4ocsPu7LKFuS5H/Auel/Q5grdn8L5wwrO4VWRv3eJroKh/Hux7Qd7f64O7qdi0XthDocf+gmtjys+Gy72M7tyf8f/O+3oKbS4CWQVTj4ZThMc9znrFnHqt2q/7pAyytTQCpk51wlzOsNvOhCueJM/jmeahaL0LuBrqngqISpnd65sgVzBcZpwK9i2Fckyt0DrZLH+NoIuvaqNhzlF+OMbAft/ylWWKCH4WUP+FKG+1LXM7ud7AA3MMbGSBxHL0/WK/INa7MB56xZKMqqyvvLLQHFTQUROJjkgkzsumdOgwZTRgIFnAZ4+vOX3/1Rtt3mAs3vdoJhL4GuKUYCnEHt908eKkWEVEs7eMk5SdSRtIsbaXO2s0dtADwg==
  template:
    metadata:
      creationTimestamp: null
      name: regcred
      namespace: default
    type: kubernetes.io/dockerconfigjson

Now that we have our sealed registry secret, we can deploy it into our cluster.

kubectl apply -f regcred-sealed.yaml

Now we can use it to pull images from our private registry.

containers:
  imagePullSecrets:
    - name: regcred

Full example #

Here is a full deployment example using all the secrets we have created.

regcred
mounting a secret
envFrom secret

---
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    service: myservice
  name: myservice
  namespace: mynamespace
spec:
  replicas: 1
  selector:
    matchLabels:
      service: myservice
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        service: myservice
    spec:
      containers:
        - envFrom:
            - secretRef:
                name: mysecret
          env:
            - name: foo
              value: bar
          image: private-registry.io/myimage:1.0.0
          name: myimage
          ports:
            - containerPort: 5000
              protocol: TCP
          resources: {}
          volumeMounts:
            - mountPath: /mysecret
              name: mysecret
      restartPolicy: Always
      volumes:
        - name: mysecret
          secret:
            secretName: mysecret
      imagePullSecrets:
        - name: regcred

Downside #

Now the main downside I see with kubeseal is that it does not provide a way to store your secrets in a way that you can access outside of your cluster. So you need to make sure that you have another solution in place to store your secrets so that you still have them if you ever were to take the cluster down or move from k8s to something else.

Overall the likelyhood of you loosing a production cluster is pretty low, so maybe it’s ok to just trust it depending on what the secrets are. Especially for things you control and can rotate anyways its fine.

kubernetes 6 months in

2024-03-26T13:20:24Z

I stumbled into kubernetes December 2023 when I was looking for a better way to self host applications. I was looking for something that didn’t require logging into a server and building and deploying like a cave man. I wanted a smoother experience than docker compose was giving me.

https://waylonwalker.com/looking-for-a-heroku-replacement/

This post turned into a list of tools that I have adopted into my k8s workflow, and plan to keep. enjoy.

Kompose #

Kompose is a great tool for gettting going and converting your docker-compose to kubernetes manifests or helm templates. It was a great tool for me to get started with, but I was afraid that it was hindering me learning more and just blindly using its output so I have tried to use it less and less. I’m now not solely leaning on it, but using it to get out quick POCs with low friction.

Kompose really helped me go 0 to 60 and get right into kubernetes with my existing docker compose files and very little change. I found it a great way to get my feet wet and learn.

Argocd #

https://argoproj.github.io/cd

Argocd is the first tool I am going to reach for in any kubernetes deployment that I work on. The number one reason is that it keeps track of all of everything internally, and if you remove something it cleans it up. Very quickly if you are not diligent you will end up with stuff laying around not sure if you need it, or where it came from.

For instance if I create a new deployment and don’t include the namespace, If I just change the yaml with the new namespace without taking it down first, that original one will be left behind and I will be stuck manually cleaning it up.

Argo Rollouts #

https://argoproj.github.io/rollouts

I’m looking into rollouts, and have played with it, but argocd really does most of what I need with low friction. I thought I needed rollouts to prevent downtime, but argo tends to do a good job of cutting over to a new pod only after its up and ready. This solves most of my problems, and I can pretty quickly roll back with low friction. If I needed to manage a high availabilty application Id look closer.

KubeSeal #

https://sealed-secrets.netlify.app/

KubeSeal is a fantastic low friction pair to argocd. The biggest benefit to kubeseal is that you can safely commit secrets straight to git. It uses a one way encryption method that is very secure and you can only decode the secrets if you gain access to the keys.

kubernetes kubeseal

💭 Configure Liveness, Readiness and Startup Probes | Kubernetes

2024-03-15T14:38:02Z

Configure Liveness, Readiness and Startup Probes

This page shows how to configure liveness, readiness and startup probes for containers. For more information about probes, see Liveness, Readiness and Startup Probes. Before you begin You need to h...

Kubernetes · kubernetes.io

What is the difference between health, liveness, readiness, and startup? This article does a great job at a full writeup description of how it works in kubernetes, here is my TLDR.

health 200 OK - I’m still responding to requests
health ERR - something happened and I cant respond to requests
liveness 200 OK - I’m ready for more work
liveness ERR - I’m still responding to requests, and i’m already working send requests to another pod, or scale up

Z-pages #

These probes are commonly deployed at /healthz and /livez endpoints.

Why the z?

z is a convention that comes from google for meta endpoints to reduce conflict with actual endpoints, and can be deployed to any application.

Note

This post is a thought. It’s a short note that I make about someone else’s content online #thoughts