Posts tagged: k8s

All posts with the tag "k8s"

33 posts latest post 2025-12-05
Publishing rhythm
Dec 2025 | 2 posts

The k3s system-upgrade controller is a fantastic tool for upgrading k3s automatically. It has done a fantastic job for me every time I’ve used it. Today I ran it on a cluster that needed to upgrade several minors and I learned that the controller does not pick up on changes to the channel url if you change from minor to minor.

The solution I came up with was to name the plan with the version it supports. Then on each patch upgrade, change both the plan name and the channel. I use gitops with argocd, it automcatically cleaned up old plans, created new plans, and the system-upgrade-controller picked up the plan and started applying immediately.

# Server plan apiVersion: upgrade.cattle.io/v1 kind: Plan metadata: name: server-plan-v1.33 # <- This is important if you want to change the channel name namespace: system-upgrade spec: concurrency: 1 cordon: true nodeSelector: matchExpressions: - key: node-role.kubernetes.io/control-plane operator: In values: - "true" serviceAccountName: system-upgrade upgrade: image: rancher/k3s-upgrade channel: https://update.k3s.io/v1-release/channels/v1.33 --- # Agent plan apiVersion: upgrade.cattle.io/v1 kind: Plan metadata: name:...

This looks like great prototyping tool for k8s. I too often ask ai to get me going with the things I need. I’ve used k8s long enough that I can generally remember all the things I need, roughly where they go, would probably forget a few things and need to iterate, but I cannot remember exactly what goes where and need examples at a minimum. I need to give this a go from desktop and see if it will work for me. Right now looking through mobile looks promising.

Interesting take on kubernetes from a front end perspective. All valid arguments to me, and really the answer to any do you need to any specific implementation of tech is probably no. We got along just fine before k8s ever existed and you still can, but its really nice in a lot of cases. If your skills lean toward backend or infrastructure I encourage you to give it a try.

There are a lot of beginner friendly k8s distros that you can setup with relative ease, kind and k0s are great for single node, If you want multi-node k3s is what I generally use. If you want a very lightweight OS that you only interact with through an api, and has a very small attack surface talos is an amazing product.

Internal, on-prem, self hosted. If you are trying to avoid the cloud for cost, rules, regulations, red tape, kubernetes is a great option to manage your container workflows yourself without needing to have a cloud budget, get approvals and sign offs on running...

I’m trying to learn proper logs, monitoring, otel, and grafana. Today I imported a bunch of pre-made k8s dashboards and made a few of my own for specific apps, and it made me want to know how I can turn my own custom dashboards into infrastructure as code. Turns out grafana makes it pretty easy to do this, if you have the grafana dashboard sidecar running. It will pick up any ConfigMap with the grafana_dashboard label and import it.

Go to Dashboards -> Pick a Dashboard -> Export -> JSON.

Diun, looks like a very interesting tool to monitor for image updates, it does not make any change, it only makes notifications. This feels like an easy start to getting image updates started with low effort, keep git ops, but requires manual updates. I see this as a tool that would be a great start and pair well with automated image updaters to ensure they are working as expected.

Keel looks interesting, I might give it a try as a simple image updater. I’m unsure if it fits my gitops patterns though. I like to keep everything defined in git, I don’t like drift outside of that so Keel might not be the thing I want.

Changing k8s Storage Class - Migration Job

I’m setting up longhorn in my homelab, and I ran into an issue where I initially setup some pvcs under longhorn, and later realized that to get longhorn to snapshot and backup I needed to hand edit volumes after the fact or change storage class. I’m all in on gitops so option 1 was not an option. So changing storageclass it is.

Now the issue is that you CANNOT mutate storageclass on a provisioned pvc, it is an immutable attribute.

This migration job will create a new pvc with the new storageclass and move the data from the old pvc to the new pvc.

...

Keycloak looks like an interesting way to setup sso. It’s part of the cncf so it’s got a good backing. I want something better for argo workflows and this might be it. I’m curious what else I can tie into it.

slow nfs performance

I’m running a two node k3s cluster at home, I thought I could simply mount an nfs share on each worker node, and essentially have the same storage accross all nodes. I’m already learning why this is not reccommended.

I’ve been running some cronjobs and argo workflows on the second node for awhile, these are things that run in the background and I don’t care if they take a bit longer to keep my master node freed up for more critical work.

I just started trying to build this site in a cronjob, It was taking 20 minutes to build, and something I noticed was that markata was taking minutes to run glob ( search for files ), normally this happens in a few ms and I never notice this step.

...

2 min read

This is a really amazing documentary of argocd. I got into k8s pretty late in the game. Which is pretty typical for me. As I went to use k8s for the first time i was using workflows, then cd. both of these tools had a level of polish that made them seem like they had been there forever and not quite as young as they actually are.

I thought it was interesting how they focused on how the name must be two syllables or less, start with a or b, logo needs to be cutesy funny and recognizable seemed interesting, but puts them at the top of lists and makes them look like they’ve been there forever.

After first setting up a new k3s instance your kubeconfig file will be located in /etc/rancher/k3s/k3s.yaml.

You cans use it from here by setting $KUBECONFIG to that file.

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Or you can copy it to ~/.kube/config

cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

If you have installed k3s on a remote server and need the config on your local machine then you will need to modify the server address to reflect the remote server.

...

I’ve started leaning in on kubernetes kustomize to customize my manifests per deployment per environment. Today I learned that it comes with a diff command.

kubectl diff -k k8s/overlays/local

You can enable color diffs by using an external diff provider like colordiff.

export KUBECTL_EXTERNAL_DIFF="colordiff -N -u"

You might need to install colordiff if you don’t already have it.

sudo pacman -S colordiff sudo apt install colordiff

Now I can try out kustomize changes and see the change with kustomize diff.

kubectl dash k

Kubernetes ships with a feature called kustomize that allows you to customize your manifests in a declarative way. It's a bit like helm, but easier to use. I...

1 min