The k3s system-upgrade controller is a fantastic tool for upgrading k3s
automatically. It has done a fantastic job for me every time I’ve used it.
Today I ran it on a cluster that needed to upgrade several minors and I
learned that the controller does not pick up on changes to the channel url if
you change from minor to minor.
The solution I came up with was to name the plan with the version it supports.
Then on each patch upgrade, change both the plan name and the channel. I use
gitops with argocd, it automcatically cleaned up old plans, created new plans,
and the system-upgrade-controller picked up the plan and started applying
immediately.
# Server plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
name: server-plan-v1.33 # <- This is important if you want to change the channel name
namespace: system-upgrade
spec:
concurrency: 1
cordon: true
nodeSelector:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: In
values:
- "true"
serviceAccountName: system-upgrade
upgrade:
image: rancher/k3s-upgrade
channel: https://update.k3s.io/v1-release/channels/v1.33
---
# Agent plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
name: age...
Posts tagged: k3s
All posts with the tag "k3s"
12 posts
latest post 2025-12-05
Publishing rhythm
slow nfs performance
I’m running a two node k3s cluster at home, I thought I could simply mount an
nfs share on each worker node, and essentially have the same storage accross
all nodes. I’m already learning why this is not reccommended.
[1]
Slow # [2]
I’ve been running some cronjobs and argo workflows on the second node for
awhile, these are things that run in the background and I don’t care if they
take a bit longer to keep my master node freed up for more critical work.
I just started trying to build this site in a cronjob, It was taking 20 minutes
to build, and something I noticed was that markata was taking minutes to run
glob ( search for files ), normally this happens in a few ms and I never
notice this step.
[3]
I just moved into the master node and the results were wild at ~30x faster
Permissions # [4]
I have seen where you can get diffent permissions on the nfs share based on
user id. Since I’m homelabbing here I only have one user per machine. As you
step into enterprise level VMs wi...
After first setting up a new k3s instance your kubeconfig file will be located
in /etc/rancher/k3s/k3s.yaml.
You cans use it from here by setting $KUBECONFIG to that file.
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
Or you can copy it to ~/.kube/config
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
If you have installed k3s on a remote server and need the config on your local
machine then you will need to modify the server address to reflect the remote
server.
scp user@<server-ip>:/etc/rancher/k3s/k3s.yaml ~/.kube/config
Warning
only do this if you don’t already have a ~/.kube/config file, otherwise copy
it to a new file and set your $KUBECONFIG env variable to use it.
Now you will need to open that file and change the server address, making sure
to keep the port number.
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ****
server: https://<server-ip>:6443
name: default
Manual Upgrades | K3s
You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version.
docs.k3s.io [1]
You can give k3s an install channel to install stable, latest, or specific versions like 1.26. This is handy to make sure that you install the same version on all of your workers.
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest <EXISTING_K3S_ENV> sh -s - <EXISTING_K3S_ARGS>
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://docs.k3s.io/upgrades/manual
[2]: /thoughts/
Uptime Kuma
A self-hosted monitoring tool
uptime.kuma.pet [1]
Uptime kuma is a fantastic self hosted [2] monitoring tool. One docker run command and you are up and running. Once you are in you have full control over checking status of urls, frequency, allowed timeouts, and a HUGE list of notification providers
docker run -d --restart=always -p 3001:3001 -v uptime-kuma:/app/data --name uptime-kuma louislam/uptime-kuma:1
I deployed it in my homelab [3] today.
[4]
Note
This post is a thought [5]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://uptime.kuma.pet/
[2]: /self-host/
[3]: /homelab/
[4]: https://twitter.com/_WaylonWalker/status/1723077941649707468
[5]: /thoughts/
External Link
thoughts.waylonwalker.com [1]
I was looking to add running kubernetes jobs to a python cli I am creating, and I found this solution, mostly thanks to ollama run mistral:7b-instruct-q4_K_M and my loose understanding of what the yaml syntax is supposed to look like for a kubernetes job. This will let me create a job in the cluster, choose the image that runs, the command that is called, and how long until the job expires and is cleaned up. While the job still exists I can go in and look at the logs, but once its ttl has expired they are gone.
from kubernetes import client, config
# Load the default kubeconfig
config.load_kube_config()
# Define the API client for batch jobs
api_instance = client.BatchV1Api()
# Create a new job object
job = client.V1Job(
api_version="batch/v1",
kind="Job",
metadata=client.V1ObjectMeta(name="myjob"),
spec=client.V1JobSpec(
ttl_seconds_after_finished=100,
template=client.V1PodTemplateSpec(
metadata=client.V1ObjectMeta(labels={"app": "myjob"}),
spec=client.V1PodSpec(
containers=[
client.V1Container(
name="myjobcontainer",
image="busybox",
command=["ls", "/"],
),
],
restart_policy="Never",
),
),
backoff_limit=1,
)...
Translate a Docker Compose File to Kubernetes Resources
What
Kubernetes · kubernetes.io [1]
kompose is a sick cli to convert docker-compose.yml to kubernetes manifest.
# install
curl -L https://github.com/kubernetes/kompose/releases/download/v1.26.0/kompose-linux-amd64 -o kompose
kompose convert
kompose convert -o deployment.yaml
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/
[2]: /thoughts/
Kubernetes Persistent Volumes with Deployment and StatefulSet
How to use Kubernetes persistent volumes with deployment and stateful set and also when you should use one or another.
Alen Komljen · akomljen.com [1]
Example of how to add a pvc to a deployment.
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://akomljen.com/kubernetes-persistent-volumes-with-deployment-and-statefulset/
[2]: /thoughts/
[1]
I was curious to see what was going on inside of my minio object storage. Great technique here by Frank to create an inspector pod, then you can do as you wish with the data.
I created the manifest as pvc-inspector.yml
apiVersion: v1
kind: Pod
metadata:
name: pvc-inspector
spec:
containers:
- image: busybox
name: pvc-inspector
command: ["tail"]
args: ["-f", "/dev/null"]
volumeMounts:
- mountPath: /pvc
name: pvc-mount
volumes:
- name: pvc-mount
persistentVolumeClaim:
claimName: pvc-name
Then used it like this.
# create pvc-inspector pod
kubectl apply -f pvc-inspector.yml
# exec into inspector
kubectl exec -it pvc-inspector -- sh
# explore data
ls /pvc
# cleanup
kubectl delete -f pvc-inspector.yml
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: /static/https://frank.sauerburger.io/2021/12/01/inspect-k8s-pvc.html
[2]: /thoughts/
External Link
stackoverflow.com [1]
In order to use k8s secrets manifest you first need to encode the data values.
echo -n 'mega_secret_key' | openssl base64
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://stackoverflow.com/questions/53394973/cant-create-secret-in-kubernetes-illegal-base64-data-at-input
[2]: /thoughts/
Can I access k3s using just kubectl (no sudo and no k3s command)
Can I access k3s using just kubectl (no sudo and no k3s command)
Reddit · reddit.com [1]
Right after installing k3s you are going to need to use sudo to use any kubectl command. The reason for this is that the default config is owned by root. To get around this you will need to make your own config and set the KUBECONFIG environment variable
To do this I used sudo one last time to copy the k3s.yaml file into my own directory and take ownership of it.
sudo cp /etc/rancher/k3s/k3s.yaml /home/waylon/.config/kube
sudo chown -R waylon:waylon ~/.config/kube
export KUBECONFIG=~/.config/kube/k3s.yaml
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://www.reddit.com/r/kubernetes/comments/cojjf5/can_i_access_k3s_using_just_kubectl_no_sudo_and/
[2]: /thoughts/
Quick-Start Guide | K3s
This guide will help you quickly launch a cluster with default options. Make sure your nodes meet the requirements before proceeding.
docs.k3s.io [1]
I recently spun up k3s in my homelab [2]. I’m trying to offload some work off of my free tier fly.io app in order to keep it free tier without crashing.
# install and start k3s
curl -sfL https://get.k3s.io | sh -
# check to see if your nodes are started
sudo kubectl get nodes
My main hiccup so far was the machine I am running on runs zfs on root, and it would not start the master node. Rather than figuring out how to make zfs play nice I just pointed k3s to a drive that is not zfs.
# manuallly
sudo k3s server -d /mnt/vault/.rancher/k3s
# without editing systemd service
sudo ln -s /mnt/vault/.rancher/k3s /var/lib/rancher/k3s
Note
This post is a thought [3]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://docs.k3s.io/quick-start
[2]: /homelab/
[3]: /thoughts/