I’m trying to learn proper logs, monitoring, otel, and grafana. Today I
imported a bunch of pre-made k8s dashboards and made a few of my own for
specific apps, and it made me want to know how I can turn my own custom
dashboards into infrastructure as code. Turns out grafana makes it pretty easy
to do this, if you have the grafana dashboard sidecar running. It will pick up
any ConfigMap with the grafana_dashboard label and import it.
Go to Dashboards -> Pick a Dashboard -> Export -> JSON.
[1]
[2]
[3]
apiVersion: v1
kind: ConfigMap
metadata:
name: my-dashboard
namespace: meta
labels:
grafana_dashboard: "1"
data:
my-dashboard.json: |
{
"annotations": {
"list": [
...
"uid": "fel2uhjhepg5ce",
"version": 3
}
References:
[1]: https://dropper.waylonwalker.com/api/file/530e8515-a72a-4341-82d7-37f6f985e327.webp
[2]: https://dropper.waylonwalker.com/api/file/d792b2db-2dcf-465f-a400-e84f199ec22d.webp
[3]: https://dropper.waylonwalker.com/api/file/684701cc-efec-4e2b-9630-c8aea7ff5b14.webp
Posts tagged: kubernetes
All posts with the tag "kubernetes"
61 posts
latest post 2026-03-23
Publishing rhythm
Diun
Receive notifications when a Docker image is updated on a Docker registry
crazymax.dev [1]
Diun, looks like a very interesting tool to monitor for image updates, it does not make any change, it only makes notifications. This feels like an easy start to getting image updates started with low effort, keep git [2] ops, but requires manual updates. I see this as a tool that would be a great start and pair well with automated image updaters to ensure they are working as expected.
Note
This post is a thought [3]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://crazymax.dev/diun/
[2]: /glossary/git/
[3]: /thoughts/
Keel
Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates
keel.sh [1]
Keel looks interesting, I might give it a try as a simple image updater. I’m unsure if it fits my gitops patterns though. I like to keep everything defined in git [2], I don’t like drift outside of that so Keel might not be the thing I want.
Note
This post is a thought [3]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://keel.sh/
[2]: /glossary/git/
[3]: /thoughts/
Changing k8s Storage Class - Migration Job
I’m setting up longhorn in my homelab [1], and I ran into an issue where I
initially setup some pvcs under longhorn, and later realized that to get
longhorn to snapshot and backup I needed to hand edit volumes after the fact or
change storage class. I’m all in on gitops so option 1 was not an option. So
changing storageclass it is.
Now the issue is that you CANNOT mutate storageclass on a provisioned pvc, it
is an immutable attribute.
Migration Job # [2]
This migration job will create a new pvc with the new storageclass and move the
data from the old pvc to the new pvc.
Existing Pods
This migration job will not work if you have a pod using the old pvc. You
will need to shutdown the pod and delete it.
# old pvc with longhorn storageclass
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: site-pvc-longhorn
namespace: waylonwalker-com
spec:
storageClassName: longhorn-backup
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
# new pvc with longhorn-ba...
AI workloads on Talos Linux
Companies are exploring how to run GPU accelerated workloads on Kubernetes.
Sidero Labs · siderolabs.com [1]
cool article for setting up talos linux with an nvidia gpu. What a wild world it we are living in where these devices that started out being only for hardcore gamers are becoming commonplace in servers and slowly entering the homelab [2] space.
Note
This post is a thought [3]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://www.siderolabs.com/blog/ai-workloads-on-talos-linux/
[2]: /homelab/
[3]: /thoughts/
-
Kelsey says several times in this interview, you don’t need kubernetes. If you are running one node you don’t need kubernetes. My question though is, would you use kubernetes? Ya I get it if you are a web developer, data scientist, backend dev, but if you are looking to bee a whole ass engineer, or infrastructure engineer, you know kubernetes, Should you use kubernetes on single node?
Note
This post is a thought [1]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: /thoughts/
poolers.postgresql.cnpg.io CRD metadata.annotations Too long · Issue #325 · cloudnative-pg/charts
Unable to deploy helm chart using ArgoCD. Getting following error Failed sync attempt to : one or more objects failed to apply, reason: CustomResourceDefinition.apiextensions.k8s.io "poolers.postgr...
GitHub · github.com [1]
I’ve never seen or needed to use a serversideapply in kubernetes before, but I ran into this same issue in my k3s homelab [2] while installing cloudnative-pg.
You can do it with argo
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
syncPolicy:
syncOptions:
- ServerSideApply=true
and you can do it with kubectl
kubectl apply --server-side --force-conflicts -f cnpg-1.25.0.yaml
Note
This post is a thought [3]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://github.com/cloudnative-pg/charts/issues/325
[2]: /homelab/
[3]: /thoughts/
Keycloak
Keycloak - the open source identity and access management solution. Add single-sign-on and authentication to applications and secure services with minimum effort.
Keycloak · keycloak.org [1]
Keycloak looks like an interesting way to setup sso. It’s part of the cncf so it’s got a good backing. I want something better for argo workflows and this might be it. I’m curious what else I can tie into it.
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://www.keycloak.org/
[2]: /thoughts/
GitHub - ngalaiko/tree-sitter-go-template: Golang template grammar for tree-sitter
Golang template grammar for tree-sitter. Contribute to ngalaiko/tree-sitter-go-template development by creating an account on GitHub.
GitHub · github.com [1]
This setup fixed my nvim syntax highlighting in helm templates.
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://github.com/ngalaiko/tree-sitter-go-template
[2]: /thoughts/
slow nfs performance
I’m running a two node k3s cluster at home, I thought I could simply mount an
nfs share on each worker node, and essentially have the same storage accross
all nodes. I’m already learning why this is not reccommended.
[1]
Slow # [2]
I’ve been running some cronjobs and argo workflows on the second node for
awhile, these are things that run in the background and I don’t care if they
take a bit longer to keep my master node freed up for more critical work.
I just started trying to build this site in a cronjob, It was taking 20 minutes
to build, and something I noticed was that markata was taking minutes to run
glob ( search for files ), normally this happens in a few ms and I never
notice this step.
[3]
I just moved into the master node and the results were wild at ~30x faster
Permissions # [4]
I have seen where you can get diffent permissions on the nfs share based on
user id. Since I’m homelabbing here I only have one user per machine. As you
step into enterprise level VMs wi...
External Link
X (formerly Twitter) · x.com [1]
switching cloud providers, theres no easy way. K8s was supposed to get us there, haha, the deep integrations with each vendor just keep locking us in
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://x.com/changelog/status/1860697708463378497
[2]: /thoughts/
-
This is a really amazing documentary of argocd. I got into k8s pretty late in the game. Which is pretty typical for me. As I went to use k8s for the first time i was using workflows, then cd. both of these tools had a level of polish that made them seem like they had been there forever and not quite as young as they actually are.
I thought it was interesting how they focused on how the name must be two syllables or less, start with a or b, logo needs to be cutesy funny and recognizable seemed interesting, but puts them at the top of lists and makes them look like they’ve been there forever.
Note
This post is a thought [1]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: /thoughts/
Provider List - The Cluster API Book
cluster-api.sigs.k8s.io [1]
Cluster API book
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://cluster-api.sigs.k8s.io/reference/providers
[2]: /thoughts/
After first setting up a new k3s instance your kubeconfig file will be located
in /etc/rancher/k3s/k3s.yaml.
You cans use it from here by setting $KUBECONFIG to that file.
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
Or you can copy it to ~/.kube/config
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
If you have installed k3s on a remote server and need the config on your local
machine then you will need to modify the server address to reflect the remote
server.
scp user@<server-ip>:/etc/rancher/k3s/k3s.yaml ~/.kube/config
Warning
only do this if you don’t already have a ~/.kube/config file, otherwise copy
it to a new file and set your $KUBECONFIG env variable to use it.
Now you will need to open that file and change the server address, making sure
to keep the port number.
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ****
server: https://<server-ip>:6443
name: default
I’ve started leaning in on kubernetes kustomize to customize my manifests per
deployment per environment. Today I learned that it comes with a diff command.
kubectl diff -k k8s/overlays/local
You can enable color diffs by using an external diff provider like colordiff.
export KUBECTL_EXTERNAL_DIFF="colordiff -N -u"
You might need to install colordiff if you don’t already have it.
sudo pacman -S colordiff
sudo apt install colordiff
Now I can try out kustomize changes and see the change with kustomize diff.
kubectl dash k
Kubernetes ships with a feature called kustomize that allows you to customize your manifests in a declarative way. It's a bit like helm, but easier to use. I...
-
Great intro into kustomize. This helped me get started with kustomize.
Note
This post is a thought [1]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: /thoughts/
kind cluster
kind [1]{.hoverlink} is a very useful tool to quickly standup and
teardown kubernetes clusters. I use it to run clusters locally. Generally
they are short lived clusters for trying, testing, and learning about
kubernetes.
Kind is Kubernetes in Docker, its very fast to get a new cluster up and
running. Other than checking a box in docker desktop it is the easiest way
currently to get a cluster up and running. I’ve used docker desktop for k8s
before I really developed on k8s and it was buggy at the time and sometimes
started and sometimes didn’t, when it didnt I had no idea how to fix it. I’d
suggest kind as the best option to get a cluster up and running locally.
Not Production # [2]
If you are looking for a production ready cluster this is not it. I really
like k3s [3]{.hoverlink}. At the time that I chose k3s it was
the most lightweight option that easily supported multi-node clusters.
Starting a kind cluster # [4]
The first step, and maybe only one that you need is to create ...
Yesterday I realized that I have overlooked the default installation method of
the sealed secrets controller for kubernetes kubeseal [1] this whole time an
jumped straight to the helm section. I spun up a quick kind cluster [2] and
had it up quickly. I can’t say this is any better or worse than helm as I have
never needed to customize the install. According to the docs you can customize
it with [[ kustomize ]] or helm.
# option if you don't have a cluster try with kind
kind create cluster
curl -L https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.27.0/controller.yaml > controller.yaml
kubectl apply -f controller.yaml
References:
[1]: /kubernetes-kubeseal/
[2]: /kind-cluster/
Where DOESN’T curl run with curl BDFL Daniel Stenberg (Changelog & Friends #49)
Daniel Stenberg shares his guiding principles for BDFL'ing curl, gives us his perspective on the state of the internet, talks financial independence, ensuring curl won't be the next XZ & more!
Changelog · changelog.com [1]
Love how Daniel just brushed off the number of netflix tv installs as small beans for curl installs. overall great episode on open source, owning a major component to technology with such a small team.
Note
This post is a thought [2]. It’s a short note that I make
about someone else’s content online #thoughts
References:
[1]: https://changelog.com/friends/49
[2]: /thoughts/