Posts tagged: containers

Today I learned that docker creates an empty /.dockerenv file to indicate that you are running in a docker container. Other runtimes like podman commonly use /run/.containerenv. kubernetes uses neither of these, the most common way to detect if you are running in kubernetes is to check for the presence of the KUBERNETES_SERVICE_HOST environment variable. There will also be a directory at /var/run/secrets/kubernetes.io/serviceaccount that contains the service account credentials if you are running in kubernetes.

Like a dufus this morning I did a hard reset on a git repo for getting I was working on a manifest for. You see I generally use argo, but occasionally I have no idea what I am doing or want yet and I start raw doggin it, fully aware that I’m going to just nuke this namespace before getting it into a proper argocd.

I was overjoyed when I found out that you can diff your manifests with live production using the kubectl diff command. It uses standard diff so you can bring all your fancy diff viewers you like.

# regular manifest
kubectl diff -f k8s/shots -n shot
# kustomize
kubectl diff -k k8s -n go-waylonwalker-com
# using a fancy diff viewer
kubectl diff -f k8s/shots -n shot | delta
# using an even fancier diff viewer
# pinkies out for this one
kubectl diff -f k8s/shots -n shot | delta --diff-so-fancy

Now I can get those changes back that I thought I lost, and apply updates with confidence knowing what is about to change.

I learned to today that setting MEMORY on your minecraft server causes the JVM to egregiously allocate all of that memory. Not setting it causes slow downs and potential crashes, but setting INIT_MEMORY and MAX_MEMORY gives us the best of both worlds. It is allowed to use more, but does not gobble it all up on startup.

In this economy we need to save all the memory we can!

Here is a non-working snippet for a minecraft server deployment in kubernetes.

      containers:
        - name: dungeon
          image: itzg/minecraft-server
          env:
            - name: EULA
              value: "true"
            - name: INIT_MEMORY
              value: "512M"
            - name: MAX_MEMORY
              value: "3G"

and in docker compose

  dungeon:
    image: itzg/minecraft-server
    environment:
      EULA: "true"
      INIT_MEMORY: "512M"
      MAX_MEMORY: "3G"

Today I learned an important lesson that you should periodically check on your kubeconfigs expiration date. It’s easy to do. You can ask for the client-certificate-data from your kubeconfig, decode it, and use openssl to get the expiration date.

kubectl config view --raw -o jsonpath='{.users[0].user.client-certificate-data}' \
  | base64 -d 2>/dev/null \
  | openssl x509 -noout -dates

The k3s system-upgrade controller is a fantastic tool for upgrading k3s automatically. It has done a fantastic job for me every time I’ve used it. Today I ran it on a cluster that needed to upgrade several minors and I learned that the controller does not pick up on changes to the channel url if you change from minor to minor.

The solution I came up with was to name the plan with the version it supports. Then on each patch upgrade, change both the plan name and the channel. I use gitops with argocd, it automcatically cleaned up old plans, created new plans, and the system-upgrade-controller picked up the plan and started applying immediately.

# Server plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: server-plan-v1.33 # <- This is important if you want to change the channel name
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: node-role.kubernetes.io/control-plane
      operator: In
      values:
      - "true"
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/v1.33
---
# Agent plan
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: agent-plan-v1.33 # <- This is important if you want to change the channel name
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: node-role.kubernetes.io/control-plane
      operator: DoesNotExist
  prepare:
    args:
    - prepare
    - server-plan
    image: rancher/k3s-upgrade
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/v1.33

I’d love to see a better way if you have a way to upgrade through minors, or manually control the minor of your cluster let me know.

If you need to target a specific k8s node in the cluster, you can use labels. You want to treat your nodes as much like cattle as you can, but sometimes budgets get in the way. You might be like me and just run any free hardware you can get in your cluster, or you might have some large storage or gpu needs that you can’t afford to put on every node in the cluster.

kubectl get nodes --show-labels

# add the bigpool label
kubectl label node k8s-1 bigpool=true
kubectl get nodes --show-labels

# remove the bigpool label
kubectl label node k8s-1 bigpool-

To use the label in a pod set spec.nodeSelector to the label that you applied.

apiVersion: v1
kind: Pod
metadata:
  name: busybox
spec:
  containers:
  - name: busybox
    image: busybox
  nodeSelector:
    bigpool: "true"