Today I Learned

I’ve been debugging a cloudflared tunnel issue in my homelab all day today, and getting really frustrated. My issue ended up being that it was running twice, once without the correct config file and another with it. I believe that cacheing may have compounded the issue.

In yesterday’s post I setup a cloudflared tunnel on my ubuntu server to expose applications running on the server to the internet. I’m setting up a new server and running cloudflared in its own vm.

setup cloudflared tunnel on ubuntu

Check that dns is pointing to the correct tunnel #

dig subdomain.example.com
traceroute subdomain.example.com

Check that the tunnel is running #

export CLOUDFLARED_TUNNEL_ID = "my-tunnel-id"

cloudflared tunnel list
cloudflared tunnel info $CLOUDFLARED_TUNNEL_ID

I run a cloudflared tunnel on my ubuntu server to expose applications running on the server to the internet. I’m setting up a new server and running cloudflared in its own vm.

Get the cloudflared binary #

sudo wget https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 -O /usr/local/bin/cloudflared

sudo chmod +x /usr/local/bin/cloudflared

#

Now setup the config directory. For the systemd service to work, the config file needs to be in /etc/cloudflared. I like to give my user rights to edit the config file without being sudo, we will do that here by creating a group cloudflared, add ourselves to the group, give ownership of /etc/cloudflared to the group, give group write access to the directory, and refresh groups.

sudo mkdir -p /etc/cloudflared
sudo groupadd cloudflared
sudo usermod -aG cloudflared $USER
sudo chown -R root:cloudflared /etc/cloudflared
sudo chmod g+w /etc/cloudflared
newgrp cloudflared

login #

Now we can log into the domain zone with cloudflared.

cloudflared tunnel login

This will give a url, follow it in a browser to log in.

cloudflared tunnel create <NAME>
mv ~/.cloudflared/cert.pem /etc/cloudflared/cert.pem
mv ~/.cloudflared/<tunnel-id>.json /etc/cloudflared/<tunnel-id>.json

config #

Now setup config. For the systemd service to work, the config file needs to be in /etc/cloudflared. The config that I have provided below will expose localhost:8000 to tester.example.com

export CLOUDFLARED_TUNNEL_ID=$(ls /etc/cloudflared/*.json | xargs -n 1 basename | sed 's/\.json$//')
mv ~/.cloudflared/${CLOUDFLARED_TUNNEL_ID}.json /etc/cloudflared/${CLOUDFLARED_TUNNEL_ID}.json
mv ~/.cloudflared/cert.pem /etc/cloudflared/cert.pem
echo "
tunnel: $(CLOUDFLARED_TUNNEL_ID)
credentials-file: /etc/cloudflared/$(CLOUDFLARED_TUNNEL_ID).json
ingress:
  - hostname: tester.example.com
    service: http://localhost:8000
  - service: 'http_status:404'
" >> /etc/cloudflared/config.yaml

dns #

Now to get a dns record for tester.example.com to point to the cloudflared tunnel.

cloudflared tunnel route dns $(CLOUDFLARED_TUNNEL_ID) tester.example.com

systemd #

Now install the systemd service.

sudo cloudflared service install

sudo systemctl status cloudflared.service
# if its not running
sudo systemctl start cloudflared.service

I’ve been playing with 3d printing some items through the slant3d api. I’ve been pricing out different prints by running a slice request through their api.

make a project #

I’ve been using uv for project management. It’s been working well for quick projects like this while making it reproducible, I’m still all in on hatch for libraries.

mkdir slantproject
cd slantproject
uv init
uv venv
. ./.venv/bin/activate
uv add httpx rich python-dotenv

Get an api key #

You will need an api key from the slant api, which currently requires a google account and a credit card to create.

# .env
#  replace with your api key from https://api-fe-two.vercel.app/
SLANT_API_KEY=sl-**

slicing an stl with teh slant api #

Then you can run the python script to price out your print. I’m not exactly sure how this compares to an order, especially when you add in different materials.

from dotenv import load_dotenv
import httpx
import os

load_dotenv()

stl_url = ''
api_key = os.environ["SLANT_API_KEY"]

api = httpx.Client(base_url="https://www.slant3dapi.com/api/slicer")

res = httpx.post(
    "https://www.slant3dapi.com/api/slicer",
    json={"fileURL": stl_url},
    headers={"api-key": api_key, "Content-Type": "application/json"},
    timeout=60,
)


print(res.json())

After first setting up a new k3s instance your kubeconfig file will be located in /etc/rancher/k3s/k3s.yaml.

You cans use it from here by setting $KUBECONFIG to that file.

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Or you can copy it to ~/.kube/config

cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

If you have installed k3s on a remote server and need the config on your local machine then you will need to modify the server address to reflect the remote server.

scp user@<server-ip>:/etc/rancher/k3s/k3s.yaml ~/.kube/config

Now you will need to open that file and change the server address, making sure to keep the port number.

apiVersion: v1
clusters:
  - cluster:
      certificate-authority-data: ****
      server: https://<server-ip>:6443
    name: default

Vim has a handy feature to format text with gq. You can use it in visual mode, give it a motion, or if you give it gqq it will format the current line. I use this quite often while writing in markdown, I do not use softwraps in vim, so gqq quickly formats my current line into a paragraph. Once I have done this for a single line one time I typically switch to the motion for around paragraph gqap to format the whole paragraph and not just the current line.

before formatting #

after formattting #

A slug is the part of the url that comes after the domain. Commonly matches the file name of a markdown file many blogging systems. These are typically human readable, unique identifiers for pages within the site.

Wikilinks are a core concept within obsidian to link to documents by Slug wrapped in double square brackets. These are commonly used within wiki site generators.

[[slug]]

Obsidian gives you a keybinding alt+enter to go to that file, but if it does not exist it will create the file for you in the root of the project. It’s a nice way to quickly make new documents.

It was not obvious to me, but if you have a wikilink such as Trying Obsidian, you can jump to the file in obsidian, just like you can with lsp go to definition, the keybinding is alt + enter.

I’ve long used copier to create all of my posts for my blog, and it works really well for my workflow. I think of a title, call a template, and give it a title. out of the box obsidian did not seem to work this way. It seems like it wants me to right click a file tree and make a new file using the tree, this is not my jam.

Here is what I came up with to replace my til template.

---
date: <% tp.file.creation_date() %>
templateKey: til
title: <%*
  const originalFileName = await tp.system.prompt("Enter file name");
  const toTitleCase = str => str.replace(
    /\w\S*/g,
    txt => txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase()
  );
  const title = toTitleCase(originalFileName);
  tR += title + '\n'; // Add the title to the template result
-%>
published: true
tags:
  -
---
<%*
const fileName = originalFileName.toLowerCase().replace(/\s+/g, '-');
const newFilePath = `pages/til/${fileName}`;
await tp.file.move(newFilePath);
-%>

<% tp.file.cursor() %>

• tR is a return value, and it gets placed directly into the place it is in the file
• to.file.cursor() creates a tab-index point so I can tab into the content

I’m giving obsidian a go as an editor for my blog and one of the main things I want to fix in my workflow is the ability to quickly drop in images. on first look through the community plugins I found Image Converter. I set it up to convert to webp and drop them in a git submodule. I may make it something other than a git repo in the future, but I’ve learned that adding images to my blog repo quickly makes it heavy and hard to clone on other machines.

Once the images are there they are pushed and deployed as their own site to cloudflare pages. I made a quick edit to my sick wikilink hover plugin for my blog. if it sees a wikilink ending in webp, convert the domain over to obsidian-assets.waylonwalker.com, and clean up the remaining "! " that the python md-it library leaves behind.

after first try I needed to increase the width from 600 to 1400, the image in this post was unreadable.

This is part of me getting set up and Trying Obsidian

I’ve had a couple of uploads to twitter fail recently and has been a pain. I tried some online converters for convenience, but none of them worked. I reached out to chatgpt and found succeess with this ffmpeg command.

ffmpeg -i input.mp4 \
  -vf "scale=trunc(oh*a/2)*2:min(720\,trunc(ih*a/2)*2)" \
  -c:v libx264 -profile:v high -level:v 4.1 \
  -b:v 3500k -maxrate 3500k -bufsize 7000k \
  -pix_fmt yuv420p \
  -c:a aac -b:a 128k -ar 44100 \
  -movflags +faststart \
  output.mp4

I’ve started leaning in on kubernetes kustomize to customize my manifests per deployment per environment. Today I learned that it comes with a diff command.

kubectl diff -k k8s/overlays/local

You can enable color diffs by using an external diff provider like colordiff.

export KUBECTL_EXTERNAL_DIFF="colordiff -N -u"

You might need to install colordiff if you don’t already have it.

sudo pacman -S colordiff

sudo apt install colordiff

Now I can try out kustomize changes and see the change with kustomize diff.

Animal well does not let you remap keys, and really doesn’t even inform you that it is keyboard compatible. I had to play around and discover the keymap, which can be a bit tricky on a 40% board. This is what I found.

• wasd - move
• space - jump / a
• enter - interact / b
• x - throw
• c - inventory
• 1 - left item / rb
• 2 - open item menu / triangle
• 3 - right item / lb

I’ve been using fastapi more and more lately and one feature I just started using is background tasks [[ thoughts-333 ]].

basic diskcache example <a href="/python-diskcache/" class="wikilink" data-title="How I setup a sqlite cache in python" data-description="When I need to cache some data between runs or share a cache accross multiple processes my go to library in python is . It&#39;s built on sqlite with just enough..." data-date="2022-03-29">How I setup a sqlite cache in python</a>

One Background Task per db entry #

I am using it for longer running tasks and I don’t want to give users the ability to spam these long running tasks with many duplicates running at the same time. And each fastapi worker will be running in a different process so I cannot keep track of work in memory, I have to do it in a distributed fashion. Since they are all running on the same machine with access to the same disk, diskcache is a good choice

What I need #

• check if a job is running
• automatically expire jobs

Less infrastructure complexity #

My brain first went to thinking I needed another service like redis running alongside fastapi for this, then it hit me that I can use diskcache.

How I used diskcache #

Here is how I used diskcache to debounce taking screenshots for a unique shot every 60 seconds.

from diskcache import Cache

jobs_cache = Cache("jobs-cache")

@shots_router.get("/shot/{shot_id}", responses={200: {"content": {"image/webp": {}}}})
@shots_router.get("/shot/{shot_id}/", responses={200: {"content": {"image/webp": {}}}})
async def get_shot_by_id(
    background_tasks: BackgroundTasks,
    request: Request,
    shot_id: int,
):
    shot = Shot.get(shot_id)
    # check if the shot exists and return it or continue to create it.



    is_running = jobs_cache.get(shot_id)

    if is_running:
        expire_time = datetime.fromtimestamp(jobs_cache.peekitem(expire_time=True)[1]) - datetime.now()
        console.print("[red]Already running store_shot: ", shot_id)
        console.print(f"[red]Can retry in {expire_time.seconds}s")
    else:
        jobs_cache.set(shot_id, True, 60)
        background_tasks.add_task(
            store_shot,
            shot_id=shot_id,
        )

Yesterday I realized that I have overlooked the default installation method of the sealed secrets controller for kubernetes kubeseal this whole time an jumped straight to the helm section. I spun up a quick kind cluster and had it up quickly. I can’t say this is any better or worse than helm as I have never needed to customize the install. According to the docs you can customize it with [[ kustomize ]] or helm.

# option if you don't have a cluster try with kind

kind create cluster

curl -L https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.27.0/controller.yaml > controller.yaml

kubectl apply -f controller.yaml

I’ve long had issues with my qmk keyboard media keys on my arch install, I always thought it was on the keyboard end. Today I learned that playerctl fixes this.

paru -S playerctl

Once it is installed all of my media keys started working right away.

I played around with it a bit more and came up with a way to display the current playing title in my notifictations.

notify-send "`playerctl metadata --format '{{lc(status)}}:{{artist}}-{{album}}-{{title}}'`"

Today I am playing around with tailwind, flexing the css muscle and learning how to build new and different layouts with it.

I created a new post template that mimics a terminal look in css where I could inject the post title, description, and other frontmatter elements.