Looking for inspiration? build [1] by pypa [2].
A simple, correct Python build frontend
References:
[1]: https://github.com/pypa/build
[2]: https://github.com/pypa
Published
All published posts
2493 posts
latest post 2026-05-11
Publishing rhythm
I was on Talk Python
After years of listening to talkpython.fm [1] I had the
honor to be part of
episode-337 [2]
to talk about Kedro for maintainable data science.
I was quite nervous to talk on a show that I helped shape my career in such a
profound way. I started my journey towards software engineering near Michaels
first few episodes. His discussions with such great developers over the years
has made an huge impact on my skill. It has always given me great advice and
topics to go deeper on.
During the episode I tried my best to let Yetu and Ivan take the spotlight as
the maintainer and chime in with my experience as a user of kedro.
Video Version # [3]
https://youtu.be/WTcjvwkXoY0
Michael made the call available on youtube as well as the audio only
podcast [2]
References:
[1]: https://talkpython.fm/
[2]: https://talkpython.fm/episodes/show/337/kedro-for-maintainable-data-science
[3]: #video-version
Check out squidfunk [1] and their project mkdocs-material [2].
Documentation that simply works
References:
[1]: https://github.com/squidfunk
[2]: https://github.com/squidfunk/mkdocs-material
I just love how some features of vim are so discoverable and memorable once you
really start to grasp it. Sorting and uniqing your files or ranges is one of
those examples for me.
" sort the file
:sort
" sort the file only keeping unique lines
:sort u
" sort a range
:'<,'> sort
" sort a range only keeping unique lines
:'<,'> sort u
I recently used this to dedupe my autogenerated links section for
rich-syntax-range-style [1].
More often I am using it to sort and uniqify objects like arrays and lists.
Here is what the markdown looks like.
* [py-tree-sitter](https://github.com/tree-sitter/py-tree-sitter)
* [rich](https://github.com/Textualize/rich)
* [@textualizeio](https://twitter.com/textualizeio)
* [rich](https://github.com/Textualize/rich)
* [another post](https://waylonwalker.com/designing-kedro-router)
* [print-register-pipelines](https://screenshots.waylonwalker.com/print-register-pipelines.webp)
* [rich](https://github.com/Textualize/rich)
* [console-print-register-pipelines](https://screenshots.waylonwalker.com/console-print-register-pipelines.webp)
* [rich](https://github.com/Textualize/rich)
* [syntax-print-register-pipelines](https://screenshots.waylonwalker.com/syn...
Today I’ve been playing with
py-tree-sitter [1] a bit and I
wanted to highlight match ranges, but was unable to figure out how to do it
with rich [2], so I reached out to
@textualizeio [3] for help.
https://twitter.com/_WaylonWalker/status/1562469770766589952
While waiting for that reply let’s show how we got this far.
imports # [4]
Lets import all the classes that we need from rich [2] and setup a console to print
to.
from rich.console import Console
from rich.syntax import Syntax
from rich.style import Style
console = Console()
some code # [5]
Now we need some code to highlight. I am going to rip my register_pipeline
from another post [6].
code = '''
from find_kedro import find_kedro
def register_pipelines(self) -> Dict[str, Pipeline]:
"""Register the project's pipeline.
Returns:
A mapping from a pipeline name to a ``Pipeline`` object.
"""
return find_kedro()
'''
print # [7]
We could simply print out the code we have as a variable, but thats a bit hard
to read.
[8]
console.print # [9]
printing with rich [2]’s console makes it a little better, but not much by default.
[10]
Syntax # [11]
We can pull from rich [2]’s syntax module to really pretty this up.
...
How to vimgrep over hidden files.
I needed to delete all build pipeline steps that were named upload docs. I
currently have about 60 projects running from the same template all running
very similar builds. In the past I’ve scripted out migrations for large
changes like this, they involved writing a python script that would load the
yaml file into a dictionary, find the corresponding steps make the change and
write it back out.
Today’s job was much simplar, just delete the step, were all steps are
surrounded by newlines. My first thought was to just open all files in vim and
run dap. I just needed to get these files:positions into my quickfix. My
issue is that all the builds reside within hidden directories by convention.
The issue # [1]
variability
After searching through all the projects it was clear that all the steps were
in their own paragraph, though I was not 100% confident enough to completely
automate it, and the word upload docs was in the paragraph.
some were a two liner
- name: upload docs
script: aws s3 ...
Some had a variation in the name
- name: upload docs to s3
script: aws s3 ...
some were more than 2 lines.
- name: upload docs
script: |
aws s3 ...
s...
[1]
a sprinter edging out his opponent by Dall-e
It’s about time to release Markata 0.3.0. I’ve had 8 pre-releases since the
last release, but more importantly it has about 3 months of updates. Many of
which are just cleaning up bad practices that were showing up as hot spots on
my pyinstrument reports
Markata started off partly as a python developer frustrated with using nodejs
for everything, and a desire to learn how to make frameworks in pluggy. Little
did I know how flexible pluggy would make it. It started out just as my blog
generator, but has turned into quite a bit more.
Over time this side project has grown some warts and some of them were now
becoming a big enough issue it was time to cut them out.
Let’s compare # [2]
I like to use my tils articles for examples and tests like this as there are
enough articles for a good test, but they are pretty short and quick to render.
mkdir ~/git/tils/tils
cp ~/git/waylonwalker.com/pages/til/ ~/tils/tils -r
cd ~/git/tils/tils
running tils on 0.2.0 # [3]
At the time of writing this is the current version of markata, so just make a
new venv and run it.
python3 -m venv .venv --prompt $(basename $PWD)
pip install markata
mark...
Check out stable-diffusion [1] by CompVis [2]. It’s a well-crafted project with great potential.
A latent text-to-image diffusion model
References:
[1]: https://github.com/CompVis/stable-diffusion
[2]: https://github.com/CompVis
Deliberative # [1]
People exceptionally talented in the Deliberative theme are best described by
the serious care they take in making decisions or choices. They anticipate
obstacles.
I am risk-adverse. I want everything well thought out and calculated before I
make any sort of change. I have never gambled in my life and just the thought
of it makes me anxious.
Aim it # [2]
I can use this as a strength to plan out potential issues and prevent them. I
do this quite often with my role in infrastructure.
I need to make sure that I use deadlines to keep this as a strength and not
hinderence.
Automation # [3]
One of the biggest ways that I utilize this skill is automation. I am all
about automating things, not just because I don’t want to do the manual work,
but I am not sure when I am going to need to do something again.
References:
[1]: #deliberative
[2]: #aim-it
[3]: #automation
Check out archlinux [1] and their project aur [2].
⚠️⚠️Experimental aur [3].git [4] mirror⚠️⚠️ (read-only mirror)
References:
[1]: https://github.com/archlinux
[2]: https://github.com/archlinux/aur
[3]: /aur/
[4]: /glossary/git/
A common meta thing that I need in python is to find the version of a package.
Most of the time I reach for package_name.__version__, but that does not
always work.
but not all projects have a __version__ # [1]
In searching the internet for an answer nearly every one of them pointed me to
__version__. This works for most projects, but is simply a convention, its
not required. Not all projects implement a __version__, but most do. I’ve
never seen it lie to me, but there is nothing stopping someone from shipping
mismatched versions.
If you maintain a project ship a __version__ # [2]
I appreciate it
While its not required its super handy and easy for anyone to remember off the
top of their head. It makes it easy to start debugging differences between
what you have vs what you see somewhere else. You can do this by dropping a
__version__ variable inside your __init__.py file.
## __init__.py
__version__ = 1.0.0
SO # [3]
stack overflow saves the day
Special thanks to this
Stack Overflow post [4]
for answering this question for me.
So what do you do… # [5]
importlib
Your next option is to reach into the package metadata of the package that you
are interested in, and this ha...
Check out gum [1] by charmbracelet [2]. It’s a well-crafted project with great potential.
A tool for glamorous shell scripts 🎀
References:
[1]: https://github.com/charmbracelet/gum
[2]: https://github.com/charmbracelet
Check out crossposter [1] by Mr-Destructive [2]. It’s a well-crafted project with great potential.
Crosspost your articles to dev.to, codenewbie.org, medium.com and hashnode.com with a single shellscript / python package
References:
[1]: https://github.com/Mr-Destructive/crossposter
[2]: https://github.com/Mr-Destructive
The one reason I switched to arch
The community, that’s it, end of post, roll the credits.
I’m a tinkerer # [1]
I am a tinkerer, I am not going to run a stock desktop manager, mostly becuase
that’s just not how my brain works. I need to tweak everything to fit my
needs. Grantid I have not spent much time in many full fledged linux desktop
environments. They are far more customizable than windows ever will be, I
absolutely love that about them. Inevitibly I end up in a situation where I
hit a wall, it just won’t do what I want it to do, or my lack of understanding
what came wtih it holds me back.
minimal # [2]
I love minimal installs. I love just building up my system from the bottom up
with things that I like, I understand, and that I can script.
I’m a noob # [3]
I spend a lot of my time in the terminal. I’d like to think I know how to use
a linux command line for software development really well, but there are a lot
of things that I still dont know all that well, mostly because I don’t need to.
The AUR # [5]...
Just starred moonlight-qt [1] by moonlight-stream [2]. It’s an exciting project with a lot to offer.
GameStream client for PCs (Windows, Mac, Linux, and Steam Link)
References:
[1]: https://github.com/moonlight-stream/moonlight-qt
[2]: https://github.com/moonlight-stream
I recently discovered moonlight-docs [1] by moonlight-stream [2], and it’s truly impressive.
Moonlight Documentation
References:
[1]: https://github.com/moonlight-stream/moonlight-docs
[2]: https://github.com/moonlight-stream
I like deresmos’s [1] project xrandr-manager [2].
Manage dual display on Linux
References:
[1]: https://github.com/deresmos
[2]: https://github.com/deresmos/xrandr-manager
If you’re into interesting projects, don’t miss out on xpadneo [1], created by atar-axis [2].
Advanced Linux Driver for Xbox One Wireless Controller (shipped with Xbox One S)
References:
[1]: https://github.com/atar-axis/xpadneo
[2]: https://github.com/atar-axis
I came across Launcher-Curseforge [1] from ShayBox [2], and it’s packed with great features and ideas.
Integrates the CF Modpack install button to any MMC based launcher
References:
[1]: https://github.com/ShayBox/Launcher-Curseforge
[2]: https://github.com/ShayBox
[1]
Recently I added two new bash/zsh aliases to make my git [2] experience just a tad
better.
trackme # [3]
Most of our work repos were recently migrated to new remote urls, we scriped
out the update to all of the repos, but I was left with a tracking error for
all of my open branches. To easily resolve this I just made an alias so that I
can just run trackme anytime I see this error.
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details
git pull <remote> <branch>
If you wish to set tracking information for this branch you can do so with:
git branch --set-upstream develop origin/<branch>
getting the branch # [4]
The following command will always return the currently checked out branch name.
git symbolic-ref --short HEAD
Injecting this into the suggested git command as a subshell gives us this
alias that when ran with trackme will automatically fix tracking for my
branch.
alias trackme='git branch --set-upstream-to=origin/$(git symbolic-ref --short HEAD)'
rebasemain # [5]
I sometimes get a bit lazy at checking main for changes before submitting any
prs, so again I made a quick shell...