Check out studioswong [1] and their project kedro-viz-glitch [2].
No description available.
References:
[1]: https://github.com/studioswong
[2]: https://github.com/studioswong/kedro-viz-glitch
Archive
All published posts
2507 posts
latest post 2026-05-29
Publishing rhythm
I’m impressed by dolt [1] from dolthub [2].
Dolt – Git [3] for Data
References:
[1]: https://github.com/dolthub/dolt
[2]: https://github.com/dolthub
[3]: /glossary/git/
I’m really excited about pyupgrade [1], an amazing project by asottile [2]. It’s worth exploring!
A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of the language.
References:
[1]: https://github.com/asottile/pyupgrade
[2]: https://github.com/asottile
I Started Streaming on Twitch
I recently started streaming on
twitch.tv/waylonwalker [1] and it’s been a
blast so far.
- python
- kedro
- Data Science
- Data Engineering
- webdev
- digital gardening
Kedro Spaceflights # [2]
It all started with
kedro/issues/606 [3], Yetu
called out for users of kedro to record themselves doing a walk through of
their tutorials. I wanted to do this, but was really stuck at the fact that
recording or editing somewhat polished vide is quite time consuming for me.
[4]
Inspiration # [5]
My introduction to twitch came from
twitch.tv/theprimeagen [6]. I watched him on
YouTube, and then decided to drop into a stream. It was so fun to watch him
live that I started following others in the science and tech category.
- twitch.tv/teej_dv [7] Brilliant neovim core dev, I learn a bunch about nvim every time I watch.
- twitch.tv/cmgriffing [8] Super Chill and engaging chat.
- twitch.tv/cassidoo [9] Fantastic discussion/chat.
- twitch.tv/anthonywritescode [10] Building the python ...
I recently discovered LuaSnip [1] by L3MON4D3 [2], and it’s truly impressive.
Snippet Engine for Neovim written in Lua.
References:
[1]: https://github.com/L3MON4D3/LuaSnip
[2]: https://github.com/L3MON4D3
The work on NearBeach [1] by nearbeach [2].
NearBeach is an open sourced project management tool, helping you keep track of your project. You can track requirements, projects and tasks
References:
[1]: https://github.com/nearbeach/NearBeach
[2]: https://github.com/nearbeach
srcards [1] by brickfrog [2] is a game-changer in its space. Excited to see how it evolves.
No description available.
References:
[1]: https://github.com/brickfrog/srcards
[2]: https://github.com/brickfrog
Check out vim-twitch-line-sign [1] by noopkat [2]. It’s a well-crafted project with great potential.
some helpers to annotate vim signs with Twitch related metadata. Inspired by: clarkio/vscode-twitch-highlighter
References:
[1]: https://github.com/noopkat/vim-twitch-line-sign
[2]: https://github.com/noopkat
Just starred flake8 [1] by PyCQA [2]. It’s an exciting project with a lot to offer.
flake8 is a python tool that glues together pycodestyle, pyflakes, mccabe, and third-party plugins to check the style and quality of some python code.
References:
[1]: https://github.com/PyCQA/flake8
[2]: https://github.com/PyCQA
Kedro Spaceflights - part 2 | Stream replay June 7, 2021
This was my seconf time ever streaming on
twitch.tv/waylonwalker [1], and I completely botched my mic 2x.
https://youtu.be/_7MwgKu-844
Links # [2]
- Spaceflights Tutorial [3]
- my spaceflights repo [4]
Notes to get started # [5]
pipx run kedro new
cd project
python -m venv .venv
source .venv/bin/activate
pip install kedro
kedro install
References:
[1]: https://twitch.tv/waylonwalker
[2]: #links
[3]: https://kedro.readthedocs.io/en/stable/03_tutorial/01_spaceflights_tutorial.html
[4]: https://github.com/WaylonWalker/kedro-spaceflights
[5]: #notes-to-get-started
🌱 Digital Gardening | gif to Mp4 | Stream replay June 4, 2021
https://youtu.be/I4VenHqIEng
Doing some Digital Gardening on stream
- Ahrefs Errors
- ahrefs large images
- Automatic gif to mp4
gif to mp4 # [1]
After this stream all gifs on my site are converted to mp4/webm if they exist.
[2]
References:
[1]: #gif-to-mp4
[2]: https://images.waylonwalker.com/tmux-navigation-2021.gif
Check out notifiers [1] by liiight [2]. It’s a well-crafted project with great potential.
The easy way to send notifications
References:
[1]: https://github.com/liiight/notifiers
[2]: https://github.com/liiight
Check out neovim-plugin [1] by norcalli [2]. It’s a well-crafted project with great potential.
No description available.
References:
[1]: https://github.com/norcalli/neovim-plugin
[2]: https://github.com/norcalli
I’m impressed by python-lsp-server [1] from python-lsp [2].
Fork of the python-language-server project, maintained by the Spyder IDE team and the community
References:
[1]: https://github.com/python-lsp/python-lsp-server
[2]: https://github.com/python-lsp
github-coauthors.nvim [1] by cwebster2 [2] is a game-changer in its space. Excited to see how it evolves.
A neovim extension for populating coauthors when comitting
References:
[1]: https://github.com/cwebster2/github-coauthors.nvim
[2]: https://github.com/cwebster2
Kedro Spaceflights - part 1 | Stream replay June 4, 2021
This was my first time ever streaming on
twitch.tv/waylonwalker [1]. I am excited to get going.
I have been streaming early in the morning while I am still waking up, so still
a bit groggy as I go.
https://youtu.be/Y07UBr9Ccjs
Kedro Spaceflights # [2]
It all started with
kedro/issues/606 [3], Yetu
called out for users of kedro to record themselves doing a walk through of
their tutorials. I wanted to do this, but was really stuck at the fact that
recording or editing somewhat polished vide is quite time consuming for me.
[4]
Notes # [5]
pipx run kedro new
cd project
python -m venv .venv
source .venv/bin/activate
pip install kedro
kedro install
References:
[1]: https://twitch.tv/waylonwalker
[2]: #kedro-spaceflights
[3]: https://github.com/kedro-org/kedro/issues/606
[4]: https://images.waylonwalker.com/kedro-issue-606.png
[5]: #notes
The work on kedro-mlflow [1] by Galileo-Galilei [2].
A kedro-plugin for integration of mlflow capabilities inside kedro projects (especially machine learning model versioning and packaging)
References:
[1]: https://github.com/Galileo-Galilei/kedro-mlflow
[2]: https://github.com/Galileo-Galilei
If you’re into interesting projects, don’t miss out on textual [1], created by Textualize [2].
The lean application framework for Python. Build sophisticated user interfaces with a simple Python API. Run your apps in the terminal and a web browser.
References:
[1]: https://github.com/Textualize/textual
[2]: https://github.com/Textualize
The work on lsp-colors.nvim [1] by folke [2].
🌈 Plugin that creates missing LSP diagnostics highlight groups for color schemes that don’t yet support the Neovim 0.5 builtin LSP client.
References:
[1]: https://github.com/folke/lsp-colors.nvim
[2]: https://github.com/folke
Using Kedro In Scripts
With the latest releases of kedro 0.17.x, it is now possible to run kedro
pipelines from within scripts. While I would not start a project with this
technique, it will be a good tool to keep in my back pocket when I want to
sprinkle in a bit of kedro goodness in existing projects.
New to Kedro # [1]
What is Kedro [2]
If your just learning about kedro check out this post walking through it
No More Rabbit Hole of Errors # [3]
as of 0.17.2
I’ve tried to do this in kedro 0.16.x, and it turned into a rabbit hole of
errors. First kedro needed a conf directory, if you tried to fake one in it
would then ask for logging setup. These errors just kept coming to the point
it wasnt worth doing and I might as well use a proper template for real
projects and stick to simple function calls for things that are not a kedro
project.
Kedro in a script # [4]
To get kedro running, you will need a pipeline, catalog, and
runner at a minimum. Those who have used kedro before the pipeline will
look v...
I’m really excited about netlify_deploy [1], an amazing project by lannonbr [2]. It’s worth exploring!
Mini Rust CLI to deploy sites to Netlify using their API
References:
[1]: https://github.com/lannonbr/netlify_deploy
[2]: https://github.com/lannonbr
I recently discovered slidev [1] by slidevjs [2], and it’s truly impressive.
Presentation Slides for Developers
References:
[1]: https://github.com/slidevjs/slidev
[2]: https://github.com/slidevjs
I’m really excited about bio [1], an amazing project by orta [2]. It’s worth exploring!
No description available.
References:
[1]: https://github.com/orta/bio
[2]: https://github.com/orta
Check out dogehouse [1] by benawad [2]. It’s a well-crafted project with great potential.
Taking voice conversations to the moon 🚀
References:
[1]: https://github.com/benawad/dogehouse
[2]: https://github.com/benawad
Silence Kedro Logs
Kedro can have a chatty logger. While this is super nice in production
so see everything that happened during a pipeline run. This can be troublesome
while trying to implement a cli extension with clean output.
Silence a Python log # [1]
First, how does one silence a python log? Python loggers can be retrieved by
the logging module’s getLogger function. Then their log level can be
changed. Much of kedro’s chattiness comes from INFO level logs. I don’t want
to hear about anything for my current use case unless it’s essential, i.e., a
failure. In this case, I set the log levels to ERROR as most errors should
stop execution anyways.
python logging levels # [2]
Level
Numeric value
CRITICAL
50
ERROR
40
WARNING
30
INFO
20
DEBUG
10
NOTSET
0
Get or Create a logger # [3]
Getting a python logger is straightforward if we know the name of the logger.
The following block will grab the logger object for the logger currently
registered under the name passed in.
logger = logging.getLog...
Looking for inspiration? python-diskcache [1] by grantjenks [2].
Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
References:
[1]: https://github.com/grantjenks/python-diskcache
[2]: https://github.com/grantjenks
The work on lookatme [1] by d0c-s4vage [2].
An interactive, terminal-based markdown presenter
References:
[1]: https://github.com/d0c-s4vage/lookatme
[2]: https://github.com/d0c-s4vage
Python Diskcahe is locked
change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed)
Running multiple processes using the same diskcache object can cause issues
with locks. As I was trying to setup a rich Live display for markata I ran
into issues where each part could not nun simultaneusly. As I had followed the
instructions from discache it was not directly aparant to me, so I had to make
a simple example to experiment and play with at a small scale.
Minimum reproducible error # [1]
Minimum reporducible error is one of my superpowers in development. I do this
very often to sus out what is really happening. My day to day work is
processing data with python, I keep a number of very small data sets handy to
break and fix. This helps separate complexities of the project and the problem.
Let’s break it # [2]
Markata has a lot going on. It’s a plugins all the way down static site
generator built in python. Trying to find the root cause through the layers ...
The work on ward [1] by darrenburns [2].
Ward is a modern test framework for Python with a focus on productivity and readability.
References:
[1]: https://github.com/darrenburns/ward
[2]: https://github.com/darrenburns
How I navigate tmux in 2021
change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed)
In 2021 I changed the way I navigate between tmux sessions big time. Now I can
create, kill, switch with ease, and generally keep work separated into logical
groups.
Update # [1]
Since making this post, I have made ~20 other posts in short form that all have
a YouTube video to go along with them you can find them all on my
tmux-playlist [2].
Chris Toomey’s [3] Tmux Course # [4]
I took Chris’s tmux course [5] in December
2020 and it was fantastic. Even as a seasoned tmux user, I learned quite a bit.
Before the course, I was proficient in navigating within each of my tmux
sessions but rarely started more than one session. A few months later, I have
adopted a lot of what I learned from Chris and made it my own.
I am now keeping projects to their own session and can move between them
fluidly with just a few keystrokes. For high-traffic projects, I have them
bound to a si...
I came across compactyl [1] from dereknheiley [2], and it’s packed with great features and ideas.
No description available.
References:
[1]: https://github.com/dereknheiley/compactyl
[2]: https://github.com/dereknheiley
I’m really excited about thinkeys [1], an amazing project by moduloindustries [2]. It’s worth exploring!
Split ortholinear custom replacement keyboard with TrackPoint for ThinkPad laptops.
References:
[1]: https://github.com/moduloindustries/thinkeys
[2]: https://github.com/moduloindustries
Trim unused git branches
Trim branches no longer on origin # [1]
git remote prune origin --dry-run
git remote prune origin
Find branches already merged # [2]
git checkout main
# list remote branches that have already been merged into main
git branch -r --merged
# list local branches that have already been merged into main
git branch --merged
References:
[1]: #trim-branches-no-longer-on-origin
[2]: #find-branches-already-merged
What is if __name__ == "__main___", and how do I use it.
change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed)
When a python module is called it is assigned the __name__ of __main__
otherwise if it’s imported it will be assigned the __name__ of the module.
Concrete example # [1]
Let’s create a module to play with __name__ a bit. We will call this module
nodes.py. It is a module that we may want to run by it’self or import and use
in other modules.
#!python
# nodes.py
if __name__ == "nodes":
import sys
import __main__
print(f"you have imported me {__name__} from {sys.modules['__main__'].__file__}")
if __name__ == "__main__":
print("you are running me as main")
I have set this module up to execute one of two if statements based on whether
the module it’self is being ran or if the module is being imported.
Note it is not common to have a if __name__ == "nodes": block, this is just
for demnonstration purposes.
running python nodes.py # [2]
Running a python script with the...
Create a Virtual File Gallery with Symlinks
Creating a directory that is a union of several directories can be achieved
with a few symlinks at the command line.
Creating a Virtual File Gallery # [1]
Here is how I am creating a virtual directory of all my projects that is a
combination of both work and not-work projects. I am creating symlinks for
every directory under ~/work and ~/git.
rm -rf ~/projects
mkdir ~/projects
ln -sf ~/work/* ~/projects
ln -sf ~/git/* ~/projects
⚠ Notice that first I am recreating the directory each time. This will ensure
that any project that is deleted from their actual directory is removed from
the virtual gallery.
Updating the gallery # [2]
Since links are always kept up to date without any extra work, all the data is
still in the same place it started. But as new directories are added to any
project directory they will not be automatically added to the virtual gallery.
- cron
- bashrc/zshrc
If you’re concerned about system resources, you can add it to a cron job to run
at a regular sch...
I recently discovered delta [1] by dandavison [2], and it’s truly impressive.
A syntax-highlighting pager for git [3], diff, grep, and blame output
References:
[1]: https://github.com/dandavison/delta
[2]: https://github.com/dandavison
[3]: /glossary/git/
Just starred monkeytype [1] by monkeytypegame [2]. It’s an exciting project with a lot to offer.
The most customizable typing website with a minimalistic design and a ton of features. Test yourself in various modes, track your progress and improve your speed.
References:
[1]: https://github.com/monkeytypegame/monkeytype
[2]: https://github.com/monkeytypegame
Check out ntangle.nvim [1] by jbyuki [2]. It’s a well-crafted project with great potential.
literate programming for neovim
References:
[1]: https://github.com/jbyuki/ntangle.nvim
[2]: https://github.com/jbyuki
If you’re into interesting projects, don’t miss out on termopen.vim [1], created by fabi1cazenave [2].
Easy integration of TUI apps in Neovim: Ranger, LF, Tig… Tetris…
References:
[1]: https://github.com/fabi1cazenave/termopen.vim
[2]: https://github.com/fabi1cazenave
karimknaebel [1] has done a fantastic job with nvim-lspinstall [2]. Highly recommend taking a look.
Provides the missing :LspInstall for nvim-lspconfig
References:
[1]: https://github.com/karimknaebel
[2]: https://github.com/karimknaebel/nvim-lspinstall
I’m impressed by nvim-lspinstall [1] from kabouzeid [2].
Provides the missing :LspInstall for nvim-lspconfig
References:
[1]: https://github.com/kabouzeid/nvim-lspinstall
[2]: https://github.com/kabouzeid
Looking for inspiration? dotfiles [1] by Conni2461 [2].
Current dotfiles and scripts
References:
[1]: https://github.com/Conni2461/dotfiles
[2]: https://github.com/Conni2461
How to Install micromamba on linux (from the comamnd line only)
I really like using conda (miniconda) as my python virtual environment [1]
manager of choice. It’s simple and it includes its own python interpreter
using the version that I specify at creation.
Mamba # [2]
from their readme [3]
---
Mamba is a reimplementation of the conda package manager in C++.
- parallel downloading of repository data and package files using multi-threading
- libsolv for much faster dependency solving, a state of the art library used in the RPM package manager of Red Hat, Fedora and OpenSUSE
- core parts of mamba are implemented in C++ for maximum efficiency
At the same time, mamba utilize the same command line parser, package
installation and deinstallation code and transaction verification routines as
conda to stay as compatible as possible.
---
Installing Micromamba # [4]
Similar to miniconda micromamba can be installed with a few lines of bash
wget -qO- https://micromamba.snakepit.net/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
./bin/...
Check out nvim-treesitter [1] and their project nvim-treesitter-textobjects [2].
No description available.
References:
[1]: https://github.com/nvim-treesitter
[2]: https://github.com/nvim-treesitter/nvim-treesitter-textobjects
telescope-project.nvim [1] by nvim-telescope [2] is a game-changer in its space. Excited to see how it evolves.
No description available.
References:
[1]: https://github.com/nvim-telescope/telescope-project.nvim
[2]: https://github.com/nvim-telescope
I came across lspsaga.nvim [1] from nvimdev [2], and it’s packed with great features and ideas.
improve neovim lsp experience
References:
[1]: https://github.com/nvimdev/lspsaga.nvim
[2]: https://github.com/nvimdev
rmagatti [1] has done a fantastic job with auto-session [2]. Highly recommend taking a look.
A small automated session manager for Neovim
References:
[1]: https://github.com/rmagatti
[2]: https://github.com/rmagatti/auto-session
Vim Wsl Clipboard
I’ve long used neovim from within windows wsl, and for far too long, I went
without a proper way to get text out of it and into windows.
wsl has access to cmd applications # [1]
wsl can access clip.exe. You can do some cool things with it, such as
cat a file into the clipboard, sending output from a command to the clipboard,
or set an autocmd group in vim to send yank to the windows clipboard.
using clip.exe # [2]
Let’s say you want to send a teammate the tail of a log file over chat. You can
tail the file into clip.exe.
tail -n 1 info.log | clip.exe
pipe streams of text into clip.exe
make it a bit more natural # [3]
I recently made mine feel a bit more natural by aliasing it to clip.
alias clip=clip.exe
pop this in your ~/.bashrc or ~/.zshrc
yanking to windows clipboard from vim # [4]
I use neovim as my daily text editor and its a pain to share code with a
teammate over chat, stack overflow, into a gist, or whatever you need. The
following snippet has been quite useful ...
I’m impressed by onebuddy [1] from Th3Whit3Wolf [2].
Light and dark atom one theme
References:
[1]: https://github.com/Th3Whit3Wolf/onebuddy
[2]: https://github.com/Th3Whit3Wolf
I’m impressed by awesome-streamerrc [1] from awesome-streamers [2].
Dotfiles for various streamers on Twitch.
References:
[1]: https://github.com/awesome-streamers/awesome-streamerrc
[2]: https://github.com/awesome-streamers