Blog

https://youtu.be/79Y-kqAiMpw Choose tree is a powerful tmux utility that provides a graphical interface to preview all sessions, windows, and panes, move between them kill them, move them and much more. The default keybinding bind-key -T prefix s choose-tree -s my preferred keybinding to open sessions and windows collapsed and Zoomed in. bind-key j choose-tree -swZ From the man page. choose-tree [-GNrswZ] [-F format] [-f filter] [-K key-format] [-O sort-order] [-t target-pane] [template] Put a pane into tree mode, where a session, window or pane may be chosen interactively from a tree. Each session, window or pane is shown on one line. A shortcut key is shown on the left in brackets allowing for immediate choice, or the tree may be navigated and an item chosen or otherwise manipulated using the keys below. -s starts with sessions collapsed and -w with windows collapsed. -Z zooms the pane. The follow‐ ing keys may be used in tree mode: Key Function Enter Choose selected i...

https://youtu.be/BMkpbfhbkKM The prefix key is an essential part of tmux, by default all of tmux’s key-bindings sit behind a prefix. This prefix is very similar to vim’s leader key. It is common for folks to change the default C-b (control b) to C-a or if they are a vim user something to match their vim leader key. set -g prefix C-Space bind Space send-prefix A few of the essential default key-bindings. % vertical split " horizontal split d detach up select up one pane down select down one pane right select right one pane left select left one pane t clock o swap panes c create window n next window p previous window A more complete list of key-bindings can be found in this gist https://gist.github.com/mzmonsour/8791835. How I navigate tmux in 2021 [1] for more information on how I navigate tmux, check out this full post References: [1]: /tmux-nav-2021/

https://youtu.be/kzgyiHap1nQ splitting panes is a core feature of tmux. It allows us to split the terminal vertically or horizontally into new panes. bind -n M-s split-window -c '#{pane_current_path}' bind -n M-v split-window -h -c '#{pane_current_path}' bind -n M-X kill-pane 🗒️ note that ‘#{pane_current_path}‘will keep the split in the same directory as it’s parent, without this it will default to your home directory. How I navigate tmux in 2021 [1] for more information on how I navigate tmux, check out this full post References: [1]: /tmux-nav-2021/

https://youtu.be/RB87EEnnMnU An ultimate productivity key-binding in tmux is one to switch to the last session. I use this to quickly get between sessions really quick. Often I am working and need to lookup a quick note, or copy something into my notes, then get back to where I was quickly. bind -n M-b switch-client -l I think of this hub and spoke model, and use last-session to quickly drive it. [1] How I navigate tmux in 2021 [2] for more information on how I navigate tmux, check out this full post References: [1]: https://images.waylonwalker.com/tmux-nav-hub-spoke.png [2]: /tmux-nav-2021/

https://youtu.be/2ZqFDsJywt8 Tmux popups are actually floating windows that you can drag around the screen. They always open in the middle (by default) when you open them, no matter where you leave them. Here are a couple of keybindings I use to open up popup windows. bind C-g display-popup -E "ta ~/git" bind -n M-g display-popup -E "tmux new-session -A -s scratch" How I navigate tmux in 2021 [1] for more information on how I navigate tmux, check out this full post References: [1]: /tmux-nav-2021/

https://youtu.be/2I8fB28zfB4 Tmux-popups are a great feature that is relatively new to tmux, many repos such as the standard ubuntu repos do not have it. Popups came in 3.2a, if your package manager does not have it, you can follow the tmux’s install instructions [1] to build from source. # open a popup scratchpad bind -n M-g display-popup -E "tmux new-session -A -s scratch" How I navigate tmux in 2021 [2] for more information on how I navigate tmux, check out this full post I use popups quite a bit in my workflow to ssh into another machine for a short period, or make a new project with a template. References: [1]: https://github.com/tmux/tmux#installation [2]: /tmux-nav-2021/

Kedro versioned datasets can be mixed with incremental and partitioned datasets to do some timeseries analysis on how our dataset changes over time. Kedro is a very extensible and composible framework, that allows us to build solutions from the individual components that it provides. This article is a great example of how you can combine these components in unique ways to achieve some powerful results with very little work. What is Kedro [1] 👆 Unsure what kedro is? Check out this post. How does our dataset change over time?? # [2] This was a question presented to me at work. We had some plots being produces as the output of our pipeline and the user wanted the ability to compare results over time. Luckily this was asked early in the project so we were able to proactively setup versioning on the right datasets. To enable this all we needed to do now was to add versioned: true and we will be able to compare results over time. Yes kedro makes it that easy to setup. set up a proje...

I recently started streaming on twitch.tv/waylonwalker [1] and it’s been a blast so far. - python - kedro - Data Science - Data Engineering - webdev - digital gardening Kedro Spaceflights # [2] It all started with kedro/issues/606 [3], Yetu called out for users of kedro to record themselves doing a walk through of their tutorials. I wanted to do this, but was really stuck at the fact that recording or editing somewhat polished vide is quite time consuming for me. [4] Inspiration # [5] My introduction to twitch came from twitch.tv/theprimeagen [6]. I watched him on YouTube, and then decided to drop into a stream. It was so fun to watch him live that I started following others in the science and tech category. - twitch.tv/teej_dv [7] Brilliant neovim core dev, I learn a bunch about nvim every time I watch. - twitch.tv/cmgriffing [8] Super Chill and engaging chat. - twitch.tv/cassidoo [9] Fantastic discussion/chat. - twitch.tv/anthonywritescode [10] Building the python ...

This was my seconf time ever streaming on twitch.tv/waylonwalker [1], and I completely botched my mic 2x. https://youtu.be/_7MwgKu-844 Links # [2] - Spaceflights Tutorial [3] - my spaceflights repo [4] Notes to get started # [5] pipx run kedro new cd project python -m venv .venv source .venv/bin/activate pip install kedro kedro install References: [1]: https://twitch.tv/waylonwalker [2]: #links [3]: https://kedro.readthedocs.io/en/stable/03_tutorial/01_spaceflights_tutorial.html [4]: https://github.com/WaylonWalker/kedro-spaceflights [5]: #notes-to-get-started

https://youtu.be/I4VenHqIEng Doing some Digital Gardening on stream - Ahrefs Errors - ahrefs large images - Automatic gif to mp4 gif to mp4 # [1] After this stream all gifs on my site are converted to mp4/webm if they exist.  [2] References: [1]: #gif-to-mp4 [2]: https://images.waylonwalker.com/tmux-navigation-2021.gif

This was my first time ever streaming on twitch.tv/waylonwalker [1]. I am excited to get going. I have been streaming early in the morning while I am still waking up, so still a bit groggy as I go. https://youtu.be/Y07UBr9Ccjs Kedro Spaceflights # [2] It all started with kedro/issues/606 [3], Yetu called out for users of kedro to record themselves doing a walk through of their tutorials. I wanted to do this, but was really stuck at the fact that recording or editing somewhat polished vide is quite time consuming for me. [4] Notes # [5] pipx run kedro new cd project python -m venv .venv source .venv/bin/activate pip install kedro kedro install References: [1]: https://twitch.tv/waylonwalker [2]: #kedro-spaceflights [3]: https://github.com/kedro-org/kedro/issues/606 [4]: https://images.waylonwalker.com/kedro-issue-606.png [5]: #notes

With the latest releases of kedro 0.17.x, it is now possible to run kedro pipelines from within scripts. While I would not start a project with this technique, it will be a good tool to keep in my back pocket when I want to sprinkle in a bit of kedro goodness in existing projects. New to Kedro # [1] What is Kedro [2] If your just learning about kedro check out this post walking through it No More Rabbit Hole of Errors # [3] as of 0.17.2 I’ve tried to do this in kedro 0.16.x, and it turned into a rabbit hole of errors. First kedro needed a conf directory, if you tried to fake one in it would then ask for logging setup. These errors just kept coming to the point it wasnt worth doing and I might as well use a proper template for real projects and stick to simple function calls for things that are not a kedro project. Kedro in a script # [4] To get kedro running, you will need a pipeline, catalog, and runner at a minimum. Those who have used kedro before the pipeline will look v...

Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while trying to implement a cli extension with clean output. Silence a Python log # [1] First, how does one silence a python log? Python loggers can be retrieved by the logging module’s getLogger function. Then their log level can be changed. Much of kedro’s chattiness comes from INFO level logs. I don’t want to hear about anything for my current use case unless it’s essential, i.e., a failure. In this case, I set the log levels to ERROR as most errors should stop execution anyways. python logging levels # [2] Level Numeric value CRITICAL 50 ERROR 40 WARNING 30 INFO 20 DEBUG 10 NOTSET 0 Get or Create a logger # [3] Getting a python logger is straightforward if we know the name of the logger. The following block will grab the logger object for the logger currently registered under the name passed in. logger = logging.getLog...

change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed) Running multiple processes using the same diskcache object can cause issues with locks. As I was trying to setup a rich Live display for markata I ran into issues where each part could not nun simultaneusly. As I had followed the instructions from discache it was not directly aparant to me, so I had to make a simple example to experiment and play with at a small scale. Minimum reproducible error # [1] Minimum reporducible error is one of my superpowers in development. I do this very often to sus out what is really happening. My day to day work is processing data with python, I keep a number of very small data sets handy to break and fix. This helps separate complexities of the project and the problem. Let’s break it # [2] Markata has a lot going on. It’s a plugins all the way down static site generator built in python. Trying to find the root cause through the layers ...

change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed) In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and generally keep work separated into logical groups. Update # [1] Since making this post, I have made ~20 other posts in short form that all have a YouTube video to go along with them you can find them all on my tmux-playlist [2]. Chris Toomey’s [3] Tmux Course # [4] I took Chris’s tmux course [5] in December 2020 and it was fantastic. Even as a seasoned tmux user, I learned quite a bit. Before the course, I was proficient in navigating within each of my tmux sessions but rarely started more than one session. A few months later, I have adopted a lot of what I learned from Chris and made it my own. I am now keeping projects to their own session and can move between them fluidly with just a few keystrokes. For high-traffic projects, I have them bound to a si...

Trim branches no longer on origin # [1] git remote prune origin --dry-run git remote prune origin Find branches already merged # [2] git checkout main # list remote branches that have already been merged into main git branch -r --merged # list local branches that have already been merged into main git branch --merged References: [1]: #trim-branches-no-longer-on-origin [2]: #find-branches-already-merged

change_speed = (speed) => [...document.querySelectorAll('video')].map(v => v.playbackRate=v.playbackRate+speed) When a python module is called it is assigned the __name__ of __main__ otherwise if it’s imported it will be assigned the __name__ of the module. Concrete example # [1] Let’s create a module to play with __name__ a bit. We will call this module nodes.py. It is a module that we may want to run by it’self or import and use in other modules. #!python # nodes.py if __name__ == "nodes": import sys import __main__ print(f"you have imported me {__name__} from {sys.modules['__main__'].__file__}") if __name__ == "__main__": print("you are running me as main") I have set this module up to execute one of two if statements based on whether the module it’self is being ran or if the module is being imported. Note it is not common to have a if __name__ == "nodes": block, this is just for demnonstration purposes. running python nodes.py # [2] Running a python script with the...

Creating a directory that is a union of several directories can be achieved with a few symlinks at the command line. Creating a Virtual File Gallery # [1] Here is how I am creating a virtual directory of all my projects that is a combination of both work and not-work projects. I am creating symlinks for every directory under ~/work and ~/git. rm -rf ~/projects mkdir ~/projects ln -sf ~/work/* ~/projects ln -sf ~/git/* ~/projects ⚠ Notice that first I am recreating the directory each time. This will ensure that any project that is deleted from their actual directory is removed from the virtual gallery. Updating the gallery # [2] Since links are always kept up to date without any extra work, all the data is still in the same place it started. But as new directories are added to any project directory they will not be automatically added to the virtual gallery. - cron - bashrc/zshrc If you’re concerned about system resources, you can add it to a cron job to run at a regular sch...

I really like using conda (miniconda) as my python virtual environment [1] manager of choice. It’s simple and it includes its own python interpreter using the version that I specify at creation. Mamba # [2] from their readme [3] --- Mamba is a reimplementation of the conda package manager in C++. - parallel downloading of repository data and package files using multi-threading - libsolv for much faster dependency solving, a state of the art library used in the RPM package manager of Red Hat, Fedora and OpenSUSE - core parts of mamba are implemented in C++ for maximum efficiency At the same time, mamba utilize the same command line parser, package installation and deinstallation code and transaction verification routines as conda to stay as compatible as possible. --- Installing Micromamba # [4] Similar to miniconda micromamba can be installed with a few lines of bash wget -qO- https://micromamba.snakepit.net/api/micromamba/linux-64/latest | tar -xvj bin/micromamba ./bin/...

I’ve long used neovim from within windows wsl, and for far too long, I went without a proper way to get text out of it and into windows. wsl has access to cmd applications # [1] wsl can access clip.exe. You can do some cool things with it, such as cat a file into the clipboard, sending output from a command to the clipboard, or set an autocmd group in vim to send yank to the windows clipboard. using clip.exe # [2] Let’s say you want to send a teammate the tail of a log file over chat. You can tail the file into clip.exe. tail -n 1 info.log | clip.exe pipe streams of text into clip.exe make it a bit more natural # [3] I recently made mine feel a bit more natural by aliasing it to clip. alias clip=clip.exe pop this in your ~/.bashrc or ~/.zshrc yanking to windows clipboard from vim # [4] I use neovim as my daily text editor and its a pain to share code with a teammate over chat, stack overflow, into a gist, or whatever you need. The following snippet has been quite useful ...