Published

Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while trying to implement a cli extension with clean output.

First, how does one silence a python log? Python loggers can be retrieved by the logging module’s getLogger function. Then their log level can be changed. Much of kedro’s chattiness comes from INFO level logs. I don’t want to hear about anything for my current use case unless it’s essential, i.e., a failure. In this case, I set the log levels to ERROR as most errors should stop execution anyways.

Getting a python logger is straightforward if we know the name of the logger. The following block will grab the logger object for the logger currently registered under the name passed in.

...

Running multiple processes using the same diskcache object can cause issues with locks. As I was trying to setup a rich Live display for markata I ran into issues where each part could not nun simultaneusly. As I had followed the instructions from discache it was not directly aparant to me, so I had to make a simple example to experiment and play with at a small scale.

Minimum reporducible error is one of my superpowers in development. I do this very often to sus out what is really happening. My day to day work is processing data with python, I keep a number of very small data sets handy to break and fix. This helps separate complexities of the project and the problem.

Markata has a lot going on. It’s a plugins all the way down static site generator built in python. Trying to find the root cause through the layers of plugin and cli modules can be a pain, but in this case building a very simple minimum reporducible error was much easier.

...

In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and generally keep work separated into logical groups.

Since making this post, I have made ~20 other posts in short form that all have a YouTube video to go along with them you can find them all on my tmux-playlist.

I took Chris’s tmux course in December 2020 and it was fantastic. Even as a seasoned tmux user, I learned quite a bit. Before the course, I was proficient in...

...

When a python module is called it is assigned the __name__ of __main__ otherwise if it’s imported it will be assigned the __name__ of the module.

Let’s create a module to play with __name__ a bit. We will call this module nodes.py. It is a module that we may want to run by it’self or import and use in other modules.

#!python # nodes.py if __name__ == "nodes": import sys import __main__ print(f"you have imported me {__name__} from {sys.modules['__main__'].__file__}") if __name__ == "__main__": print("you are running me as main")

I have set this module up to execute one of two if statements based on whether the module it’self is being ran or if the module is being imported.

...

Creating a directory that is a union of several directories can be achieved with a few symlinks at the command line.

Here is how I am creating a virtual directory of all my projects that is a combination of both work and not-work projects. I am creating symlinks for every directory under ~/work and ~/git.

rm -rf ~/projects mkdir ~/projects ln -sf ~/work/* ~/projects ln -sf ~/git/* ~/projects

⚠ Notice that first I am recreating the directory each time. This will ensure that any project that is deleted from their actual directory is removed from the virtual gallery.

...