-
markata search
A side effect of [[ markata-didyoumean ]] is that we are able to implement some rudimentary search based on slug, title, tags, and description. ![screenshot-202
-
Markata DidYouMean
Coming in Markata 0.9.1 is far better documentation. i.e. Documentation that actually exists for everything. As part of poking around I realized that I often g
-
๐ญ Models - Pydantic
I came accross fromattributes today it allows creation of pydantic models from objects such as a sqlalchemy Base Mod
-
๐ญ Top Python libraries of 2024 | Tryolabs
Really good listicle of new modern top python libraries from 2024. Very well done article with images, links, and an a
-
๐ญ Choosing color palettes โ seaborn 0.13.2 documen...
Good overview of seaborn color palettes. They have all sorts of different types, some designed to purposfully give each
-
I fixed lists in my blog
A small improvement to my blog today. I was checking out my [[ now ]] post and noticed that lists have left padding but no discs or numbers. I popped open my `
-
nvim-manager
bump I recently built a cli application as a nearly-one-shot-app called [nvim-manager](https://github.com/waylonwalker/nvim-manager). It manages your nvim dotfi
-
one-shot-s3-cleanup
So I made a mistake in an app I am working on and ended up creating thumbnails of thumbnails, and thumbnails of thumbnails of thumbnails... etc. I was able to d
-
๐ญ cyclotruc/gitingest: Replace 'hub' with 'ingest'...
Gitingest has a python package on pypi that you can run with uvx, and it accepts the same arguments as the web version,
-
๐ญ kitze ๐ on X: "holy fk I thought package.json wa...
kitze hating on python packaging in new ways. Python packaging has a lot of quirks that can make it infuriating. Not
-
๐ญ 4 Tips for Building a Production-Ready FastAPI B...
Great list of 4 tips for running fastapi applications. Keep routes small Fat routers with all of the logic built
-
๐ญ Building Python tools with a one-shot prompt usi...
This is really interesting, the lazy uv scripts are really becoming quite appealing, especially for something like this
-
๐ญ PEP 723 โ Inline script metadata | peps.python.o...
PEP 723 is what is inspiring all of these lazy self installing python scripts, Authored by the author of hatch and pyapp
-
๐ญ Switching from virtualenvwrapper to direnv, Star...
I've kinda fallen out of using direnv now that a lot of my projects use hatch, I generally just hatch shell into them.
-
๐ญ Lazy self-installing Python scripts with uv
I really like Trey's steps to making an executable python script with uv his old process seems to be the same with a ne
-
thoughts 0.0.4
This is such an improvement to the backend of my website it warrants a blog post of celebration. For far too long I've been dealing with a tiny ass edit f
-
๐ญ Add a healthcheck to your FastAPI app | Pype.dev
Nice example of adding a healthcheck to fastapi, and integrating it with docker. Don't forget to include curl in the in
-
๐ญ watchfiles
Simple, modern and high performance file watching and code reload in python.
-
price an stl print on slant3d
I've been playing with 3d printing some items through the slant3d api. I've been pricing out different prints by running a slice request through thei
-
๐ญ Production-ready Python Docker Containers with u...
Hynek has a sick dockerfile setup for using uv in python projects.
-
๐ญ Proper handling of None in WHERE condition ยท Iss...
SQLModel models ship with an is, and isnot that you can use to compare to None without pesky linters complaining.
-
๐ญ Document how to provide a negative number as an ...
Today I learned that you cannot pass negative integers as values to typer. in this case contextsettings={"ignoreunkn
-
๐ญ Will McGugan on X: "New Version of #Textual rele...
This new demo of textual is wildly awesome, so many widgets and features being added into the main library. The themes
-
๐ญ Pin versions of dependencies ยท Issue #2200 ยท Koz...
weazyprint was throwing me some errors, turns out that it's currently not compatible with the latest pydyf package. my
-
k3s config after first install
-
๐ญ Installing Python - uv
uv now can install python for you. uv is inspired by rust's cargo, an all in one fast package and installation mana
-
๐ญ python is simultaneously at the top and the bott...
Hard to argue this take, happy to see that its at the top. With it being such an old language its amazing that it still
-
๐ญ FastHX
Very interesting approach to htmx and fast api. It uses separate decorators for returning template partials and json th
-
๐ญ FastHX
Very interesting approach to htmx and fast api. It uses separate decorators for returning template partials and json th
-
diskcache as debounce
I've been using fastapi more and more lately and one feature I just started using is background tasks [[ thoughts-333 ]]. !!! seealso basic diskcache e
-
๐ญ Background Tasks - FastAPI
fastapi comes with a concept of background tasks which are functions that can be ran in the background after a function
-
๐ญ markdown-it-pyrs ยท PyPI
markdown it py running in rust claims to be 20x faster. I'll definitely look into this if markdown it py is ever a bott
-
๐ญ DiskCache API Reference โ DiskCache 5.6.1 docume...
diskcache has a peekitem method that allows you to lookup the expiretime of a cached item without changing it. I recen
-
๐ญ PyAnnotate: Auto-generate PEP-484 annotations
pyannotate is a tool that uses runtime types to suggest type annotations to use. Test runners such as pytest can be use
-
๐ญ tusharsadhwani/yen: Create virtual environments ...
> Create virtual environments for any Python version, without needing Python installed.
-
๐ญ Episode #323 Best practices for Docker in produc...
Great listen for anyone interested in productionizing python code with docker. Itamar brings up some Don't trust bas
-
๐ญ xxHash - Extremely fast non-cryptographic hash a...
> xxHash is an extremely fast non-cryptographic hash algorithm, working at RAM speed limit. It is proposed in four flavo
-
๐ญ xxhash ยท PyPI
I hit an issue with markata where even though a bunch of articles were cached, the site build was still slow because I w
-
Latest Page in Markata
I just implemented a latest blog post link in Markata by asking for the first post slug from the blog feed. The implementation uses the jinja_md plugin to rend
-
sick wikilink hover
Today I set up some sick wikilink hover effects using tailwind see [[ a-case-for-tailwindcss ]]. When you hover over them they show an image preview of the link
-
markata
This post is a work in progress. Markata is the static site generator that I created to build my website [[ about-this-site ]]. I built it for me and I enjoy u
-
thinking about static sites in 2024
actions build k8s build fastapi sqlite
-
๐ญ Fields - Pydantic
exclude=True and repr=False is a good pydantic combination for secret attributes such as user passwords, or hashed p
-
๐ญ Hatch v1.10.0 - Hatch
Hatch be flyin. This new release of hatch includes support for the new package installer uv which is just mind blowin
-
๐ญ rich.panel โ Rich 13.6.0 documentation
This is my go to rich response container for clis written in python. It creates a nice box around the content on the sc
-
๐ญ Handling Errors - FastAPI
This page shows how to customize your fastapi errors. I found this very useful to setup common templates so that I can
-
control minio token access
To allow access only to the <bucket>, you can pass add the Resource field to the User Policy when you create a new token.
-
How to List Sqlite tables
You can inspect sqlite tables with the sqlite shell. note that you get into the shell with `sqlite3 database.db` I also learned that `.tables`, `.index` and `.s
-
copier trust
I recently had to update my copier-gallery command to trust my own templates because some of them have shell scripts that run afterwards. Be warned that this c
-
ipython f2
Today I accidentally ran f2 in ipython to discover that it opens your $EDITOR! I use this feature quite often in zsh, it is bound to `<c-e>` for me, and s
-
sqlmodel indexes
I've really been enjoying using sqlmodel for my projects that need a database. One thing that I definitely lacked on for too long was indexing my database.
-
python inline snapshot
`inline-snapshot` is a new tool that I am trying out for python testing. It takes snapshots of your outputs and places them inline with the test. Here is the m
-
๐ญ inline-snapshot
This is a cool snapshot testing tool that automatically creates, and updates test values for you. Starting with some te
-
Composing Typer clis
Typer makes it easy to compose your cli applications, like you might with a web router if you are more familiar with that. This allows you to build smaller app
-
One Day Build - Play Outside
Inspired by Adam Savage and his One Day builds on youtube. I often build things, and want to make them generally useful for others and over configure out of th
-
jinja macros
I am working on a page for [htmx-patterns](https://htmx-patterns.waylonwalker.com) and I ran into a situation with lots of duplication. Especially when i am us
-
jinja loop variable and htmx
jinja has a loop variable that is very handy to use with htmx. Whether you want to implement a click to load more or an infinite scroll this loop variable is v
-
fastapi jinja url_for with query params
Out of the box Starlette does not support url_for with query params. When trying to use url_for with query params it throws the following error. In my searchin
-
setting up a kind cluster with argocd installed
Kind (Kubernetes in Docker) is a tool that makes it easy to create and tear down local clusters quickly. I like to use it to test out new workflows. Argocd is
-
feeds
I have several feeds that you can consume my content with. Many posts will cross between them, so if you would like to follow a certain stream of content in yo
-
๐ญ fastapi decorators
I've been using these decorators to modify the behavior of specific routes. It will do things like 404 admin only route
-
๐ญ fastapi https url_for
jinja's urlfor in fastapi does not account for https by default, there is probably a better way, but this is a way t
-
๐ญ My New Python Lsp Setup
I figured out the killer combination for python lsp servers, ruff and jedi! ruff does all of the diagnostics and format
-
๐ญ flake8-to-ruff ยท PyPI
Underrated python library to on board ruff, or just use it on a project where its not the norm. ruff claims that its 99
-
๐ญ Creating One-To-Many Relationships in Flask-SQLA...
Great example from Anthony showing how easy it is to practice building database orm models and playing with them in a re
-
๐ญ python - Concepts of backref and back_populate i...
Today I came across some sqlalchemy models that created some relationships, some used backref some used backpopula
-
๐ญ Read a Range of Data - LIMIT and OFFSET - SQLMod...
Today I was running some sqlmodel queries through the sqlalchemy orm. Admittedly I've not done enough orm queries befor
-
๐ญ Template Designer Documentation โ Jinja Document...
html code generated by my jinja templates generally look half garbage because of indents and whitespace all over the pla
-
thoughts on unit tests
 Theo's response puts a lot of my feelings about unit testi
-
๐ญ We Finally Agree On Unit Tests - YouTube
Theo's response puts a lot of my feelings about unit testing into words. It's crazy how cargo culty it becomes that the
-
๐ญ Mastodon.py โ Mastodon.py 1.8.1 documentation
Mastadon.py is a python api client for mastadon that makes it easy to cross post to mastadon. python from mastodon
-
๐ญ FastAPI - dependency inside Middleware? - Stack ...
After struggling to get dependencies inside of middleware I learned that you can make global dependencies at the app lev
-
๐ญ Handling Errors - FastAPI
This page shows how to customize your fastapi errors. I found this very useful to setup common templates so that I can
-
๐ญ logs with FastAPI and Uvicorn ยท Issue #1508 ยท ti...
Setting an additional log handler to the uvicorn logger for access logs in fastapi was not straightforward, but This pos
-
๐ญ How to group FastAPI endpoints in Swagger UI?
Setting tags in your fastapi endpoints will group them in the docs. You can also set some metadata around the t
-
๐ญ Show some equivalent list comprehensions in filt...
I often want to reach for non existing list comprehensions in jinja 2, Here are a few nice equivalents. python a: {
-
๐ญ DataDog/ddqa: Datadog's QA manager for releases ...
DataDog ddqa is building out a textual app and deploying it with pyapp. They have CI setup to fully build and cross com
-
๐ญ Path Operation Advanced Configuration - FastAPI
Excluding routes from fastapi docs, can be done from the route configuration using includeinschema. This is
-
๐ญ Protect API docs behind authentication? ยท Issue ...
You can protect your fastapi docs behind auth so that not only can certain roles not run certain routes, but they cannot
-
Stripe Cancellations in FastAPI and SQLModel
Today I am working on fokais.com, trying to get to a point where I can launch by workig through stripe integrations. This is my first time using stripe, so ther
-
Looking for a Heroku replacement, What I found was...
I've long hosted my personal blog as a static site on waylonwalker.com. It's all markdown, converted to html, and shipped as is. It's been great,
-
๐ญ Will McGugan on X: "Just a wee thing that came u...
Textual is so sick, Will just made a live markdown editor in the terminal!
-
๐ญ Sebastiรกn Ramรญrez on X: "Now @FastAPI has 65k+ G...
Fastapi passes flask in GitHub stars! [: with Session(engine) as
-
๐ญ Center things - Textual
How to center things in textual. Textual has a very unique way of styling text user interfaces for the terminal using cs
-
๐ญ s3-tree ยท PyPI
Super useful way to show a tree view of an s3 bucket's structure! python pip install s3-tree python s3-t
-
๐ญ python - SQLAlchemy ORDER BY DESCENDING? - Stack...
How to sort results from a sqlalchemy based orm. python .orderby(model.Entry.amount.desc()) I needed this to
-
๐ญ Using Rich Inspect to interrogate Python objects...
I love rich inspect. It's one of my most often used features of rich. It gives you a great human readable insight into
-
๐ญ > htmx ~ The json-enc Extension
json-enc extension converts url encoded form values into json encoded data, this is very useful for fastapi to have the
-
๐ญ Header Parameters - FastAPI
Getting request headers in fastapi has a pretty nice stetup, it allows you to get headers values as function arguments,
-
๐ญ 1j01/textual-paint: :art: MS Paint... in your te...
1j01 created a complete working clone of ms paint in the terminal using the textual framework
-
๐ญ sqlite-utils now supports plugins
As the title states sqlite-utils now supports plugins. I dug in just a bit and Simon implemented this completely with e
-
๐ญ Form Data - FastAPI
Getting form data inside of fastapi was not intuitive to me at first. Everything I had used in fastapi leaned on pydanti
-
๐ญ pywebview
I am creating this post from a desktop app that I created in 3 lines. python import webview webview.createwindo
-
๐ญ Column INSERT/UPDATE Defaults โ SQLAlchemy 1.4 D...
sqlalchemy serverdefaults end up as defaults in the database when new values are inserted. python t = Table( "
-
๐ญ Template Designer Documentation โ Jinja Document...
A feature of jinja that I just discovered is including sub templates. Here is an example from the docs. html {% in
-
๐ญ Templates - FastAPI
A guide to add Jinja2Templates to fastapi.
-
๐ญ python 3.x - FastAPI redirection for trailing sl...
I am trying to use htmx on a new fastapi site for my thoughts, and have been hitting this error. js Mixed Content:
-
๐ญ Static Files - FastAPI
Mounting static files in fastapi. python from fastapi import FastAPI from fastapi.staticfiles import StaticFiles a
-
๐ญ First-class session support in FastAPI ยท Issue #...
Here is a snippet provided by @tiangolo to store the users jwt inside of a session cookie in fatapi. This was written i
-
๐ญ Harlequin SQL IDE - DuckDB
Harlequin is a pretty sweet example of what textual can be used to create. Its a terminal based sql ide for DuckDB.
-
๐ญ Python API - DuckDB
To persist data in duckdb you need to first make a connection to a duck db database. python con = duckdb.connect('f
-
๐ญ SQL on Pandas - DuckDB
duckdb can just query any pandas dataframe that is in memory. > I tried running it against a list of objects and got th
-
๐ญ pytest-subtests ยท PyPI
pytest-subtests is a package to register multiple subtests within a similar test function.
-
๐ญ doyensec/wsrepl: WebSocket REPL for pentesters
Very inspiring textual project to check out how they set up the ui. Their intro video has a pretty epic dev experience.
-
๐ญ Doyensec on Twitter: "Announcing wsrepl, the Web...
wsrepl is an epic websocket repl built in python on the textual framework.
-
๐ญ Filter Data - WHERE - SQLModel
When fetching pydantic models from the database with sqlmodel, and you cannot select your item by id, you probably need
-
๐ญ URL Decoding query strings or form parameters in...
In order to turn url encoded links back into links that I would find in the database of my thoughts project I need to ur
-
๐ญ Python Sucks And I LOVE It | Prime Reacts - YouT...
Prime reviews an article with some hot takes about python being slow and quirky, but good enough for a lot of things. E
-
๐ญ Flask Trailing Slashes 404
In flask apps I often get a 404 for routes with a trailing slash. This Stack Overflow post shows how to configure flask
-
๐ญ python script is not found
When setting up a new machine, vm, docker image you might be installing command line tools from places like pip. They w
-
Python scandir ignores hidden directories
The next version of markata will be around a full second faster at building it's docs, that's a 30% bump in performance at the current state. This per
-
Pycon 2023
Keynote Speaker - James Powell I don't want to be an expert python developer. [https://www.youtube.com/watch?v=iKzOBWOHGFE](https://www.youtube.com/watch?v
-
Getting Started with Pydantic
Pydantic is a Python library for serializing data into models that can be validated with a deep set of built in valitators or your own custom validators, and de
-
Playing Star Wars Text Adventure with a 10 yr old
The following is a playthrough of Star Wars Text Adventure with a 10 yr old.The following is a playthrough of StarThe following is a playthrough of Star
-
Pydantic and singledispatch
I was reading about [pydantic-singledispatch](https://www.gidware.com/reducing-complexity-with-pydantic-singledispatch/) from Giddeon's blog and found it v
-
pipx textual devtools
I really like having global cli command installed with pipx. Since textual `0.2.x` (the css release) is out I want to be able to pop into textual devtools easi
-
textual app devtools
I am working through the textual tutorial, and I want to put it in a proper cli that I can pip install and run the command without `textual run --dev app.py`. T
-
caching python virtual environments in GitHub Acti...
-
Markata GitHub Pages
11ty https://www.rockyourcode.com/how-to-deploy-eleventy-to-github-pages-with-github-actions/ hugo puts it in the base url https://gohugo.io/getting-started/con
-
Markata Supports Jinja Plugins 0.5.0.dev2
Markata now allows you to create jinja extensions that will be loaded right in with nothing more than a `pip install`. From the Changelog The entry for 0.5.0.de
-
Using Django Rest Framework inside react
-
Getting Started with Django REST framework
In my adventure to learn django, I want to be able to setup REST api's to feed into dynamic front end sites. Potentially sites running react under the hood
-
Markata now uses hatch
Markata now uses hatch as its build backend, and version bumping tool. `setup.py`, and `setup.cfg` are completely gone. 
-
Trying out django
I have no experience in django, and in my exploration to become a better python developer I am dipping my toe into one of the most polished and widely used web
-
how to escape variables in jinja templates
While updating my site to use Markata's new configurable head I ran into some escaping issues. Things like single quotes would cause jinja to fail as it w
-
I've made my ipython config too complicated, let's...
When I am developing python code I often have a repl open alongside of it running snippets ofcode as I go. Ipython is my repl of choice, and I hace tricked it
-
markata extend in configuration
{.more-cinematic} A long needed feature of
-
Create a new Python Project with the Hatch Cli
I'm really getting into using hatch as my go to build system, and I am really liking it so far. I am slowly finding new things that just work really well
-
Versioning Python Projects with Hatch
Hatch has an amazing versioning cli for python packages that just works. It takes very little config to get going and you can start bumping versions without wo
-
From Markdown to Blog with Markata | lightning tal...
Markata is a great python framework that allows you to go from markdown to a full website very quickly. You can get up and running with nothing more than Markd
-
I spoke at Python Web Conf 2022
I spoke at python webconf in March 2022 about how I deploy this blog on a continuous basis. Building this blog has brought me a lot of benefits. I have a set o
-
Packages I Maintain
Markata I open sourced the static site framework that I use to build [my-blog](https://waylonwalker.com/) among other side projects. It's a plugins all the
-
I turned on 2FA for all of my pypi packages
I got all the pypi packages that I own behind 2 factor authentication. ๐ช Recently this really made it's rounds in the python news since pypi was requiring
-
I was on Talk Python
After years of listening to [talkpython.fm](https://talkpython.fm/) I had the honor to be part of [episode-337](https://talkpython.fm/episodes/show/337/kedro-fo
-
use vim to remove duplicates
I just love how some features of vim are so discoverable and memorable once you really start to grasp it. Sorting and uniqing your files or ranges is one of th
-
Highlighting text ranges with Rich | python
Today I've been playing with [py-tree-sitter](https://github.com/tree-sitter/py-tree-sitter) a bit and I wanted to highlight match ranges, but was unable t
-
Highlighting text ranges with Rich | python
{% for year in markata.map("date.year", filter='published')|unique %} {{ year }} {% for post in markata.map('post', filter="p
-
markata 0.3.0 is 15-20% faster

-
get the version of any python package | python api
A common meta thing that I need in python is to find the version of a package. Most of the time I reach for `package_name.__version__`, but that does not always
-
Running My Blog on 3.11-dev
3.10.5: 109.441 3.11-dev: 108.856
-
Manage your displays with xrandr manager
 xrandr is a great cli to manage your windows
-
IPython q to exit
So many terminal applications bind `q` to exit, even the python debugger, its muscle memory for me. But to exit ipython I have to type out `exit<ENTER>`.
-
Copier Slugify | python templating | using cookiec...
It's no secret that I love automation, and lately my templating framework of choice has been copier. One hiccup I recently ran into was having spaces in m
-
Textual has devtools
Textual has devtools in the upcoming css branch, and its pretty awesome! It's still early Textual is still very early and not really ready for prime time,
-
Python atexit
totally guessed at this post's date I'm still trying to understand this one, but this is how you force a python object to stop atexit.
-
The one pip config you need to have
Whenever you are installing python packages, you should always use a virtual environment. pip makes this easy to follow by adding some configuration to pip. re
-
pyenv no module named '_sqlite3'
I've been trying to adopt pyenv for a few months, but have been completely blocked by this issue on one of the main machines I use. Whenever I start up ip
-
Using Different versions of python with pipx | pye...
{.hoverlink} Idea to blog post i
-
Python sys.excepthook
Sometimes you just want python to do something else when you hit an exception, maybe that's fire a text, slack message, email, or system notification like
-
pygame events are queued | Don't make this mistake
pygame events are stored in a queue, by default the most suggested way shown in all tutorials "`pumps`" the queue, which removes all the messages. sta
-
Display Sprites in Pygame | Load and Blit
One of the most essential concepts of pygame to start making a game you will need to understand is loading images and blitting them to the screen. **blit** stan
-
Dunk is my new diff pager
[Dunk](https://github.com/darrenburns/dunk) is a beautiful git diff tool built on top of [rich](https://github.com/Textualize/rich). Browsing through twitter th
-
Pygame Boilerplate Apr 2022
I'm poking a bit into gamedev. Partly to better understand, partly because it's stretching different parts of my brain/skillset than writing data pip
-
Did my site build just go down?
My personal Site build went down last week, and I was unable to publish a new article. This is the process I went through to get it back up and running quickly
-
Don't inherit from python BaseException, Here's wh...
I ran into a PR this week where the author was inheriting what BaseException rather than exception. I made this example to illustrate the unintended side effec
-
How I make cache-keys from python objects
When I need a consistent key for a pythohn object I often reach for `hashlib.md5` It works for me and the use cases I have. diskcache Yesterday we talked about
-
How I setup a sqlite cache in python
When I need to cache some data between runs or share a cache accross multiple processes my go to library in python is `diskcache`. It's built on sqlite wi
-
Cache a python function with lru_cache
The easiest way to speed up any code is to run less code. A common technique to reduce the amount of repative work is to implement a cache such that the next t
-
Set User Agent on pandas read_csv
I keep a small [cars.csv](https://waylonwalker.com/cars.csv) on my website for quickly trying out different pandas operations. It's very handy to keep aro
-
Get Webpage with python requests
Python's requests library is one of the gold standard apis, designed by Kenneth Reitz. It was designed with the user perspective in mind first and impleme
-
Style Lookatme Slides a bit more Personal
I recently gave a talk at python web conf 2022, and one of the things I did when I should have been working on my presentation was workig on how my presentation
-
How I load Markdown in Python
I use a package [eyeseast/python-frontmatter](https://github.com/eyeseast/python-frontmatter){.hoverlink} to load files with frontmatter in them. Its a handy p
-
Python's Dict Union Operator | Pep 584
Today I was watching the python web conf 2022 and saw [@davidbujic](https://twitter.com/davidvujic) use the new Dict Union Operator Live on stage during his [Fu
-
How I Present Markdown Slides from the Terminal
I love the freedom of writing in markdown. It allows me to write content from the comfort of my editor with very little focus on page style. It turns out that
-
How I read Files in Python
When I need to read contents from a plain text file in python I find the easiest way is to just use `Pathlib`.
-
How I glob for Files in Python
A very common task for any script is to look for files on the system. My go to method when globbing for files in python is to use pathlib. Setup I setup a dire
-
pytest-mock Basics
Last Thursday I learned about `pytest-mock` at a local python meetup. The presenter showed how he uses `pytest-mock` for his work, and it was kinda eye opening
-
Python Walrus Inside List Comprehension
Python 3.8 came out two and a half years ago and I have yet to really lean in on the walrus operator. Partly because it always seemed like something kinda sill
-
Make Kedro Runs Beautiful
Kedro rich is a very new and unstable (it's good, just not ready) plugin for kedro to make the command line prettier. Install kedro rich There is no pypi p
-
Open Files with Nvim Remote
I recently found a really great [plugin](https://github.com/mhinz/neovim-remote) by [mhinz](https://github.com/mhinz) to open files in neovim from a different t
-
Textual Popup Hack
As I am toying around with textual, I am wanting some popup user input to take over. Textual is still pretty new and likely to change quite significantly, so I
-
GitHub Markdown now Supports Mermaid Diagrams
Mermaid diagrams provide a way to display graphs defined as plain text. Some markdown renderers support this as a plugin. GitHub now supports it. example You c
-
Glances can watch docker processes
Glances is a system monitor with a ton of features, including docker processes. I have started using portainer to look at running docker processes, its a great
-
Glances webui with pipx
Glances has a pretty incredible webui to view system processes and information like htop, or task manager for windows. The nice thing about the webui is that it
-
Run glances without install with pipx
Glances is a fully featured system monitoring tool written in python. Out of the box it's quite similar to htop, but has quite a few more features, and ca
-
Nested requirements.txt in python
python requirements text files can in fact depend on each other due to the fact that you can pass pip install arguments right into your `requirements.txt` file.
-
View AWS event bridge rules with visidata
Reading eventbridge rules from the command line can be a total drag, pipe it into visidata to make it a breeze. I just love when I start thinking through how to
-
Python string of letters is a string of letters, b...
In python, a string is a string until you add special characters. In browsing twitter this morning I came accross this tweet, that showed that you can use `is`
-
Read stderr from python subprocess.Popen
I often run shell commands from python with Popen, but not often enough do I set up error handline for these subprocesses. It's not too hard, but it can b
-
Using a Python Markdown ast to Find All Paragraphs
In looking for a way to automatically generate descriptions for pages I stumbled into a markdown ast in python. It allows me to go over the markdown page and g
-
python dict get
For an embarassingly long time, til today, I have been wrapping my dict gets with key errors in python. I'm sure I've read it in code a bunch of time
-
Find all Headings with BeautifulSoup
BeautifulSoup is a DOM like library for python. It's quite useful to manipulate html. Here is an example to find_all html headings. I stole the regex fr
-
Lambda Function as a Kedro Node
I keep my nodes short and sweet. They do one thing and do it well. I turn almost every DataFrame transformation into its own node. It makes it must easier to
-
Add New Dependencies to Your Kedro Project
As you work on your kedro projects you are bound to need to add more dependencies to the project eventually. Kedro uses a fantastic command `pip-compile` under
-
Practice making pipelines with kedro
I am a huge believer in practicing your craft. Professional athletes spend most of their time honing their skills and making themsleves better. In Engineering
-
Tmux hotkey for copier templates
I have added a hotkey to my copier template setup to quickly access all my templates at any time from tmux. At any point I can hit `<c-b><c-b>`, th
-
Markata Filters as Telescope Pickers in Neovim
I often pop into my blog from neovim with the intent to look at just a single series of posts, `til`, `gratitude`, or just see todays posts. [Markata](https://m
-
Copier Tasks | Python templating post run task
Copier allows you to run post render tasks, just like cookiecutter. These are defined as a list of `tasks` in your `copier.yml`. They are simply shell commands
-
Gradual Typing in Python
I've referenced a video from Anthony Sotile in passing conversation several times. Walking through his gradual typing process has really helped me underst
-
Python Reverse Sluggify
In order to make an auto title plugin for markata I needed to come up with a way to reverse the slug of a post to create a title for one that does not explicitl
-
Get Python docstring with ast
Getting docstrings from python's ast is far simpler and more reliable than any method of regex or brute force searching. It's also much less intimida
-
Python Respect the .gitignore
Many tools such as ripgrep respect the `.gitignore` file in the directory it's searching in. This helps make it incredibly faster and generally more intui
-
Rename Python Variables with nvim
I don't use refactoring tools as much as I probably should. mostly because I work with small functions with unique names, but I recently had a case where
-
Python Find Available Port
When running a python process that requires a port it's handy if there is an option for it to just run on the next avaialble port. To do this we can use t
-
python functools total ordering
functools.total_ordering makes adding all of six of the rich comparison operators to your custom classes much easier, and more likely that you remember all of t
-
Implement --pdb in a python cli
Adding a `--pdb` flag to your applications can make them much easier for those using it to debug your application, especially if your applicatoin is a cli appli
-
Python Enum
Python comes with an enum module for creating enums. You can make your own enum by inheriting importing and inheriting from Enum. auto incrementing Enum values
-
Installing Pipx on Ubuntu
I recently paired up with another dev running windows with Ubuntu running in wsl, and we had a bit of a stuggle to get our project off the ground because they w
-
Using Copier Answers to rerun templates quickly
The copier answers file is a key component to making your templates re-runnable. Let's look at the example for my setup.py. Inside of my `[[ _copier_conf.
-
Making a Textual Widget from a Rich Renderable
Once you have made your sick looking cli apps with rich, eventually you are going to want to add some keybindings to them. Currently Textual, also written by [
-
pipx examples
count lines of code
-
Copier Templates
%%include til/copier_endops %%include til/copier-template-variables %%include til/copier-answers
-
Changing copier template strings (_endops)
I was completely stuck for awhile. copier was not replacing my template variables. I found out that adding all these `_endops` fixed it. Now It will support
-
copier template variables
I've been looking for a templating tool for awhile that works well with single files. My go to templating tool `cookiecutter` does not work for single fil
-
Running Kedro on Ubuntu 21.10 Impish Indri
I just installed a brand new Ubuntu 21.10 Impish Indri, and wanted a kedro project to play with so I did what any good kedroid would do, I went to my command li
-
A Minimal Pluggy Example
Pluggy makes it so easy to allow users to modify the behavior of a framework without thier specific feature needing to be implemented in the framework itself. I
-
My first impressions with pyenv
pyenv provides an easy way to install almost any version of python from a large list of distributions. I have simply been using the version of python from the o
-
Installing packages with ansible only if they do n...
Part of my neovim setup requires having the `black` python formatter installed and callable. I install it with `pipx` so that I don't have to manage a vir
-
Adding __rich__ methods to python classes
Adding a `__render__` method that returns a rich renderable to any python class makes it display this output if printed with rich. This also includes being nes
-
Smoother Python with automatic imports | pyflyby
This is not a flaky works half the time kind of plugin, it's a seriously smooth editing experience. I've just started using pyflyby, and it is solid
-
Kedro Course
find all nodes with raw in the name use parameters make and use a logger use find-kedro in spaceflights slide in a new node vizualize your pipeline find slow no
-
kedro catalog create
I use `kedro catalog create` to boost my productivity by automatically generating yaml catalog entries for me. It will create new yaml files for each pipeline,
-
Using Nix to manage my Python Interpreter
I recently started playing with nix. goals automatically select correct python version per project activating one doesn't bleed into the other Installing n
-
Just Ask Ipython for help
It happens to the best of us We can't all remember every single function signature out there, it's just not possible. If you want to stay productive
-
Setting Parameters in kedro
Parameters are a place for you to store variables for your pipeline that can be accessed by any node that needs it, and can be easily changed by changing your e
-
Writing your first kedro Nodes
https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let's make some nodes with some vanilla data structures. import node You will need to i
-
Running your Kedro Pipeline from the command line
Running your kedro pipeline from the command line could not be any easier to get started. This is a concept that you may or may not do often depending on your
-
kedro Virtual Environment
Avoid serious version conflict issues, and use a virtual environment anytime you are running python, here are three ways you can setup a kedro virtual environme
-
Kedro Pipeline Create
Kedro pipeline create is a command that makes creating new pipelines much easier. There is much less boilerplate that you need to write yourself. https://youtu
-
Kedro Install
Kedro comes with an `install` command to install and manage all of your projects dependencies. https://youtu.be/IWimEs-hHQg cd into your project directory and a
-
Kedro Git Init
Immediately after `kedro new`, before you start running `kedro install` or your first line of code the first thing you should always do after getting a new kedr
-
Kedro New
[https://youtu.be/uqiv5LAiJe0](https://youtu.be/uqiv5LAiJe0){.youtube-embed} Kedro new is simply a wrapper around the cookiecutter templating library. The kedr
-
What is Kedro
Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user a way to build pipelines that automatica
-
Incremental Versioned Datasets in Kedro
Kedro versioned datasets can be mixed with incremental and partitioned datasets to do some timeseries analysis on how our dataset changes over time. Kedro is a
-
Manage many git repos with ease
mu-repo
-
I Started Streaming on Twitch
I recently started streaming on [twitch.tv/waylonwalker](https://www.twitch.tv/waylonwalker) and it's been a blast so far. python kedro Data Science Data E
-
Upcoming Stream
[https://stackoverflow.com/questions/16720541/python-string-replace-regular-expression](https://stackoverflow.com/questions/16720541/python-string-replace-regul
-
Kedro Spaceflights - part 2 | Stream replay June 7...
This was my seconf time ever streaming on [twitch.tv/waylonwalker](https://twitch.tv/waylonwalker), and I completely botched my mic 2x. [https://youtu.be/_7MwgK
-
๐ฑ Digital Gardening | gif to Mp4 | Stream replay J...
https://youtu.be/I4VenHqIEng Doing some Digital Gardening on stream Ahrefs Errors ahrefs large images Automatic gif to mp4 gif to mp4 After this stream all gifs
-
Kedro Spaceflights - part 1 | Stream replay June 4...
This was my first time ever streaming on [twitch.tv/waylonwalker](https://twitch.tv/waylonwalker). I am excited to get going. I have been streaming early in th
-
Comprehensive guide to creating kedro nodes
The Kedro node is an essential part of the pipeline. It defines what catalog entries get passed in, what function gets ran, and the catalog entry to save the re
-
Creating pypi-list with kedro
I had an idea come to me via twitter. Short one word name packages are becoming hard to find on pypi. Short one word readable package names that are not a play
-
Using Kedro In Scripts
With the latest releases of kedro `0.17.x`, it is now possible to run kedro pipelines from within scripts. While I would not start a project with this techniqu
-
Silence Kedro Logs
Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while try
-
python lsp setup
Setting up python with the native nvim>0.5 lsp was mr lsp-config https://github.com/neovim/nvim-lspconfig pyls#190 https://github.com/palantir/python-languag
-
Python Diskcahe is locked
Running multiple processes using the same diskcache object can cause issues with locks. As I was trying to setup a rich Live display for markata I ran into iss
-
Vim Fugitive
Add current file and commit with diff in a split :on[ly] _C-W o_ :on[ly] will make the current buffer the only one on the screen. This is super helpful as many
-
What is if __name__ == "__main___", and how do I u...
When a python module is called it is assigned the `__name__` of `__main__` otherwise if it's imported it will be assigned the `__name__` of the module. Con
-
Custom Kedro Logger
DRAFT -
-
How to Install micromamba on linux (from the comam...
I really like using conda (`miniconda`) as my python virtual environment manager of choice. It's simple and it includes its own python interpreter using t
-
kedro replit
I am trying to see what an embeded replit
-
Zev Averbach Interview
Zev Averbach, _Frustrated spreadsheet jockey to software developer at 36_ Q: Tell me about your journey as a spreadsheet jockey into Data Engineering? A: First
-
Pytest capsys
Testing print/log statements in pytest can be a bit tricky, capsys makes it super easy, but I often struggle to find it. capsys capsys is a builtin pytest fixtu
-
Building Rich a Dev Server
**Draft Post** I've really been digging [@willmcgugan's](https://twitter.com/willmcgugan) [rich](https://github.com/willmcgugan/rich) library for crea
-
fix crlf for entire git repo
Final Result
-
Automatic Conda Environments
I have automated my process to create virtual environments in my python projects, here is how I did it. I've really been digging my new tmux session manage
-
How I Review Pipeline Code
I have started doing more regular PR's on my teams [Kedro](https://waylonwalker.com/what-is-kedro) pipelines. I generally take a two phase approach to the
-
Kedro pipeline_registry.py
With the realease of `kedro==0.17.2` came a new module in the project template `pipeline_registry.py`. Here are some notes that I learned while playing with th
-
๐ Pluggable Architecture with Python
pytest has open sourced their amazing plugin framework `pluggy`, it allows library authors to give their users a way to modify the libaries behavior without nee
-
Create Og Image Covers Using Python
There are so many tutorials out there for creating OG images with `nodejs` or `puppeteer`, but I have yet to see many using python. PIL Template Text Overlays
-
โ How Python Tools Are Configured
There are various ways to configure python tools, config files, code, or environment variables. Let's look at a few projects that allow users to configure
-
Minimal Kedro Pipeline
How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It's only a total of 35 lines of python,
-
Markdown Cli
This is a post that may be a work in progress for awhile, Its a collections of thoughts on managing my blog, but could be translated into anythiung that is just
-
Blog Data With Python
Generating an api for a blog is much simpler than one might expect with python. Markdown Frontmatter Fill in the blanks fix missing data Fast
-
Kedro - My Data Is Not A Table
In python data science/engineering most of our data is in the form of some sort of table, typically a DataFrame from a library like pandas, spark, or dask. Data
-
Quickly Change Conda Env With Fzf
Changing conda environments is a bit verbose, I use a function with fzf that both lists environments and selects the one I want in one go. Conda I have used con
-
Minimal Python Package
What does it take to create an installable python package that can be hosted on pypi? What is the minimal python package setup.py my_module.py This post is some
-
Custom Ipython Prompt
I've grown tired of the standard ipython prompt as it doesn't do much to give me any useful information. The default one gives out a line number that
-
Ipython-Config
I use my ipython terminal daily. It's my go to way of running python most of the time. After you use it for a little bit you will probably want to setup
-
Ipython Ninjitsu
?docstring ??sourcecode %run %debug %autoreload %history autoformat %reset !shell commands ?docstring Stop going to google everytime your stuck and stay in your
-
Automating my Post Starter
One thing we all dread is mundane work of getting started, and all the hoops it takes to get going. This year I want to post more often and I am taking some ste
-
Windowing Python Lists
In python data science we often will reach for pandas a bit more than necessary. While pandas can save us so much there are times where there are alternatives t
-
Testing Data Pipelines
Thinking through testing best practices for Data Pipelines
-
reasons-to-kedro
There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processing DataFrames from many data sources shou
-
What's New in Kedro 0.16.6
This version of kedro releases a new set of supported deployment options and the spaceflights pipeline is officially added as a starter alias.
-
Designing a "Router" for kedro
nodes_global I released a router-like plugin for kedro back in April 2020. This was not the first design, the idea actually came from one of the QB folks who t
-
Reclaim memory usage in Jupyter
Today I ran into an issue where we had a one-off script that just needed to work, but it was just chewing threw memory like nothing. It started with a colleague
-
Strip Trailing Whitespace from Git projects
A common linting error thrown by various linters is for trailing whitespace. I most often use flake8. I generally have [pre-commit]([https://waylonwalker.com/
-
Three things to Automate with Python using Pandas
Here are three things that I see my non programming counterparts doing every single day. These really sum up so much of what folks do within an office. So man
-
How to Install miniconda on linux (from the comman...
Installing miniconda from only the command line can be a bit of an intimidating task your first time. Lets walk through how to do it.
-
What's New in Kedro 0.16.4
If we take a look at the [release notes](https://github.com/kedro-org/kedro/blob/master/RELEASE.md) I see one **major** feature improvement on the list, auto-di
-
Integration testing with Python, TestProject.io, a...
As I continue to build out [waylonwalker.com](https://waylonwalker.com/) I sometimes run into some errors that are not caught because I do not have good testing
-
๐ Practice Python Online
When learning a new skill it's important to practice along the way. In order for me to show up to practice I need to make it easy to show up. An easy way t
-
Kedro Catalog
I am exploring a kedro catalog meta data hook
-
How python tools configure
mypy Mypy's config parser seems to be one of the most complex. This is likely in part to it having the largest backwards compatability of all projects tha
-
๐ Parsing RSS feeds with Python
I am looking into a way to replace my google reader experience that I had back in 2013 before google took it from us. I am starting by learning how to parse fe
-
SLIDES - understanding python \*args and \*\*kwarg...
Python `*args` and `**kwargs` are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions f
-
Gracefully adopt kedro, the catalog
Why use kedro catalog? While using the catalog alone will not reap all of the benefits of the framework, it does get you and your project ready for the full fra
-
How to find things in your kedro catalog
kedro 0.16.2 just dropped last week with a long-awaited feature... **catalog search**! I went as far as monkey patching this into each of my projects. I work
-
How Kedro handles your inputs
Passing inputs into kedro is a key concept. Understanding how it accepts a single catalog key as input is quite trivial that easily makes sense, but passing a
-
016
Recieving `**kwargs`
-
015
Recieving `*args` **order matters**
-
018
Sending `**kwargs`
-
017
Sending `*args` **order matters**
-
understanding python \*args and \*\*kwargs
Python `*args` and `**kwargs` are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions f
-
010
load remote _data_ with **pandas**
-
011
Load _data_ from database into **pandas**
-
009
Combine a directory of _csv's_ with **pandas**
-
012
**autoreload** your imports in ipython for โก fast development ๐ add this to your `~/.ipython/profile_default/ipython_config.py.`
-
005
**pre-commit** setup is _simple_ **Try** it yourself
-
006
Setup **pre-commit** for _isort_ _includes automatic_ .isort-config
-
pre-commit is awesome
I recently discovered the โจ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really
-
002
**kedro-static-viz** 0.3.0 just launched with _hooks_ support ๐
-
Kedro Static Viz 0.3.0 is out with Hooks Support
[kedro-static-viz](https://github.com/WaylonWalker/kedro-static-viz) is out with support for the newly released hooks feature. This means that you can have `ke
-
001
**practice** building pipelines with _#kedro_ today
-
Create Custom Kedro Dataset
Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For t
-
Interrogate is a pretty awesome, brand new, cli fo...
dead simple docstring coverage for your python project
-
creating the kedro-preflight hook
Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro `0.16.0`. They allow you to hook into `catalog_created`,`pipeline_run`, and `node_run
-
๐ Kedro Preflight Notes
run checks before running the pipeline
-
python-deepwatch
Is it possible to deep watch a single python function for changes? Shallow Watch keeping track of a python functions hash is quite simple. There is a`__hash__`
-
TIL: Bind arguments to dynamically generated lambd...
This past week I had a really weird bug in my [kedro](https://kedro.readthedocs.io/) pipeline. For some reason data running through my pipeline was coming out
-
Four Github Actions for Python
If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools gener
-
Variables names don't need their type
So often I see a variables `type()` inside of its name and it hurts me a little inside. Tell me I'm right or prove me wrong below. Examples Pandas `DataFr
-
Create New Kedro Project
This is a quickstart to getting a new [kedro](https://kedro.readthedocs.io) pipeline up and running. After this article you should be able to understand how to
-
Long variable names are good
๐ท๏ธ Long variable names are a good thing. Self documenting code is more important than poorly documented code. Simply adding a few characters to your variable na
-
What is Kedro
[[ what-is-kedro ]] This is my original what-is-kedro article. There is a brand new one Kedro is an open source data pipeline framework. It provides guardrail
-
simple click 2
-
simple click
Add helpful cli to your python libraries... All of them!
-
SqlAlchemy Models
My Notes about using sqlalchemy models
-
Building Cli apps in Python
learning about building cli apps in python
-
Kedro
My Notes about using kedro
-
๐ Packages to Investigate Notes
jmespath Tabnine Bulwark |-|-| |github: |[https://github.com/zaxr/bulwark](https://github.com/zaxr/bulwark)| I definitely want to try this out with kedro. Bulwa
-
Debugging Python
Debugging Python
-
Just Use Pathlib
Pathlib is an amazing cross-platform path tool.
-
Filtering Pandas
Filtering pandas DataFrames many different ways.
-
Pyspark
I have been using pyspark since March 2019, here are my thoughts.
-
Making good documentation in python
Tools Sphinx Portray I just started using portray and it is amazingly simple to use! Methodology
-
Quick Progress Bars in python using TQDM
Quick Progress Bars in python using TQDM
-
Clean up Your Data Science with Named Tuples
If you are a regular listener of [TalkPython](https://talkpython.fm) or PythonBytes you have hear Michael Kennedy talk about Named Tuples many times, but what a
-
Background Tasks in Python for Data Science
This post is intended as an extension/update from [background tasks in python](https://waylonwalker.com/background-1/). I started using `background` the week t
-
๐ Bash Notes
Waylon Walker's Bash Notes
-
Autoreload in Ipython
Autoreload in python
-
Python Tips
Dictionaries Unpacking **kwargs func(**input) locals().update(d)
-
Generating Readme Tables From Pandas
none
-
Pycon 2018 Roundup
none
-
My favorite pandas pattern
My favorite pandas pattern I work with a lot of transactional timeseries data that includes categories. I often want to create timeseries plots with each catego
-
background tasks in python
none
-
Pycon 2017 Roundup
none