Posts tagged: python

I’m really getting into using hatch as my go to build system, and I am really liking it so far. I am slowly finding new things that just work really well. hatch new is one of those things that I didn’t realize I needed until I had it. [1] creating new versions created by myself with stable diffusion [2] ❯ pipx run hatch new --help Usage: hatch new [OPTIONS] [NAME] [LOCATION] Create or initialize a project. Options: -i, --interactive Interactively choose details about the project --cli Give the project a command line interface --init Initialize an existing project -h, --help Show this message and exit. Note! I am running all of these commands with pipx. I like to use pipx for all of my system level cli applications. To emphasis this point in the article I am going to use pipx run hatch, but you can pipx install hatch then just run hatch from there. Interacively create a new project # [3] Running hatch new -i will ask let you interactivly choose details about the project, such as the project’s name. pipx run hatch new -i After running and naming the project Hatch New we end up with the following filetree. ❯ tree . . ├── hatch_new │   ├── __about__.py │   └── __in...

Hatch has an amazing versioning cli for python packages that just works. It takes very little config to get going and you can start bumping versions without worry. [1] creating new versions created by myself with stable diffusion project layout # [2] For trying out the hatch version cli let’s make a simple project with the terrible name pkg. ❯ tree . . ├── pkg │   ├── __about__.py │   └── __init__.py ├── pyproject.toml └── README.md 1 directory, 4 files pyproject.toml # [3] The main hero of this post is the pyproject.toml. This is what defines all of our PEP 517 [4] style project setup. [project] name = "pkg" description = "Show how to version packages with hatch" readme = "README.md" dynamic = [ "version", ] [build-system] requires = [ "hatchling>=1.4.1", ] build-backend = "hatchling.build" [tool.hatch.version] path = "pkg/__about__.py" statically versioning # [5] project.version It is possible to set the version number inside the pyproject.toml statically. This is fine if you just want to version your package manually, and not through the hatch cli. [project] name = "pkg" version = "0.0.0" # ... Statically versioning in pyproject.toml will not work with hatch ...

Markata is a great python framework that allows you to go from markdown to a full website very quickly. You can get up and running with nothing more than Markdown. It is also built on a full plugin architecture, so if there is extra functionality that you want to add, you can create a plugin to make it behave like you want. Full transparancy… I built markata. The talk # [1] The talk is live on YouTube. Make sure you check out the other videos from the conference. There were quite a few quality talks that deserve a watch as well. https://youtu.be/Wq9YBamSgs0 [2] References: [1]: #the-talk [2]: https://stable-diffusion.waylonwalker.com/000221.1721400243.webp

I spoke at python webconf in March 2022 about how I deploy this blog on a continuous basis. Building this blog has brought me a lot of benefits. I have a set of custom curated notes to help describe a problem and how to solve it to me. At theis point it’s not uncommon to google an Issue I am having and finding my own blog with exactly the solution I need at the top. I also bump into people from time to time that recognize me from the blog, its a nice conversation starter, and street cred. The Talk # [1] The talk recently released on Youtube, you can watch it without having a ticket to the conference for free. There were a bunch of other talks that you should check out too! https://youtu.be/JDZVjDCTKHE References: [1]: #the-talk

I got all the pypi packages that I own behind 2 factor authentication. 💪 Recently this really made it’s rounds in the python news since pypi was requiring critical package maintainers to have 2FA on and even offering them hardware tokens to help them turn this on. I feel like this caused a bit of confusion as turning on 2FA does not mean that you need to do anything different to deploy a package, and it DOES NOT require a hardware token. You can continue using your favorite 2FA app. You might wonder what this means for my projects. It means that to edit any sensitive content such as pull a new api token, add/remove maintainers, or deleting a release I need to use a TOPT (time based one time password) application such as Google Authenticator, Microsoft Authenticator, Authy, or FreeOTP. This has very little change to my overall workflow as my CI system still automatically deploys for me with the same api token as before. This is one small thing that maintainers can do to prevent supply chain attacks on their projects that they put so much work into. Login # [1] When I log in I now get this extra screen asking for an auth token. [2] My packages # [3] Once I turned on 2FA...

I just love how some features of vim are so discoverable and memorable once you really start to grasp it. Sorting and uniqing your files or ranges is one of those examples for me. " sort the file :sort " sort the file only keeping unique lines :sort u " sort a range :'<,'> sort " sort a range only keeping unique lines :'<,'> sort u I recently used this to dedupe my autogenerated links section for rich-syntax-range-style [1]. More often I am using it to sort and uniqify objects like arrays and lists. Here is what the markdown looks like. * [py-tree-sitter](https://github.com/tree-sitter/py-tree-sitter) * [rich](https://github.com/Textualize/rich) * [@textualizeio](https://twitter.com/textualizeio) * [rich](https://github.com/Textualize/rich) * [another post](https://waylonwalker.com/designing-kedro-router) * [print-register-pipelines](https://screenshots.waylonwalker.com/print-register-pipelines.webp) * [rich](https://github.com/Textualize/rich) * [console-print-register-pipelines](https://screenshots.waylonwalker.com/console-print-register-pipelines.webp) * [rich](https://github.com/Textualize/rich) * [syntax-print-register-pipelines](https://screenshots.waylonwalker.com/syn...

Today I’ve been playing with py-tree-sitter [1] a bit and I wanted to highlight match ranges, but was unable to figure out how to do it with rich [2], so I reached out to @textualizeio [3] for help. https://twitter.com/_WaylonWalker/status/1562469770766589952 While waiting for that reply let’s show how we got this far. imports # [4] Lets import all the classes that we need from rich [2] and setup a console to print to. from rich.console import Console from rich.syntax import Syntax from rich.style import Style console = Console() some code # [5] Now we need some code to highlight. I am going to rip my register_pipeline from another post [6]. code = ''' from find_kedro import find_kedro def register_pipelines(self) -> Dict[str, Pipeline]: """Register the project's pipeline. Returns: A mapping from a pipeline name to a ``Pipeline`` object. """ return find_kedro() ''' print # [7] We could simply print out the code we have as a variable, but thats a bit hard to read. [8] console.print # [9] printing with rich [2]’s console makes it a little better, but not much by default. [10] Syntax # [11] We can pull from rich [2]’s syntax module to really pretty this up. ...

{% for year in markata.map(“date.year”, filter=‘published’)|unique %} {{ year }} # [1] {% for post in markata.map(‘post’, filter=“published and date.year == “+year|string, sort=‘date’) %} - [{{ post.title }} - {{ post.date.month }}/{{ post.date.day }}](/{{ post.slug }}) {% endfor %} {% endfor %} References: [1]: #-year-

[1] a sprinter edging out his opponent by Dall-e It’s about time to release Markata 0.3.0. I’ve had 8 pre-releases since the last release, but more importantly it has about 3 months of updates. Many of which are just cleaning up bad practices that were showing up as hot spots on my pyinstrument reports Markata started off partly as a python developer frustrated with using nodejs for everything, and a desire to learn how to make frameworks in pluggy. Little did I know how flexible pluggy would make it. It started out just as my blog generator, but has turned into quite a bit more. Over time this side project has grown some warts and some of them were now becoming a big enough issue it was time to cut them out. Let’s compare # [2] I like to use my tils articles for examples and tests like this as there are enough articles for a good test, but they are pretty short and quick to render. mkdir ~/git/tils/tils cp ~/git/waylonwalker.com/pages/til/ ~/tils/tils -r cd ~/git/tils/tils running tils on 0.2.0 # [3] At the time of writing this is the current version of markata, so just make a new venv and run it. python3 -m venv .venv --prompt $(basename $PWD) pip install markata mark...

[1] References: [1]: https://images.waylonwalker.com/polybar-gpu-stat.png

A common meta thing that I need in python is to find the version of a package. Most of the time I reach for package_name.__version__, but that does not always work. but not all projects have a __version__ # [1] In searching the internet for an answer nearly every one of them pointed me to __version__. This works for most projects, but is simply a convention, its not required. Not all projects implement a __version__, but most do. I’ve never seen it lie to me, but there is nothing stopping someone from shipping mismatched versions. If you maintain a project ship a __version__ # [2] I appreciate it While its not required its super handy and easy for anyone to remember off the top of their head. It makes it easy to start debugging differences between what you have vs what you see somewhere else. You can do this by dropping a __version__ variable inside your __init__.py file. ## __init__.py __version__ = 1.0.0 SO # [3] stack overflow saves the day Special thanks to this Stack Overflow post [4] for answering this question for me. So what do you do… # [5] importlib Your next option is to reach into the package metadata of the package that you are interested in, and this ha...

[1] xrandr is a great cli to manage your windows in a linux distro using x11, which is most of them. The issue is that I can never remember all the flags to the command, and if you are using it with something like a laptop using a dock the names of all the displays tend to change every time you redock. This makes it really hard to make scripts that work right every time. Homepage # [2] Check out the deresmos/xrandr-manager [3] for more details on it. installation # [4] xrander-manager is a python cli application that is simply a nice interface into xrandr. So you must have xrandr already installed, which is generally just there on any x11 window manager, I’ve never had to install it. As with any python cli that is indended to be used as a global/system level cli application I always install them with pipx. This automates the process of creating a virtual environment [5] for xrandr-manager for me, and does not clutter up my system packages with its dependencies that may eventually clash with another that I want to use. # prereqs (xrandr, pipx) pipx install xrandr-manager set main monitor # [6] First if your main display is not set to the correct monitor set your main dis...

So many terminal applications bind q to exit, even the python debugger, its muscle memory for me. But to exit ipython I have to type out exit<ENTER>. This is fine, but since q is muscle memory for me I get this error a few times per day. ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ <ipython-input-1-2b66fd261ee5>:1 in <module> │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ NameError: name 'q' is not defined After digging way too deep into how IPython implements its ExitAutoCall I realized there was a very simple solution here. IPython automatically imports all the scripts you put in your profile directory, all I needed was to create ~/.ipython/profile_default/startup/q.py with the following. q = exit It was that simple. This is not a game changer by any means, but I will now see one less error in my workflow. I just press q<Enter> and I am out, without error.

It’s no secret that I love automation, and lately my templating framework of choice has been copier. One hiccup I recently ran into was having spaces in my templated directory names. This makes it harder to run commands against as you need to escape them, and if they end up in a url you end up with ugly %20 all over. Cookiecutter has the solution # [1] Yes the solution comes from a competing templating framework. I install copier with pipx, so I need to inject cookiecutter in to my copier environment to use the slugify filter. pipx inject copier cookiecutter If you are using a normal virtual environment [2] you can just pip install it. pip install copier cookiecutter add the extension to your template # [3] copier.yml Now to enable the extension you need to declare it in your copier.yml file in your template. _jinja_extensions: - cookiecutter.extensions.SlugifyExtension Use it | slugify # [4] use-it Now to use it, anywhere that you want to slugify a variable, you just pipe it into slugify. ❯ tree . . ├── copier.yml ├── README.md └── {{ site_name|slugify }} └── markata.toml.jinja 1 directory, 3 files Here is a slimmed down version of what the copier.yml looks like. ...

Textual has devtools in the upcoming css branch, and its pretty awesome! It’s still early # [1] Textual is still very early and not really ready for prime time, but it’s quite amazing how easy some things such as creating keybindings is. The docs are coming, but missing right now so if you want to use textual be ready for reading source code and examples. On to the devtools # [2] As [3]@willmcgugan [4] shows in this tweet it’s pretty easy to setup, it requires having two terminals open, or using tmux, and currently you have to use the css branch. https://twitter.com/willmcgugan/status/1531294412696956930 Why does textual need its own devtools # [5] Textual is a tui application framework. Unlike when you are building cli applications, when the tui takes over the terminal in full screen there is no where to print statement debug, and breakpoints don’t work. getting the css branch # [6] In the future it will likely be in main and not need this, but for now you need to get the css branch to get devtools. git clone https://github.com/Textualize/textual git fetch --alll git checkout css install in a virtual environment [7] # [8] Now you can create a virtual environment, fee...

totally guessed at this post’s date I’m still trying to understand this one, but this is how you force a python object to stop atexit. import atexit class Server: def __init__( self, auto_restart: bool = True, directory: Union[str, "Path"] = None, port: int = 8000, ): if directory is None: from markata import Markata m = Markata() directory = m.config["output_dir"] self.directory = directory self.port = find_port(port=port) self.start_server() atexit.register(self.kill) def start_server(self): import subprocess self.cmd = [ "python", "-m", "http.server", str(self.port), "--directory", self.directory, ] self.proc = subprocess.Popen( self.cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE, ) self.start_time = time.time() def kill(self): self.auto_restart = False self.proc.kill() def __rich__(self) -> Panel: if not self.proc.poll(): return Panel( f"[green]serving on port: [gold1]{self.port} [green]using pid: [gold1]{self.proc.pid} [green]uptime: [gold1]{self.uptime} [green]link: [gold1] http://localhost:{self.port}[/]", border_style="blue", title="server", ) else: if self.auto_restart: self.start_server() return Panel(f"[red]...

Whenever you are installing python packages, you should always use a virtual environment. pip makes this easy to follow by adding some configuration to pip. require-virtualenv # [1] Pip is the pacakage tool for python. It installs third-party packages and is configurable. One of the configuration settings that I highly reccommend everyone to add is require-virtualenv. This will stop pip from installing any packages if you have not activated a virtualenv. why # [2] python packages often require many different dependencies, sometimes packages are up to date and sometimes they require different versions of dependencies. If you install everything in one environment its easy to end up with version conflict issues that are really hard to resolve, especially since your system environment cannot easily be restarted. PIPX my one exception # [3] My one exception that I put in my system level packages is pipx. pipx is very handy as it manages virtual environments for you and is intended for command line utilities that would end up in your system env or require you to manually manage virtual environments without it. pip config # [4] Your pip config might be found in either ~/.pip/pi...