All posts with the tag "python"
Here are some CFP’s that I used for PyOhio 2022.
https://pretalx.com/pyohio-2022/cfp
Markata is a plugins all the way down static site generator, that covers all the things you need to go from markdown to a blog site out of the box. Since it’s plugins all the way down you can also rip out all the default plugins, and do something completely different with the lifecycle.
...
Sometimes you just want python to do something else when you hit an exception, maybe that’s fire a text, slack message, email, or system notification like I wanted.
I am working on a quick and dirty python script designed to take screenshots and land them on my website in a single hotkey. With it being designed to run with a hotkey, if it were to error I would not see it.
I could have gone down a logging route, but honestly this is meant to be quick, dirty, and work on my system for me. I just want to get it in my system notification.
Python exposes sys.excepthook for just this case. Here is what I ended up doing to fire a system notification as well as printing the message. Yaya a log would be mroe appropriate, but this is designed to just get done quick and do the job I want it to do.
pygame events are stored in a queue, by default the most suggested way shown in all tutorials “pumps” the queue, which removes all the messages.
You don’t necessarily need a full boilerplate to start looking at events, you just just need to pygame.init() and to capture any keystrokes you need a window to capture them on, so you will need a display running.
import pygame pygame.init() pygame.display.set_mode((854, 480))
Let’s use pygames normal event.get method to get events.
events = pygame.event.get()
printing the events reveal this
...
One of the most essential concepts of pygame to start making a game you will need to understand is loading images and blitting them to the screen.
blit stands for block image transfer, to me it feels a lot like layering up layers/images in photoshop or Gimp.
I started by making a spotlight in Gimp, by opening a 64x64 pixel image and painting the center with a very soft brush.
This is what it looks like
...
Dunk is a beautiful git diff tool built on top of rich.
Browsing through twitter the other day I discovered it through this tweet by _darrenburns.
https://twitter.com/_darrenburns/status/1510350016623394817
Before I dive in deep, I do want to mention that Dunk is super new and beta at this point. I am making it my default pager, because I know what I am doing and can quickly shift back...
...
I’m poking a bit into gamedev. Partly to better understand, partly because it’s stretching different parts of my brain/skillset than writing data pipelines does, but mostly for the experience of designing them with my 9yo Wyatt.
I’ve seen several pygame boilerplate templates, but they all seem to rely heavily on globl variables. That’s just not how I generally develop anything. I want a package that I can pip install, run, import, test, all the good stuff.
What currently have is a single module starter package that is on github so that I can install it and start building games with very little code.
Since it’s a package on GitHub you can install it with the git+ prefix.
...
My personal Site build went down last week, and I was unable to publish a new article. This is the process I went through to get it back up and running quickly.
Classic IT fix, rerun it and see if you get the same error. Everyone is busy and when you have your build go down you are probably busy doing something else. My first step is often to simply click rerun right from GitHub actions. Sometimes this will fix it, and sometimes it doesn’t. It’s an easy fix to run in the meantime you are not focused on fixing it.
Also worth a check to see if GitHub is having a hiccup or not. This error felt pretty obviously not GitHub’s fault, but it’s a good one to check when you run into a weird unexplainable error.
Check github status for any downtime issues with actions.
...
I ran into a PR this week where the author was inheriting what BaseException rather than exception. I made this example to illustrate the unintended side effects that it can have.
Try running these examples in a .py file for yourself and try to kill them with control-c.
Since things such as KeyboardInterrupt are created as an exception that inherits from BaseException, if you except BaseException you can no longer KeyboardInterrupt.
from time import sleep while True: try: sleep(30) except BaseException: # ❌ pass
If you except from exception or something than inherits from it you will be better off, and avoid unintended side effects.
...
When I need a consistent key for a pythohn object I often reach for hashlib.md5 It works for me and the use cases I have.
Yesterday we talked about setting up a persistant cache with python diskcache. In order to make this really work we need a good way to make consistent cache keys from some sort of python object.
How I setup a sqlite cache in python
does not work
...
When I need to cache some data between runs or share a cache accross multiple processes my go to library in python is diskcache. It’s built on sqlite with just enough cacheing niceties that make it very worth it.
Install diskcache into your virtual environement of choice using pip from your command line.
python -m pip install diskcache
There are a couple of different types of cache, Cache, FanoutCache, and DjangoCache, you can read more about those in the docs
from diskcache import Cache cache = FanoutCache('.mycache', statistics=True)
Adding to the cache only needs a key and value.
...
The easiest way to speed up any code is to run less code. A common technique to reduce the amount of repative work is to implement a cache such that the next time you need the same work done, you don’t need to recompute anything you can simply retrieve it from a cache.
The easiest and most common to setup in python is a builtin functools.lru_cache.
from functools import lru_cache @lru_cache def get_cars(): print('pulling cars data') return pd.read_csv("https://waylonwalker.com/cars.csv", storage_options = {'User-Agent': 'Mozilla/5.0'})
Any time you have a function where you expect the same results each time a function is called with the same inputs, you can use lru_cache.
when same *args, **kwargs always return the same value
...
I keep a small cars.csv on my website for quickly trying out different pandas operations. It’s very handy to keep around to help what a method you are unfamiliar with does, or give a teammate an example they can replicate.
I recently switched hosting from netlify over to cloudflare. Well cloudflare does some work to block certain requests that it does not think is a real user. One of these checks is to ensure there is a real user agent on the request.
This breaks my go to example dataset.
pd.read_csv("https://waylonwalker.com/cars.csv") # HTTPError: HTTP Error 403: Forbidden
What’s weird is, requests still works just fine! Not sure why using urllib the way pandas does breaks the request, but it does.
...
Python’s requests library is one of the gold standard apis, designed by Kenneth Reitz. It was designed with the user perspective in mind first and implementation second. I have heard this called readme driven development, where the interface the user will use is laid out first, then implemented. This makes the library much mor intuitive than if it were designed around how it was easiest to implement.
Requests is on pypi and can be installed into your virtual environtment with pip.
python -m pip install requests
Requests makes getting content from a web url as easy as possible.
import requests r = requests.get('https://waylonwalker.com/til/htmx-get/') article = r.content
...
I recently gave a talk at python web conf 2022, and one of the things I did when I should have been working on my presentation was workig on how my presentation looked… classic procrastination technique.
Lets use this section to show what it looks like as I change my styles.
from markata import Markata Markata() markata.run()
☝ This is how my website is built
This is what the above slide looks like in lookatme.
...
I use a package eyeseast/python-frontmatter{.hoverlink} to load files with frontmatter in them. Its a handy package that allows you to load files with structured frontmatter (yaml, json, or toml).
It’s on pypi, so you can install it into your virtual environment with pip.
python -m pip install python-frontmatter
Frontmatter is a handy way to add metadata to your plain text files. It’s quite common to have yaml frontmatter in markdown. All of my blog posts have yaml frontmatter to give the post metadata such as post date, tags, title, and template. dev.to is a popular developer blogging...
...
Today I was watching the python web conf 2022 and saw @davidbujic use the new Dict Union Operator Live on stage during his Functional Programming talk. This operator was first introduced into python 3.9 with pep584.
I’ve long updated dicts through the use of unpacking. Note that the last item always wins. It makes it pretty easy to make user overrides to default configurations. With pep584 landing in python 3.9 we can now leverage the | operator to achieve the same result.
default_config = {'url': 'https://example.com', 'assets_dir': 'static' } user_config = {'url':...
...
I love the freedom of writing in markdown. It allows me to write content from the comfort of my editor with very little focus on page style. It turns out that markdown is also a fantastic tool for creating slides.
I will most often just present right from the terminal using lookatme. Presenting from the terminal lets me see the results quick right from where I am editing. It also allows me to pop into other terminal applications quickly.
I sometimes also use reveal.js, but that’s for another post. It is handy that it lives in the browser and is easier to share.
I leverage auto slides when I write my slides in markdown. The largest heading, usually an h2 for me, becomes the new slide marker. Otherwise my process is not much different, It just becomes a shorter writing style.
...
When I need to read contents from a plain text file in python I find the easiest way is to just use Pathlib.
A very common task for any script is to look for files on the system. My go to method when globbing for files in python is to use pathlib.
I setup a directory to make some examples about globbing. Here is what the directory looks like.
❯ tree . . ├── content │ ├── hello.md │ ├── hello.py │ ├── me.md │ └── you.md ├── readme.md ├── README.md ├── READMES.md └── setup.py
1 directory, 8 files
Pathlib is a standard library module available in all LTS versions of python at this point.
...
Last Thursday I learned about pytest-mock at a local python meetup. The presenter showed how he uses pytest-mock for his work, and it was kinda eye opening. I knew what mocking was, but I had not seen it in this context.
Watching him use pytest-mock I realized that mocking was not as hard as I had made it out to be. You can install pytest-mock, use the mocker fixture, and patch objects methods with what you want them to be.
pytest-mock is out on pypi and can be installed with pip.
python -m pip install pytest-mock
Sometimes I fall victim to making these posts nice and easy to follow. It takes more steps than just pip install, you need a place to practice in a nice sandbox. Here is how I make my sandboxes.
...