Posts tagged: python

I’ve been trying to adopt pyenv for a few months, but have been completely blocked by this issue on one of the main machines I use. Whenever I start up ipython I get the following error. ImportError: No module named '_sqlite3 I talked about why and how to use pyenv along with my first impressions in this post [1] pyenv/issues/678 # [2] According to #678 [3] I need to install libsqlite3-dev on ubuntu to resolve this issue. install libsqlite3-dev # [4] libsqlite3-dev can be installed using apt sudo apt install libsqlite3-dev But wait…. # [5] When I make a fresh env and install ipython I still get the same error and I am still not able to use ipython with pyenv. ImportError: No module named '_sqlite3 re-install python # [6] After having this issue for awhile an coming back to #678 [3] several times I realized that libsqlite3-dev needs to be installed while during install. pyenv install 3.8.13 I think I had tried this several times, but was missing the -y option each time. You gotta read errors like this, I am really good at glossing over them. [7] Let’s never have this issue again. # [8] When you spend months living with little errors like this and finally fix it, it...

I really like the super clean look of no status menus, no url bar, no bookmarks bar, nothing. Don’t get me wrong these things are useful, but honestly they take up screen real estate and I RARELY look at them. What I really want is a toggle hotkey. I found this one from one of DT’s youtube video’s. I can now tap xx and both the status bar at the botton and the address bar at the top disappear. # ~/.config/qutebrowser/config.py config.bind("xb", "config-cycle statusbar.show always never") config.bind("xt", "config-cycle tabs.show always never") config.bind( "xx", "config-cycle statusbar.show always never;; config-cycle tabs.show always never", )

When you first start qutebrowser It will create some config files in your home directory for you, but they will be empty. Config # [1] As far as I know qutebrowser will create this default config out of the box for you, if it doesn’t, then somehow it just appeared for me 😁. ❯ tree ~/.config/qutebrowser /home/waylon/.config/qutebrowser ├── autoconfig.yml ├── bookmarks │   └── urls ├── config.py ├── greasemonkey └── quickmarks 2 directories, 5 files Why convert # [2] You might want to confvert if you are more comfortable with the python config, or if like me you just want config in one place and you are stealing configuration options from others who have thiers in config.py. Convert to py # [3] References: [1]: #config [2]: #why-convert [3]: #convert-to-py

I am getting ready to do some timeseries analysis on a git [1] repo with python, my first step is to figure out a way to list all of the git commits so that I can analyze each one however I want. The GitPython library made this almost trivial once I realized how. from git import Repo repo = Repo('.') commits = repo.iter_commits() This returns a generator, if you are iterating over them this is likely what you want. commits # <generator object Commit._iter_from_process_or_stream at 0x7f3307584510> The generator will return git.Commit objects with lots of information about each commit such as hexsha, author, commited_datetime, gpgsig, and message. next(commits) # <git.Commit "d125317892d0fab10a36638a2d23356ba25c5621"> References: [1]: /glossary/git/

GitPython is a python api for your git [1] repos, it can be quite handy when you need to work with git from python. Use Case # [2] I recently made myself a handy tool for making screenshots in python and it need to do a git commit and push from within the script. For this I reached for GitPython. How I Quickly Capture Screenshots directly into My Blog [3] Installation # [4] GitPython is a python library hosted on pypi that we will want to install into our virtual environments using pip. pip install GitPython Create a Repo Object # [5] Import Repo from the git library and create an instance of the Repo object by giving it a path to the directory containing your .git directory. from git import Repo repo = Repo('~/git/waylonwalker.com/') Two interfaces # [6] from the docs It provides abstractions of git objects for easy access of repository data, and additionally allows you to access the git repository more directly using either a pure python implementation, or the faster, but more resource intensive git command implementation. I only needed to use the more intensive but familar to me git command implementation to get me project off the ground. There is a good tutorial [...

Python, click install Edit the System Environment Variables Environment Variables button Add the following path to your users Path Variable C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\dotnet\;C:\Users\quadm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts;

Sometimes you just want python to do something else when you hit an exception, maybe that’s fire a text, slack message, email, or system notification like I wanted. I am working on a quick and dirty python script designed to take screenshots and land them on my website in a single hotkey. With it being designed to run with a hotkey, if it were to error I would not see it. I could have gone down a logging route, but honestly this is meant to be quick, dirty, and work on my system for me. I just want to get it in my system notification. sys.excepthook # [1] Python exposes sys.excepthook for just this case. Here is what I ended up doing to fire a system notification as well as printing the message. Yaya a log would be mroe appropriate, but this is designed to just get done quick and do the job I want it to do. def notify_exception(type, value, tb): traceback_details = "\n".join(traceback.extract_tb(tb).format()) msg = f"caller: {' '.join(sys.argv)}\n{type}: {value}\n{traceback_details}" print(msg) Popen( f'notify-send "screenshot.py hit an exception" "{msg}" -a screenshot.py', shell=True, ) sys.excepthook = notify_exception 0 / 0 References: [1]: #sysexcepthook

pygame events are stored in a queue, by default the most suggested way shown in all tutorials “pumps” the queue, which removes all the messages. start up pygame # [1] You don’t necessarily need a full boilerplate [2] to start looking at events, you just just need to pygame.init() and to capture any keystrokes you need a window to capture them on, so you will need a display running. import pygame pygame.init() pygame.display.set_mode((854, 480)) get some events # [3] Let’s use pygames normal event.get method to get events. events = pygame.event.get() printing the events reveal this [ <Event(1541-JoyDeviceAdded {'device_index': 0, 'guid': '030000005e0400008e02000010010000'})>, <Event(4352-AudioDeviceAdded {'which': 0, 'iscapture': 0})>, <Event(4352-AudioDeviceAdded {'which': 1, 'iscapture': 0})>, <Event(4352-AudioDeviceAdded {'which': 2, 'iscapture': 0})>, <Event(4352-AudioDeviceAdded {'which': 0, 'iscapture': 1})>, <Event(4352-AudioDeviceAdded {'which': 1, 'iscapture': 1})>, <Event(32774-WindowShown {'window': None})>, <Event(32777-WindowMoved {'x': 535, 'y': 302, 'window': None})>, <Event(32770-VideoExpose {})>, <Event(32776-WindowExposed {'window': None})>, <Ev...

One of the most essential concepts of pygame to start making a game you will need to understand is loading images and blitting them to the screen. blit stands for block image transfer, to me it feels a lot like layering up layers/images in photoshop or Gimp. Loading an image # [1] I started by making a spotlight in Gimp, by opening a 64x64 pixel image and painting the center with a very soft brush. [2] This is what it looks like Now we can load this into pygame. import pygame img = pygame.image.load("assets/spotlight.png") Converting to the pygame colorspace # [3] To make pygame a bit more efficient we can convert the image to pygames colorspace once when we load it rather than every time we blit it onto another surface. import pygame # convert full opaque images img = pygame.image.load("assets/spotlight.png").convert() # convert pngs with transparancy img = pygame.image.load("assets/spotlight.png").convert_alpha() blitting # [4] To display the image onto the screen we need to use the blit method which needs at least two arguments, something to blit and a position to blit it at. screen = pygame.display.set_mode(self.screen_size) screen.blit( img, (0, 0),) note blit...

Dunk [1] is a beautiful git [2] diff tool built on top of rich [3]. Browsing through twitter the other day I discovered it through this tweet by _darrenburns [4]. https://twitter.com/_darrenburns/status/1510350016623394817 Dunk is beta # [5] Before I dive in deep, I do want to mention that Dunk is super new and beta at this point. I am making it my default pager, because I know what I am doing and can quickly shift back if I need to, no sweat. If you are a little less comfortable with the command line, terminal, or reading any issues that might come up, it might be best if you just pipe into Dunk when you want to use it. The author really cautions the use of it as your default pager this early, I’m just showing that it’s possible, and I’m trying it. He notes that it might have some issues especially with partially staged files. try it # [6] You can try it with pipx. git diff | pipx run dunk install it # [7] If you like it, you can install it with pip or pipx, I prefer pipx for cli applications like this. pipx install dunk set it as your default pager # [8] You can configure dunk as your default pager with the command line, or by editing your .gitconfig file. git con...

I’m poking a bit into gamedev. Partly to better understand, partly because it’s stretching different parts of my brain/skillset than writing data pipelines does, but mostly for the experience of designing them with my 9yo Wyatt. pygame boilerplates # [1] I’ve seen several pygame boilerplate templates, but they all seem to rely heavily on globl variables. That’s just not how I generally develop anything. I want a package that I can pip install, run, import, test, all the good stuff. My current starter # [2] What currently have is a single module starter package that is on github so that I can install it and start building games with very little code. Installation # [3] Since it’s a package on GitHub you can install it with the git [4]+ prefix. pip install git+https://github.com/WaylonWalker/pygame-starter Example Game # [5] You can make a quick game by inheriting from Game, and calling .run(). This example just fills the screen with an aqua color, but you can put all of your game logic in the game method. from pygame_starer import Game class MyGame(Game): def game(self): self.screen.fill((128, 255, 255)) if __name__ == "__main__": game = MyGame() game.run() The st...

My personal Site build went down last week, and I was unable to publish a new article. This is the process I went through to get it back up and running quickly. Is it a fluke? # [1] Classic IT fix, rerun it and see if you get the same error. Everyone is busy and when you have your build go down you are probably busy doing something else. My first step is often to simply click rerun right from GitHub actions. Sometimes this will fix it, and sometimes it doesn’t. It’s an easy fix to run in the meantime you are not focused on fixing it. Is GitHub having issues? # [2] Also worth a check to see if GitHub is having a hiccup or not. This error felt pretty obviously not GitHub’s fault, but it’s a good one to check when you run into a weird unexplainable error. Check github status [3] for any downtime issues with actions. Build Down # [4] Alright down to the error message I got. The error is pretty obvious that somewhere I am trying to import a non-existing module from click. Run markata build --no-pretty Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/markata", line 33, in <module> sys.exit(load_entry_point('markata==0.1.0', 'console_scripts...

I ran into a PR this week where the author was inheriting what BaseException rather than exception. I made this example to illustrate the unintended side effects that it can have. Try running these examples in a .py file for yourself and try to kill them with control-c. You cannot Keybard interrupt # [1] Since things such as KeyboardInterrupt are created as an exception that inherits from BaseException, if you except BaseException you can no longer KeyboardInterrupt. from time import sleep while True: try: sleep(30) except BaseException: # ❌ pass except from Exception or higher # [2] If you except from exception or something than inherits from it you will be better off, and avoid unintended side effects. from time import sleep while True: try: sleep(30) except Exception: # ✅ pass This goes with Custom Exceptions as well # [3] When you make custom exceptions expect that users, or your team members will want to catch them and try to handle them if they can. If you inherit from BaseException you will put them in a similar situation when they use your custom Exception. class MyFancyException(BaseException): # ❌ ... class MyFancyException(Exception): # ✅ ... Ref...

When I need a consistent key for a pythohn object I often reach for hashlib.md5 It works for me and the use cases I have. diskcache # [1] Yesterday we talked about setting up a persistant cache with python diskcache. In order to make this really work we need a good way to make consistent cache keys from some sort of python object. How I setup a sqlite cache in python [2] hash # [3] does not work My first thought was to just hash the files, this will give me a unique key for each. This will work, and give you a consistant key for one and only one given python process. If you start a new interpreter you will get different keys. waylonwalker.com on  main [$✘!?] via  v5.1.5  v3.8.0 (waylonwalker.com) ❯ ipython waylonwalker ↪main v3.8.0 ipython ❯ hash("waylonwalker") -3862245013515310359 waylonwalker ↪main v3.8.0 ipython ❯ hash("waylonwalker") -3862245013515310359 waylonwalker ↪main v3.8.0 ipython ❯ exit waylonwalker.com on  main [$✘!?] via  v5.1.5  v3.8.0 (waylonwalker.com) ❯ ipython waylonwalker ↪main v3.8.0 ipython ❯ hash("waylonwalker") -83673051278873734 here is a snapshot of my terminal proving that you can get the same hash in one session, but it changes whe...

When I need to cache some data between runs or share a cache accross multiple processes my go to library in python is diskcache. It’s built on sqlite with just enough cacheing niceties that make it very worth it. install diskcache # [1] Install diskcache into your virtual environement of choice using pip from your command line. python -m pip install diskcache setup the cache # [2] There are a couple of different types of cache, Cache, FanoutCache, and DjangoCache, you can read more about those in the docs [3] from diskcache import Cache cache = FanoutCache('.mycache', statistics=True) Adding to the cache # [4] Adding to the cache only needs a key and value. cache.add('me', 'waylonwalker' ) Set the expire time # [5] Optionally you can set the seconds before it expires. The cache invalidation tools like this is what really makes diskcache shine over using raw sqlite or any sort of static file. cache.add('me', 'waylonwalker', expire=60) tagging # [6] Diskcache supports tagging entries added to the cache. # add an item to the cache with a tag cache.add('me', 'waylonwalker', expire=60, tag='people') This seems to let you do a few new things like getting items from the cach...