GitHub Stars

Let’s make a vim command to automatically collect all the links in these posts at the end of each article. Regex confuses the heck out of me… I don’t have my regex liscense, but regex can be so darn powerful especially in an editor.

Before you run someone’s regex from the internet that you don’t fully understand, check your git status and make sure you are all clear with git before you wreck something

Something that I have always appreciated form Nick Janetakis is his links section. I often try to gather up the links at the end of my posts, but often end up not doing it or forgetting.

Searchng through the internet I was able to find an article from Vitaly Parnas called

...

I came across awesome-tuis from rothgar, and it’s packed with great features and ideas.

List of projects that provide terminal user interfaces

If you ever end up on a linux machine that just does not have enough ram to suffice what you are doing and you just need to get the job done you can give it some more swap. You can look up reccomendations for how much swap you should have this is more about just trying to get your job done when you are almost there, but running out of memory on the hardware you have.

You can put this where you wish, for this example I am going to pop it into /swap

sudo fallocate -l 4G /swap sudo chmod 600 /swap sudo mkswap /swap sudo swapon /swap

make sure that your swap is on #

You can make sure that your swap is working by using the free command, I like using the -h flag to get human readable numbers.

❯ free -h total used free shared buff/cache available Mem: 15Gi 5.5Gi 4.9Gi 458Mi 5.2Gi 9.3Gi Swap: 4.0Gi 0B 4.0Gi

Mar 18, 2022

A very common task for any script is to look for files on the system. My go to method when globbing for files in python is to use pathlib.

I setup a directory to make some examples about globbing. Here is what the directory looks like.

❯ tree . . ├── content │ ├── hello.md │ ├── hello.py │ ├── me.md │ └── you.md ├── readme.md ├── README.md ├── READMES.md └── setup.py

1 directory, 8 files

Pathlib is a standard library module available in all LTS versions of python at this point.

...

Setting up your git pager to your liking can help you navigate diffs and logs much more efficiently. You can set it to whatever pager you like so that your keys feel nice and smooth and your fingers know exactly what to do. You might even gain a few extra features.

You can set the pager right from your command line with the following command.

git config --global core.pager 'more'

You can also set your pager by editing your global .gitconfig file which by default is set to ~/.gitconfig.

[core] pager = more

Color #

In my experience you need to turn colors off with nvim. bat handles them and looks good either way, but nvim will be plain white and display the color codes as plain text if color is on.

...

If you have ever mistyped a git command very close to an existing one you have likely seen this message.

❯ git chekout dev git: 'chekout' is not a git command. See 'git --help'. The most similar command is checkout

Automatically run the right one #

What you might not have known is that you can configure git to just run this command for you.

# Gives you 0.1 seconds to respond git config --global help.autocorrect 1 # Gives you 1 seconds to respond git config --global help.autocorrect 10 # Gives you 5 seconds to respond git config --global help.autocorrect 50

Fat Fingers Gone #

Now when you typo a git command it will autmatically run after the configured number of tenths of a second.

...

yq is a command line utility for parsing and querying yaml, like jq does for json.

I love that all of these modern tools built in go and rust, just give you a zipped up executable right from GitHub releases, but it’s not necessarily straight forward how to install them. yq does one of the best jobs I have seen, giving you instructions on how to get a specific version and install it.

I use a bunch of these tools, and for what its worth I trust the devs behind them to make sure they don’t break. This so far has worked out well for me, but if it ever doesn’t I can always pick an older version.

Since I am all trusting of them I just want the latest version. I do not want to update a shell script with new versions, or even care about what then next version is, I just want it. Luckily you can script the release page for the latest version on all that I have came accross.

...

Last Thursday I learned about pytest-mock at a local python meetup. The presenter showed how he uses pytest-mock for his work, and it was kinda eye opening. I knew what mocking was, but I had not seen it in this context.

Watching him use pytest-mock I realized that mocking was not as hard as I had made it out to be. You can install pytest-mock, use the mocker fixture, and patch objects methods with what you want them to be.

pytest-mock is out on pypi and can be installed with pip.

python -m pip install pytest-mock

What I actually did #

Sometimes I fall victim to making these posts nice and easy to follow. It takes more steps than just pip install, you need a place to practice in a nice sandbox. Here is how I make my sandboxes.

...

Python 3.8 came out two and a half years ago and I have yet to really lean in on the walrus operator. Partly because it always seemed like something kinda silly (my use cases) to require a python version bump for, and partly because I really didn’t understand it the best. Primarily I have wanted to use it in comprehensions, but I did not really understand how.

Now that Python 3.6 is end of life, and most folks are using at least 3.8 it seems time to learn and use it.

:=

The assignment operator in python is more commonly referred to as the walrus operator due to how := looks like a walrus. It allows you to assign and use a variable in a single expression.

...

I’m impressed by markdown-styles from mixu.

Markdown to static HTML generator and multiple CSS themes for Markdown

Kedro rich is a very new and unstable (it’s good, just not ready) plugin for kedro to make the command line prettier.

There is no pypi package yet, but it’s on github. You can pip install it with the git url.

pip install git+https://github.com/datajoely/kedro-rich

Kedro run #

You can run your pipeline just as you normally would, except you get progress bars and pretty prints.

kedro run

Kedro catalog #

Listing out catalog entries from the command line now print out a nice pretty table.

...

So worktrees, I always thought they were a big scary things. Turns out they are much simpler than I thought.

no special setup

I thought you had to be all in or worktrees or normal git, but not both. When I see folks go all in on worktrees they start with a bare repo, while its true this is the way you go all in, its not true that this is required.

Making a worktree is as easy as making a branch. It’s actually just a branch that lives in another place in your filesystem.

...

nicoddemus has done a fantastic job with pytest-rich. Highly recommend taking a look.

pytest + rich integration (proof of concept)

Has no upstream branch errors in git can be such a damn productivity killer. You gotta stop your flow and swap over the branch, there is a config so that you don’t have to do this.

If you have not yet configured git to always push to the current branch, you will get a has no upstream branch error if you don’t explicitly set it.

Let’s show an example

git checkout -b feat/ingest-inventory-data git add conf/base/catalog.yml git commit -m "feat: ingest inventory data from abc-db" git push

You will be presented with the following error.

...

I write many of these posts from a 10 year old desktop that sits in my office these days. It does a very fine job running all of the things I need it to for my side work, but sometimes I want a mobile setup. I don’t really want to spend the $$ on a new laptop just for the few times I want to be somewhere else in the house. What I do have though is a chromebook.

I’ve tried to get the chromebook into my workflow in the past, but have failed. Much because by the time I got all of my tools up and running in the linux vm it was taking up quite a bit of space on the device and made it harder for others to use as a chromebook.

Today I am giving it a second try, but this time with ssh.

Before doing anything I checked to see if sshd is already running. Using the following command.

...

patrick-kidger has done a fantastic job with mkposters. Highly recommend taking a look.

Make posters from Markdown files.

Mermaid gives us a way to style nodes through the use of css, but rather than using normal css selectors we need to use style <nodeid>. This also applies to subgraphs, and we can use the name of the subgraph in place of the nodeid.

graph TD; a --> A A --> B B --> C style A fill:#f9f,stroke:#333,stroke-width:4px style B fill:#f9f,stroke:#333,stroke-width:4px subgraph one a end style one fill:#BADA55

produces the following graph

style one fill:#BADA55

I recently found a really great plugin by mhinz to open files in neovim from a different tmux split, without touching neovim at all.

neovim-remote is not a neovim plugin at all, it’s a python cli that you can install with pip. Unlike the repo suggests, I use pipx to install nvr.

pipx install neovim-remote

How I use it #

I have this added to my .envrc that is in every one of my projects....

...

Mermaid provides some really great ways to group or fence in parts of your graphs through the use of subgraphs.

Here we can model some sort of data ingest with some raw iot device and our warehouse in different groups.

graph TD; subgraph raw_iot a end subgraph warehouse A --> B B --> C end graph TD;

subgraph raw_iot a end subgraph warehouse A --> B B --> C end 

connecting subgroups #

If we want to connect them, we can make a connection between a and A outside of the subgraphs.

graph TD; subgraph raw_iot a end a --> A subgraph warehouse A --> B B --> C end graph TD;

subgraph raw_iot a end a --> A subgraph warehouse A --> B B --> C end 

separation of concerns #

It’s also possible to specify subgraphs separate from where you...

I’m impressed by pype.dev from pypeaday.

my blog?