All Posts

3d Printing Dovetails Experiment

I hit an issue with 3d printing oversized parts that I have not hit before. I'm working on some jigs for an upcoming woodworking project that will involve a lot of repetition. We want to utilize some dowel joinery and jigs for consistency. These parts will be up to 20in in length this is much larger than my print bed. I've fit things together before Here's where I went wrong, I wasn't really thinking through my previous applications. They've all been slip fit, primarily print in place joints

python extras are for shipping

Python has two ways of adding optional dependencies to your projects pyproject.toml file and . dependency-groups for development Dependency grooups are used when working on the project, they do not ship with the project, users cannot select to install them with the project. These are for things like running tests, linting, or docs. You might want to run these in ci, or keep your dev machines tight. For the most part you can probably keep these in . Depending on your team, fluency, and to

Shots Get An Upgrade To the Main Feed

Today I updated my jinja templates and feeds to include shots posts in the main feed. These posts will prominently show the image in the feed with a small post for each one. screenshot-2025-09-21T01-19-39-934Z.png Shots are instagram style imge posts inspired by https://justin.searls.co/shots/.

zmk two hand hold down

I don't know about you, but I don't re-flash my keyboard enough to ever remember where I put the bootloader. Sometimes its the last thing I think about in a refactor and I end up cornering myself into a place where I cant get into that layer anymore. I've started putting hardware switches on my newer builds, but some older builds don't have a hardware one, so it requires disassembly and jumping the microcontroller. Even when I have one though, I gotta flip my board over and its annoying some

Why Make a Website in 2025

Inspired by Jim and Dave It's Fun If it's not something that you enjoy, you might as well move on there are far better ways to spend your time in 2025. Only the weirdos read this shit anymore, the masses have long moved on to curated social media feeds, and on to chatting with llms. I enjoy spending some time in the digital garden every once in awhile tweaking templates, creating [[ markata ]] plugins to do interesting things. I enjoy expressing my thoughts, or at least shouting them into th

store

I've been 3d printing for years, and create a lot of my own designs. Many of them are hyper specific to me, but for the ones that I think others might find useful I will be putting up here for sale, you can buy prints that will show up to your door in a few days. I'm leaning on the slant3d print farm so this depends on your proximity to them. The prints are typically black petg, if you would like a different color reach out to me and I will see what other options we have, or for an additional

When To Vibe Code

I enjoyed this post from Theo and think it deserves re-iterated, revisited, and to remind myself of some of these things. https://youtu.be/6TMPWvPG5GA?si=guQem4R8dLOMBntP&t=1356 The skill/read spectrum screenshot-2025-09-03T00-32-08-321Z.png The first diagram describes that there has become a spectrum of agentic coding from vibe coding where you don't ready anything, to looking at everything in detail, across a group of people who don't have a clue what the code says to people who could do it wa

Keycap Cut Away

I was curious how/if my custom keycap design was hitting my switches. So I set out to find out what the fitup inside of this assembly looks like, but not theoretically, a fully sliced view into their fit up in the flesh. The set up To setup for this cut, I flooded the edge of a 2x4 with hot glue, and inserted the cap such that the step was tangent with the edge. This way I could use the edge as a guide to cut one side off and leave the stem in tact. I took a handsaw to it and filed it smooth.

I'm Out On Agents

Its the year 2025 and we are only a few years into having 6 months to live before ai takes our jobs, and the big push right now is agents, managing agents. I will fully concede to I'm not doing it right, or a future state gets better than where we are right now, but right now they kinda suck. !!! Note Transparency Chat Chat is what really kicked off ai uses and goes back as old as computers, but it always sucked. Then chatgpt rocked the world with the biggest launch day in history and showed u

trying forgejo

WIP

blender donut tutorial 2025

First Launch screenshot-2025-07-25T13-29-06-029Z.png my setup screenshot-2025-07-25T13-30-22-340Z.png Add shift A bump resolution scale edit -> preferences -> interface -> resolution scale 1.25 screenshot-2025-07-25T13-35-20-580Z.png orbit without middle mouse button screenshot-2025-07-25T13-36-32-690Z.png viewport shading rendered screenshot-2025-07-25T13-37-52-174Z.png

command palettes are overrated

Command palettes are slow, and overrated, you should treat yourself better. You probably installed VSC*** out of the box and your co-workers see you using the mouse and reprimanded you as they should. Mouse usage is not OK if you are a software dev, you should have the cheap ass free mouse that came with your cousins dell machine five years ago and only use if for emergencies. If you want to be fast you cannot do that by moving cursors to imprecise locations and clicking with your hand. You a

just fucking use kubernetes

You want to run containers? JUST FUCKING USE KUBERNETES. !!! tip "Obvious satire" If you don't like harsh language this is not the post for you. Obviously ripping off motherfuckingwebsite . !!! warning "ThIs is AI SLoP" If you don't like if you can fuck off to the next post, I'm having fun here, but satire is not my strong suit and needed some help. "But it’s complicated!" Shut up. Close twitter and fucking do something. Life is complicated. You know what else is complicated? Email. DNS. L

markata parallel render

principal-engineer-at-meta

Jake Bolam principal engineer at Meta, has some of the best career advice for those looking to become principal or just be better at their craft. This video was such a banger I had to bring it in as a full post, and not just a thought. It was a random YouTube auto play, something that I probably wouldn't have clicked on given title an thumbnail, but turned out to be very impactful. Jake is such a smart guy with a lot of great insights, and I can tell he thinks really quick on his feet, he jus

csv

perfect

Perfect is a made up word that humans use to describe something that is above average, or works really well for them. The idea of perfection is fleeting, as you think more deeply about something, you can continue to chase the idea of perfection to unimaginable senses. Sometimes perfect simply means good enough . Could there be something better, Always , but at what cost. If I spent 10 more minutes on this post would it be better, maybe, but I might fuck it up. If I spent my lifetime stu

just

I just never quite understood why the word can send people over the top. I get it when you don't know someone, you don't have history with them, and they come in saying you are doing something wrong. When you say "just," you're skipping over all the invisible complexity. You're assuming the problem is simple, and that the person asking for help hasn't already considered the obvious. You're not seeing the constraints: Legacy code Business requirements Team conventions Time, budget, or technical

kraft

Kraft is the family gaming server that we started early 2025. Repopulating Villagers are really hard to get gong. We have this huge villager tower on the server, we nearly every bed filled, and one day we logged in and there were three. Not sure what happened until I witnessed one of them jump maybe 3 blocks down and off himself. I checked my last two villagers and they were each on one or two hearts left. We almost lost every single villager on the server. villager-tower-needs-repopulated.m

my nvim spellcheck setup

I've gone too long without a proper spellcheck setup in nvim. I know it's there, I just don't use it, I don't have the right keybinds, like I do with [[vim-date]], to make it work, and its clunky. Default keybinds z= show spell suggestions zg add word to dictionary zw remove word from dictionary ]s jump to next misspelled word [s jump to previous misspelled word I really struggle with bracketed keybinds, they don't flow for me. I have to shift into it and hit two keys, you cant just pop throug

wants

Inspired by mara.town/wants want but do not enjoy New hardware that cannot be repaired Disciplining Children Nice landscaping which requires regular maintenance enjoy but do not want breakfast Fancy things Manual Labor in moderation Vacations to far away places

Linux Is About Choice

This Luke Smith video came across my feed Linux, Bitcoin: When Tech Projects Become "Too Popular..." Don't forget the goal. . It's interesting to hear his perspective about Linux, FOSS, Free Software being the end goal, and that we are loosing sight of the goal. This sentiment really aligns with the early FOSS movement from Stallman, but was this ever the goal? Taken over by apathetic interests Luke talks about these projects getting taken over by people with no passion for the original goal

tinyapps

I'm working on replacing my usage of google inline search apps with real apps, these are ones that I create and host on my own homelab. The first three that I created are mostly chatgpt based, with a bit of hand edit after the fact, uploaded to minio and become an app on my k8s-pages !!! Note renamed The original title of this post was "Replacing Google Search Apps With Self Hosted Web Apps" I'm leaning on [[web-wakelock]] to keep the screen on while these apps are running, primarily clos, time

m9a devlog 1

It's sad to see textualize.io close the doors, but textual is still alive and maintained as a n open source project. I tried to use it very early, and struggled, this was before docs and tutorials really existed, before a lot of the widgets and components existed. Then as we all do I got busy and moved on to other things in life and did not have the capacity to build TUIs. I like tuis I like tuis, I like staying in the terminal. I use lf daily to move files around when I want something more

hollow knight home row layout

I just made it past 100% in my main hollow knight run, so now I will allow myself to get silksong when it comes out. I did this with a little bit of YT guidance, but mostly just figuring it out. I only just discovered the [[reznormichael-hollow-knight-completion-check]] which got me an extra 2% for a few items I must have got and not saved on, because I was sure I had them. Controller Hollow Knight is a game that can be played with keyboard or controller, You can use analog stick for movements

fix feed descriptions

Today I fixed a bug in markata that has been occurring for a few months where the description for posts come out as None if coming from cache, the issue was a pretty simple check and pull properly from cache. This fixes all the descriptions in feeds and metadata on the post. Better description While in there we went ahead and improved our get_description to more accurately return plain text without escaped characters, remove cutoff words, and add an elipsis if the description cuts off the text.

portal-platformer-devlog-1

Here is the current state of my platformer yet to really be named, I want to make something in between hollow knight and portal. Starting I made one once in make code arcace on a pybadge. It was quick and dirty, but fun to work on. It had the basic of blocks that I could move, blocks i could put a portal onto, and a goal for each level. Some levels you can just walk through and some levels required you to really think about where to place the portal. History So this version of the game is a l

backup distrobox image

Today I'm upgrading my distrobox, but don't want to end up in a situation where I can't get anything done becauase I work out of my distrobox.

Changing k8s Storage Class - Migration Job

I'm setting up longhorn in my homelab, and I ran into an issue where I initially setup some pvcs under longhorn, and later realized that to get longhorn to snapshot and backup I needed to hand edit volumes after the fact or change storage class. I'm all in on gitops so option 1 was not an option. So changing storageclass it is. Now the issue is that you CANNOT mutate storageclass on a provisioned pvc, it is an immutable attribute. Migration Job This migration job will create a new pvc with the

homelab drive ids

Drive Bay 1 ata-ST4000VN008-2DR166_ZDHBZSWZ +-------------------------------------------------------------------------+ | [ Power] [ Reset ] | +-------------------------------------------------------------------------+ | [ BAY 5 ] 3TB WD30EFRX WMC4N0D3J9R7 ext4 /mnt/sdf4 | +-------------------------------------------------------------------------+ | [ BAY 4 ] 14TB EXOS ZTM09R9N zfs main pool mirror /mnt/main |

THPS 1 Downtown Gaps

screenshot-2025-02-28T00-44-32-696Z.png screenshot-2025-02-28T00-45-05-468Z.png Previously [[ thps-1-skate-park-gaps ]] Big Ass THPS1-5-BigAss.mp4 Billboard Grind THPS1-5-BillboardGrind.mp4 BS Gap THPS1-5-BSGap.mp4 BS Grind THPS1-5-BSGrind.mp4 Burly Deck Gap THPS1-5-BurlyDeckGap.mp4 Car Ollie THPS1-5-CarOllie.mp4 THPS1-5-CarPlant.mp4 Cheesy Deck Gap THPS1-5-CheesyDeckGap.mp4 Death Grind THPS1-5-DeathGrind.mp4 Deck Gap THPS1-5-DeckGap.mp4 Dirty Rail THPS1-5-DirtyRail.mp4 Glass Gap THPS1-5-GlassGa

THPS 1 Skate Park Gaps

image Previously [[ thps-1-mall-gaps ]] Next [[ thps-1-downtown-gaps ]] Acid Drop THPS1-4-AcidDrop.mp4 Across The Park THPS1-4-AcrossThePark.mp4 Around The Bend THPS1-4-AroundTheBend.mp4 HP Transfer THPS1-4-HPTransfer.mp4 Light Grind THPS1-4-LightGrind.mp4 Over The Box THPS1-4-OverTheBox.mp4 Over The Pipe THPS1-4-OverThePipe.mp4 Over the Rafters THPS1-4-OverTheRafters.mp4 Pool 2 Walkway THPS1-4-Pool2Walkway.mp4 Pool Hip THPS1-4-PoolHip.mp4 Pool Rail Trans THPS1-4-PoolRailTrans.mp4 Rafter Rail TH

Trailmakers Pioneers

Trailmakers 2.0 Pioneers was released today as an update to trailmakers. It's a brand new campaign mode that requires you to mine resources to get materials to craft your vehicles rather than salvage parts like you so in the Stranded Campaign. There also seems to be a more linear play through of the story with the frogitts, unlike stranded where it let you do anything completely open world, as long as you had the equipment to go there. Four Frogitts Out of the gate Wyatt came in with four frog

THPS 1 School Gaps

On the road to Gap Master achievement I got all School Gaps in THPS1. This is a super fun level, I'll have to admit that I had to look up at least a quarter of them, and could not find them myself, mostly due to the naming, all of the rail tricks were actually ledges and I was looking for big rails. None of the warehouse gaps were manuals so I had no idea that the downhill and accross the gym would be manual tricks. Previously [[ thps-1-warehouse-gaps ]] next [[ thps-1-mall-gaps ]] screenshot

THPS 1 Mall Gaps

This is one of the most memorable levels for me in THPS1 from when I had originally played it. Surprisingly there aren't as many gaps in it as I thought there would be, there are definitely a lot that you can make up on your own. Anyways here's my playthrough of all the mall gaps in THPS1 on my way to gap master. Previously [[ thps-1-school-gaps ]] Next [[ thps-1-skate-park-gaps ]] image Huge Stair Set Off A Mezzanine THPS1-3-HugeStairSetOffAMezzanine.mp4 Coffee Grind THPS1-3-CoffeeGrind.mp4 T

THPS 1 Warehouse Gaps

On the road to Gap Master achievement I got all Warehouse Gaps in THPS1. This is this classic warehouse level that everyone thinks of when THPS is mentioned. Such a good level, compact, easy to play, with some fun gaps that might take a few tries to hit. Next up [[ thps-1-school-gaps ]] image Big Rail THPS1-1-BigRail.mp4 Channel Gap THPS1-1-ChannelGap.mp4 Deck 2 Rail THPS1-1-Deck2Rail.mp4 Face Plant THPS1-1-FacePlant.mp4 High Rail THPS1-1-HighRail.mp4 Holy Shi THPS1-1-HolyShi.mp4 Kicker 2 Led

fragmention

This post is still WIP. ..... https://indieweb.org/fragmention##Challenges I've been digging through David Bushell's blog over the past day, he has some really good ideas about blogging and webdev. One really interesting post I came accross is url-fragment-text-directives . I've long had id's linked on my headings, though sometimes broken, or now showing the link, I've done my best to include them. Fragmentions extend this to allow any text to be linkable like this. fragmentioner ui: https://

fixed long standing nvim startup error

Here's the diff, this is it. The error On first install of my dotfiles I'm presenting with this flashbang of an error filling the screen with red background. Its kinda hard to read, I'm not deep into lua and reading their tracebacks. It pops up in this pager that if I scroll too far it quits and the error is gone before I know what it is or how it got there. image For the longest time it just felt like it randomly showed up without much warning. I sent ai at the issue I tried some chatgpt and

Setting up 4G Backup with Google Fi and Netgear LM...

I connected my home network to have 4G bakcup with Google Fi and Netgear LM1200. Goodle Fi offers free data-only sim cards that you can order from their service. It takes a couple of days, and a new sim arrives in the mail free of charge. It does pull data from your account, so if you are not on an unlimited plan be careful of how much you let go through the sim. I've owned this for a few years now, but it's been disconnected for a good six months or so. I'm not s[e what happened, but it sto

ahrefs-cleanup-2024

This post is a big work in progress, expect it to keep getting better. Initial Score image 404 Not Found, generate a page for each tag image Title too long image 404 Not Found, comma separated tags Another hit on 404's caused by tags, was tag parsing from thoughts into posts, this cause links to the full comma separated list of tags rather than one per tag. image You can see on the website the whole dang set of tags was being treated as a single tag. image Broken images I'm not fully sure what h

dst session one

Wyatt is also documenting his run, while we aren't playing in the same world we are playing together. https://wyattbubbylee.com/dst-forever-world/ Loading into the world I always select the character Woodie . He comes with an axe called Lucy she has some dark humor (but not too dark), I like her. She also does not wear out so you always have a weapon and way to chop wood with you. dst-1-1.mp4 First encounter with a Tall Bird On the first day I made a look around my home biome, and ran into

markdown split panel

Today I was playing with markdown split panels. I want to be able to compare and constrast occasionually, today the inspiration hit to do this using admonitions. screenshot-2025-02-04T02-28-46-750Z.png {.more-cinematic} {.rounded-xl } !!! Note Mobile Users 🔄 The Markdown This is what I am going for, one admonition that is easy to remember, that nests inside of itself , and I can put as much markdown on the inside that I want. Here is the result of that markdown. !!! vsplit I Have two opinions {

Make MinIO Access Key

Today in my homelab I wanted to setup a new service that needed a MinIO access key. So I created a new user and a new access key with the MinIO CLI rather than poking through the ui like I have before. screenshot-2025-02-03T02-13-38-628Z.png {.more-cinematic} Global Level vs User Level The MinIO CLI has two levels of access, global and user level. Most of the commands in this post will have several ways to do similar tasks that would potentially work. We are going to prefer to use the user leve

markta 0.8.0

I realize that I never did a post on markata 0.8.0, so here it is. 0.8.0 was released on Jan 2, 2024, just over a year ago at this point. This was the release that we got pydantic support, and multi feeds. screenshot-2025-02-01T02-37-59-690Z.png {.more-cinematic} Pydantic Support Was kinda big Markata is leaning on pydanttic for configuration and Post models. These models are filled with validators such that you can give it an empty markdown post and it will figure out some pretty sane defaul

markata search

A side effect of [[ markata-didyoumean ]] is that we are able to implement some rudimentary search based on slug, title, tags, and description. screenshot-2025-01-31T14-50-00-094Z.png {.more-cinematic} I made a video of the early work on using this side effect to search for content on markata.dev. Replay markata-search-1.mp4 This was first implemented to solve 404 pages giving users a way to try to find the page that they tried to access. But it turned out to be a decent way to search through t

Markata DidYouMean

Coming in Markata 0.9.1 is far better documentation. i.e. Documentation that actually exists for everything. As part of poking around I realized that I often go to look up the docs for a plugin and forget that the path is , sometimes I might try or or or I might even forget the plugin name exactly and try something like and get a 404. So I added a plugin to markata that takes care of this. screenshot-2025-01-31T14-53-31-264Z.png {.more-cinematic} I made a quick recording of this earl

cold builds

Here are a list of some cold builds from my site. I've ran this site for a long time and would like to have some references to go back to, and wish I had kept a few profiles of cold builds laying around to compare with. The time is now lets keep some cold build links around for reference. screenshot-2025-01-31T21-23-56-725Z.png {.more-cinematic} 01/27/2025 https://ec314b08.waylonwalker-com.pages.dev/_profile/ - 429s - markata==0.9.0.dev5 has md_video open cache on every post issue https://27f1

pesos

Pesos is the act of Publish Elsewhere Syncicate to Own Site. It is an indieweb concept that I recently started applying to my own site. here does it skip again !!! note See https://indieweb.org/PESOS for more information, they have a ton of information about the indieweb In short it is the concept of pulling data from other sites that you use and republishing it to your own site. This gives a single source of information for you, and protection against sites and apis changing or rug pulling.

hover z-index and positioning

I broke my [[ sick-wikilink-hover ]] recently in a refactor, today I did some diving in to figure out what happened. Before As you can see in the screenshot below, the link is in a list of links, and when the hover image pops up it sits behind all of the other text. The z-index of the list-item is supposed to be raised above the others on hover. image Manually setting z-index to 20 in the inspector I noticed this message from devtools, "The position: static property prevents z-index from havin

analytics

I've been posting on this site since 2016, when there was a huge rollout of layoffs at the company I was working at at the time. In fear of being laid off personal blog and side projects was at the top of the suggestions so off I I started looking at what I needed to do to start building my resume, having a went. This site is built on [[ markata ]], more about that in the [[ colophon ]]. screenshot-2025-01-31T21-23-13-643Z.png Total Posts over time Total Cumulative Posts Over Time Post Contribu

I fixed lists in my blog

A small improvement to my blog today. I was checking out my [[ now ]] post and noticed that lists have left padding but no discs or numbers. I popped open my added these lines and rebuilt. And rebuild, which I actually do with [[ casey-just ]], but showing the command here for clarity. WHY the applies {.bg-blue-900 .border-r-8 .border-black .rounded-xl .max-w-xl .p-6 .font-bold} Dude, you know that @apply is an anti-pattern in tailwind right? {.bg-green-900 .border-l-8 .border-black .rounded-x

a night of rounds

Tonight my son and I played rounds, I pulled clips of all the final rounds of each set of rounds. We play hardly any games with guns in them, this is one of the few exceptions, it feels quite whimsical, fun, and strategic, and not at all realistic or first person. The strategy part really brings me back to the fps games I would play with friends in my teens. Round 1 - Orange I thought I had him on this one. I had this rare power up that allows me to shot through walls. It came down to the ve

wondercraft session one

My son and I are starting a new Minecraft server today called wondercraft. setup We are running a few mods on this server notably FTB Ultimine for less grindy resource gathering, Sodium for shaders, and Universal Shops. image We are runninng on Minecraft 1.20.1, I think this is because it was the latest supported by Universal Shops, its at least one major release old at this point. image I grabbed a few shaders, the ones that you see in this post are rethinking voxels. image spawn We initially

/now

This page is meant to be simple view into what is happening in my life right now. Inspired by @NowNowNow . and Derek Sivers. Last Update 2025-05-22 14:27:47 Working on [[ninesui]] Working on [[krayt]] learning observability with grafana Fairly happy with [[markata]], at the moment, less focus and some small fixes The blog is fully k8s, love the experience replacing Google apps with self hosted apps [[replacing-google-search-apps-with-self-hosted-web-apps]] Doing a second run through [[hollow-kn

one-shot-s3-cleanup

So I made a mistake in an app I am working on and ended up creating thumbnails of thumbnails, and thumbnails of thumbnails of thumbnails... etc. I was able to delete them all, but I wanted to make a one-shot script to do it. screenshot-2025-01-31T21-20-56-934Z.png One Shot App I got the idea of the one shot app from Simon Willison and replicated his setup in a chatgpt project screenshot-2025-01-12T18-16-57-443Z.png Initial Prompt s3_cleanup Here is the result of the s3_cleanup script. It ende

nvim-manager

I recently built a cli application as a nearly-one-shot-app called nvim-manager . It manages your nvim dotfiles install. screenshot-2025-01-31T21-21-40-707Z.png {.more-cinematic} Why {.chat-left} Don't we have stow? {.chat-right} Ya, thats not enough. {.chat-left} Why not?? {.chat-right} Inevitably shit goes sideways and I break my vim install. How is nvim manager any better nvim-manager allows you to install pinned versions of your dotfiles, your friends dotfiles, and distros in ~/.config. Thi

we beat trailmakers

Today we broke through the atmosphere in trailmakers ready for pickup from those back at mission conrol. I haven't posted a lot of gaming content to by blog, idk why, but I think I am going to start, especially for the ones that are good memories with my son. Today we hit 100% of the required parts required to build the space ship and leave the atmosphere of the planet that the story is set in. Trailmakers is a game where you design and build your own vehicles. The campaign mode standed is a s

/start

Welcome to waylonwalker.com, my small corner of the internet. I currently have {{ markata.feeds.archive.posts | length }} posts published, here are some links to help you get started around here. 2fcdafc0-f152-4fa9-ac91-799acd9084d3-239-1.webp {.more-cinematic} Feeds I have quite a few different feeds that you can browse or subscribe to in your rss reader, you can find them on my [[ feeds ]] page. Slash posts [[ slashes ]] Slash pages {.hoverlink} are some evergreen pages that I will do my best

/colophon

Colophon a page that describes how the site is made, with what tools, supporting what technologies Author Waylon Walker's Profile Picture All posts on this site are written by Waylon Walker , the typical content has changed and evolved over time. I go back and make a few corrections, but for the most part things stay pretty much as they were published originally. see more in [[ about ]] tech This site is a static site build with my own static site generator [[ markata ]], [[ thoughts ]] or as

slow nfs performance

I'm running a two node k3s cluster at home, I thought I could simply mount an nfs share on each worker node, and essentially have the same storage accross all nodes. I'm already learning why this is not reccommended. screenshot-2025-01-31T21-18-43-543Z.png Slow I've been running some cronjobs and argo workflows on the second node for awhile, these are things that run in the background and I don't care if they take a bit longer to keep my master node freed up for more critical work. I just sta

setting up ucore-zfs

I just setup my oldest hardware on the newest hotest server distro ucore-zfs. This is a gateway FX6860 manufactured in 2010. Immutable is the future My current boot log shows that I first started daily driving bazzite back in August 2024. I've been hapily using it since my arch install was plaugued with a crippling display driver error, or something that would lock the display for minutes every 30s or so, it became unusable. I switched because this is what I put my son on and it was working gr

thoughts 0.0.4

This is such an improvement to the backend of my website it warrants a blog post of celebration. For far too long I've been dealing with a tiny ass edit form on thoughts. I tend to not edit them, and try to get them right in one go. This is kinda the point of a thought, its a quick post meant to be the size of a tweet, but sometimes I'm leaving thoughts on a video or long post and want to make sure I have a good save point, but I just keep the thing in draft and hope I don't loose if for far

New-caps-for-porta-john

Today I dropped some new caps in the porta-john. I need to do a bunch of posts on zmk and my journey building the porta-john, I've been rocking it for at least two weeks now pretty solid and I am absolutely loving it. More on that later. KLP-Lame-Keycaps The model of the caps are KLP-Lame by braindefender , they are super rounded and smooth. They look really good on camera and feel good to the finders. new-caps-for-porta-john-20240805193001765.webp I printed these at home on my ender3 s1 pr

blogging in 2024

Over the past years couple of years blog traffic has been tanking, for some time I thought it might have just been me, but then i heard it from Dave Rupert on Shop Talk show, and it made sense. LLMs and AI is slowly killing the internet as we know it. My metrics I dug into my emails from the google search console team and found two results nearly one year apart. Impressions are only 60% of its value a year ago while clicks are only 35% of where they were a year ago. I'm getting less impressio

Trying Obsidian

I am giving obsidian a try, this is a test post to learn the flow. Something that has been really hard for me for a long time is images, I don't include a lot of images just because it takes so much longer just to get the post out. I store them in a separate repo, I need to resize and compress them for the web so they aren't so big. Images are easy This is my current wallpaper. trying-obsidian-20240731135759007.webp I took he screenshot and just pasted it in. I have more about my process in th

animal well codes

fish room fish bubbles ul d dl l dl ur ul jellyfish d l d l d l u r egg room bar dir given bar codes top r mid dr bot dl calculated bar codes num dir 1 r 2 dr 3 d 4 dl 5 l 6 ul 7 u 8 ur bar code direction num bar 1 top 2 mid 3 top mid 4 bot 5 top bot 6 mid bot 7 top mid bot 8 empty egg room decode 6 3 3 6 3 1 3 4 2 4 3 2 8 3 5 8 _ 3 5 7 5_ __ 5 2 5 6 3 1 3 3 1 3 6 4 4 6 4 2 4 6 8 6 4 8 7 3 7 6 5 7 6 __ egg room decode dir ul dd __ dd ul dd rr dd __ dl dr dl dd __ dr ur__ dd ll ur __ dd ll

kind cluster

kind {.hoverlink} is a very useful tool to quickly standup and teardown kubernetes clusters. I use it to run clusters locally. Generally they are short lived clusters for trying, testing, and learning about kubernetes. Kind is Kubernetes in Docker, its very fast to get a new cluster up and running. Other than checking a box in docker desktop it is the easiest way currently to get a cluster up and running. I've used docker desktop for k8s before I really developed on k8s and it was buggy at t

What I'm learning in 2024

2024 has been a learning fueled year, Diving deep into things I never would have previously thought I would. It's been a bit of a mix of the 🔥hot twitter trends, and exactly what tech twitter tells you not to do. It just goes to show community is great, the tech community is filled with strong opinions, but you need to think about what really makes sense for you, your career and your customers (or lack there of). tech k8s tailwind fastapi htmx jinja opnsense successful one day builds play-outs

about this site

I registered waylonwalker.com and started making content for it in 2017 after a big industry downturn in 2016 that left me scared for what would happen if I were laid off. The company I was working for at the time did it's first major downsizing in history sending many really good engineers out to look for new opportunities in a world flooded with many in the same situation. This was very similar to what happened to the tech industry in 2024. This was very similar to what happened to the tech

Refactoring one line links into wikilinks

Previously I had setup a feature of my website to expand one line links into a card. This was not a standard, even to the point that some formatters wrap the links with , thus breaking my custom plugin. Moving to the wikilink standard will allow my markdown posts to work accross more site builders without custom integrations. [[ expand-one-line-links ]] What is a wikilink Wikilinks are standard to a lot of wikis written in markdown. markdown-it-wikilinks {.hoverlink} The wikilink syntax is a

markata

This post is a work in progress. Markata is the static site generator that I created to build my website [[ about-this-site ]]. I built it for me and I enjoy using it. I know everying it can do and I can extend it to do more easily. I have set it up for some friends to also use it and am proud that it helps them publish their content. It's a meme to create your own static site generator to make your website. Yes its funny, I don't recommend it if your not ready for the level of work that com

Your LinkedIn is Garbage

Your linkedin link sits at the top of your resume, its one of the first things I see when I open your resume, but yet it gives me no more information that the damn resume you sent me. Save that space on your resume for something useful. So you want that /in/me on your resume Fine if you want it on your resume make it actually useful for someone reading your resume. Actually post something If I am reading resume's and I actually take the time to look at your linkedin I want to see you post someth

my linked in work history

My linked in work history is empty. I made up a position about content developer that tracks how long I've been blogging. I think i did this because LinkedIn requires it. Either way this is public knowledge and fine sharing. Social Engineering If you have taken any security class for your job seriously you have already been told not to share your work with most companies to the public, this is private information that only opens you up for social engineering attacks against that company. I ca

sick wikilink hover

Today I set up some sick wikilink hover effects using tailwind see [[ a-case-for-tailwindcss ]]. When you hover over them they show an image preview of the link that you are going to. I cant find where I have seen this but it comes from some docs sites. I'll finish this article later, just excited to see it up.

One Day Build - Play Outside

Inspired by Adam Savage and his One Day builds on youtube. I often build things, and want to make them generally useful for others and over configure out of the gate. This project is purely for me inspired by a need I have. play-outside !How-To This post will not directly show how to make a weather app, but document the process that I went through to make mine. It will show the tools that I used to make it, and the final result. The Situation It often goes in our house ask dad while he is bus

feeds

I have several feeds that you can consume my content with. Many posts will cross between them, so if you would like to follow a certain stream of content in your rss reader thats what this is for. !!! Note TLDR If you are looking for everything look here https://waylonwalker.com/archive/rss.xml All This is a feed of everything that I have published on my site to date. web rss til - daily(ish) This is a feed of all of my TIL's (Today I Learned). These are generally short single topic posts tha

Thoughts

These are generally my thoughts on a web page or some sort of url, except a rare few don't have a link. These are dual published off of my thoughts.waylonwalker.com site. It's a fully dynamically rendered site 2000's style. Posts are stored in a database and instantly available. Almost all of the posts were written in a small field within a chrome extension that I built for it. These posts are intended to in two ways. One, link building for the author. I hope that I give the people helpi

kubernetes 6 months in

I stumbled into kubernetes December 2023 when I was looking for a better way to self host applications. I was looking for something that didn't require logging into a server and building and deploying like a cave man. I wanted a smoother experience than docker compose was giving me. https://waylonwalker.com/looking-for-a-heroku-replacement/ {.hoverlink} This post turned into a list of tools that I have adopted into my k8s workflow, and plan to keep. enjoy. Kompose Kompose is a great tool for g

poc is not product

A poc is not a product. I started focais, not in a rush, but as something that I already had a POC for and thought it would be easy. I wanted to build tools to make creating blog posts like this one easier. I stared with a tool that takes screenshots of websites. POC (proof of concept) For the poc, I made a single fastapi endpoint that takes a url and returns a screenshot of the page. It converts the url into a key that I can lookup to see if I have the shot, if I don't I go get it. With

thoughts on unit tests

thoughts on unit tests cover image Theo's response puts a lot of my feelings about unit testing into words. Many of us have grown up in this world preaching unit testing. We often hear these statements "Everything must be unit tested, tests make code more maintainable." In reality when we are not writing complex low level code unit tests are probably the wrong approach. thought 192, a thought about theo's reaction to prime's unit testing Most of us are assemblers So much of software engineerin

Looking for a Heroku replacement, What I found was...

I've long hosted my personal blog as a static site on waylonwalker.com. It's all markdown, converted to html, and shipped as is. It's been great, I've moved it from GitHub Pages, to Netlify, tried Vercel for a minute, and have landed on Cloudflare Pages. Each migration has not really been that hard, it's just pointing ci to a different host after the site has built. What about server side Now the part that I have struggled with is how to cheaply host a server rendered application that can just l

why-is-postgres-default

Serious question. No one ever got fired for choosing PostgreSQL But, why. It's the most loved db, right? Right? Maybe it's time to rethink it. Don't get me wrong, if I need a relational db as a service, PostgreSQL is going to be my first choice, but why do I need to run a separate application for it? Tutorials use sqlite Why is that? Because there is nothing else to stand up. Nothing else to maintain. And you probably already have it installed on just about anything that has a battery. SQLite ru

A Case For Tailwindcss

I was watching @theprimeagen recently and I think he sold me on using tailwindcss. The thing about tailwind is that it is not a big component library, it's a set of css classes mapped to a few (usually one) style. All css classes are shitty, so you might as well use someone else's shitty css classes on all your projects rather than thinking you're being smart with a new set of classes that you will hate in 6 months when you come back to the project. roughly quoted from memory of @theprimeagen I

Make the easy things easy

It's so easy to get out of rhythm, get busy, and drop the ball on some things that you really want to do or should do. This blog is a good example. I took some time off for some family reasons, but have taken a long time to get back to it simply because I am out of rhythm. As I am trying to get back into the rhythm there is some tooling that I have set up for it that I completely forgot about that feel good to use again. Repetitive Tasks Simple Repetitive Tasks that I have to do often can just f

Underground Bases with Wyatt

Playing minecraft with Wyatt today he started a server all on his own and had me join. All vanilla, only one rule, underground bases. underground-bases-1.webp I spawned into the server and it was already night time. I gathered up some wood on my way down a tree, and was attacked by zombies before I could get any tools, so I ran up another tree and crafted a crafting table. underground-bases-2.webp Now to follow the rules, it's time to head underground to build my base. underground-bases-3.webp I

I took a break

Life comes in waves, and sometimes you need to set down some of your projects to focus on others. For the first part of 2023 I've really had a lot of family stuff to focus on, we also are pretty new homeowners and are still trying to get our new to us house cleaned up and modernized. Side Projects You can see in my growing list of repos that I have poked around on quite a few side projects over the past few months. This has been quite relaxng to me, mostly things that I use to learn from, but a

Playing Star Wars Text Adventure with a 10 yr old

The following is a playthrough of Star Wars Text Adventure with a 10 yr old.The following is a playthrough of StarThe following is a playthrough of Star

Pydantic and singledispatch

I was reading about pydantic-singledispatch from Giddeon's blog and found it very intersting. I'm getting ready to implement pydantic on my static site generator markata , and I think there are so uses for this idea, so I want to try it out. The Idea Let's set up some pydantic settings. We will need separate Models for each environment that we want to support for this to work. The whole idea is to use and type hints to provide unique execution for each environment. We might want something like

minecraft documentary

This is my first time journaling a Minecraft hardcore world, my son Wyatt is also documenting his journey in a survival world on wyattbubbylee.com . Day 0 init minecraft-doc-day-0.webp I logged into a brand new hardcore world. I was welcomed by a great Acacia biome spawn full of resources. I quickly cut my first tree, crafted an axe and set out to find my first sheep. I was able to find enough sheep for a bed, several cows and pigs. I crafted a set of wooden tools, and farmed out a wheat farm ti

Packages I Maintain

Markata I open sourced the static site framework that I use to build my-blog among other side projects. It's a plugins all the way down static site generator, that makes me happy to use. {% gh_repo_list_topic "waylonwalker", "markata" %} Repos used to build this blog my-blog is built on a number of small repos. I set it up this way so that creating content is fast and easy to do. I don't have to worry about carrying around large images with my lightweight text files just to make some posts. {%

I was on Talk Python

After years of listening to talkpython.fm I had the honor to be part of episode-337 to talk about Kedro for maintainable data science. I was quite nervous to talk on a show that I helped shape my career in such a profound way. I started my journey towards software engineering near Michaels first few episodes. His discussions with such great developers over the years has made an huge impact on my skill. It has always given me great advice and topics to go deeper on. During the episode I tried m

The one reason I switched to arch

The community, that's it, end of post, roll the credits. I'm a tinkerer I am a tinkerer, I am not going to run a stock desktop manager, mostly becuase that's just not how my brain works. I need to tweak everything to fit my needs. Grantid I have not spent much time in many full fledged linux desktop environments. They are far more customizable than windows ever will be, I absolutely love that about them. Inevitibly I end up in a situation where I hit a wall, it just won't do what I want it t

Using Different versions of python with pipx | pye...

"cell shaded, long, full body, shot of a cybernetic blue soldier with glowing pink eyes, llustration, post grunge, cinebatic dramatic atmosphere, sharp focus, pink glowing volumetric lighting, concept art by josan gonzales and wlop, by james jean, Victo ngai, David Rubín, Mike Mignola, Laurie Greasley, highly detailed, sharp focus,alien,Trending on Artstation, HQ, deviantart, art by artgem" -s50 -W832 -H416 -C7.5 -Ak_lms -S70567464 I love using pipx for automatic virtual environment management o

LIVE-REPLAY - Python dev | Markata todoui | 4/6/20...

https://youtu.be/-42A5210HYo Super fun steam Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/waylonwalker We worked on markata todoui, a command tui trello board written in python using only markdown files to store the data. I love markdown and I want to make this my workflow. During this stream we get RAIDED by TEEJ_DV! and chat about tmux a bit before calling the changes to markata-tui good and signing off. dotfiles: https://github.com/WaylonWalker/devtainer today's project:

How I Quickly Capture Screenshots directly into My...

When I am creating blog posts it's often helpful to add screenshots to them to illustrate what I see on my screen. Sometimes I lack good screenshots in my posts because it just takes more effort than I have in the moment, and I prioritize making content over making perfect content. Making Screenshots When I have something to take a screenshot of, I need to take the shot, optimize the image, often convert it to a better format, publish it, and create a the img tag in my blog. take screenshot opti

Copier < 6.0.0b0 considered dangerous

Copier is a fantastic templating library written in python, but older versions have a dangerous bug if you are using it inside of existing directories. !!UPDATE As of May 15, 2022, the stable release of copier now includes these changes, if you have not already make sure you update. This is a PSA I Use copier several times per day and get fantastic benefit from this project, this post is not intended to crap all over copier in any way, but is rather a PSA for other users who do use copier like I

PyOhio CFP's

Here are some CFP's that I used for PyOhio 2022. https://pretalx.com/pyohio-2022/cfp {.hoverlink} Idea to blog post in minutes - Shorter Markata is a plugins all the way down static site generator, that covers all the things you need to go from markdown to a blog site out of the box. Since it's plugins all the way down you can also rip out all the default plugins, and do something completely different with the lifecycle. Lets build a whole blog site in 5 minutes. Add Kedro to your Pandas Workfl

Smoother Python with automatic imports | pyflyby

This is not a flaky works half the time kind of plugin, it's a seriously smooth editing experience. I've just started using pyflyby, and it is solid so far. I have automatic imports on every save of a python file in neovim, and automatic imports on every command in ipython. I can't tell you how pumped I am for this, and how good its felt to use over the past few weeks. It's glorious. YouTube video Listen to me rant on how great pyflyby is https://youtu.be/2QW5DJiEJH4 {.youtube-embed} Give the

You must use augroup with autocmd in vim | Here's ...

If you are running vim autocmd's without a group, you're killing your performance. Granted your probably not sourcing your vimscript files with autocmd's too often, but every time you source that vimscript you are adding another command that needs to run redundantly. https://youtu.be/2ITTn4Dl0lc This is what I had Not silky smooth For WAAY too long I have had something like this in my vimrc or init.vim. It formats my python for me on every save, works great except if I source my dotfiles mor

Code Review from the comfort of vim | Diffurcate

I often review Pull requests from the browser as it just makes it so easy to see the diffs and navigate through them, but there comes a time when the diffs get really big and hard to follow. That's when its time to bring in the comforts of vim. https://youtu.be/5NKaZFavM0E Plugins needed This all stems from the great plugin by AndrewRadev . It breaks a down into a project. So rather than poping into a pager from git diff, you can pipe to diffurcate and it will setup a project in a tmp directo

Setup a yaml schema | yamlls for a silky smooth se...

I've gone far too long without a good setup for editing yaml files, I am missing out on autocomplete and proper diagnostics. This ends today as I setup yaml-language-server in neovim. https://youtu.be/xo4HrFoKF4c {.youtube-embed} The video for this one is part of a challenge-playlist I put out for myself to constantly improve my dotfiles for all of December. init.vim I have my setup to only source other modules, if you want everything in a single config, feel free to do as you wish. I broke mi

Open files FAST from zsh | or bash if thats your t...

https://youtu.be/PQw_is7rQSw I am often in a set of tmux splits flying back and forth, accidentally close my editor, so when I come back to that split and hit my keybinds to edit files I enter them into zsh rather than into nvim like I intended. Today I am going to sand off that rough edge and get as similar behavior to nvim as I can with a couple of aliases. Make sure you check out the YouTube video to see all of my improvements. what's an alias If you have never heard of an alias before it's

How linux users install a text editor

In honor of the neovim 0.6.0 release, I decided to do a funny skit installing neovim, and fix up my install script in the process as part of my challenge to fix up my dotfiles. I ran into one snag where I was not updating the repo that I cloned. I moved it to the directory I now keep third-party git repos and set it to update with ansible. https://youtu.be/64oKLphhBuo The thing that took me the longest to realize was.... I had a path issue pointing me to an old install of the appimage over the

30 days dotfile ricing

https://youtu.be/Jq1Y48F_rOU {.youtube-embed} I am challenging myself to 30 days of dotfile ricing. I have been on linux desktop for a few months now and have a pretty good workflow going, I have the coarse edits done to my workflow, but it has some rough edges that need sanded down. It's time to squash some of those little annoyances that still exist in my setup. This is primarily going to be focused on productivity, but may have a few things to just look better. This will comprise heavily of a

JUT | Read Notebooks in the Terminal

Trying to read a .ipynb file without starting a jupyter server? jut has you covered. https://youtu.be/t8AvImnwor0 watch the video version of this post on YouTube install is packaged and available on pypi so installing is as easy as pip installing it. installing jut with pip ! This is my first time including snippets of the video in the article like this, let me know what you think! examples running jut examples what are all the commands available for jut? Take a look at the help of the cli

Waylon Walker

👋 Hi, I am Waylon Walker. Husband, Father of two, and creator things on the web, learning, and teaching others. I play lots of Minecraft, make art, and skate everyday with my kids. I finish the day binging episodes of Big Bang Theory with my wife. I believe in a decentralized platform where everyone has their own space on this internet to share their thoughts and ideas. I created my blogging platform from scratch to learn about building tools with pluggy and diskcache. I was frustrated with

kedro catalog create

I use to boost my productivity by automatically generating yaml catalog entries for me. It will create new yaml files for each pipeline, fill in missiing catalog entries, and respect already existing catalog entries. It will reformat the file, and sort it based on catalog key. https://youtu.be/_22ELT4kja4 {.youtube-embed} [[ what-is-kedro ]] 👆 Unsure what kedro is? Check out this post. Running Kedro Catalog Create The command to ensure there are catalog entries for every dataset in the pass

nvim conf 2021 | IDE's are slow | Waylon Walker

https://youtu.be/E18m4KkJUnI {.youtube-embed} Slides 👇 welcome Other possible titles Using Vim as a Team Lead I 💜 Tmux Why I stopped using @code Get there fast How I vim It's ok Use a graphical IDE if it works for you. Trick it out vim is so well integrated into the terminal, take advantage It wasn't working for me anymore dozens of instances As a team lead I bounce betweeen a dozen projects a per day https://pbs.twimg.com/media/FAEmRjYUcAUk2eR?format=jpg&name=large {.hoverlink} Move With Intent

Just Ask Ipython for help

It happens to the best of us We can't all remember every single function signature out there, it's just not possible. If you want to stay productive while coding without the temptation to hit YouTube or Twitter. Use the built in help. Here are 5 ways to get help without leaving your terminal. https://youtu.be/TZrRAP-9UMk Docstrings In any python repl you can access the docstring of a function by calling for . In Ipython we can even get some syntax highlighting with the . Source Code Sometim

Hacktoberfest 2021?

Are you participating in Hacktoberfest 2021? I got my start with open source with Hacktoberfest 2018. I remember being so nervous for those first pr's. I have continued to participate every year, and it feels good to give back. It made a big change on me Something that it really did for me was to lower the hurdle to contribute regularly. I am not a huge contributor to open source. I am not necessarily a regular maintainer of any large project, but I do often raise issues when I see things as a u

I made a neovim plugin

I've slowly adding more and more lua functions into my neovim configuration, and recently I noticed a pattern for a class of functions that reach out to run shell commands that can be abstracted away. https://youtu.be/8m5ipBuopPU {.youtube-embed} Telegraph.nvim Check out the project readme for the most up to date details on the plugin itself. Motivation I want a simple way to make remaps into shell commands that can open new tmux windows, popups, or just run a command with context from the edit

Setting Parameters in kedro

Parameters are a place for you to store variables for your pipeline that can be accessed by any node that needs it, and can be easily changed by changing your environment. Parameters are stored in the repository in yaml files. https://youtu.be/Jj5cQ5bqcjg {.youtube-embed} [[ what-is-kedro ]] 👆 Unsure what kedro is? Check out this post. parameters files You can have multiple parameters files and choose which ones to load by setting your environment. By default kedro will give you a and para

Writing your first kedro Nodes

https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let's make some nodes with some vanilla data structures. import node You will need to import node from kedro.pipeline to start creating nodes. func The is a callable that will take the and create the . inputs / outputs Inputs and outputs can be None, a single catalog entry as a string, mutiple catalog entries as a List of strings, or a dictionary of strings where the key is the keyword argument of the func and the value is

Running your Kedro Pipeline from the command line

Running your kedro pipeline from the command line could not be any easier to get started. This is a concept that you may or may not do often depending on your workflow, but its good to have under your belt. I personally do this half the time and run from ipython half the time. In production, I mostly use docker and that is all done with this cli. https://youtu.be/ZmccpLy-OEI {.youtube-embed} [[ what-is-kedro ]] 👆 Unsure what kedro is? Check out this post. Kedro run To run the whole darn proj

kedro Virtual Environment

Avoid serious version conflict issues, and use a virtual environment anytime you are running python, here are three ways you can setup a kedro virtual environment. https://youtu.be/ZSxc5VVCBhM {.youtube-embed} conda venv pipenv conda I prefer to use conda as my virtual environment manager of choice as it give me both the interpreter and the packages I install. I don't have to rely on the system version of python or another tool to maintain python versions at all, I get everything in one tool. st

Kedro Install

Kedro comes with an command to install and manage all of your projects dependencies. https://youtu.be/IWimEs-hHQg cd into your project directory and activate env You must start by having your kedro project either cloned down from an existing project or created from kedro new. Then activate your environment. [[ kedro-new ]] this post covers kedro new [[ kedro-environment ]] This post covers creating your virtual environment for kedro install kedro Make sure you have kedro installed in your cur

Kedro Git Init

Immediately after , before you start running or your first line of code the first thing you should always do after getting a new kedro template created is to . https://youtu.be/IGba3ytf_6U git init Its as simple as these three commands to get started. I don't care if this project is for learning, if it will never have a remote or not, use git.

Kedro New

https://youtu.be/uqiv5LAiJe0 {.youtube-embed} Kedro new is simply a wrapper around the cookiecutter templating library. The kedro team maintains a ready made template that has everything you need for a kedro project. They also maintain a few kedro starters, which are very similar to the base template. [[ what-is-kedro ]] Unsure what kedro is, Check out yesterdays post on What is Kedro. pipx I reccomend using when running kedro new. is designed for system level cli tools so that you do not

What is Kedro

Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user a way to build pipelines that automatically take care of io through the use of abstract that the user specifies through entries. These entries are loaded, ran through a function, and saved by . The order that these are executed are determined by the , which is a DAG . It's the 's job to manage the execution of the . https://youtu.be/Wf4rnFsaFFU {.youtube-embed}

How I Kedro

https://youtu.be/bw5_FWDVRpU Ubuntu I recently switched over to using Ubuntu, it works well pretty much out of the box for me. I am using gnome with a dark theme. Gnome Terminal I am still using the built in default gnome terminal, it just works. It does all the things that I need it to do. It supports transparency renders my fonts and allows me to highlight things well. One Dark Theme dotfiles You can find my dotfiles on github. Feel free to read through and take anything that you find use

tmux show-messages

https://youtu.be/LLk94fKpGg4 {.youtube-embed} As we push the limits of tmux further and further you are bound to end up in a situation where you are mashing down a hotkey and it's just not doing what you want it to do, and you have no idea why. is a tmux command that can be used to show what tmux is actually doing behind the scenes. This might highlight any hot key conflicts you might have in your . man page for show-messages In case you wnat more information about show-messages, here is the

tmux ta

https://youtu.be/nT0FA1RNZEs {.youtube-embed} Now your creating, jumping, and killing sessions like a boss. You are slicing through projects with ease, let me show you one more thing that can be the cream on top of this silky smooth setup we have been working towards. Chris Toomey's Tmux Course This script is simply my fork of Chris Toomey's script straight out of his course. It helps us create or jump to project specific sessions with ease. a directory of projects My version of the script

tmux display-message

https://youtu.be/utfLA6L8o5s {.youtube-embed} You've got some long running tasks, and you're trying to stay productive and knock tasks off that board, but you keep finding that your processes finish and you stay on other tasks for longer than you should. You were in the flow and just did not check back in on it. With you can send yourself a notification when that long running task is complete. from the man page Here is a snippet of from the tmux man page. I rarely need to do anything othe

killing tmux

https://youtu.be/QWPyYx54JbE {.youtube-embed} Now it's time to switch gears, we are onto a different part of our day and there are just too many sessions running and we need to clean up shop. kill-server One viable option is to nuke the whole dang thing. I actually do this more than you might think. save and commit your work diligently before kill-session A more reasonable option might be to kill a single session. choose-tree Killing sessions one by one from the command line can be a bit tedio

tmux has-session

https://youtu.be/XucVVgGmesM {.youtube-embed} I see you there, trying to script out your tmux layouts. Tryig to get each project setup just perfect with a script, but you keep stumbling over yourself with error messages The tmux command is a handy tool to prevent this error message when scripting your tmux layouts. command line The command is pretty straight forward, you simply ask tmux if the session name you are looking for exists. [[ tmux-nav-2021 ]] for more information on how I naviga

tmux list-keys

https://youtu.be/Y1MYmL8ZolE {.youtube-embed} Tmux list keys can be a useful tool to help remind you of what kebindings you have setup. You can search for them and scroll just like in tmux copy-mode. command line You can call list-keys from the command line but the interface is not very usable by itself. It might be nice to mix with grep or a pager in some circumstances. tmux command line Running from within the tmux command line puts you into a much more pleasant . default keybinging By de

tmux status-bar

https://youtu.be/mvgM8UH11_U {.youtube-embed} The tmux status bar can be a handy tool to remind yourself where you are within tmux. It can also include a bunch of system information like battery status, cpu, mem, whatever you can get from the command line. Honestly I like to keep it minimal, and actually keep it turned off most of the time. I find that it helps a little bit for others to follow along if I keep it on in certain circumstances. show the status bar You can set a hotkey to show o

tmux source-file

https://youtu.be/dDq0depPrTs {.youtube-embed} So you have been tricking out that , you're looking for a silky smooth workflow that lets you fly through tmux with super speed, but every time you tweak out that you have to restart your whole session. Not amymore, Let's add this to the bottom of our tmux.conf so that you can see everytime it gets sourced. command We can run this command from your shell to re-source your changed It also works from the tmux command line. tmux hotkey It's very com

tmux fzf session jumper

https://youtu.be/DkJ9rb85LC0 {.youtube-embed} Quickly getting between tmux splits is critical skill for productivity. You can get by with or session for awhile, but if you have more than about three session you need something a bit more targeted. Full Screen selector I have used this fzf one keybinding for quite awhile, honestly I did not make it up, and cannot remember where it came from. It will open up a session picker in a new full screen window. Popup selector Like with many of my key

tmux next/prev session

https://youtu.be/8kZnjHPYnKU {.youtube-embed} Now that we are splitting up work into their own sessions, lets talk about how to navigate between them without the command line. Navigating sessions is what kept me using a too many splits and windows workflow for far too long. It was rough, I was always tripping over panes and windows until I got too frustrated and just ran to start on a new blank slate. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux start application

https://youtu.be/c-a2Bnv_NJ0 {.youtube-embed} Scripting tmux to open up specific applications can be intimidating your first time. It can be tricky to get it to start in the right directory. If you are trying to assign applictaions to a keybinding it can be easy to mess up and have weird things happen every time your gets sourced. Open htop in an above split I used this one for a number of years to take a quick peek into my systems performance while a memory intensive task was running. 🗒️ no

tmux rename session

https://youtu.be/WRLRiQDjVIA {.youtube-embed} So you have been working on your tmux workflow, you've dropped a too many window workflow for scoping work that belongs together into separate sessions, but you cannot remember what session your work is in. If your diligent you have named your window when you created it, but sometimes its intent has changed or your were just plain too lazy at the time for the extra characters needed to name it. Don't worry we can still give that session a descriptiv

tmux targeted session

https://youtu.be/5KE7Il7SOEk {.youtube-embed} This is something that I made up but use every single day, this is what keeps much of what is on my blog or my teams private work wiki going. I have a few very important directories that I have assigned directly to a hotkey for fast session switching. [[ tmux-new-session ]] This one is building off of yeserday's new-session post, make sure you check that one out as well. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this

tmux detach

https://youtu.be/A1qx3tNKDdA {.youtube-embed} tmux detach is a handy tmux command that will quit your current session while keeping it running. The full name of the comamnd is , is a shorthand. default keybinding I have mine bound to where mod is alt. https://waylonwalker.com/tmux-nav-2021/ {.hoverlink} for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist to see all of the videos in this series.

tmux new-session

https://youtu.be/LbQNdCAUogE {.youtube-embed} This one starts a new chapter in our series that is going to open up a whole new set of workflow productivity options, understanding how the command is a critical command in our adventure into tmux glory. This is going to open the door for some seriously game changing hotkeys and scripting. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist to see all of the videos

tmux attach

https://youtu.be/JQ0yDCVu44E attach is one of the most useful features of tmux. If you have no interest in tmux for pane and window management, you should use tmux for this. It can be a life saver if you ever get disconnected from the host machine or accidently close your terminal you can connect right back into the session you were just in using attach. attach this command will simply attach back to tmux if you are ever disconnected If you ever run long running tasks on a remote machine by ss

Git in Depth Notes

These are my notes from taking @nnja's FEM course git-in-depth . requirements > than 2.0 creating a git

tmux ls

https://youtu.be/LY41GLn_DGg {.youtube-embed} tmux ls will list the sessions that you have running within the tmux server if tmux is currently running. This is handy to combine with commands such as . [[ tmux-attach ]] [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist to see all of the videos in this series.

tmux command line

https://youtu.be/SNu-4IrkjAs {.youtube-embed} So far we have covered a lot of tmux commands and how they map to keybindings but these same commands can be executed at the command line. From the command line Let's make a popup that displays our git status for 5s or until we close it manually. We can run the following command at the command line, in a split. From the tmux command line Or we can open the tmux command line and run it from tmux's built in command line, which is very similar to bim E

tmux copy-mode

https://youtu.be/-ypY_-VmBKk {.youtube-embed} tmux copy-mode is a tmux mode that lets you scroll, search, copy, and jump your way through a pane. There are a ton of keybindings for copy-mode, the main ones you will need to know are for searching down for searching up, for next item, for starting a selection, and enter to copy the selection. Arrow keys will be used for navigation unless you have specified vi mode, then it will be . Default keybinding to get into copy mode is . If you a

tmux join-pane

https://youtu.be/Vm5rRtcVXLw {.youtube-embed} Join-pane allows you to join panes that you have broken away from your window, or created in a different window to the window you want it in. As far as I know there is not a default keybinding for it. Before you can join a pane you must first have a pane marked to join. Once you mark a pane, go back to the window you want to join it to and join-pane. My keybindings, you must add this to your file to use them. [[ tmux-nav-2021 ]] for more informat

tmux break-pane

https://youtu.be/ICL609F2xnc {.youtube-embed} Break-pane is a handy tmux command when your layout gets too cramped and you want to just move a split into its own window. Calling does exactly that, it creates a for you and moves your currently selected split into that window Default key binding for [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux zoom

https://youtu.be/Rn6mOarCQ-Y {.youtube-embed} Zooming into the current split in tmux is a valuable tool to give yourself some screen real estate. These days I am almost always presenting, streaming, or pairing up with a co-worker over a video call. Since I am always sharing my screen I am generally zoomed in to a level that is just a bit uncomfortable, so anytime I make a split it is really uncomfortable, being able to zoom into the split I am focused on is a big help, and also help anyone wat

tmux new-window

https://youtu.be/YRPZBv-iYyE {.youtube-embed} New window as it sounds makes new windows in tmux. Windows are kind of like tabs. They are another screen within your sessions that you can name and make new panes in. Default key bindings for creating and navigating windows in tmux. As always I have rebound these keys because I generally prefer a single keystroke over the prefix plus keybinding approach that tmux gives by default. When I started using tmux I did almost everything in one giant sess

tmux slect-pane

https://youtu.be/CPZJZjN9YTY {.youtube-embed} These are my MOST often used keybindings that I use in tmux. They allow me to jump between splits with ease with a vim style layout. I can hold mod and jump between panes with a familiar arrow key. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux rotate-window

https://youtu.be/06z5qf81ofo {.youtube-embed} Rotate window is the main way that I navigated tmux before I learned . It allows you to change your focused pane, or rotate the position of the panes easily. Default keybindings My keybindings look just a bit different than the default ones, I do not like needing to hit prefix for every command, especially for repeated commands. I set a similar keybinding to the default one that uses mod instead of prefix. [[ tmux-nav-2021 ]] for more information o

tmux select-layout

https://youtu.be/F0mHnwTrNNc {.youtube-embed} When you get many splits going in tmux sometimes its time for a new layout. There are four layout strategies that I use, main-vertical, main-horizontal, even-vertical, even-horizontal. Almost always I am useing the main ones with mod plus a or mod plus shift a keybindings. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux resize-panes

https://youtu.be/hpFYE2LU7xc {.youtube-embed} Resizing panes in tmux can be quite difficult in default tmux, I use a set of keybingings to help resize panes in the rare occasions that I do need just a bit more space. I set the keybinding to the same as my split navigation bindings but shifted. They are very vim like (h,j,k,l). Most often when I need to resize panes I just grab the edge of the pane with my mouse. Yes the mouse, its not that often that I actually need to change the size of a pan

tmux choose-tree

https://youtu.be/79Y-kqAiMpw {.youtube-embed} Choose tree is a powerful tmux utility that provides a graphical interface to preview all sessions, windows, and panes, move between them kill them, move them and much more. The default keybinding my preferred keybinding to open sessions and windows collapsed and Zoomed in. From the man page. https://waylonwalker.com/tmux-nav-2021/ {.hoverlink} for more information on how I navigate tmux, check out this full post

tmux prefix

https://youtu.be/BMkpbfhbkKM {.youtube-embed} The prefix key is an essential part of tmux, by default all of tmux's key-bindings sit behind a prefix. This prefix is very similar to vim's leader key. It is common for folks to change the default (control b) to or if they are a vim user something to match their vim leader key. A few of the essential default key-bindings. A more complete list of key-bindings can be found in this gist https://gist.github.com/mzmonsour/8791835 . [[ tmux-nav-2021

tmux splitting panes

https://youtu.be/kzgyiHap1nQ {.youtube-embed} splitting panes is a core feature of tmux. It allows us to split the terminal vertically or horizontally into new panes. 🗒️ note that '#{pane_current_path}'will keep the split in the same directory as it's parent, without this it will default to your home directory. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux last session

https://youtu.be/RB87EEnnMnU {.youtube-embed} An ultimate productivity key-binding in tmux is one to switch to the last session. I use this to quickly get between sessions really quick. Often I am working and need to lookup a quick note, or copy something into my notes, then get back to where I was quickly. I think of this hub and spoke model, and use to quickly drive it. hub and spoke [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux floating popups

https://youtu.be/2ZqFDsJywt8 {.youtube-embed} Tmux popups are actually floating windows that you can drag around the screen. They always open in the middle (by default) when you open them, no matter where you leave them. Here are a couple of keybindings I use to open up popup windows. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post

tmux popups

https://youtu.be/2I8fB28zfB4 {.youtube-embed} Tmux-popups are a great feature that is relatively new to tmux, many repos such as the standard ubuntu repos do not have it. Popups came in 3.2a, if your package manager does not have it, you can follow the tmux's install instructions to build from source. [[ tmux-nav-2021 ]] for more information on how I navigate tmux, check out this full post I use popups quite a bit in my workflow to ssh into another machine for a short period, or make a new p

Incremental Versioned Datasets in Kedro

Kedro versioned datasets can be mixed with incremental and partitioned datasets to do some timeseries analysis on how our dataset changes over time. Kedro is a very extensible and composible framework, that allows us to build solutions from the individual components that it provides. This article is a great example of how you can combine these components in unique ways to achieve some powerful results with very little work. [[ what-is-kedro ]] 👆 Unsure what kedro is? Check out this post. How

I Started Streaming on Twitch

I recently started streaming on twitch.tv/waylonwalker and it's been a blast so far. python kedro Data Science Data Engineering webdev digital gardening Kedro Spaceflights It all started with kedro/issues/606 , Yetu called out for users of kedro to record themselves doing a walk through of their tutorials. I wanted to do this, but was really stuck at the fact that recording or editing somewhat polished vide is quite time consuming for me. kedro-issue-606 Inspiration My introduction to twitch c

Kedro Spaceflights - part 2 | Stream replay June 7...

This was my seconf time ever streaming on twitch.tv/waylonwalker , and I completely botched my mic 2x. https://youtu.be/_7MwgKu-844 Links Spaceflights Tutorial my spaceflights repo Notes to get started

🌱 Digital Gardening | gif to Mp4 | Stream replay J...

https://youtu.be/I4VenHqIEng Doing some Digital Gardening on stream Ahrefs Errors ahrefs large images Automatic gif to mp4 gif to mp4 After this stream all gifs on my site are converted to mp4/webm if they exist. tmux-navigation-2021

Kedro Spaceflights - part 1 | Stream replay June 4...

This was my first time ever streaming on twitch.tv/waylonwalker . I am excited to get going. I have been streaming early in the morning while I am still waking up, so still a bit groggy as I go. https://youtu.be/Y07UBr9Ccjs Kedro Spaceflights It all started with kedro/issues/606 , Yetu called out for users of kedro to record themselves doing a walk through of their tutorials. I wanted to do this, but was really stuck at the fact that recording or editing somewhat polished vide is quite time co

Using Kedro In Scripts

With the latest releases of kedro , it is now possible to run kedro pipelines from within scripts. While I would not start a project with this technique, it will be a good tool to keep in my back pocket when I want to sprinkle in a bit of kedro goodness in existing projects. New to Kedro [[ what-is-kedro ]] If your just learning about kedro check out this post walking through it No More Rabbit Hole of Errors as of 0.17.2 I've tried to do this in kedro and it turned into a rabbit hole of erro

Silence Kedro Logs

Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while trying to implement a cli extension with clean output. Silence a Python log First, how does one silence a python log? Python loggers can be retrieved by the module's function. Then their log level can be changed. Much of kedro's chattiness comes from INFO level logs. I don't want to hear about anything for my current use case unless i

Python Diskcahe is locked

Running multiple processes using the same diskcache object can cause issues with locks. As I was trying to setup a rich Live display for markata I ran into issues where each part could not nun simultaneusly. As I had followed the instructions from discache it was not directly aparant to me, so I had to make a simple example to experiment and play with at a small scale. Minimum reproducible error Minimum reporducible error is one of my superpowers in development. I do this very often to sus ou

How I navigate tmux in 2021

In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and generally keep work separated into logical groups. Update Since making this post, I have made ~20 other posts in short form that all have a YouTube video to go along with them you can find them all on my tmux-playlist . Chris Toomey's Tmux Course I took Chris's tmux course in December 2020 and it was fantastic. Even as a seasoned tmux user, I learned quite a bit. Before the cou

Trim unused git branches

Trim branches no longer on origin Find branches already merged

What is if __name__ == "__main___", and how do I u...

When a python module is called it is assigned the of otherwise if it's imported it will be assigned the of the module. Concrete example Let's create a module to play with a bit. We will call this module . It is a module that we may want to run by it'self or import and use in other modules. I have set this module up to execute one of two if statements based on whether the module it'self is being ran or if the module is being imported. Note it is not common to have a block, this is just

Create a Virtual File Gallery with Symlinks

Creating a directory that is a union of several directories can be achieved with a few symlinks at the command line. Creating a Virtual File Gallery Here is how I am creating a virtual directory of all my projects that is a combination of both work and not-work projects. I am creating symlinks for every directory under and . ⚠ Notice that first I am recreating the directory each time. This will ensure that any project that is deleted from their actual directory is removed from the virtual ga

How to Install micromamba on linux (from the comam...

I really like using conda ( ) as my python virtual environment manager of choice. It's simple and it includes its own python interpreter using the version that I specify at creation. Mamba from their readme Mamba is a reimplementation of the conda package manager in C++. parallel downloading of repository data and package files using multi-threading libsolv for much faster dependency solving, a state of the art library used in the RPM package manager of Red Hat, Fedora and OpenSUSE core parts

Vim Wsl Clipboard

I've long used neovim from within windows wsl, and for far too long, I went without a proper way to get text out of it and into windows. wsl has access to cmd applications wsl can access clip.exe. You can do some cool things with it, such as cat a file into the clipboard, sending output from a command to the clipboard, or set an autocmd group in vim to send yank to the windows clipboard. using clip.exe Let's say you want to send a teammate the tail of a log file over chat. You can tail the file

kedro replit

I am trying to see what an embeded replit

Pytest capsys

Testing print/log statements in pytest can be a bit tricky, capsys makes it super easy, but I often struggle to find it. capsys capsys is a builtin pytest fixture that can be passed into any test to capture stdin/stdout. For a more comprehensive description check out the docs on capsys using capsys Simply create a test function that accepts capsys as an argument and pytest will give you a capsys opject.

Building Rich a Dev Server

Draft Post I've really been digging @willmcgugan's rich library for creating TUI like interfaces in python. I've only recently started to take full advantage of it. Dev Server I am working on a project in which I want to have a dev server running continuously in the background. I really like dev servers theat automatically chooose an unused port and list out the running pid so that I can kill it if I need to. automatic port number auto-restart display ( port, pid, uptime ) finding the port I

Site Down During Build

Recently I noticed a new netlify site of mine was down while I was checking to see if new content was live. Later found out this was consistent after each and every push the site would go gown as soon as I hit push, and would not come back until the build finished. Is this normal? Do other Netlify sites go down during build??? Short Answer NO. All of my google fu lead me to believe I was alone and none of my other sites do this. Digging into my build My deploy script ends with the following. Aft

Kedro pipeline_registry.py

With the realease of came a new module in the project template . Here are some notes that I learned while playing with this new module. migrating to create a file create a function in that mirrors the register_pipelines method from your module do not bring the decorator remove register_pipelines method on your class You should now have something that looks like this in your . pipeline_registry only works in Conflict Resolution What happens If I register pipelines in both places I w

Stand With Your Team

People who are quick to toss team members under a bus are not well trusted or highly thought of and it will lead to some toxic team dynamics. Building Steam While collaborating on any project there are going to be decisions made that aren't necessarily your favorite, during a summer internship my mentor made some decisions that I was not on board with, but I accepted his wisdom and moved forward with little push back. Full Spead Ahead During a review, leadership showed interest in the option tha

Blogging For Me

I create this blog with one person in mind, me. There are others like me This is not completely selfish, as there are likely many others out there that think similarly to me. Everyone comes from different backgrounds and varying levels of experience. In no way do you need to be an expert to create content others will benefit from. Accurate I am as accurate as possible. I don't know everything, and If I waited for that to happen I would never post, or write at such a high level no one else (in

Minimal Kedro Pipeline

How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It's only a total of 35 lines of python, 8 in and 27 in . 📝 Note this is only a composable pipeline, not a full project, it does not contain a catalog or runner. Minimal Kedro Pipeline I have everything for this post hosted in this gihub repo , you can fork it, clone it, or just follow along. Installation Caveats This repo represents the minimal amount of structure to build a ked

Markdown Cli

This is a post that may be a work in progress for awhile, Its a collections of thoughts on managing my blog, but could be translated into anythiung that is just a collection of markdown. Listing things posts tags draft posts data frontmatter filepath content template html render content Markdown.Markdown support extentsions frontmatter cleaning. provide ways to hook in or clean up the frontmatter Markata.Markata methods load render save Markata.Post methods load render save Markata plugins befor

My Content Strategy For 2021

I am making another push in 2021 to get my content out in the world and meeting users where they are. See how I plan to execute. Platforms waylonwalker.com Twitter DEV hashnode Medium LinkedIn Anchor Markdown My content is written in markdown, all markdown. I find that markdown does a really great job at getting out of the way and letting ideas flow onto the page. I am never fussing with fonts and formatting while physically writing posts. Not that I don't spend way more time than I need to twea

Quickly Edit Posts

Recently I automated starting new posts with a python script. Today I want to work on the next part that is editing those posts quickly. [[ automating-my-post-starter ]] Check out this post about setting up my posts with python 🐍 Enter Bash For the process of editing a post I just need to open the file in vim quickly. I dont need much in the way of parsing and setting up the frontmatter. I think this is a simple job for a bash script and fzf. change to the root of my blog fuzzy find the post

Gitui is a blazing fast terminal git interface

Gitui is a terminal-based git user interface (TUI) that will change the way that you work with git. I have been a long-time user of the git cli, and it's been hard to beat, mostly because there is nothing that keeps my fingers on the keyboard quite like it, except which comes with some great ways to very quickly walk through a git project. installation Go to their [releases]https://github.com/extrawurst/gitui/releases) page, download the latest build, and pop it on your PATH. I have the follo

Kedro - My Data Is Not A Table

In python data science/engineering most of our data is in the form of some sort of table, typically a DataFrame from a library like pandas, spark, or dask. DataFrames are the heart of most pipelines These containers for data contain many convenient methods to manipulate table like data structures. Sometimes we leverage other data types, namely vanilla types like lists and dicts, or even numpy data types. [[ what-is-kedro ]] unfamiliar with kedro, check out this post Sometimes datasets are not t

Quickly Change Conda Env With Fzf

Changing conda environments is a bit verbose, I use a function with fzf that both lists environments and selects the one I want in one go. Conda I have used conda as a virtual environment tool for years now. I started using conda for its simplicity to install packages on windows, but now that has gotten so much better and it's been years since I have run a command. I'm sure that I could use a different environment manager, but it works for me and makes sense. What environment manager do you u

Minimal Python Package

What does it take to create an installable python package that can be hosted on pypi? What is the minimal python package setup.py my_module.py This post is somewhat inspired by the bottle framework, which is famously created as a single python module. Yes, a whole web framework is written in one file. Directory structure setup.py name The name of the package can contain any letters, numbers, "_", or "-". Even if it's for internal/personal consumption only I usually check for discrepancy with p

Vim Replace Visual Star

Replacing text based on whats in the current search register is a quite handy tool that I use often. I believe I picked this tip up from Nick Janetakis, check out his YouTube channel for some amazing vim tips. https://www.youtube.com/watch?v=fP_ckZ30gbs {.hoverlink} If there is one thing that I Like most about vim it's the ability to hack on it and make it work well for you. Replacing text in vim Vim can often be a bit verbose, but that's ok because we can hack on it, and make our own shortcuts

If Tmux

I do much of my work from tmux, I love it so much that I want to setup some functionality that puts me in tmux even if I didn't ask for it. Bash Function Bash function to check if the shell is in a tmux session. Using the bash function I often open up vim to do some quite edits, but before I know it I have several splits open and I need access to another shell utility, but I forgot to start in tmux. This function makes sure tht I start in tmux everytime. Using to ensure vim is opened in tmux.

Save Vim Macro

If you are like me, you have created a macro or two that is pure glory, and you forget how you made it after a day or so, or you immediately want to store it away as a custom keybinding. As with most things with vim, it's easy to do once you understand it. Creating a Macro One of the earliest things we all learn to do in vim is to create macros, custom sets of functionality stored in a register that can be replayed later. To create a macro, get into normal mode, then type followed by a letter

Live Substitution In Neovim

Replacing text in vim can be quite frustrating especially since it doesn't have live feedback to what is changing. Today I was watching Josh Branchaud's Vim-Unalphabet series on Youtuve and realized that his vim was doing this and I had to have it. https://twitter.com/_WaylonWalker/status/1346081617199198210 How to do it I had to do a bit of searching and found a great post from vimcasts that shows exactly how to get the live search and replace highlighting using :h inccommand Add this to you

Newsboat

Web browsers are a black hole of productivity. I try to use them as little as possible when it is time to focus. I try to use , , or with ipython, or --help at the command line as much as possible. What about that time I am trying to see what my online friends are posting on their sites? I used to used google reader quite heavily before that was taken down. Newsboat I am going to give a terminal rss reader a try for a bit and see how that goes for me. I have really struggled to get into

Large Refactor At The Command Line

As projects grow patterns that worked early on break and we need to change things to make the project easier to work with, and more welcoming to new developers. git Before you start mucking up a project with wild commands at the terminal check that you have a super clean git status. We may make some mistakes and need a way to undo 100's files and git makes it really easy if you start with a clean history. If we are ready to begin work we should see a response like this. It would also be wise to

Ipython-Config

I use my ipython terminal daily. It's my go to way of running python most of the time. After you use it for a little bit you will probably want to setup a bit of your own configuration. install ipython Activate your virtual environment of choice and pip install it. Any time you are running your project in a virtual environment, you will need to install ipython inside it to access those packages from ipython. You are using a virtual environment right? Virtual environments like venv or conda ca

Custom Ipython Prompt

I've grown tired of the standard ipython prompt as it doesn't do much to give me any useful information. The default one gives out a line number that only seems to add anxiety as I am working on a simple problem and see that number grow to several hundred. I start to question my ability 🤦‍♂️. Configuration If you already have an ipython config you can move on otherwise check out this post on creating an ipython config. [[ ipython-config ]] The Dream Prompt I want something similar to the stars

Automating my Post Starter

One thing we all dread is mundane work of getting started, and all the hoops it takes to get going. This year I want to post more often and I am taking some steps towards making it easier for myself to just get started. When I start a new post I need to cd into my blog directory, start neovim in a markdown file with a clever name, copy some frontmatter boilerplate, update the post date, add tags, a description, and a cover. Todo List for starting a post frontmatter template Title slug tags date

Windowing Python Lists

In python data science we often will reach for pandas a bit more than necessary. While pandas can save us so much there are times where there are alternatives that are much simpler. The itertools more-itertools` are full of cases of this. This post is a walkthrough of me solving a problem with rather than reaching for a for loop, or pandas. I am working on a one-line-link expander for my blog. I ended up doing it, just by modifying the markdown with python. I first split the post into line

Adding Audio to my blog posts

This is episode 1 of the Waylon Walker Audio experience, posts from waylonwalker.com {.hoverlink} in audio form. So I have had this idea for awhile to add audio to my blog posts. The idea partly comes from the aws blog, if you have ever been on their blog you will have noticed that they have a voiced by amazon polly section. What to Expect Honestly I don't know this is all new to me and I dont have much to go off of. For now its a test that may or may not work out. I will say that the time th

gatsby-remark-embedder

Inspired by discourse's link expansion I am rolling out expansions for one line links on the blog waylonwalker . I was able to find a gatsby plugin gatsby-remark-embedder that expands one line links for social cards for popular platforms like twitter and YouTube through a repose from Kyle Mathews to my tweet. https://twitter.com/kylemathews/status/1329817928666005504 Use Cases This covers a couple of use cases I have with very little effort. Twitter YouTube install This was super quick and sim

Expand One Line Links

I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in vanilla markdown in GitHub. I have been using a snippet that puts HTML into the markdown. While this works, it's more manual/difficult for me does not look the best, and does not read well as Goals for new card The new card should be fully automated to expand with title, description, and cover image. Bonus if I am able to attach a comment behind it. fully automated card expa

Find and Replace in the Terminal.

grepr grepd CocSearch

Resume Tips

customize for the job Why are you a good fit? What will you bring to the role? Give real outcomes give real experience Stop tech vomiting if you link to GitHub Make a profile readme Guide me to your best work have some activity if you link to LinkedIn Provide some benefit that is not on your resume Have a logical flow of experience (dont make me hunt for past experience) Keep it under 2 pages Who you know. Reference real experience Deployed 12 data pipelines with over 500 nodes to process 200GB

Codeit Bro Interview

profile image use this profile image Please share your professional role as a data scientist? [Also feel free to share about your personal projects, publications, etc.] I graduated with a Mechanical Engineering Degree 8 years ago. Much of my work early in my career was wrapped around analyzing larger datasets for my group to understand quality, drive changes to improve quality or prove that quality was already good. Three years ago I made the switch to Data Science and have loved every minute o

reasons-to-kedro

There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processing DataFrames from many data sources should be considering a pipeline framework. Kedro is a great option that provides many benefits for teams to collaborate, develop, and deploy data pipelines [[ what-is-kedro ]] Starter Template Kedro makes it super easy to get started with their cli that utilizes cookiecutter under the hood. [[ create-new-kedro-project ]] read more about how

Reasons to Kedro

Reasons to Kedro collaboration Sharable catalog small nodes over monolithic notebooks catalog easily load anything without needing to run No need to write read/write code pipeline No need to keep execution order in your head easily run a slice of a pipeline plugins pip install make your own hooks flexible expandable cli Reasons Not to Kedro Already utilizing another DAG framework Data is not in a widely supported format Micro short-lived project Large Project / Deadline Use a lower profile proje

Reading List

Latest Post [[ latest ]] STOP LEAVING Browser Tabs open and save them here! jbrancha til The Video Course Launch that Made Me Think photo prism box python library kedro on hn How can a Data Scientist refactor Jupyter notebooks towards production-quality code? Sourcing vs executing in Bash Should We Follow The Open-Closed Principle? Create multi-dimensional arrays in pure Python: The Correct Way Beware of These 9 Red Flags in a Developer Interview How to Overcome Impostor Syndrome as a Developer

What's New in Kedro 0.16.6

Kedro 0.16.6 is out! Let's take a look through the release notes Deployment Docs This is really exciting to see more deployment options coming from the kedro team. It really shows the power of the framework. The power of some of these orchestrations options is incredible. Argo Prefect Kubeflow Batch SageMaker Most of them hinge on a sweet combination of the kedro cli, docker image, and the pipeline knowing your nodes dependencies. Argo, Prefect, and Kubeflow have an interesting technique where

A brain dump of stories

I started making stories as kind of a brain dump a few times per day and posting them to [LinkedIn](https://www.linkedin.com/in/waylonwalker/(https://www.linkedin.com/in/waylonwalker/). Here are the last 11 days of stories. I store all the stories on my website with the hopes of doing something with them on my own platform eventually. For now it makes it easy to make these posts. Stories 10-10-2020 - 10-21-2020

Fix git commit author

I was 20 commits into a hackoberfest PR when I suddenly realized they they all had my work email on them instead of my personal email 😱. This is the story of how I corrected my email address on 19 individual commits after already submitting for a PR. Change the email for this repo Prepare for rebasing start the rebase 🛠 Fix First wrong Commit Fix all commits Done ReCap Change the email for this repo stop the bleeding Before anything else set the email correctly! Prepare for rebasing First thing

Designing a "Router" for kedro

nodes_global I released a router-like plugin for kedro back in April 2020. This was not the first design, the idea actually came from one of the QB folks who taught me kedro nearly a year before. We were assembling our pipelines with something called . It worked fairly well but did have some issues around being set as a global variable. But... One thing in particular that it did not lend itself well to was being able to create a packagable pipeline that I could pip install and append into an

Reclaim memory usage in Jupyter

Today I ran into an issue where we had a one-off script that just needed to work, but it was just chewing threw memory like nothing. It started with a colleague asking me How do I clear the memory in a Jupyter notebook, these are the steps we took to debug the issue and free up some memory in their notebook. How do I clear the memory in a Jupyter notebook? Pre check the status of memory There are a number of ways that you can check the amount of memory on your system. The easiest is not necessa

Strip Trailing Whitespace from Git projects

A common linting error thrown by various linters is for trailing whitespace. I most often use flake8. I generally have [pre-commit]( https://waylonwalker.com/pre-commit-is-awesome hooks setup to strip this, but sometimes I run into situations where I jump into a project without it, and my editor lights up with errors. A simple fix is to run this one-liner. One-Liner to strip whitespace bash read more about how [[pre-commit-is-awesome]]

Chrome Extensions I use

There are many useful chrome extensions out there. I probably have way too many installed, here are four that I am currently using. This post was inspired from Chris over at daily-dev-tips LastPass Stylus Vimium hypothesis LastPass Love it or hate it passwords are hard to manage. Everyone needs a password manager to avoid the dreaded password reuse, and to be able to quickly rotate them with a service. I use lastpass, thus it's browser extension is my most used extension. lastpass Stylus Sty

Creating Reusable Bash Scripts

Bash is a language that is quite useful for automation no matter what language you write in. Bash can do so many powerful system-level tasks. Even if you are on windows these days you are likely to come across bash inside a cloud VM, Continuous Integration, or even inside of docker. I have three techniques that help me write more composable bash scripts. functions Arguments positional arguments All Arguments Error Handling main script Functions Break scripts down into reusable components Functio

Three things to Automate with Python using Pandas

Here are three things that I see my non programming counterparts doing every single day. These really sum up so much of what folks do within an office. So many of us dabble in or become power users of spreadsheets without knowing there is an alternative out there that can save us time, automate boring things, and allow us to open up our minds for the part that we add value, Thinking about the data. Focus on Value Add Operations Lets face it, stitching together spreadsheets is zero value add by

How to Install miniconda on linux (from the comman...

miniconda is a python distribution from continuum. It's a slimmed-down version of their very popular anaconda distribution. It comes with its own environment manager and has eased the install process for many that do not have a way to compile c-extensions. It made it much easier to install the data science stack on windows a few years ago. These days windows are much better than it was back then at compiling c-extensions. I still like its environment manager, which installs to a global directory

How to crush amazing posts on DEV

This post was inspired by a comment I left on @dsteenman's post. {% post dsteenman/how-long-should-a-blogpost-be-2k6n %} Most of the time I prefer short as I am more likely to read the whole thing. If its setup as a series I am more likely to work my way through the whole series in a matter of a few sessions. Just my preference I will say though there are certain articles that fit well to the long format. They are articles that folks tend to come back to often as a reference again and again.

Black Tech Pipeline

I was particularly inspired by @chantastic episode 103 of the react podcast with @ParissAthena. They spoke about the black tech pipeline as well as Diversity, Equity, and Inclusion. Pariss is quite an inspiration. She has done so much work to create a better place for POC in tech. I like that not only is she helping them get jobs but acting as a mentor for their first few months on the job to make sure that they are able to find their place and fit in. Based on an episode of react podcast. 🎙 Lis

Review of the git-auto-commit-action

It's a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template. Check out the repo for git-auto-commit-action . It's a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template. This has been by far the easiest way to commit back to a repo that I have seen. Other patterns often require fully setting up the git

What's New in Kedro 0.16.4

If we take a look at the release notes I see one major feature improvement on the list, auto-discovery of hooks. This one comes a bit surprising as it was just casually mentioned in #435 auto enabled plugins mentioned in issue 435 Think pytest As mentioned in #435 this is the model that pytest uses. Not all plugins automatically start doing things right out of the box but require a CLI argument. simplicity It feels a bit crazy that simply installing a package will change the way that your

Integration testing with Python, TestProject.io, a...

!!! Caution None of the testproject.io urls resolve anymore in JAN 2025, I removed all of the broken links. As I continue to build out waylonwalker.com I sometimes run into some errors that are not caught because I do not have good testing implemented. I want to explore some integration testing options using GitHub's actions. Running integration tests will not prevent bugs from happening completely, but it will allow me to quickly spot them and rollback. 🤔 What to test first? The very first t

New Machine for developing Tests with TestProject....

Today I setup a new machine on Digital Ocean to use with TestProject.io, Here are my installation notes. envsubst < .github/ci/docker-compose.yml > docker-compose.yml

🐍 Practice Python Online

When learning a new skill it's important to practice along the way. In order for me to show up to practice I need to make it easy to show up. An easy way to show up to practice with python is to use an online repl. With these you can try out something quick. Sometimes I see snippets from blogs or tweets and I need to try the out for myself to really understand. When learning a new skill it's important to practice along the way. In order for me to show up to practice I need to make it easy to sh

Kedro Catalog

I am exploring a kedro catalog meta data hook, these are some notes about what I am thinking. Process metadata will be attached to the dataset object under a attribute metadata will be updated metadata will be empty until a pipeline is ran with the hook on optionally a function to add metadata will be added metadata will be stored in a file next to the meta Problems This Hook Should solve what datasets have a columns with in the name what datasets were updated after last tuesday which pipe

How python tools configure

mypy Mypy's config parser seems to be one of the most complex. This is likely in part to it having the largest backwards compatability of all projects that I looked at. mypy/config_parser flake8 options/config.py black black portray only uses pyproject.toml portray/config.py interrogate only uses pyproject.toml

Edit On GitHub

I recently added a button to my blog, and subsequently my posts on DEV.to . It's the best thing that I have done for it in a while. It makes it so easy to do quick edits. finding errors I refer back to my old posts quite a bit, sometimes I find errors in them. Honestly most of the time its too much effort to load up my editor make the change and and . It's not much, but when I am referring to my own post generally I am just trying to get something done and don't have time for that. The slu

Why use a cms

When first learning to code its very common to hard code everything right into the code. This happens with most folks in just about any language. Whether its HTML or markdown for front end content, or even hardcoding parameters in our backend languages like python, or node.js. 🤷‍♀️ What's wrong with hard coding everything? Hard coding everything right into your code makes it really hard for non-technical collaborators to join. It makes it nearly impossible to hand websites off to clients without

🐍 Parsing RSS feeds with Python

I am looking into a way to replace my google reader experience that I had back in 2013 before google took it from us. I am starting by learning how to parse feeds with python, and without much previous knowledge, it proved to be much easier than anticipated thanks to the library. This is how I used python to parse rss and setup my own custom feed. Install Install the feedparser library. Get the content The feed object The feed is a feedparser.FeedParserDict. For all intents and purposes this

Reader-2020

Inputs The input will be a yaml file containing a list of you want to stay up to date with. Inside each item will be a url, and weight. Types rss feed (primary source) youtube feed Stack Overflow tags GitHub repo activity pypi release dev.to post Twitter Search # user will need an api key Methodology Each url will be pulled in and parsed into a standard data scructure. Some items may yield special feaures, a schemaless/nosql datastructure may be best. Pipeline will decide to how to weight p

🤓 What's on your GitHub Profile

I ran this post on dev.to and got a great response of great examples, check it out . dev.to whats-on-your-github-profile The GitHub profile feature just went live for a subset of users. Simply creating a repo named after your username, and clicking share to Profile on the sidebar will create a custom profile that shows up just above your pinned projects. I am still trying to figure out what to put on mine, but this is what I have so far. I feel like mine is a bit big at the moment, I don't li

🙋‍♂️ Can Anyone Explain Twitter Cards to me?

Can someone explain how or why twitter cards render differently from device to device? I do understand that twitter cards a built from meta tags, the full list can be found in their docs Rendered on Mobile Mobile Looks fine. rendered card Not Rendered on Desktop On Desktop it is not picking up the image. not rendered card Twitter Card Validator The Validator renders the card correctly. I tried the official twitter card validator , as well as heymeta.com , and metatags.io . All look good. re

How I Built My GitHub Profile

I ran a discussion on dev that collected quite a list of examples in the comment section. So many great calls to action, animations, memes, and weird tricks. dev.to whats-on-your-github-profile My current profile Waylon Walkers GitHub profile social icons Upload all of your icons to the repo in a directory such as or , then link them with a attribute like below. I used html for mine, not sure if you can set the in markdown. note I did add a bit of (non-breaking-whitespace) between m

SLIDES - understanding python \*args and \*\*kwarg...

Python and are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packed and unpacked into lists or dictionaries. Beware though, this power can lead to some really unreadable/unusable code if done wrong. I generally post these as a carousel on LinkedIn based on a full article. Let mw know what you think of it shown inside of a blog @_waylonwalker . python args and kwargs

Gracefully adopt kedro, the catalog

Why use kedro catalog? While using the catalog alone will not reap all of the benefits of the framework, it does get you and your project ready for the full framework eventually. For me the full benefit of the catalog comes when you combine it with the pipeline and dont even touch read/write steps at all. Taking a step into kedro by adopting the catalog first will give you a way to organize all of your data loads in one place, and stop manually writing read/write code, which can be different fo

How to find things in your kedro catalog

kedro 0.16.2 just dropped last week with a long-awaited feature... catalog search ! I went as far as monkey patching this into each of my projects. I work jump between a few really big projects that have tons of datasets. Being able to quickly search for what I need is so useful. The Catalog The kedro data catalog is a key component to the kedro framework. It handles all data loading and saving for you. It is configurable and hackable. Having all your data connections listed in one place

How Kedro handles your inputs

Passing inputs into kedro is a key concept. Understanding how it accepts a single catalog key as input is quite trivial that easily makes sense, but passing a list or dictionary of catalog entries can be a bit confusing. *args/**args review Check out this post for a review of how work in python. [[ python-args-kwargs ]] python args and kwargs article by @_waylonwalker All Kedro inputs are catalog Entries When kedro runs your pipeline it uses the catalog to imperatively load your data, mea

My first eight years as a working professional.

This day 8 years ago I started my first day as a Mechanical Engineer. I am so grateful for this journey that I have been able to have. There is no way that I could have planned this journey from the beginning. Keep Learning My initial career plans were down a completely different path. I have been very flexible in taking on a new career path. I have been eager to learn new things and respond to life changes that I never would have imagined. Life Changes Very severe chronic health issues from my

Refactoring your blog urls

I just did a quick refactoring of my JAMStack blog urls. Some didn't fit with my style, some had that I wanted to switch to , and others were ridiculously long. I've been using forestry as my CMS, I write many of my posts there, and sometimes it picks some crazy file names (based on my titles). It was time to refactor. [[ refactor-in-cli ]] When refactorings similar to this get really big I often need to do some project wide find an replace, I usually do this right from the command line. 🖊

Master No More

It's been a long time coming. We use some very harsh language within tech so much sometimes that we become numb to it. It's time to do my very small part in this movement and purge this language from my active repos starting with this blog right here. [[ refactor-in-cli ]] this post follows my method of refactoring code bases from the command line, read more about that in this article. c-s-f First off browsing through the content of my blog I found many references to master. I cannot complete

understanding python \*args and \*\*kwargs

Python and are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packed and unpacked into lists or dictionaries. Beware though, this power can lead to some really unreadable/unusable code if done wrong. Python and are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function a

Building kedro.dev

Follow along the Journey as I build out kedro.dev . Building a Community I have really enjoyed my own personal journey as I have started to build all of my data pipeline projects with the kedro framework. I want to start building a place to share resources with the community. I want to see this community grow and flourish. They say in front end web development if you are not using a framework you end up building one. That's exactly what I was doing before I started using kedro. I want to b

pre-commit is awesome

I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really it's easy. It will automatically run checks for you. In some cases, it will even automatically fix them for you. Out of the box, it will do things like automatically trim extra whitespace, fix file endings, and ensure file sizes are not too large for git. I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it

Kedro Static Viz 0.3.0 is out with Hooks Support

kedro-static-viz is out with support for the newly released hooks feature. This means that you can have automatically deploy a full gatsby site keeping your visualization always up to date. Even though it is a static site there is no functionality lost. The only thing that's missing is the flask server. With kedro-static-viz you can deploy your visualization to a number of static hosting providers such as GitHub pages free of charge with wicked fast performance ⚡ It's Fast Even though

Create Configurable Kedro Hooks

There are two main ways to create kedro hooks, with modules and classes. Each one still uses the same verbiage as the function/method names. Class hooks seem a bit special as they give you a way to configure them so that they are a bit more generally useful. [[ what-is-kedro ]] If you are completely unsure what kedro is be sure to check out my what is kedro post Installation .create a new environment manager of choice. Here I will use . Then we will install from pypi. Create a sample proj

Brainstorming Kedro Hooks

This post is a 🧠 branstorming work in progress. I will likely use it as a storage location/brain dump of hook ideas. What is Kedro 🤔 If you are completely unsure what kedro is be sure to check out my what is kedro post after_catalog_created filepath replacer bucket replacer before_pipeline_run preflight check that data exists run run mypy run interrogate run flake8 after_pipeline_run Great Expectations send email send slack before_node_run after_node_run Great Expectations save stats/meta da

How to get Dev Comments from an article Url

I want to incorporate some of the wonderful comments, \U0001F495, \U0001F984, and \U0001F516's that I have been getting on dev.to on my website. I have dabbled once or twice with no avail this time I am taking notes on my journey, so follow along and let's get there together. By the end of this post, I will have a way to get comments from posts on the client-side thanks to the wonderfully open dev.to API. I want to incorporate some of the wonderful comments, 💕, 🦄, and 🔖's that I have been gett

Four github actions for your website

GitHub's actions are a new GitHub feature that will trigger GitHub to spin up a virtual machine and run some tasks with some special access to your repo. It can interact with comments/issues, it can clone your repo, You can explicitly pass in secrets so that it can commit back to the repo or deploy to another service. The environment may be a Linux, windows, or even a mac machine. I believe this is wildly incredible for the open-source community, putting these tools in the same place that we are

Adding google fonts to a gatsbyjs site

stack overflow link

Create Custom Kedro Dataset

Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For the most part you just need to tell Kedro what connector to use and its filepath. When running Kedro takes care of all of the read/write, you just reference the catalog key. But what is happening behind the scenes Under the hood there is an that each connector inherits from. It sets up a lot of the behind the scenes structure for us so

Interrogate is a pretty awesome, brand new, cli fo...

As usual while listening to python bytes 181 I heard of a tool that I had to try out right away! This thing is 🔥 hot off the press folks, we're talking the first release only 3 weeks ago. Its something that the python community needed years ago, and it belongs in your CI today . I had tried several tools that tried to do docstring coverage in the past but they were a bit cumbersome and were quickly forgotten about. Not interrogate, its dead simple! Nothing I have tried has come close to being

drawing ascii boxes

When creating cli's I often want some nice full-width character. I find it tough to find them, and when I do half the time it is an image or something that cannot be copied 👿. I rarely get very complex with my semi-manual ASCII art. I can do 98% of what I need with bars and corners. Using some simple full-width characters can really give your cli a nice clean look. Example I'd say 50% of what I need is just a full-width horizontal bar to give some visual flair or separation. Bars Square Corn

creating the kedro-preflight hook

Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro . They allow you to hook into , , and (nouns). With a , or (adjective). This really reminds me of reacts lifecycle hooks, that let you hook into various state of react web components. This is going to make kedro so extendable by the community. I am super pumped to see what the community is able to do with this ability. kedro hooks are an exciting upcoming feature of kedro . They allow you to hook into , , and

📝 Kedro Preflight Notes

This is a very rough idea for a kedro package to prevent time lost to get partway through a pipeline run only to realize that you dont have access to data or resources. Must Haves check that inputs exist or are of a type to skip (sql) Good to haves check that all input and output databases are accessible with good credentials check for s3 bucket access check for spark install Implementation run params

Maintianing multiple git remotes

📢 Announcing find-kedro

is a small library to enhance your kedro experience. It looks through your modules to find kedro pipelines, nodes, and iterables (lists, sets, tuples) of nodes. It then assembles them into a dictionary of pipelines, each module will create a separate pipeline, and being a combination of all pipelines. This format is compatible with the kedro format. Python package Test Build-Docs Motivation is a ✨ fantastic project that allows for super-fast prototyping of data pipelines, while yielding

Explicit vs Implicit Returns in Javascript

Often when reading through javascript examples you will find some arrow functions use parentheses while others use braces . This key difference is that parentheses will implicitly return the last statement while braces require an explicit return statement. It is important to understand the difference between them because it is likely that you will find code examples of both and trying to edit code written differently than you're used to may have unintended consequences. Arrow functions Arro

Twitter deepdives

Inspired by Chris Achard My ideas Python List comps Classes Inheritance Background Click Lambdas Kedro Cataloging Custom datasets Reusable pipelines find-kedro Learn kedro in 5 days Email course inspired by learn d3 in 5 days Mail Share your knowledge Practice Practice in public Make practice easy Share your notes Digital Gardening Own your content Build your audience Be nice Have empathy Learn your way Continuous learning

python-deepwatch

Is it possible to deep watch a single python function for changes? Shallow Watch keeping track of a python functions hash is quite simple. There is a method attached to every python function. Calling it will return a hash of the function. If the function changes the hash will change. Using hashlib provides a consistent hash. Now we have a consistent way to hash function code. Deep hashing Find dependencies setup a function in a module with a dependency Create Generic module importer by filepath

TIL: Bind arguments to dynamically generated lambd...

This past week I had a really weird bug in my kedro pipeline. For some reason data running through my pipeline was coming out completely made no sense, but if I manually request raw data outside of the pipeline it matched expectations. NOTE While this story is about a kedro pipeline, it can be applied anywhere closures are put into an iterable. Debugger to the rescue After a few days of looking at it off and on, I pinpointed that it was all the way down in the raw layer. Right as data is com

Four Github Actions for Python

If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have such a simple cli I have opted to use the cli for most of these, that way I know exactly what is happening and have more control over it if I need. If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have such a simpl

Variables names don't need their type

So often I see a variables inside of its name and it hurts me a little inside. Tell me I'm right or prove me wrong below. Examples Pandas are probably the worst offender that I see Sometimes vanilla structures too! Edge Cases? It's so common when you need to get inside a data structure in a special way that itsn't provided by the library.... I am not exactly sure of a good way around it. Containers are plural Always name your containers plural, so that naming while iterating is simple. Befo

Send Emails with GitHub Actions

Here is one useful thing that you can do with GitHub actions no matter what language you use, send email. You might want to know right away when your ci passes. You might want to give your team a nice pat on the back when a new release is deployed. There might be subscribers wanting to see the latest release notes in their inbox as soon as the latest version is deployed. Whatever it is, its pretty easy to do with an action right out of the actions marketplace. Mail on Star Here is a silly ex

What Are GitHub Actions

GitHub actions are an amazing tool that allows us to run code based on triggers inside of our repo. Their is a large and growing community of actions inside the marketplace to use with very little effort. Best of all they are free for public repositories, and private repos have a very generous free tier. I have been diving deep into Github actions for about a month now and they are wicked good! They allow you to run any sort of arbitrary code based on events in your repo, webhooks, or schedul

Getting Started with GitHub Actions

Github actions are written in configuration files using the YAML syntax. YAML is a superset of JSON. Most YAML can be expressed inline with JSON syntax. Similar to python YAML is whitespace driven by whitespace rather than brackets tags. The argument for using YAML for configuration files such as actions is that it is more human-readable and editable. It's much easier to see the whitespace layout than it is to get closing brackets correct. For actions, I believe this is mostly true. I don'

Today I learned `git diff feature..main`

Today I learned how to diff between two branches. Sometimes we get a little happy and mistakenly commit something that we just can't figure out. This is a good way to figure out what the heck has changed on the current branch compared to any other branch. Example Let's create a new directory, initialize git and toss some content into a readme. After all of that, we have a git repository on our local machine with a single file that contains the following. Create a branch and ✍ edit Let's chec

Create New Kedro Project

This is a quickstart to getting a new kedro pipeline up and running. After this article you should be able to understand how to get started with kedro . You can learn more about this Hello World Example in the docs 🧹 Install Kedro 🛢 Create the Example Pipeline 💨 Run the example 📉 Show the pipeline visualization Create a Virtual Environment I use conda to control my virtual environments and will create a new environment called with the following command. note the latest compatible vers

What is YOUR Advice for New Data Scientists

Learn the business Learn Git Your code does not need to be amazing Keep Learning Learn Git You dont have to start out as a git wizard with the cleanest possible commit history. At first dont let yourself get too wrapped up in it, the most important part is that you make commits. You will find needs for more advanced stuff later. Get comfortable with this, then learn how to , , , etc... Your code does not need to be amazing Get the job done. Keep it in small bite size pieces. Make readable

Do You Hoist

I am working through Wes Bos's beginnerjavascript.com/ I just hit module 18 on hoisting. It's something that I always knew was there, Its not something I typically see used or use myself. Do you Hoist? Do you have any use cases that you use hoising? Why? It seems like a really cool feature in any language that uses it, but I dont really notice it in use. What is Hoising There are many articles that cover this in far more depth, but its the idea that variable declarations and functions are de

What is Kedro

[[ what-is-kedro ]] This is my original what-is-kedro article. There is a brand new one Kedro is an open source data pipeline framework. It provides guardrails to set your project up right from the start without needing to know deeply how to setup your own python library for data pipelining. It includes really great ways to manipulate and . This article will cover the 10K view of kedro, future articles will dive deper into each one. kedro is an open-source data pipeline framework. It pr

Custom Scrollbar Design

Getting a custom scrollbar on your site makes it stand out a bit compared to the very plain stock one that are on most sites. This is how I set mine up on my gatsby site. Inspired by Wes Bos's new uses.tech I wanted a custom scrollbar on my personal site. I had tried to do it in the past, but gave up after it was not working. Looking at the Source Since uses.tech is open source I jumped on github, searched for scroll and found this layout.js . Copy it to my own component My first step was

Don’t waste your time learning everything

"Don’t waste your time learning everything." Inspired by this linkedIn post I felt that this comment was very powerful. Here are my 2 cents. Be Productive Stick to what you know, and learn a little bit of something new every day. If what you know is how to use Excel like a boss, don't fee ashamed that you are missing something. Be proud and use what you know. Don't Stagnate Take small steps enhance what you know now with something new that you get you closer to where you want to be. If you nee

2020 waylonwalker.com rebrand

Moving into 2020 I have been really leaning on using purple as my theme color everywhere more and more. Its time for an update to my personal site, not just because it feels plain, not just because the cover art I am using for dev.to doesn't fit my current card layout, but because I feel inspired and I want to. Starting point  This is what we are working with. It has been my card design for

I just added react-headroom to my site

It was so easy to get a professional looking navbar with just 3 lines of code. This package seriously is so usable on mobile it is ridiculous. I found this package from day-4 of the 100 days of gatsby challenge. It is by the wonderful man who brought us gatsbyjs Kyle Mathews, so you know its gotta be good. install react-headroom installation is easy Import Headroom There was no instructions for es6 style imports that are common with gatsbyjs sites like mine, but it was intuitive to figure out.

Serverless things to investigate

Social Automation Zappier Free only one step is free can take rss to social Automate.io Form submission https://getform.io Free to get started https://formkeep.com/?#pricing Free 7 day trial $4.99 cheapest after

I finally fixed my Styled-Components in gatsby.js

I finally fixed my Styled-Components in gatsby.js. I am starting a redesign of my website. I have started cross posting to dev.to more regularly. With that I have been making more detailed cover images at the recommended . These images get cut off on my own site, which is a bit ridiculous to have my own content not look right on my own site. But before we start a heavy redesign I have a small issue that has plagued the site for at least a year! I have a small issue that has plagued the si

Should I switch to Zeit Now

Netlify I have happily had my personal site waylonwalker.com hosted on netlify for nearly 2 years now. In fact I have hosted about a dozen different toy projects to play with on there, 4 of which have gone far enough to get a custom domain name. They are fast to deploy and consistently do so on every to main. Zeit I have recently started playing with zeit again. I really like their cli tool, its dead simple and makes sense. I tried the netlify one early on and dont think I really gave it

git push without setting upstream

Finally after years of hand typing out a full I found there is a setting to automatcally push to the current branch. More realisitically I just did a let git yell at me, and copying the suggestion. git config This one setting will now to the current branch without yelling at you that your upstream does not match your current branch. This helps me ship chnages faster as I am constantly chnaging projects and branches.

Out of Space

This morning I logged into my machine and was nearly out of space 64GB miniconda3! 5GB conda cache 4GM pip cache 34GB docker Find it screenshot-2025-02-12T22-32-14-298Z.png {.more-cinematic} These are the commands that I often use to reclaim space. Its so easy to fill up small vm's in the cloud, or in my case today let your dev machine go way too long without a good cleanup. Show Remaining Space on Drives This shows us where to start and gives a baseline of how much space we have reclaimed. sho

Personal URL shortener with Netlify Redirects

I love using URL shorteners to easily share links without hitting character limits, but they loose their meaning. Services like bit.ly will save my links for me so that I can find them, but I would rather them to be easy to remember. https://bit.ly/2ruLwQz does not roll of the tongue so well. 301 🤸‍♀️ I recently discovered a really cool feature of netlify that I have always looked past, . It is so simple cool and powerful, every netlify site should do this! But how 🤷‍♀️ simply add a file to

simple click

cli tools are super handy and easy to add to your python libraries to supercharge them. Even if your library is not a cli tool there are a number of things that a cli can do to your library. Example Ideas Things a cli can do to enhance your library. 🆚 print version 🕶 print readme 📝 print changelog 📃 print config ✏ change config 👩‍🎓 run a tutorial 🏗 scaffold a project with cookiecutter 🖱 Click Click is the most popular python cli tool framework for python. There are others, some old, some new c

cmd.exe tips

I spend a lot of my time at the terminal for my daily work, mostly in Linux or wsl. One big reason for using wsl over cmd.exe is the ease of walking through history that fzf provides. This week we had a windows bug in a cli and I was stuck in vanilla cmd.exe 😭 > Cmder First off if you are stuck using cmd.exe, do yourself a favor and get cmder. It makes life just a bit easier. It is super confugurable and comes with several power ups that make it a bit more enjoyable than cmd.exe. History F

What is something you should have learned or under...

Mine is the python debugger. I was a long holdout thinking that print statements were sufficient. That was untill I started having errors crop up in functions that took minutes to run. The thing that I most notably wish I would have known about is post_mortem. Example Debug with iPython/Jupyter Vanilla Debug More For more information about the debugger checkout the real python article. https://realpython.com/python-debugging-pdb/ Also keep a bookmark of the table of pdb commands from the articl

Supercharge Zsh Startup

I have been using oh-my-zsh successfully for about 2 years now. But lately my startup time has been really bothersome. It has grown to the point where it was taking about 5.5s to startup a shell! This is ok if I am going to spend some time in here for awhile and do some work that benefits from all of the autocompletions, plugins, and shortcuts that oh-my-zsh brings. But to only jump in to run a handful of commands is infuriating. 📑 My Setup I believe the real issue is io speed on wsl. I hav

Keep Location List Closed

Vim's (neovim in my case) location list can provide some very useful information while developing. Mine gives me information about linting and type checking errors with fairly little config. Generally, it sits nicely at the bottom of the screen and barely affects me. Other times, especially while zoomed way in during a presentation, it just gets in the way. location list eats the screen Location List eating up the screen while I am zoomed in and trying to live code Toggling the location list

SqlAlchemy Models

Make a connection Make a session Make a Base Class Make your First Model Make your own Base Class to inherit From Use the Custom Base Class

Building Cli apps in Python

Packages Click Inputs Click primarily takes two forms of inputs Options and arguments. I think of options as keyword argument and arguments as regular positional arguments. Option typically aliased with a shorthand ('-v', '--verbose') **From the Docs To get the Python argument name, the chosen name is converted to lower case, up to two dashes are removed as the prefix, and other dashes are converted to underscores. Argument positional required no help text supplied by click Yaspin Yaspin Gif Cl

Kedro

See all of my kedro related posts in [[ kedro-feed ]]. #kedrotips I am tweeting out most of these snippets as I add them, you can find them all here #kedrotips . 🗣 Heads up Below are some quick snippets/notes for when using kedro to build data pipelines. So far I am just compiling snippets. Eventually I will create several posts on kedro. These are mostly things that I use In my everyday with kedro. Some are a bit more essoteric. Some are helpful when writing production code, some are useful mo

📝 Packages to Investigate Notes

jmespath Tabnine Bulwark |-|-| |github: | https://github.com/zaxr/bulwark | I definitely want to try this out with kedro. Bulwark is a package for convenient property-based testing of pandas dataframes, supported for Python 3.5+. Example

Debugging Python

Using pdb

Just Use Pathlib

Pathlib is an amazing cross-platform path tool. Import Create path object Current Directory Users Home Directory module directory Others Let's create a path relative to our current module. Check if files exist Make Directories rename files List files Glob Files recursively Write

Custom Python Exceptions

Custom Exceptions

Filtering Pandas

query Good for method chaining, i.e. adding more methods or filters without assigning a new variable. masking general purpose, this is probably the most common method you see in training/examples isin capable of including multiple strings to include contains Good For partial matches MASKS anything that we put inside of square brackets can be set as a variable then passed in. Operators & - and ~ - not | - or AVAILABLE and NAME AVAILABLE or NAME AVAILABLE and not NAME

Digital Ocean

I love digital ocean for it's simplicity and its commitment to open source.

Quick Progress Bars in python using TQDM

tqdm is one of my favorite general purpose utility libraries in python. It allows me to see progress of multipart processes as they happen. I really like this for when I am developing something that takes some amount of time and I am unsure of performance. It allows me to be patient when the process is going well and will finish in sufficient time, and allows me to 💥 kill it and find a way to make it perform better if it will not finish in sufficient time. for more gifs like these follow me

Clean up Your Data Science with Named Tuples

If you are a regular listener of TalkPython or PythonBytes you have hear Michael Kennedy talk about Named Tuples many times, but what are they and how do they fit into my data science workflow. Example As you graduate your scripts into modules and libraries you might start to notice that you need to pass a lot of data around to all of the functions that you have created. For example if you are running some analysis utilizing , , and data. You may need to calculate total revenue, inventory

Background Tasks in Python for Data Science

This post is intended as an extension/update from background tasks in python . I started using the week that Kenneth Reitz released it. It takes away so much boilerplate from running background tasks that I use it in more places than I probably should. After taking a look at that post today, I wanted to put a better data science example in here to help folks get started. This post is intended as an extension/update from background tasks in python . I started using the week that Kenneth R

📝 Bash Notes

Bash is super powerful. File System Full Show Remaining Space on Drives show largest files in current directory Move files then symlink them Fuzzy One Liners edit in vim cat a file bash execute git add git reset Kill a process Finding things Files fd-find is amazing for finding files, it even respects your file 😲. Install with . ++Vanilla Bonus Content ** show matching text ** ++Vanilla Bonus ** show file names only ** ++Vanilla Bonus Recursively Replace text ++Vanilla Bon

Autoreload in Ipython

I have used for several years now with great success and 🔥 rapid reloads. It allows me to move super fast when developing libraries and modules. They have made some great updates this year that allows class modules to be automatically be updated. What I like about autoreload 🔥 Blazing Fast 💥 Keeps me in the comfort of my text editor 👏 Allows me to use Jupyter when I need 👟 Extremely Reliable One of the biggest benefits that I find is that it shortens the distance between my module/library co

Keyboard Driven VSCode

Throw that mouse Away its time to setup some keyboard shortcuts. These sortcuts were the baseline for switching from tmux/vim to vscode. Most folks posts I was able to find gave great tips on replacing vim, but very few have focused on the hackability of tmux. tmux allows me to rapidly fire up a workspace, create new windows and splits. Then When I switch tasks I can leave that workspace open and and jump right back in later exactly where I left off. There is nothing quite like it. The shor

Realistic Git Workflow

My git workflow based on real life. Its not always clean and simple. sometimes things get messy The Clean Path pull 👉 branch 👉 format 👉 work👉 add 👉 commit 👉 pull 👉 rebase 👉 push Pull As complicated as that seems it is pretty straight forward. When you sit down to work the first thing you do is to pull down the teams latest working "develop" branch from git. Branch Next create a new branch with a name that will remind you of what you are working on. For your own sanity choose something d