So many terminal applications bind After digging way too deep into how IPython implements its It was that simple. This is not a game changer by any means, but I will now
2022-6-13
til
It Yes the solution comes from a competing templating framework. I install copier with pipx, so I need to inject cookiecutter in to my copier If you are using a normal virtual environment you can just pip install it. Now to enable the extension you n
2022-6-11
til
I love getting faster in my workflow, something I have recently added in is Check out what I created here.
2022-6-10
til
Textual has devtools in the upcoming css branch, and its pretty awesome Textual is still very early and not really ready for prime time, but it As https://twitter.com/willmcgugan/status/1531294412696956930 Textual is a tui application framework. Unl
2022-6-9
til
Whenever you are installing python packages, you should always use a virtual Pip is the pacakage tool for python. It installs third-party packages and is python packages often require many different dependencies, sometimes packages My one exception
2022-6-1
til
I I talked about why and how to use pyenv along with my first impressions in According to libsqlite3-dev When I make a fresh env and install ipython I still get the same error and I am After having this issue for awhile an coming back to I think I h
2022-5-31
til
Sometimes you have a pretty old branch you are trying to merge into and you are The first step is to make sure your local copy of the branch you are moving It Merge the changes from main into Now is where the merge conflict may have started. If you
2022-5-24
til
A few of my friends and I all just borked our neovim configs during a plug https://twitter.com/pypeaday/status/1524882893914398732 Lucky for me I did it on a home machine that I only occasionally edit from, Snapshotting ensures that you install the s
2022-5-13
til
https://youtu.be/-42A5210HYo Super fun steam Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/waylonwalker We worked on markata todoui, a command tui trello board written in python using only markdown files to store the data. I love
2022-5-12
blog-post
I really like the super clean look of no status menus, no url bar, no bookmarks
2022-5-11
til
I am often editing my own scripts as I develop them. I want to make a better Currently I am combining for now lets use my todo command as an example On first pass I made a bash function to do exactly what I have been doing. The Note, I use bash fun
2022-5-10
til
I am getting ready to do some timeseries analysis on a git repo with python, my This returns a generator, if you are iterating over them this is likely what The generator will return
2022-5-9
til
I was editing some blog posts over ssh, when I ran into This is the error message I was seeing. The fix ended up being pretty simple, but quite a ways down this This is what it looks like when it asks for the passphrase.
2022-5-8
til
Sometimes you get a PR on a project, but cannot review it without wrecking your This will create a new directory
2022-5-4
til
GitPython I recently made myself a handy tool for making screenshots in python and it https://waylonwalker.com/screenshot-to-blog/ GitPython Import Repo from the git library and create an instance of the from the docs It provides abstractions of git
2022-4-30
til
Python, click install Edit the System Environment Variables Environment Variables button Add the following path to your users Path Variable
2022-4-18
til
Sometimes you just want python to do something else when you hit an exception, I am working on a quick and dirty python script designed to take screenshots I could have gone down a logging route, but honestly this is meant to be quick, Python exposes
2022-4-10
til
I recently was unable to boot into my home Linux Desktop, it got stuck at https://twitter.com/ There Normally you have 6 TTY ctrl+alt+F1: login screen ctrl+alt+F2: Desktop ctrl+alt+F3: TTY 3 ctrl+alt+F4: TTY 4 ctrl+alt+F5: TTY 5 ctrl+alt+F6: TTY 6 In
2022-4-8
til
pygame events are stored in a queue, by default the most suggested way You don Let printing the events reveal this Let printing the events reveals that there are no events, well i Even simple games don Store events for each frame in memory. I thought
2022-4-7
til
One of the most essential concepts of pygame to start making a game you will I started by making a spotlight in Gimp, by opening a 64x64 pixel image and This is what it looks like Now we can load this into pygame. To make pygame a bit more efficient
2022-4-6
til
From the same Author that brought us command line essentials like You can install from one of the Something I often do to blend colors together is add a little alpha to Here is one from the docs that show how you can generate a color palette from I
2022-4-5
til
Browsing through twitter the other day I discovered it through this https://twitter.com/ Before I dive in deep, I do want to mention that Dunk is super new and beta at The author really cautions the use of it as your default pager this early, I He no
2022-4-4
til
I I What currently have is a single module starter package that is on github Since it You can make a quick game by inheriting from Game, and calling Here is what the current
2022-4-3
til
This morning I was trying to install a modpack on my minecraft server after So I Then I go back to my server and download the modpack with wget. Now I can unzip my mods into the I run the minecraft server with docker, which is setup to mount the htt
2022-4-2
til
My personal Site build went down last week, and I was unable to publish a new Classic IT fix, rerun it and see if you get the same error. Everyone is busy Also worth a check to see if GitHub is having a hiccup or not. This error felt Check Alright
2022-4-1
til
I ran into a PR this week where the author was inheriting what BaseException Try running these examples in a Since things such as If you except from exception or something than inherits from it you will be When you make custom exceptions expect tha
2022-3-31
til
When I need a consistent key for a pythohn object I often reach for Yesterday we talked about setting up a persistant cache with python diskcache. https://waylonwalker.com/til/python-diskcache/ My first thought was to just hash the files, this will g
2022-3-30
til
When I need to cache some data between runs or share a cache accross multiple Install diskcache into your virtual environement of choice using pip from your command line. There are a couple of different types of cache, Adding to the cache only needs
2022-3-29
til
The easiest way to speed up any code is to run less code. A common technique The easiest and most common to setup in python is a builtin functools.lru Any time you have a function where you expect the same results each time a when same lru lru lru
2022-3-28
til
I keep a small I recently switched hosting from netlify over to cloudflare. Well cloudflare This breaks my go to example dataset. What After a bit of googling I realize that this is a common thing, and that setting Now this works again, but it feel
2022-3-27
til
Python Requests is on pypi and can be installed into your virtual environtment with pip. Requests makes getting content from a web url as easy as possible. Requests can handle any web request and is not limited to only html. Here are There is way mo
2022-3-26
til
I recently attended Start with some html boilerplate, pop in a script tag to add the Save this as If you don Now the page has a button that is ready to replace itself, notice the To make it a bit clearer here is what the file tree looks like after I
2022-3-25
til
I recently gave a talk at python web conf 2022, and one of the things I did Lets use this section to show what it ☝ This is how my write markdown build site publish This is what the above slide looks like in lookatme. The way I write my slides I wa
2022-3-24
til
I use a package It Frontmatter is a handy way to add metadata to your plain text files. It Here is the exact frontmatter for this post you are reading on my site. yaml is the most commmon, but If you want a good set of examples of yaml Here is how I
2022-3-23
til
Today I was watching the python web conf 2022 and saw I https://waylonwalker.com/python-args-kwargs/ More on unpacking in this post. With the release there is also a new update syntax Are you writing libraries/applications that are only going to be
2022-3-22
til
I love the freedom of writing in markdown. It allows me to write content from I will most often just present right from the terminal using I sometimes also use reveal.js, but that I leverage auto slides when I write my slides in markdown. The large
2022-3-21
til
When I need to read contents from a plain text file in python I find the
2022-3-20
til
Let Before you run someone Something that I have always appreciated form Searchng through the internet I was able to find an article from Here is my interpretation of the code I took from Vitaly So far it is working for me and saving me a few seconds
2022-3-19
til
If you ever end up on a linux machine that just does not have enough ram to You can put this where you wish, for this example I am going to pop it into You can make sure that your swap is working by using the https://waylonwalker.com/reset-ipython I
2022-3-18
til
A very common task for any script is to look for files on the system. My go to I setup a directory to make some examples about globbing. Here is what the 1 directory, 8 files Pathlib is a standard library module available in all LTS versions of pyt
2022-3-17
til
Setting up your git pager to your liking can help you navigate diffs and logs You can set the pager right from your command line with the following command. You can also set your pager by editing your global In my experience you need to turn colors
2022-3-16
til
If you have ever mistyped a git command very close to an existing one What you might not have known is that you can configure git to just run Now when you typo a git command it will autmatically run after the I
2022-3-15
til
yq I love that all of these modern tools built in go and rust, just give you a I use a bunch of these tools, and for what its worth I trust the devs behind Since I am all trusting of them I just want the latest version. I do not want I wrote or stol
2022-3-14
til
Last Thursday I learned about Watching him use pytest-mock is out on pypi and can be installed with pip. Sometimes I fall victim to making these posts nice and easy to follow. It I just wanted to do something that was worth mocking, the first thin
2022-3-14
til
Python 3.8 came out two and a half years ago and I have yet to really lean in Now that Python 3.6 is end of life, and most folks are using at least The assignment operator in python is more commonly referred to as the walrus This example from the do
2022-3-12
til
Kedro rich is a very new and unstable (it There is no pypi package yet, but it You can run your pipeline just as you normally would, except you get progress Listing out catalog entries from the command line now print out a nice pretty Go to the
2022-3-11
til
So worktrees, I always thought they were a big scary things. Turns out they I thought you had to be all in or worktrees or normal git, but not both. When Making a worktree is as easy as making a branch. It The worktree that you create is considere
2022-3-10
til
Has no upstream branch errors in git can be such a damn productivity killer. If you have not yet configured git to always push to the current branch, you Let You will be presented with the following error. To resolve this fatal error your first optio
2022-3-9
til
I write many of these posts from a 10 year old desktop that sits in my I Today I am giving it a second try, but this time with ssh. Before doing anything I checked to see if sshd is already running. Using Both returned nothing so I know that its not
2022-3-8
til
Mermaid gives us a way to style nodes through the use of css, but rather than produces the following graph style one fill:#BADA55
2022-3-7
til
I recently found a really great I have this added to my In my workflow I open a tmux session for each project, so this First open neovim, but with the If you try to run
2022-3-6
til
Mermaid provides some really great ways to group or fence in parts of your Here we can model some sort of data ingest with some raw iot device and our If we want to connect them, we can make a connection between a and A outside of It
2022-3-5
til
If you have ever ran
2022-3-4
til
Since GitHub started supporting mermaid in their markdown I wanted to The docs kinda just jumped right into their mermaid language and really You just write mermaid syntax in a div with a class of mermaid on The above gets me this diagram. This feel
2022-3-3
til
There is GNU coreutils command called Here are some examples of making temp directories in different places, my Here is a sample script that shows how to capture the tempdir as a variable and You must have at least 3 trailing X My randomm samples I
2022-3-2
til
Once you give a branch the big D ( Checkout is your savior, all you need is the commit hash. We have all done this, you give branch the big D only to realize it was Notice the hash is right there is the hash of your commit. You can use If you have
2022-3-1
til
Listing all the deleted files in all of git history can be done by These various commands will show all files that were ever deleted on The reflog can be super powerful in finding lost files here, as it only
2022-2-28
til
Git commands such as You can find the full description by searching for Open up a git repo and play around with this, here are some example that
2022-2-27
til
As I am toying around with textual, I am wanting some popup user input The main issue is that when you are in a textual app, it kinda owns the textual is still very beta Part of this comes down to the fact that textual is still very beta and So the s
2022-2-26
til
Big announcement recently that obs studio now builds out to a flatpak, I did not have flatpak installed so the first thing I had to do was get Once I had flatpak, I was able to get obs installed with the following Once Installed it fired right up for
2022-2-25
til
Mermaid diagrams provide a way to display graphs defined as plain text. You can define nodes like this in mermaid, and GitHub will now render
2022-2-24
til
Git has a built in way to rebase all the way back to the beginning of
2022-2-23
til
Glances is a system monitor with a ton of features, including docker processes. I have started using portainer to look at running docker processes, its a great You will need to install glances to use the glances webui. We can still use You will be p
2022-2-22
til
Git reflog can perform some serious magic in reviving your hard work You must git commit I really like to practice these techniques before I need to use them so This is what I did to revive a dropped Here was the final reflog that shows all of my gi
2022-2-21
til
Glances has a pretty incredible webui to view system processes and information The nice thing about the webui is that it can be accessed from a remote system. You will need to install glances to use the glances webui. We can still use You will be pr
2022-2-20
til
Right inside the git This epiphany deepens my understanding of git, and lets me understand that most Here are some git commands for you to try out on your own that are all pretty If I am looking for a missing file, I might want to leverage Here is
2022-2-19
til
Glances is a fully featured system monitoring tool written in python. Out of Once you run this you will be in a tui application similar to htop. You can
2022-2-18
til
python requirements text files can in fact depend on each other due to Lets create two requirements files in a new directory to play with. Then add the following to each requirements file. Installing requirements This is covered in the
2022-2-17
til
In my adventure to put more homelab in docker, I moved our modded So far I have found all of our mods from under the hood docker is using wget to get the mod. The link you click I am using docker compose, it makes the command much easier to start, C
2022-2-16
til
Reading eventbridge rules from the command line can be a total drag, pipe it I just love when I start thinking through how to parse a bunch of json at the
2022-2-15
til
Anyone just starting out their vim customization journey is bound to run into this error. I If you run You still need to map your remaps with a : if you do not close it with a If you can close the
2022-2-14
til
The default keybinding for copy-mode To do this I just popped open my I have a full video on copy-mode you can find here. https://waylonwalker.com/tmux-copy-mode/
2022-2-13
til
In python, a string is a string until you add special characters. In browsing twitter this morning I came accross this tweet, that showed that https://twitter.com/bascodes/status/1492147596688871424 I popped open ipython to play with this. I could c
2022-2-12
til
One thing about moving to a tiling window manager like awesome wm or i3 is that Running Here is what I had to do to connect my headphones. Here is the output of the help menu on my machine, it seems pretty straight note ctrl revers to the bluetooth
2022-2-11
til
I often run shell commands from python with Popen, but not often enough To get the stderr we must get it from the proc, read it, and decode the Now that we can read the
2022-2-10
til
Samba is an implementation of the smb protocol that allows me to setup network I think the homelab is starting to intrigue me enought to dive into the path of To get goind I am going to make a directory Install samba, open the firewall, and edit the
2022-2-9
til
A super useful tool when doing PR It
2022-2-8
til
I In all these years, I My buddy Nic has been sharing me some of his homelab setup, and it I ended up running the first thing in the Readme that included a volume With the following stored as my To start the server we open up the terminal in this di
2022-2-7
til
Installing rust in your own ansible playbook will make sure that you can I try to install everything that I will want to use for more than just a This is how rust reccomends that you install it on Ubuntu. First update The first thing I do in my play
2022-2-6
til
In looking for a way to automatically generate descriptions for pages I It When I originally wrote this post, I did not realize at the time that
2022-2-5
til
Creating a minimal config specifically for git commits has made running The other thing that is engrained into my muscle memory is Here is the config that has taken ~/.config/nvim/init-git.vim ~/.config/nvim/git-plugins.vim ~/.gitconfig
2022-2-4
til
For an embarassingly long time, til today, I have been wrapping my dict Lets consider this example for prices of supplies. Here we set a variable of What I would always do is try to get the key, and if it failed on KeyError, I What I noticed Nic doe
2022-2-3
til
I was listening to Since many things still want to see a version number, there is one automatic
2022-2-2
til
BeautifulSoup is a DOM like library for python. It Lets make a sample.html file with the following contents. It mainly has Lets import our packages, read in our And what we get is a list of I recently added a heading
2022-2-1
til
I keep my nodes short and sweet. They do one thing and do it well. I Here are two examples, the first one Many times I just want to get the data in as fast as possible, learn Note: try not to take the idea of a one liner too far. If your
2022-1-31
til
2022-1-30
til
I Within the past year I have added some tools to my personal setup that are not Like any Once I had this file, I stowed it into Always stow your dotfiles, don Note, the reason that it is a Next run this command to add the Once you have done this
2022-1-30
til
Today I discovered a sweet new cli for compressing images. First the main feature of squoosh is a What is even cooler is that once you have settings you are happy with and are I copied the command that it gave me for converting to webp, and set it u
2022-1-29
til
As you work on your kedro projects you are bound to need to add more Before you start mucking around with any changes to dependencies make sure that New requirements get added to a requirements.in file. If you need to specify Here I added the popula
2022-1-28
til
I am a huge believer in practicing your craft. Professional athletes Go to your playground directory, and if you don Install pipx in your system python. This is one of the very few, and From inside your I use this quite often to try out new things
2022-1-27
til
One of the first things I noticed broken in my terminal based workflow moving First off you need to get I have tmux setup to automatically copy any selection I make to the clipboard To get my yanks to go to the system clipboard in neovim, I just add
2022-1-26
til
With the latest version of minecraft it requires a very new, possibly I was getting this error trying to get a 1.12.2 forge server running. Caused by: java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast
2022-1-25
til
I have added a hotkey to my copier template setup to quickly access all my I
2022-1-24
til
I often pop into my blog from neovim with the intent to look at just a To tie these into a Telescope picker you add the command as the NOTE telescope treates each word as a string, do not wrap an extra
2022-1-23
til
Copier allows you to run post render tasks, just like cookiecutter. These are The example I have below runs an I have put the script in
2022-1-22
til
I https://youtu.be/Rk-Y71P Run Mypy as is, don Next we will add The final stage to this series is to add Make sure that you watch Anthony https://www.youtube.com/watch?v=Rk-Y71P
2022-1-21
til
In order to make an auto title plugin for markata I needed to come up ! Here I have a To turn this into a markata plugin I put it into a pre
2022-1-20
til
I really appreciate that in linux anything can be scripted, including I set my default wallpaper with Leaning in on feh, we can use fzf to pick a wallpaper from a directory I have mine alias
2022-1-19
til
Getting docstrings from python First you need to load in some python code as a string, and parse it with You can then use To get all of the functions docstrings we can use ast.walk docs: Recursively yield all descendant nodes in the tree starting a
2022-1-18
til
Many tools such as ripgrep respect the Editors like vscode often do not include files that are .gitignored in pathspec pathspec
2022-1-17
til
I don I first tried the nvim lsp rename, and it failed, Then I pip installed Once you have rope installed you can call rename on the variable.
2022-1-16
til
When running a python process that requires a port it
2022-1-15
til
functools.total From the Docs: The class must define one of Here is an example using the Enum I was working on the other day.
2022-1-14
til
Adding a
2022-1-13
til
Converting markdown posts to pdf on ubuntu takes a few packages from the Here is an image of what converting this article over to a pdf looks
2022-1-12
til
Python comes with an enum module for creating enums. You can make your Enum values can be auto incremented by importing auto, and calling Enum
2022-1-11
til
I recently paired up with another dev running windows with Ubuntu running in Open up a terminal and get your required system dependencies using the apt I like running things like this through an ansible-playbook as it give me some Here is a clip of m
2022-1-10
til
Stow is an incredible way to manage your dotfiles. It works by managing When using stow its easiest to keep your dotfiles directory (you may name it Then each application directory should reflet the same diretory structure as you Here is a simple ex
2022-1-9
til
The copier answers file is a key component to making your templates Inside of my Inside my copier.yml I have setup my Once I change the I After rendering the template we have the following content in our This is where I was most stuck, primarily b
2022-1-8
til
Once you have made your sick looking cli apps with rich, eventually you are Install them from the command line. Import make a .py file and import them in it. If you return your rich renderable out of class that inherits from You At this point It prob
2022-1-7
til
I It turns out they are tabs, and you can get rid of the little leading
2022-1-6
til
tmux popups can be sized how you like based on the % width of the
2022-1-5
til
I was completely stuck for awhile. copier was not replacing my template !
2022-1-4
til
I By default copier uses double square brackets for its variables. note Before running copier we need to tell copier what variables to ask for, I prefer to install cli tools that I need globally with pipx, this When running note The resulting files
2022-1-3
til
I just installed a brand new Ubuntu 21.10 Impish Indri, and wanted a But what I got back was not what I expected This is weird, why cant I run kedro new with pipx? Lets try pip. Same issue. https://waylonwalker.com/what-is-kedro/ Curious what kedro
2022-1-2
til
Pluggy makes it so easy to allow users to modify the behavior of a framework I As long as the framework document the hooks that are available and what it Installing pluggy is just like most python applications, install python, make At the time I star
2022-1-1
til
One of the most useful skills you can acquire to make you faster at I Passing the flag Giving
2021-12-31
til
pyenv provides an easy way to install almost any version of python from I needed to install an older version of python on ubuntu I Based on the Readme it looked like I needed to install using homebrew,so this https://waylonwalker.com/til/installing-h
2021-12-30
til
Installing brew on linux proved quite easy and got pyenv running for me I had never used homebrew before, honestly I thought it was a mac only That was it, now homebrew is working. Starting a new shell and running
2021-12-29
til
When I first moved to vim from and ide like vscode or sublime text one of my You will need a way to run another process alongside vim, here are a couple use background jobs c-z to send a job to the background fg to bring it back use a second terminal
2021-12-28
til
Many command line tools can output a list of files, this is quite powerful. Telescope is the fuzzy file finder I use every day inside of neovim. Its pretty This brings up a normal Telescope picker with results from the Adding more arguments can be
2021-12-27
til
Finding hidden files using Telescope as you fuzzy file finder is not too I have this keymap set to help me list out all files including hidden see the
2021-12-26
til
Lately I This is one of those things that can be a total pain to get right on make sure your user fonts directory exists chech if the font you want exists on your machine download and unzip fonts into the fonts directory repeat 2-3 for all the fonts
2021-12-25
til
Part of my neovim setup requires having the re-installing a bunch of things that are already installed can be quite check if the command is installed with register that step ignore if that step fails add a https://www.youtube.com/watch?v=MCFg6-W5S
2021-12-24
til
Adding a
2021-12-23
til
Fugitive comes with a pretty sick way to commit files and see the diff at the example of a verbose commit in fugitive
2021-12-22
til
This is not a flaky works half the time kind of plugin, it I can https://youtu.be/2QW5DJiEJH4 Give the video a watch, I did not have noise-cancelling on in obs. My pyflyby is hosted on pypi, so you can get it with pip. I have had no issues If you Se
2021-12-20
blog-post
If you are running vim autocmd https://youtu.be/2ITTn4Dl0lc For Lets create a new file called I What you need to do is clear out all commands in the augroup with Now this is what I have in my dotfiles for a silky smooth setup that does not
2021-12-8
blog-post
I often review Pull requests from the browser as it just makes it so easy to see https://youtu.be/5NKaZFavM0E This all stems from the great plugin by First to quickly checkout PR Next I have a few aliases setup for checking diffs. The first one chec
2021-12-4
blog-post
I https://youtu.be/xo4HrFoKF4c The video for this one is part of a I have my You will need the following plugins. I use plug, if you don Make sure that you have nvim Again if you don Next up is the heart of this post, the lsp-config.lua. This one
2021-12-3
blog-post
https://youtu.be/PQw I am often in a set of tmux splits flying back and forth, accidentally close my Make sure you check out the YouTube video to see all of my improvements. If you have never heard of an alias before it Here are the new aliases that
2021-12-2
blog-post
In honor of the neovim 0.6.0 release, I decided to do a funny skit installing https://youtu.be/64oKLphhBuo The thing that took me the longest to realize was.... I had a path issue https://neovim.io/
2021-11-30
blog-post
https://youtu.be/Jq1Y48F I am challenging myself to 30 days of dotfile ricing. I have been on linux This is primarily going to be focused on productivity, but may have a few Follow the
2021-11-29
blog-post
Trying to read a .ipynb file without starting a jupyter server? jut has you https://youtu.be/t8AvImnwor0 watch the video version of this post on jut ! Take a look at the help of the There is some good information on the projects Don jut nbconvert
2021-11-20
blog-post
I use https://youtu.be/ https://waylonwalker.com/what-is-kedro/ 👆 Unsure what kedro is? Check out this post. The command to ensure there are catalog entries for every dataset in the passed Create Fills in new dataset entries with the default datase
2021-11-15
blog-post
https://youtu.be/E18m4KkJUnI Using Vim as a Team Lead I 💜 Tmux Why I stopped using @code Get there fast How I vim Use a graphical IDE if it works for you. vim is so well integrated into the terminal, take advantage As a team lead I bounce betweeen a
2021-10-29
blog-post
We can https://youtu.be/TZrRAP-9UMk In any python repl you can access the docstring of a function by calling for In Ipython we can even get some syntax highlighting with the Sometimes the docstrings are not good enough, and don The more common way
2021-10-10
blog-post
I https://youtu.be/8m5ipBuopPU Check out the project I want a simple way to make remaps into shell commands that can open new tmux For example I want to make remaps to do things like open the current file in lookatme. The main goal here is that rema
2021-10-5
blog-post
Parameters are a place for you to store variables for your pipeline that can be https://youtu.be/Jj5cQ5bqcjg https://waylonwalker.com/what-is-kedro/ 👆 Unsure what kedro is? Check out this post. You can have multiple parameters files and choose which
2021-9-19
blog-post
https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let You will need to import node from kedro.pipeline to start creating nodes. The Inputs and outputs can be None, a single catalog entry as a string, mutiple Sometimes in our pipeli
2021-9-13
blog-post
Running your kedro pipeline from the command line could not be any easier to https://youtu.be/ZmccpLy-OEI https://waylonwalker.com/what-is-kedro/ 👆 Unsure what kedro is? Check out this post. To run the whole darn project all we need to do is fire up
2021-8-24
blog-post
Avoid serious version conflict issues, and use a virtual environment anytime https://youtu.be/ZSxc5VVCBhM conda venv pipenv I prefer to use conda as my virtual environment manager of choice as it give me stores environment in a root directory i.e. c
2021-8-23
blog-post
Kedro comes with an https://youtu.be/IWimEs-hHQg You must start by having your kedro project either cloned down https://waylonwalker.com/kedro-new/ this post covers kedro new https://waylonwalker.com/kedro-environment/ This post covers creating your
2021-8-21
blog-post
Immediately after https://youtu.be/IGba3ytf Its as simple as these three commands to get started. I don
2021-8-20
blog-post
https://youtu.be/uqiv5LAiJe0 Kedro new is simply a wrapper around the cookiecutter templating library. The https://waylonwalker.com/what-is-kedro/ Unsure what kedro is, Check out yesterdays post on What is Kedro. I reccomend using The kedro team do
2021-8-18
blog-post
Kedro is an unopinionated Data Engineering framework that comes with a somewhat https://youtu.be/Wf4rnFsaFFU https://waylonwalker.com/what-is-kedro-1/ This is an updated version of my original what-is-kedro article If you are doing a series of operat
2021-8-17
blog-post
https://youtu.be/bw5 I recently switched over to using Ubuntu, it works well pretty much out of the I am still using the built in default gnome terminal, it just works. It does One Dark Theme You can find my I use zsh as my default shell. I like to
2021-8-16
blog-post
https://youtu.be/LLk94fKpGg4 As we push the limits of tmux further and further you are bound to end up in a show-messages In case you wnat more information about show-messages, here is the man page. https://waylonwalker.com/tmux-nav-2021/ for more in
2021-8-14
blog-post
https://youtu.be/nT0FA1RNZEs Now your creating, jumping, and killing sessions like a boss. You are slicing This script is simply my fork of Chris Toomey My version of the By default I have my projects open with a vertical split, vim is on top, with
2021-8-13
blog-post
https://youtu.be/utfLA6L8o5s You Here is a snippet of With Without setting the target-pane https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post Also check out the full YouTube
2021-8-12
blog-post
https://youtu.be/QWPyYx54JbE Now it One viable option is to nuke the whole dang thing. I actually do this more save and commit your work diligently before A more reasonable option might be to kill a single session. Killing sessions one by one from
2021-8-11
blog-post
https://youtu.be/XucVVgGmesM I see you there, trying to script out your tmux layouts. Tryig to get each The The command is pretty straight forward, you simply ask tmux if the session name https://waylonwalker.com/tmux-nav-2021/ for more information
2021-8-9
blog-post
https://youtu.be/Y1MYmL8ZolE Tmux list keys can be a useful tool to help remind you of what kebindings you You can call list-keys from the command line but the interface is not very Running By default tmux comes with You can see the additional flag
2021-8-8
blog-post
https://youtu.be/mvgM8UH11 The tmux status bar can be a handy tool to remind yourself where you are within You can set a hotkey to show or hide the status bar. I really want a minimal status bar with very little bling, I want it to get out I want my
2021-8-7
blog-post
https://youtu.be/dDq0depPrTs So you have been tricking out that Let We can run this command from your shell to re-source your changed It also works from the tmux command line. It This is my preferred way of re-sourcing my https://waylonwalker.com/
2021-8-6
blog-post
https://youtu.be/DkJ9rb85LC0 Quickly getting between tmux splits is critical skill for productivity. You I have used this fzf one keybinding for quite awhile, honestly I did not make Like with many of my keybindings I have swapped this one out for
2021-8-5
blog-post
https://youtu.be/8kZnjHPYnKU Now that we are splitting up work into their own sessions, lets talk about how https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post Also check out the full YouTube
2021-8-4
blog-post
https://youtu.be/WRLRiQDjVIA So you have been working on your tmux workflow, you Let There is a default keybinding that you can use I https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post Also
2021-8-3
blog-post
https://youtu.be/c-a2Bnv Scripting tmux to open up specific applications can be intimidating your first I used this one for a number of years to take a quick peek into my systems 🗒️ note that the With the new tmux popup windows I really like the flo
2021-8-3
blog-post
https://youtu.be/5KE7Il7SOEk This is something that I made up but use every single day, this is what keeps https://waylonwalker.com/tmux-new-session/ This one is building off of yeserday https://waylonwalker.com/tmux-nav-2021/ for more information on
2021-8-2
blog-post
https://youtu.be/LbQNdCAUogE This one starts a new chapter in our series that is going to open up a whole https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post Also check out the full YouTube
2021-8-1
blog-post
https://youtu.be/JQ0yDCVu44E attach is one of the most useful features of tmux. If you have no interest in this command will simply attach back to tmux if you are ever disconnected If you ever run long running tasks on a remote machine by sshing int
2021-7-31
blog-post
These are my notes from taking @nnja git --version
2021-7-31
blog-post
https://youtu.be/LY41GLn tmux ls will list the sessions that you have running within the tmux server if https://waylonwalker.com/tmux-attach/ https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full pos
2021-7-30
blog-post
https://youtu.be/SNu-4IrkjAs So far we have covered a lot of tmux commands and how they map to keybindings Let Or we can open the tmux command line and run it from tmux 🗒️ note that the tmux command is called by default when inside of tmux. Finally w
2021-7-29
blog-post
https://youtu.be/-ypY tmux copy-mode is a tmux mode that lets you scroll, search, copy, and jump your Default keybinding to get into copy mode is If you are a vim user you will likely want to use vi style keys, add this to full list of copy-mode key
2021-7-28
blog-post
https://youtu.be/Vm5rRtcVXLw Join-pane allows you to join panes that you have broken away from your window, Before you can join a pane you must first have a pane marked to join. Once you My keybindings, you must add this to your https://waylonwalke
2021-7-27
blog-post
https://youtu.be/ICL609F2xnc Break-pane is a handy tmux command when your layout gets too cramped and you Default key binding for https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post
2021-7-26
blog-post
https://youtu.be/Rn6mOarCQ-Y Zooming into the current split in tmux is a valuable tool to give yourself some Default key bindings for zooming the current split I have rebound this to match the default binding with mod+z rather so that I https://waylo
2021-7-25
blog-post
https://youtu.be/YRPZBv-iYyE New window as it sounds makes new windows in tmux. Windows are kind of like Default key bindings for creating and navigating windows in tmux. As always I have rebound these keys because I generally prefer a single When I
2021-7-24
blog-post
https://youtu.be/CPZJZjN9YTY These are my MOST often used keybindings that I use in tmux. They allow me to https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post
2021-7-23
blog-post
https://youtu.be/06z5qf81ofo Rotate window is the main way that I navigated tmux before I learned Default keybindings My keybindings look just a bit different than the default ones, I do not like https://waylonwalker.com/tmux-nav-2021/ for more infor
2021-7-22
blog-post
https://youtu.be/F0mHnwTrNNc When you get many splits going in tmux sometimes its time for a new layout. https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post
2021-7-21
blog-post
https://youtu.be/hpFYE2LU7xc Resizing panes in tmux can be quite difficult in default tmux, I Most often when I need to resize panes I just grab the edge of the pane with my https://waylonwalker.com/tmux-nav-2021/ for more information on how I naviga
2021-7-20
blog-post
https://youtu.be/79Y-kqAiMpw Choose tree is a powerful tmux utility that provides a graphical interface to The default keybinding my preferred keybinding to open sessions and windows collapsed and Zoomed in. From the man page. https://waylonwalker.co
2021-7-19
blog-post
https://youtu.be/BMkpbfhbkKM The prefix key is an essential part of tmux, by default all of tmux A few of the essential default key-bindings. A more complete list of key-bindings can be found in this gist https://gist.github.com/mzmonsour/8791835. ht
2021-7-18
blog-post
https://youtu.be/kzgyiHap1nQ splitting panes is a core feature of tmux. It allows us to split the terminal 🗒️ note that https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post
2021-7-17
blog-post
https://youtu.be/RB87EEnnMnU An ultimate productivity key-binding in tmux is one to switch to the last session. I use this to quickly get between sessions really quick. Often I am working and need to lookup a quick note, or copy something into my n
2021-7-16
blog-post
https://youtu.be/2ZqFDsJywt8 Tmux popups are actually floating windows that you can drag around the screen. They always open in the middle (by default) when you open them, no matter where you leave them. Here are a couple of keybindings I use to ope
2021-7-15
blog-post
https://youtu.be/2I8fB28zfB4 https://waylonwalker.com/tmux-nav-2021/ for more information on how I navigate tmux, check out this full post
2021-7-14
blog-post
Kedro versioned datasets can be mixed with incremental and partitioned datasets https://waylonwalker.com/what-is-kedro/ 👆 Unsure what kedro is? Check out this post. This was a question presented to me at work. We had some plots being produces To en
2021-7-8
blog-post
I recently started streaming on python kedro Data Science Data Engineering webdev digital gardening It all started with My introduction to twitch came from After taking a jump in with the spaceflights tutorial I realized that it really I started stor
2021-6-12
blog-post
This was my seconf time ever streaming on
2021-6-7
blog-post
https://youtu.be/I4VenHqIEng Doing some Digital Gardening on stream Ahrefs Errors ahrefs large images Automatic gif to mp4 After this stream all gifs on my site are converted to mp4/webm if they exist.
2021-6-6
blog-post
This was my first time ever streaming on https://youtu.be/Y07UBr9Ccjs It all started with
2021-6-4
blog-post
With the latest releases of kedro https://waylonwalker.com/what-is-kedro/ If your just learning about kedro check out this post walking through it I To get kedro running, you will need a pipeline, catalog, and 👆 Above is the minimal setup to get a k
2021-5-23
blog-post
Kedro can have a chatty logger. While this is super nice in production First, how does one silence a python log? Python loggers can be retrieved by | Level | Numeric value | Getting a python logger is straightforward if we know the name of the l
2021-5-20
blog-post
In 2021 I changed the way I navigate between tmux sessions big time. Now I can Since making this post, I have made ~20 other posts in short form that all have a YouTube video to go along with them you can find them all on my I took Chris I am now ke
2021-5-11
blog-post
When a python module is called it is assigned the Let I have set this module up to execute one of two if statements based on whether Note it is not common to have a Running a python script with the command This will print out https://waylonwalker
2021-5-7
blog-post
2021-5-7
blog-post
Creating a directory that is a union of several directories can be achieved Here is how I am creating a virtual directory of all my projects that is a ⚠ Notice that first I am recreating the directory each time. This will ensure Since links are alway
2021-5-5
blog-post
I really like using conda ( Mamba is a reimplementation of the conda package manager in C++. parallel downloading of repository data and package files using multi-threading libsolv for much faster dependency solving, a state of the art library used i
2021-4-21
blog-post
I wsl can access clip.exe. You can do some cool things with it, such as Let pipe streams of text into clip.exe I recently made mine feel a bit more natural by aliasing it to clip. pop this in your ~/.bashrc or ~/.zshrc I use neovim as my daily text
2021-4-17
blog-post
I am trying to see what an embeded replit
2021-4-15
blog-post
Testing print/log statements in pytest can be a bit tricky, capsys makes it capsys is a builtin pytest fixture that can be passed into any test to capture Simply create a test function that accepts capsys as an argument and pytest
2021-4-5
blog-post
I I am working on a project in which I want to have a dev automatic port number auto-restart display I am very novice at best when it comes to sockets, the following function came I am going super basic here and just running Optionally if you wante
2021-3-27
blog-post
With the realease of create a register_pipelines register hook_impl class You should now have something that looks like this in your pipeline I was not able to find any official documentation on how conflict resolution kedro automatically merges pi
2021-3-20
blog-post
People who are quick to toss team members under a bus are not well trusted or While collaborating on any project there are going to be decisions made that During a review, leadership showed interest in the option that I wanted to go I walked out so
2021-1-22
blog-post
I create this blog with one person in mind, me. This is not completely selfish, as there are likely many others out there that I am as accurate as possible. I don Generally I know that I am directionally accurate for most of the article. In General
2021-1-21
blog-post
How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It 📝 Note this is only a composable pipeline, not a full project, it does not contain a catalog or runner. I have everything for this po
2021-1-20
blog-post
This is a post that may be a work in progress for awhile, Its a collections of posts tags draft posts frontmatter filepath content template html Markdown.Markdown support extentsions provide ways to hook in or clean up the frontmatter load render sav
2021-1-20
blog-post
I am making another push in 2021 to get my content out in the world and meeting users where they are. See how I plan to execute. waylonwalker.com Twitter DEV hashnode Medium LinkedIn Anchor My content is written in markdown, all markdown. I find th
2021-1-19
blog-post
Recently I automated starting new posts with a python script. Today I want to https://waylonwalker.com/automating-my-post-starter Check out this post about setting up my posts with python 🐍 For the process of editing a post I just need to open the f
2021-1-18
blog-post
Gitui is a terminal-based git user interface (TUI) that will change the way Go to their It opens blazing fast. Sometimes I edit a number of files and want to commit them one at a time, this By default, Generally, I pop open w/s: to toggle focus
2021-1-17
blog-post
In python data science/engineering most of our data is in the form of some sort These containers for data contain many convenient methods to manipulate table https://waylonwalker.com/what-is-kedro/ unfamiliar with kedro, check out this post There are
2021-1-14
blog-post
Changing conda environments is a bit verbose, I use a function with fzf that I have used conda as a virtual environment tool for years now. I started using What environment manager do you use for python? Conda environments are stored in a central lo
2021-1-11
blog-post
Replacing text based on whats in the current search register is a quite handy https://www.youtube.com/watch?v=fP If there is one thing that I Like most about vim it Vim can often be a bit verbose, but that I have a keybinding in my In command mode
2021-1-10
blog-post
What does it take to create an installable python package that can be hosted on pypi? setup.py my This post is somewhat inspired by the bottle framework, which is famously created as a single python module. Yes, a whole web framework is written in o
2021-1-10
blog-post
I do much of my work from tmux, I love it so much that I want to setup some Bash function to check if the shell is in a tmux session. I often open up vim to do some quite edits, but before I know it I have several Using I am not quite sure if this i
2021-1-9
blog-post
Replacing text in vim can be quite frustrating especially since it doesn https://twitter.com/ I had to do a bit of searching and found a great post from I believe that this is a neovim only feature, add it into your Check out Josh Branchaud https://
2021-1-4
blog-post
If you are like me, you have created a macro or two that is pure glory, and you forget how you made it after a day or so, or you immediately want to store it away as a custom keybinding. As with most things with vim, it One of the earliest things we
2021-1-4
blog-post
Web browsers are a black hole of productivity. I try to use them as little as I am going to give a terminal rss reader a try for a bit and see how that goes I installed with the reccomended snap for Ubuntu. Running help for newsboat directed me towa
2021-1-2
blog-post
As projects grow patterns that worked early on break and we need to change things to make the project easier to work with, and more welcoming to new developers. Before you start mucking up a project with wild commands at the terminal check that you h
2020-12-30
blog-post
I use my ipython terminal daily. It Activate your virtual environment of choice and pip install it. Any time you You are using a virtual environment right? Virtual environments like venv or When you install ipython you start out with no config at a
2020-12-20
blog-post
I If you already have an ipython config you can move on otherwise check out this https://waylonwalker.com/ipython-config/ I want something similar to the starship prompt I am using in the python version active environment git branch This is my zsh pr
2020-12-20
blog-post
One thing we all dread is mundane work of getting started, and all the hoops it When I start a new post I need to cd into my blog directory, start neovim in a frontmatter template Title slug tags date cover description create markdown file open in ne
2020-12-11
blog-post
In python data science we often will reach for pandas a bit more than necessary. While pandas can save us so much there are times where there are alternatives that are much simpler. The itertools This post is a walkthrough of me solving a problem wi
2020-12-10
blog-post
This is episode 1 of the Waylon Walker Audio experience, posts from So I have had this idea for awhile to add audio to my blog posts. The idea Honestly I don I will say that the time that I have available for clean audio is a bit limited One thing t
2020-11-25
blog-post
I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in vanilla markdown in GitHub. I have been using a snippet that puts HTML into the markdown. While this works, it The new card shou
2020-11-18
blog-post
Inspired by discourse https://twitter.com/kylemathews/status/1329817928666005504 This covers a couple of use cases I have with very little effort. Twitter YouTube This was super quick and simple to setup, the only thing that was extra was to Thats it
2020-11-18
blog-post
2020-11-12
blog-post
customize for the job Why are you a good fit? What will you bring to the role? Give real outcomes give real experience Stop tech vomiting if you link to GitHub Make a profile readme Guide me to your best work have some activity if you link to LinkedI
2020-11-6
blog-post
use this profile image Please share your professional role as a data scientist? I graduated with a Mechanical Engineering Degree 8 years ago. Much of my work Three years ago I made the switch to Data Science and have loved every minute of What are
2020-11-2
blog-post
There are many reasons that you should be using kedro. If you are on a team of https://waylonwalker.com/what-is-kedro Kedro makes it super easy to get started with their cli that utilizes https://waylonwalker.com/create-new-kedro-project read more a
2020-11-1
blog-post
collaboration Sharable catalog small nodes over monolithic notebooks catalog easily load anything without needing to run No need to write read/write code pipeline No need to keep execution order in your head easily run a slice of a pipeline plugins p
2020-10-31
blog-post
https://waylonwalker.com/latest/ STOP LEAVING Browser Tabs open and save them here Jungle Scout - Julia Evans - git course - https://ianthehenry.com/posts/how-to-learn-nix/python3-alpha/ https://twitter.com/willmcgugan/status/1531294412696956930 h
2020-10-28
blog-post
Kedro This is really exciting to see more deployment options coming from the kedro team. It really shows the power of the framework. The power of some of these orchestrations options is incredible. Most of them hinge on a sweet combination of the ke
2020-10-25
blog-post
I started making stories as kind of a brain dump a few times per day and I store all the stories on my website with the hopes of doing something with
2020-10-21
blog-post
I was 20 commits into a hackoberfest PR when I suddenly realized they they all had my work email on them instead of my personal email 😱. This is the story of how I corrected my email address on 19 individual commits after already submitting for a PR
2020-10-17
blog-post
I released a router-like plugin for kedro back in April 2020. This was not the first design, the idea actually came from one of the QB folks who taught me kedro nearly a year before. We were assembling our pipelines with something called One thing
2020-10-8
blog-post
Today I ran into an issue where we had a one-off script that just needed to It started with a colleague asking me How do I clear the memory in a Jupyter How do I clear the memory in a Jupyter notebook? There are a number of ways that you can check th
2020-10-1
blog-post
A common linting error thrown by various linters is for trailing whitespace. I read more about pre-commit
2020-9-30
blog-post
There are many useful chrome extensions out there. I probably have way too many installed, here are four that I am currently using. This post was inspired from Chris over at Love it or hate it passwords are hard to manage. Everyone needs a passwor
2020-9-13
blog-post
Bash is a language that is quite useful for automation no matter what language I have three techniques that help me write more composable bash scripts. Functions in bash are quite simple. They are something that I wish I would have Source the file to
2020-8-13
blog-post
Here are three things that I see my non programming counterparts doing every single day. These really sum up so much of what folks do within an office. So many of us dabble in or become power users of spreadsheets without knowing there is an altern
2020-8-11
blog-post
miniconda is a python distribution from continuum. It Installing miniconda on Linux can be a bit tricky the first time you do it completely from the terminal. The following snippet will create a directory to install miniconda into, download the lates
2020-8-10
blog-post
Learn Kedro in 5 days vm install python editor kedro new kedro viz catalog filter catalog load data fsspec pipeline nodes filter pipeline run partial pipeline kedro docker GitHub Actions hooks custom datasets modular pipelines
2020-8-8
blog-post
This post was inspired by a comment I left on @dsteenman {% post dsteenman/how-long-should-a-blogpost-be-2k6n %} Most of the time I prefer short as I am more likely to read the whole thing. If its setup as a series I am more likely to work my way th
2020-8-7
blog-post
I was particularly inspired by @chantastic episode 103 of the react podcast with @ParissAthena. They spoke about the black tech pipeline as well as Diversity, Equity, and Inclusion. Pariss is quite an inspiration. She has done so much work to create
2020-8-3
blog-post
It Check out the repo for It This has been by far the easiest way to commit back to a repo that I have seen. Other patterns often require fully setting up the git user and everything. While it You must give it a commit message and thats it. Optio
2020-8-3
blog-post
If we take a look at the This one comes a bit surprising as it was just casually mentioned in As mentioned in It feels a bit crazy that simply installing a package will change the way that your pipeline gets executed. I do like that it requires ju
2020-8-1
blog-post
As I continue to build out Running integration tests will not prevent bugs from happening completely, but it will allow me to quickly spot them and rollback. The very first thing that comes to my mind is anything that is loaded or ran client-side.
2020-7-27
blog-post
Today I setup a new machine on Digital Ocean to use with TestProject.io, Here are my installation notes. envsubst
2020-7-25
blog-post
When learning a new skill it When learning a new skill it Here are three different options that I have used in the past to try out something at some various levels. I am sure there are plenty more, but these are three that I have tried. I am not cove
2020-7-25
blog-post
I am exploring a kedro catalog meta data hook, these are some notes about what I am thinking. metadata will be attached to the dataset object under a metadata will be updated metadata will be empty until a pipeline is ran with the hook on optionall
2020-7-24
blog-post
Mypy only uses pyproject.toml only uses pyproject.toml
2020-7-21
blog-post
I recently added a button to my blog, and subsequently my posts on I refer back to my old posts quite a bit, sometimes I find errors in them. The slug that I am getting from gatsby is formatted as GitHub makes it super easy to form a URL that puts y
2020-7-18
blog-post
When first learning to code its very common to hard code everything right into Hard coding everything right into your code makes it really hard for The cms generally come with a rich content editor that feels more like Even when you are developing fo
2020-7-14
blog-post
I am looking into a way to replace my google reader experience that I had back This is how I used python to parse rss and setup my own custom feed. Install the feedparser library. The feed is a feedparser.FeedParserDict. For all intents and purposes
2020-7-13
blog-post
The input will be a yaml file containing a list of rss feed (primary source) youtube feed Stack Overflow tags GitHub repo activity pypi release dev.to post Twitter Search # user will need an api key Each url will be pulled in and parsed into a stand
2020-7-12
blog-post
Can someone explain how or why twitter cards render differently from device to device? I do understand that twitter cards a built from meta tags, the full list can be found in their Mobile Looks fine. On Desktop it is not picking up the image. The
2020-7-11
blog-post
I ran this post on dev.to and got a great response of great examples, The GitHub profile feature just went live for a subset of users. Simply creating a repo named after your username, and clicking share to Profile on the sidebar will create a cust
2020-7-11
blog-post
I ran a discussion on dev that collected quite a list of examples in the comment section. So many great calls to action, animations, memes, and weird tricks. Upload all of your icons to the repo in a directory such as Aligning things in the center
2020-7-10
blog-post
Python I generally post these as a carousel on LinkedIn based on a full article. Let mw know what you think of it shown inside of a blog See the full article
2020-7-2
blog-post
While using the catalog alone will not reap all of the benefits of the framework, it does get you and your project ready for the full framework eventually. For me the full benefit of the catalog comes when you combine it with the pipeline and dont e
2020-6-29
blog-post
kedro 0.16.2 just dropped last week with a long-awaited feature... The kedro data catalog is a key component to the kedro framework. It handles all data loading and saving for you. It is configurable and hackable. Having all your data connections
2020-6-22
blog-post
Passing inputs into kedro is a key concept. Understanding how it accepts a Check out this post for a review of how https://waylonwalker.com/python-args-kwargs/ When kedro runs your pipeline it uses the catalog to imperatively load your data, meanin
2020-6-19
blog-post
This day 8 years ago I started my first day as a Mechanical Engineer. I am so My initial career plans were down a completely different path. I have been very flexible in taking on a new career path. I have been eager to learn new things and respond t
2020-6-19
blog-post
I just did a quick refactoring of my JAMStack blog urls. Some didn https://waylonwalker.com/refactor-in-cli When refactorings similar to this get really big I often need to do some My post urls are based on the file name of my markdown file, so I ca
2020-6-11
blog-post
It https://waylonwalker.com/refactor-in-cli/ this post follows my method of refactoring code bases from the command line, First off browsing through the content of my blog I found many references to master. I cannot completely whole-sale find and re
2020-6-11
blog-post
Python Python * When recieving variables as a Never add Generally I find If your Here Here the function signature makes it clear what Inversely we can send a list of things as individual arguments by Just like Here is a function accepting
2020-6-10
blog-post
I recently discovered the ✨ awesomeness that is pre-commit. I steered away I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really it It comes wi
2020-6-5
blog-post
Follow along the Journey as I build out I have really enjoyed my own personal journey as I have started to build all of my data pipeline projects with the kedro framework. I want to start building a place to share resources with the community. I w
2020-6-5
blog-post
Even though it is a static site there is no functionality lost. The only thing that Even though it Kedro viz is a fantastic kedro plugin that allows you to visualize your data pipeline. Kedro allows you to quickly build production-ready pipelines w
2020-5-28
blog-post
There are two main ways to create kedro hooks, with modules and classes. Each Class hooks seem a bit special as they give you a way to configure them so that they are a bit more generally useful. https://waylonwalker.com/what-is-kedro If you are com
2020-5-23
blog-post
This might be useful
2020-5-22
blog-post
This post is a 🧠 branstorming work in progress. I will likely use it as a If you are completely unsure what kedro is be sure to check out my filepath replacer bucket replacer preflight check that data exists run run mypy run interrogate run flake8
2020-5-22
blog-post
I want to incorporate some of the wonderful comments, I want to incorporate some of the wonderful comments, 💕, 🦄, and 🔖 dev.to has an open API that allows us to easily get comments as HTML. They have their API hosted at Here we can see that going
2020-5-20
blog-post
GitHub GitHub actions can give you confidence that your site is up and running, with the latest JavaScript packages, does not have broken links, and can even take screenshots of what your website looks like on different screen sizes and operating sys
2020-5-18
blog-post
2020-5-17
blog-post
Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For the most part you just need to tell Kedro what connector to use and its filepath. When run
2020-5-15
blog-post
As usual while listening to This thing is 🔥 hot off the press folks, we Nothing I have tried has come close to being this good It runs documentation coverage for your python project. It allows you to set the minimum amount of docstring coverage for
2020-5-15
blog-post
When creating cli I rarely get very complex with my semi-manual ASCII art. I can do 98% of what I need with bars and corners. Using some simple full-width characters can really give your cli a nice clean look. I As I was putting this together I stu
2020-5-12
blog-post
Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro kedro hooks are an exciting upcoming feature of kedro https://waylonwalker.com/what-is-kedro/ If you are completely unsure what kedro is be sure to check out my what is kedro
2020-5-10
blog-post
This is a very rough idea for a kedro package to prevent time lost to get partway through a pipeline run only to realize that you dont have access to data or resources. check that inputs exist or are of a type to skip (sql) check that all input and o
2020-5-9
blog-post
2020-5-7
blog-post
find-kedro kedro When working on larger pipeline projects, it is advisable to break your project down into different sub-modules which requires knowledge of building python libraries, and knowing how to import each module correctly. While this is no
2020-5-4
blog-post
Often when reading through javascript examples you will find some arrow Arrow functions are one-liner functions in javascript that have two main syntactical ways to create the code block. with parentheses and braces. Let Here is an example of an ar
2020-5-3
blog-post
Inspired by Chris Achard List comps Classes Inheritance Background Click Lambdas Cataloging Custom datasets Reusable pipelines find-kedro Email course inspired by learn d3 in 5 days Share your knowledge Practice Practice in public Make practice easy
2020-5-1
blog-post
Is it possible to deep watch a single python function for changes? keeping track of a python functions hash is quite simple. There is a Using hashlib provides a consistent hash. Now we have a consistent way to hash function code. setup a function in
2020-4-27
blog-post
This past week I had a really weird bug in my After a few days of looking at it off and on, I pinpointed that it was all the way down in the raw layer. Right as data is coming off of the database. For this I already had existing I have dynamically
2020-4-27
blog-post
If you are developing python packages and using GitHub here are four actions If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have suc
2020-4-13
blog-post
So often I see a variables Pandas Sometimes vanilla structures too It Always name your containers plural, so that naming while iterating is simple. Before I start fights 🥊 in code review, am I inline here or just being pedantic?
2020-4-8
blog-post
Here is one useful thing that you can do with GitHub actions no matter what language you use, send email. You might want to know right away when your ci passes. You might want to give your team a nice pat on the back when a new release is deployed.
2020-3-31
blog-post
Github actions are written in configuration files using the YAML syntax. YAML Can I just say that I learned more than I realized about YAML by writing this In YAML or JSON, the most basic containers for data are The start of an array container is s
2020-3-16
blog-post
GitHub actions are an amazing tool that allows us to run code based on triggers I have been diving deep into Github actions for about a month now and they are wicked good some interaction to your repository The online editor for actions is pretty am
2020-3-16
blog-post
Today I learned how to diff between two branches. Sometimes we get a little Let After all of that, we have a git repository on our local machine with a single file Let At this point we have one commit. Things are really straightforward, and our di
2020-3-3
blog-post
This is a quickstart to getting a new 🧹 Install 🛢 Create the Example Pipeline 💨 Run the example 📉 Show the pipeline visualization I use conda to control my virtual environments and will create a new environment called Options I try to keep my base
2020-3-2
blog-post
Learn the business Learn Git Your code does not need to be amazing Keep Learning You dont have to start out as a git wizard with the cleanest possible commit history. At first dont let yourself get too wrapped up in it, the most important part is th
2020-2-26
blog-post
I am working through Wes Bos Do you have any use cases that you use hoising? Why? It seems like a really cool feature in any language that uses it, but I dont really notice it in use. There are many articles that cover this in far more depth, but i
2020-2-25
blog-post
https://waylonwalker.com/what-is-kedro This is my original what-is-kedro article. There is a brand new one Kedro is an open source data pipeline framework. It provides guardrails to set Currently, 💎 📉 🏗 Inside this core library is a data catalo
2020-2-24
blog-post
Getting a custom scrollbar on your site makes it stand out a bit compared to Inspired by Wes Bos Since My first step was to take his css and copy it into a styled component for my entire layout, but it failed. I do not fully understand why. None o
2020-2-21
blog-post
" Inspired by this I felt that this comment was very powerful. Here are my 2 cents. Stick to what you know, and learn a little bit of something new every day. If what you know is how to use Excel like a boss, don Take small steps enhance what you kn
2020-2-19
blog-post
Moving into 2020 I have been really leaning on using purple as my theme color ![ This is what we are working with. It has been my card design for at least a plain cover art does not fit I am not digging the date on cover art that also has text I hav
2020-2-17
blog-post
It was so easy to get a professional looking navbar with just 3 lines of code. installation is easy There was no instructions for es6 style imports that are common with gatsbyjs Simply wrap your existing content, Nav in my case, with the I think thi
2020-2-11
blog-post
Free only one step is free can take rss to social Free to get started Free 7 day trial $4.99 cheapest after
2020-2-10
blog-post
I finally fixed my Styled-Components in gatsby.js. I am starting a redesign of I have a small issue that has plagued the site for at least a year I have been using styled components in my gatsby.js site for about a year now. And it has been plagued
2020-2-8
blog-post
I have happily had my personal site I have recently started playing with zeit again. I really like their cli tool, its dead simple and makes sense. I tried the netlify one early on and dont think I really gave it much of a chance. I was able to ba
2020-2-6
blog-post
Finally after years of hand typing out a full This one setting will now
2020-2-4
blog-post
This morning I logged into my machine and was nearly out of space 64GB miniconda3 5GB conda cache 4GM pip cache 34GB docker Photo by These are the commands that I often use to reclaim space. Its so easy to fill up small vm This shows us where to st
2020-2-1
blog-post
I love using URL shorteners to easily share links without hitting character I recently discovered a really cool feature of netlify that I have always looked past, simply add a Now with shorter links we have more space for our content without needin
2020-1-29
blog-post
cli tools are super handy and easy to add to your python libraries to supercharge them. Even if your library is not a cli tool there are a number of things that a cli can do to your library. Things a cli can do to enhance your library. 🆚 print versio
2020-1-29
blog-post
I spend a lot of my time at the terminal for my daily work, mostly in Linux or wsl. One big reason for using wsl over cmd.exe is the ease of walking through history that fzf provides. This week we had a windows bug in a cli and I was stuck in vanil
2020-1-23
blog-post
Mine is the python debugger. I was a long holdout thinking that print statements were sufficient. That was untill I started having errors crop up in functions that took minutes to run. The thing that I most notably wish I would have known about is po
2020-1-10
blog-post
I have been using oh-my-zsh successfully for about 2 years now. But lately my I believe the real issue is io speed on wsl. I have some remote servers with Quick side note: your zsh config is controled by your Time the Move your Time the fastest s
2020-1-3
blog-post
Vim Location List eating up the screen while I am zoomed in and trying to live code Through some google search I found the culprit was syntastic. It has an I want to keep the location list open automatically most of the time, but when I don First,
2020-1-1
blog-post
2019-12-21
blog-post
Click primarily takes two forms of inputs Options and arguments. I think of options as keyword argument and arguments as regular positional arguments. typically aliased with a shorthand ( ** To get the Python argument name, the chosen name is conver
2019-11-11
blog-post
I am tweeting out most of these snippets as I add them, you can find them all here Below are some quick snippets/notes for when using kedro to build data pipelines. So far I am just compiling snippets. Eventually I will create several posts on kedro
2019-11-2
blog-post
jmespath Tabnine |-|-| I definitely want to try this out with kedro. Bulwark is a package for convenient property-based testing of pandas dataframes, supported for Python 3.5+.
2019-10-14
blog-post
2019-10-1
blog-post
Pathlib is an amazing cross-platform path tool.
2019-9-26
blog-post
2019-9-25
blog-post
Good for method chaining, i.e. adding more methods or filters without assigning a new variable. general purpose, this is probably the most common method you see in training/examples capable of including multiple strings to include Good For partial ma
2019-9-24
blog-post
I love digital ocean for it
2019-9-22
blog-post
tqdm is one of my favorite general purpose utility libraries in python. It for more gifs like these follow me on twitter TQDM also has a convenience function called trange that wraps the range function with a tqdm progress bar automatically. There i
2019-9-18
blog-post
If you are a regular listener of As you graduate your scripts into modules and libraries you might start to notice that you need to pass a lot of data around to all of the functions that you have created. For example if you are running some analysis
2019-9-11
blog-post
This post is intended as an extension/update from This post is intended as an extension/update from I use it in more places than I probably should Before we get into it, I want to make a shout out to Kenneth Reitz for making this so easy. Kenneth
2019-9-10
blog-post
Bash is super powerful. * * There are so many options inside of https://waylonwalker.com/refactor-in-cli/ I use these replace commands heavily when doing large refactorings. I like this one when there is not a good cli into config files and I need t
2019-9-8
blog-post
I have used 🔥 Blazing Fast 💥 Keeps me in the comfort of my text editor 👏 Allows me to use Jupyter when I need 👟 Extremely Reliable One of the biggest benefits that I find is that it shortens the distance between my module/library code and test code
2019-9-8
blog-post
Throw that mouse Away its time to setup some keyboard shortcuts. These sortcuts were the baseline for switching from tmux/vim to vscode. Most folks posts I was able to find gave great tips on replacing vim, but very few have focused on the hackabili
2019-7-7
blog-post
My git workflow based on real life. Its not always clean and simple. As complicated as that seems it is pretty straight forward. When you sit down to work the first thing you do is to Next create a new branch with a name that will remind you of w
2019-5-27
blog-post
Testing out forestry.io I have been playing with the netlify cms for a while now, and it has been a decent experience, but I Forestry.io has this amazing feature to create By far my favorite feature is I have a use case for a photography site wher
2019-4-23
blog-post
rebase git commit --amend rage quit git reset HEAD~n removes modifications keeps hitsory of changes and undoes them git checkout HEAD~n -- keeps modifications removes history --SOFT --HARD --Mixed locally before push after push after push
2019-2-5
blog-post
rebase git commit --amend rage quit git reset HEAD~n removes modifications keeps hitsory of changes and undoes them git checkout HEAD~n -- keeps modifications removes history --SOFT --HARD --Mixed locally before push after push after push
2019-2-5
blog-post
I am addicted to the process of learning and improving my skills nearly to a fault. The reason I say nearly is because my addiction is fueled with results. I crave the output of my work enhance the work of others. I jump with joy as I see users gain
2019-2-2
blog-post
This morning I log into my VCS and check activity on my projects to find that Clone the repo, note it must be a Curl down the Run the script, and push the updates. Delete the I hope this helps someone, or future me who needs to fix their informat
2019-1-21
blog-post
This is a follow up to my previous post This year I will become more productive, by intensionally working on a well thought out plan, learning the right technologies, and leave behind a positive legacy. Last year I was able to make some great stride
2019-1-12
blog-post
2018 was a year of many ups and downs, and learning to deal with a whole new At home the doctor appointments keep piling in. While I am learning to deal Here are the goals I had listed out for 2018 and how I faired at completing Continue Take 2 10m
2019-1-5
blog-post
Today I want to take some time to talk about the things that make me happy in my work environment. This is completely free-flow off the cuff, but are things that I do that make me happy, not having them would definitely be a deal breaker for me . A
2019-1-1
blog-post
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere
2018-5-8
blog-post
I recently finished up the flexbox-zombies course to learn more about flexbox, and to become proficient with it. I can truly say that this course has changed the way that I create layouts. Flexbox is very intuitive now. What this course does reall
2018-4-22
blog-post
I
2000-1-15
til