Blog

Netlify I have happily had my personal site waylonwalker.com hosted on netlify for nearly 2 years now. In fact I have hosted about a dozen different toy projects to play with on there, 4 of which have gone far enough to get a custom domain name. They are fast to deploy and consistently do so on every to main. Zeit I have recently started playing with zeit again. I really like their cli tool, its dead simple and makes sense. I tried the netlify one early on and dont think I really gave it much of a chance. I was able to backup a site we were modifying by saving everything locally (literally control+s) and running in the command line. While I was on the site I realized that when switching to gatsby v2 I had deployed it to now.sh while testing, before cutting over to the updated one on netlify. It has been building every version since without issue! It has been building every version since without issue! What I am using gatsby forestry.io I am using gatsby to build my site, and I do use f…

Finally after years of hand typing out a full I found there is a setting to automatcally push to the current branch. More realisitically I just did a let git yell at me, and copying the suggestion. git config This one setting will now to the current branch without yelling at you that your upstream does not match your current branch. This helps me ship chnages faster as I am constantly chnaging projects and branches.

This morning I logged into my machine and was nearly out of space 64GB miniconda3! 5GB conda cache 4GM pip cache 34GB docker Find it These are the commands that I often use to reclaim space. Its so easy to fill up small vm’s in the cloud, or in my case today let your dev machine go way too long without a good cleanup. Show Remaining Space on Drives This shows us where to start and gives a baseline of how much space we have reclaimed. show largest files in current directory Next keep drilling into directories that are big and running this command to see whats big inside of it. When you find somethign that you are willing to part with it and check to see if you have enough reclaimed yet. Honestly I rarely bother unless the directory is in the GB’s of space. A super simple filter for that is to just grep for G. conda How Many? As a first baseline lets see how many enviroments we are starting with. I started with 71. Yeah I have had this machine for 2 years, and dont regularly remove them.…

cli tools are super handy and easy to add to your python libraries to supercharge them. Even if your library is not a cli tool there are a number of things that a cli can do to your library. Example Ideas Things a cli can do to enhance your library. 🆚 print version 🕶 print readme 📝 print changelog 📃 print config ✏ change config 👩‍🎓 run a tutorial 🏗 scaffold a project with cookiecutter 🖱 Click Click is the most popular python cli tool framework for python. There are others, some old, some new comers that make take the crown. For now Click is the gold standard if you want to make a powerful cli quickly. If you are dependency conscious and dont need a lot of tooling, use argparse. Project Structure ❯ cli.py ✨ init.py For our simple_click library can be left empty. It is here purely to signify that simple_click is a library. It is likely that you will import other modules here that need to reside at the top level of your library api, your cli does not need to be at the top of of your api.…

I love using URL shorteners to easily share links without hitting character limits, but they loose their meaning. Services like bit.ly will save my links for me so that I can find them, but I would rather them to be easy to remember. https://bit.ly/2ruLwQz does not roll of the tongue so well. 301 🤸‍♀️ I recently discovered a really cool feature of netlify that I have always looked past,. It is so simple cool and powerful, every netlify site should do this! But how 🤷‍♀️ simply add a file to the root of your your published site with the following format. The trick I found with my gatsby site was that it needed to be in my static directory, not root. Next you just put space separated links on separate lines. #’s can be used for comments. 🙌 Share those short links Now with shorter links we have more space for our content without needing to use a service like bit.ly that makes our links unreadable.

I spend a lot of my time at the terminal for my daily work, mostly in Linux or wsl. One big reason for using wsl over cmd.exe is the ease of walking through history that fzf provides. This week we had a windows bug in a cli and I was stuck in vanilla cmd.exe 😭 > Cmder First off if you are stuck using cmd.exe, do yourself a favor and get cmder. It makes life just a bit easier. It is super confugurable and comes with several power ups that make it a bit more enjoyable than cmd.exe. History F7 - Scroll through history F8 - Search history based Example.bat The next simple technique is to save your commands into a.bat file. Any valid command ran with cmd.exe can be saved into a bat file and called again later by running it in the terminal. save your command use f7/f8 to get your command back add at the end, hit the home key and add echo to the front. Do not wrap with quotes. This is not bash. » append type not cat To ensure that you got the command right… and didn’t forget that you were in…

Mine is the python debugger. I was a long holdout thinking that print statements were sufficient. That was untill I started having errors crop up in functions that took minutes to run. The thing that I most notably wish I would have known about is post_mortem. Example —-> 3 print(f’{msg}\n’ * repeats) 4 Debug with iPython/Jupyter Vanilla Debug More For more information about the debugger checkout the real python article. https://realpython.com/python-debugging-pdb/ Also keep a bookmark of the table of pdb commands from the article https://realpython.com/python-debugging-pdb/#essential-pdb-commands Debug Session

I have been using oh-my-zsh successfully for about 2 years now. But lately my startup time has been really bothersome. It has grown to the point where it was taking about 5.5s to startup a shell! This is ok if I am going to spend some time in here for awhile and do some work that benefits from all of the autocompletions, plugins, and shortcuts that oh-my-zsh brings. But to only jump in to run a handful of commands is infuriating. 📑 My Setup I believe the real issue is io speed on wsl. I have some remote servers with similar configs that are 10x faster or more, loading in 100s of milliseconds rather than seconds. Sourcing all of the individual plugin files are just too much for it. 💨 How Fast can it be Quick side note: your zsh config is controled by your ~/.zshrc file. This file can source other files, load plugins, or run literally anything. Time the initial time Move your ~/.zshrc config file. Time the fastest startup possible with nothing in your ~/.zshrc config file. Move your ~/.z…

Vim’s (neovim in my case) location list can provide some very useful information while developing. Mine gives me information about linting and type checking errors with fairly little config. Generally, it sits nicely at the bottom of the screen and barely affects me. Other times, especially while zoomed way in during a presentation, it just gets in the way. Location List eating up the screen while I am zoomed in and trying to live code Toggling the location list Through some google search I found the culprit was syntastic. It has an feature. We can turn it off by setting. Keybindings I want to keep the location list open automatically most of the time, but when I don’t want it to keep opening it’s generally detrimental. Trying to live code while the location list keeps taking up the whole screen is not cool. First, create a function that will toggle both the location list and syntactic together. This binding will allow me to use from normal mode to toggle the location list. I am starti…

tqdm is one of my favorite general purpose utility libraries in python. It allows me to see progress of multipart processes as they happen. I really like this for when I am developing something that takes some amount of time and I am unsure of performance. It allows me to be patient when the process is going well and will finish in sufficient time, and allows me to 💥 kill it and find a way to make it perform better if it will not finish in sufficient time. @waylonwalker Add a simple Progress bar! convenience TQDM also has a convenience function called trange that wraps the range function with a tqdm progress bar automatically. notebook support There is also notebook support. If you are bouncing between ipython and jupyter I recomend importing from the auto module. Autoreload in Ipython If you are using notebooks you should enable ipython autoreload 👆

If you are a regular listener of TalkPython or PythonBytes you have hear Michael Kennedy talk about Named Tuples many times, but what are they and how do they fit into my data science workflow. Example As you graduate your scripts into modules and libraries you might start to notice that you need to pass a lot of data around to all of the functions that you have created. For example if you are running some analysis utilizing,, and data. You may need to calculate total revenue, inventory on hand. You may need to pass these data sets into various models to drive production or pricing based on predicted volumes. Load data Here we setup functions that can load data from the sales database. Assume that we also have similar functions to and. Create Metrics Here we create our first function to calculate some metrics. There are likely many of these functions that repeat a similar pattern. They use similar data and have their own custom logic for calculations and joins. Furthermore these functi…

This post is intended as an extension/update from background tasks in python. I started using the week that Kenneth Reitz released it. It takes away so much boilerplate from running background tasks that I use it in more places than I probably should. After taking a look at that post today, I wanted to put a better data science example in here to help folks get started. This post is intended as an extension/update from background tasks in python. I started using the week that Kenneth Reitz released it. It takes away so much boilerplate from running background tasks that I use it in more places than I probably should. After taking a look at that post today, I wanted to put a better data science example in here to help folks get started. I use it in more places than I probably should Before we get into it, I want to make a shout out to Kenneth Reitz for making this so easy. Kenneth is a python God for all that he has given to the community in so many ways, especially with his ideas in bu…

I have used for several years now with great success and 🔥 rapid reloads. It allows me to move super fast when developing libraries and modules. They have made some great updates this year that allows class modules to be automatically be updated. What I like about autoreload 🔥 Blazing Fast 💥 Keeps me in the comfort of my text editor 👏 Allows me to use Jupyter when I need 👟 Extremely Reliable One of the biggest benefits that I find is that it shortens the distance between my module/library code and test code inside of a terminal/notebook. Now I primarily use jupyter notebooks for the presentation aspect. I develop code from the comfort of my editor with all of the tools I have setup, and run the functions in a notebook to get the output. From there I might do some aggregations or plots, but the 🥩 meat of development is done outside of jupyter. Now I primarily use jupyter notebooks for the presentation aspect. Enabling Autoreload 📐 config This is a short script that I use to setup ipytho…

Throw that mouse Away its time to setup some keyboard shortcuts. These sortcuts were the baseline for switching from tmux/vim to vscode. Most folks posts I was able to find gave great tips on replacing vim, but very few have focused on the hackability of tmux. tmux allows me to rapidly fire up a workspace, create new windows and splits. Then When I switch tasks I can leave that workspace open and and jump right back in later exactly where I left off. There is nothing quite like it. The shortcuts listed here make the transition a bit better. The worst thing I found when using vscode at first was no way to switch between the terminal and editor without the mouse. This first set of keybindings solve that issue. The worst thing I found when using vscode at first was no way to switch between the terminal and editor without the mouse. See-also I have an updated article in my tmux workflow How I navigate tmux in 2021 Alt+[hjkl] navigation ⬅ jump to left split alt+h ⬇ jump to terminal from edi…

My git workflow based on real life. Its not always clean and simple. sometimes things get messy The Clean Path pull 👉 branch 👉 format 👉 work👉 add 👉 commit 👉 pull 👉 rebase 👉 push Pull As complicated as that seems it is pretty straight forward. When you sit down to work the first thing you do is to pull down the teams latest working “develop” branch from git. Branch Next create a new branch with a name that will remind you of what you are working on. For your own sanity choose something descriptive. It is easy to get too many similar branches going and forget which branch is which. Format If you know which files in existance that you will be editing before you start work it is a good idea to format them in a commit early on to keep your working commits separate from formatting. This will make it easier for reviewers to distinguish from your changes and formatting fixes. If your team agrees to a consistent formatting logic, sticks to it and always remembers to run the linting/fixing tools…

Testing out forestry.io Sorry Netlify CMS I still ♥️ your product dont be forestry is simple I have been playing with the netlify cms for a while now, and it has been a decent experience, but I really struggle configuring it. Forestry is so simple to setup. My favorite part is that I can code up my gatsby.js site, storing all editable text in markdown, and come back later and add the CMS based on existing documents. Configuration is Simple Forestry.io has this amazing feature to create create based on existing document 🤯. This is great because it sets up the config for me without error. And If I really want to come back later to customize it more I have that option, too. Multi-File Gallery I have a use case for a photography site where the owner wants to be able to show off sample photos of each type of work she does. I got it working in the netlify cms, although it was not very user friendly. Everything was nested in an accordian 😢. Next I looked into forestry.io. I pointed forestry.i…

This is my story into data science. The Journey Begins I am addicted to the process of learning and improving my skills nearly to a fault. The reason I say nearly is because my addiction is fueled with results. I crave the output of my work enhance the work of others. I jump with joy as I see users gain insights they could have never imagined before. My mouth starts watering as I see their boring repetitive data mining activites be completed in a matter of seconds, opening up their mind to focus on their expertise. The day I stop learning will be the day that I start looking for another career path. It’s not all about winning. ~Mom This happened to me in 2014. I have a mechanical engineering degree and had a really good position at the time. I owned full engineering control of a small subset of engine components. The problem was that Everything is so proprietary and hardly documented the process of learning did not click with me. I felt like I had learned a lot about the product early…

This morning I log into my VCS and check activity on my projects to find that someone else has been very active on my projects fo the last few weeks. I quicklyhover over the missing avatar to find that It’s Me. What’s wrong here, why do I look like two different people throughout the day! upon further investigation I see the issue. while setting up a new terminal environment I mistyped my email address by one character. After much searching and a few failed attempts I was able to fix it by following an article no longer available (2021) from https://help.github.com/articles. Bare Clone Clone the repo, note it must be a clone. git -author-rewrite Curl down the script and edit the following variables Run the script, and push the updates. Cleanup Delete the repo from your local machine. I hope this helps someone, or future me who needs to fix their information in git.

The year of intenionality This is a follow up to my previous post 2018-retrospective professional This year I will become more productive, by intensionally working on a well thought out plan, learning the right technologies, and leave behind a positive legacy. Productivity Last year I was able to make some great strides in my productivity and focus. This year I want to be able to bring it up a notch. Intentionality plays a big part in this. Taking some time to sit down and think about the tasks you realistically want to complete for the day, then focusing on those tasks one at a time. I started off the year last year using the pomodoro system to focus on one task and one task only for 25 minutes at a time. This worked really well for me but I quickly fell off of the train and fell into my old trap of reacting to the loudest customer over the items I put intensionally on my todo list. A kanban system is key to keeping it intentional. I currently use a very simple trello system, with ver…