Hello, I am Waylon Walker.

There are so many tutorials out there for creating OG images with nodejs or puppeteer, but I have yet to see many using

People who are quick to toss team members under a bus are not well trusted or highly thought of and it will lead to some

I create this blog with one person in mind, me. There are others like me This is not completely selfish, as there are li

There are various ways to configure python tools, config files, code, or environment variables. Let's look at a few pro

This is a post that may be a work in progress for awhile, Its a collections of thoughts on managing my blog, but could b

How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It's o

I am making another push in 2021 to get my content out in the world and meeting users where they are. See how I plan to

Docs https://kedro.readthedocs.io/en/stable/04_kedro_project_setup/01_dependencies.html?highlight=install pip-tools pip-

Generating an api for a blog is much simpler than one might expect with python. Markdown Frontmatter Fill in the blanks

Recently I automated starting new posts with a python script. Today I want to work on the next part that is editing tho

Gitui is a terminal-based git user interface (TUI) that will change the way that you work with git. I have been a long-t

In python data science/engineering most of our data is in the form of some sort of table, typically a DataFrame from a l

Changing conda environments is a bit verbose, I use a function with fzf that both lists environments and selects the one

What does it take to create an installable python package that can be hosted on pypi? What is the minimal python package

Replacing text based on whats in the current search register is a quite handy tool that I use often. I believe I picked

I do much of my work from tmux, I love it so much that I want to setup some functionality that puts me in tmux even if I

Replacing text in vim can be quite frustrating especially since it doesn't have live feedback to what is changing. Today

If you are like me, you have created a macro or two that is pure glory, and you forget how you made it after a day or so

Web browsers are a black hole of productivity. I try to use them as little as possible when it is time to focus. I try

cleanup tags tags ui post -> slides podcast -> audio visualizer

As projects grow patterns that worked early on break and we need to change things to make the project easier to work wit

I've grown tired of the standard ipython prompt as it doesn't do much to give me any useful information. The default on

I use my ipython terminal daily. It's my go to way of running python most of the time. After you use it for a little b

?docstring ??sourcecode %run %debug %autoreload %history autoformat %reset !shell commands ?docstring Stop going to go

One thing we all dread is mundane work of getting started, and all the hoops it takes to get going. This year I want to

In python data science we often will reach for pandas a bit more than necessary. While pandas can save us so much there

This is episode 1 of the Waylon Walker Audio experience, posts from waylonwalker.com in audio form. So I have had this

I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in

Inspired by discourse's link expansion I am rolling out expansions for one line links on the blog

Kedro Plugin Idea for creating Dags quickly

Thinking through testing best practices for Data Pipelines

A few notes for my next refactoring of waylonwalker.com

notes about find and replace techniques

customize for the job Why are you a good fit? What will you bring to the role? Give real outcomes give real experience

use this profile image Please share your professional role as a data scientist? [Also feel free to share about your pe

There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processi

Reasons to Kedro collaboration Sharable catalog small nodes over monolithic notebooks catalog easily load anything with

Things to read later

This version of kedro releases a new set of supported deployment options and the spaceflights pipeline is officially added as a starter alias.

I started making stories as kind of a brain dump a few times per day and posting them to LinkedIn. Here are the last 11 days of stories

I was 20 commits into a hackoberfest PR when I suddenly realized they they all had my work email on them instead of my personal email 😱.

Nesting loops inside of each other in python makes for much harder code to understand, it takes more brain power to unde

I released a router-like plugin for kedro back in April 2020. This was not the first design, the idea actually came from one of the QB folks who taught me kedro nearly a year before.

bash while read file; do echo $(git log --pretty=format:%ad -n 1 --date=raw -- $file) $file; done < <(git ls-tree -r --n

Today we ran into an issue where we had a one-off script that just needed to work, but it was just chewing threw memory like nothing.

A common linting error thrown by various linters is for trailing whitespace. I most often use flake8. Having an automated way to fix linting errors such as trailing whitespace is invaluable.

There are many useful chrome extensions out there. I probably have way too many installed, here are four that I am currently using.

Bash is a language that is quite useful for automation no matter what language you write in. Bash can do so many powerful system-level tasks. Even if you are on windows these days you are likely to come across bash inside a cloud VM, Continuous Integration, or even inside of docker.

Here are three things that I see my non programming counterparts doing every single day. These really sum up so much of what folks do within an office.

Installing miniconda from only the command line can be a bit of an intimidating task your first time. Lets walk through how to do it.

In my upcoming free kedro course, you can learn how to start building pipelines in 5 days.

Here are a few of my top tips to help make dev.to posts more readable, and drive better engagement.

It's a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template.

I was particularly inspired by @chantastic episode 103 of the react podcast with @ParissAthena. They spoke about the black tech pipeline as well as Diversity, Equity, and Inclusion. Pariss is quite an inspiration. She has done so much work to create a better place for POC in tech. I like that not only is she helping them get jobs but acting as a mentor for their first few months on the job to make sure that they are able to find their place and fit in.

If we take a look at the release notes I see one **major** feature improvement on the list, auto-discovery of hooks.

As I continue to build out https://waylonwalker.com/ I sometimes run into some errors that are not caught becuase I do not have good testing implemented. I want to explore some integration testing options using GitHub actions.

Today I setup a new machine on Digital Ocean to use with TestProject.io, Here are my installation notes.

When learning a new skill it's important to practice along the way. In order for me to show up to practice I need to make it easy to show up. An easy way to show up to practice with python is to use an online repl. With these you can try out something quick. Sometimes I see snippets from blogs or tweets and I need to try the out for myself to really understand.

I am exploring a kedro catalog meta data hook

I recently added a button to my blog, and subsequently my posts on [DEV.to](https://dev.to/waylonwalker). It's the best thing that I have done for it in a while.

When first learning to code its very common to hard code everything right into the code. This happens with most folks in just about any language. Whether its html or markdown for front end content, or even hardcoding parameters in our backend languages like python, or node.js.

I am looking into a way to replace my google reader experience that I had back in 2013 before google took it from us. I am starting by learning how to parse feeds with python, and without much previous knowledge it proved to be much easier than anticipated thanks to the `feedparser` library.

Notes about my reader idea

Can someone explain how or why twitter cards render differently from device to device? I do understand that twitter cards a built from meta tags, the full list can be found in their

Comment on the DEV.to thread to show off what is on your GitHub profile.

Python `*args` and `**kwargs` are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packed and unpacked into lists or dictionaries. Beware though, this power **can** lead to some really unreadable/unusable code if done wrong.

My Notes about using kedro

kedro 0.16.2 just dropped last week with a long-awaited feature... catalog search! I went as far as monkey patching this into each of my projects. I work jump between a few really big projects that have tons of datasets. Being able to quickly search for what I need is so useful.

This day 8 years ago I started my first day as a Mechanical Engineer. I am so grateful for this journey that I have been able to have. There is no way that I could have planned this journey from the beginning.

Passing inputs into kedro is a key concept. Understanding how it accepts a single catalog key as input is quite trivial that easily makes sense, but passing a list or dictionary of catalog entries can be a bit confusing.

I just did a quick refactoring of my JAMStack blog urls. Some didn't fit with my style, some had `_` that I wanted to switch to `-`, and others were rediculously long. I've been using forestry as my CMS, I write many of my posts there, and sometimes it picks some crazy file names (based on my titles). It was time to refactor.

It's been a long time coming. We use some very harsh language within tech so much sometimes that we become numb to it. It's time to do my very small part in this movement and purge this language from my active repos starting with this blog right here.

Python `*args` and `**kwargs` are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packed and unpacked into lists or dictionaries. Beware though, this power **can** lead to some really unreadable/unusable code if done wrong.

This is my journey to building up the community page.

I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really it's easy. It will automatically run checks for you. In some cases, it will even automatically fix them for you. Out of the box, it will do things like automatically trim extra whitespace, fix file endings, and ensure file sizes are not too large for git.

kedro-static-viz is out with support for the newly released hooks feature. This means that you can have `kedro-static-viz` automatically deploy a full gatsby site `before_pipeline_run` keeping your visualization always up to date.

There are two main ways to create kedro hooks, with modules and classes. Each one still uses the same verbiage as the function/method names. Class hooks seem a bit special as they give you a way to configure them so that they are a bit more generally useful.

This post is a 🧠 branstorming work in progress, I will likely use it as a storage location/brain dump of hook ideas.

This might be useful https://github.com/nfl/react-helmet/issues/146 https://github.com/gatsbyjs/gatsby/issues/13136

I want to incorporate some of the wonderful comments, 💕, 🦄, and 🔖's that I have been getting on dev.to on my website. I have dabbled once or twice with no avail this time I am taking notes on my journey, so follow along and let's get there together. By the end of this post, I will have a way to get comments from posts on the client-side thanks to the wonderfully open dev.to API.

You're awesome! Thanks for subscribing to my newsletter.

GitHub actions can give you confidence that your site is up and running, with the latests JavaScript packages, does not have broken links, and can even take screenshots of what your website looks like on different screen sizes and operating systems.

https://stackoverflow.com/questions/47488440/how-do-i-add-google-fonts-to-a-gatsby-site

Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For the most part you just need to tell Kedro what connector to use and its filepath. When running Kedro takes care of all of the read/write, you just reference the catalog key.

dead simple docstring coverage for your python project

I always struggle to find a good reference of ascii characters. This is a compilation of my most wanted ascii characters.

Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro `0.16.0`. They allow you to hook into `catalog_created`,`pipeline_run`, and `node_run`(nouns). With a `before`, or `after` (adjective). This really reminds me of reacts lifecycle hooks, that let you hook into various state of react web components. This is going to make kedro so extendable by the community. I am super pumped to see what the community is able to do with this ability.

run checks before running the pipeline

bash git remote -v git remote add gitlab <url> git push gitlab main

kedro is an amazing project that allows for super-fast prototyping of data pipelines, yet yielding production-ready pipelines. find-kedro enhances this experience by adding a pytest-like node discovery eliminating the need to bubble up pipelines through modules.

Often when reading through javascript examples you will find some arrow functions use parentheses `()` while others use braces `{}`. This key difference is that parentheses will implicitly return the last statement while braces require an explicit return statement. It is important to understand the difference between them because it is likely that you will find code examples of both and trying to edit code written differently than you're used to may have unintended consequences.

Inspired by Chris Achard My ideas Python List comps Classes Inheritance Background Click Lambdas Kedro Cataloging Cus

This past week I had a really weird bug in my [kedro](https://kedro.readthedocs.io/) pipeline. For some reason data running through my pipeline was coming out completely made no sense, but if I manually request raw data outside of the pipeline it matched expectations.

Is it possible to deep watch a single python function for changes? Shallow Watch keeping track of a python functions has

If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have such a simple cli I have opted to use the cli for most of these, that way I know exactly what is happening and have more control over it if I need.

So often I see a variables `type()` inside of its name and it hurts me a little inside. Tell me I'm right or prove me wrong below.

One useful action that you can utilize no matter what content your repo has is sending email.

GitHub actions use YAML to configure your workflows. What gets ran, When it runs, What it runs on. Let's discuss YAML and the structure needed for GitHub actions.

GitHub actions are an amazing tool that allows us to run code based on triggers inside of our repo. Their is a large and growing community of actions inside the marketplace to use with very little effort. Best of all they are free for public repositories, and private repos have a very generous free tier.

Show up For Practice how I practice What Works for You

Sometimes we get a little `git add . && git commit -m "WIP"` happy and mistakenly commit something that we just cant figure out. This is a good way to figure out what the heck has changed on the current branch compared to any other branch.

Getting up and going with a brand new [kedro](https://kedro.readthedocs.io) project is super simple, thanks to the help of the `kedro new` command. The ability to add an example pipeline from the start makes it that much easier to get going and have a template to follow for your own projects.

What is YOUR Advice for New Data Scientists

Do you have any use cases that you use hoising? Why? It seems like a really cool feature in any language that uses it, but I dont really notice it in use.

Inspired by https://m.soundcloud.com/front-end-happy-hour/episode-095-shot-for-shot-one-on-ones

Kedro is an open source data pipeline framework. It provides guardrails to set your project up right from the start without needing to know deeply how to setup your own python library for data pipelining. It includes really great ways to manipulate `catalogs` and `pipelines`. This article will cover the 10K view of kedro, future articles will dive deper into each one.

🏷️ Long variable names are a good thing. Self documenting code is more important than poorly documented code. Simply a

Getting a custom scrollbar on your site makes it stand out a bit compared to the very plain stock one that are on most sites. This is how I set mine up on my gatsby site.

Make the change easy Make the easy change Martin Fowler When to refactor

Stick to what you know, and learn a little bit of something new every day.

I documented the story of my 2020 waylonwalker.com redesign on my blog. This post walks through the tools and inspiration for the card design.

It was so easy to get a professional looking navbar with just 3 lines of code. This thing is so usable on mobile.

Social Automation Zappier Free only one step is free can take rss to social Automate.io * Form submission https://getf

I finally fixed my Styled-Components in gatsby.js

Should I switch to Zeit Now. Netlify build times are starting to creep in.

git config --global push.default current

Out of Space! How to remove 65 conda environments in one command.

Personal URL shortener with Netlify Redirects

Add helpful cli to your python libraries... All of them!

cmd.exe tips

Add an rss feed to your Gatsby Site

What is something that you recently learned that you wish you would have learned or understood earlier?

I have been using oh-my-zsh successfully for about 2 years now. But lately my startup time has been really bothersome. It has grown to the point where it was taking about **5.5s** to startup a shell! This is ok if I am going to spend some time in here for awhile and do some work that benefits from all of the autocompletions, plugins, and shortcuts that oh-my-zsh brings. But to only jump in to run a handful of commands is infuriating.

none

Vim's (neovim in my case) location list can provide some very useful information while developing. Mine gives me inform

My Notes about using sqlalchemy models

What DataScientists Should Know About S3

learning about building cli apps in python

My Notes about using kedro

jmespath Tabnine Bulwark |-|-| |github: |https://github.com/zaxr/bulwark| I definitely want to try this out with kedro

Debugging Python

Pathlib is an amazing cross-platform path tool.

Custom Python Exceptions

Filtering pandas DataFrames many different ways.

I started using aws in March 2019. Here are some of my notes. aws-cli ec-2 SageMaker S3

I love digital ocean for it's simplicity and its commitment to open source.

Tools Sphinx Portray I just started using portray and it is amazingly simple to use! Methodology

I have been using pyspark since March 2019, here are my thoughts.

Quick Progress Bars in python using TQDM

This post is intended as an extension/update from [background tasks in python](https://waylonwalker.com/background-1/). I started using `background` the week that Kenneth Reitz released it. It takes away so much boilerplate from running background tasks that I use it in more places than I probably should. After taking a look at that post today, I wanted to put a better data science example in here to help folks get started.

Autoreload in python

Waylon Walker's Bash Notes

Reimagining my tmux configuration inside of vscode.

My git workflow based on real life. Its not always clean and simple.

Creating Posts from forestry.io.

How to setup a data science project in python.

This is my story into data science.

2019 goals

happiness in the workplace

none

none

none

none

none

none

none

none

none

none

none

none

none

none

none