Avoid serious version conflict issues, and use a virtual environment anytime you are running python, here are three ways you can setup a kedro virtual environment. https://youtu.be/ZSxc5VVCBhM I prefer to use conda as my vir
2023-8-19
https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let's make some nodes with some vanilla data structures. You will need to import node from kedro.pipeline to start creating nodes. The
2021-9-13
Kedro comes with an install command to install and manage all of your projects dependencies. https://youtu.be/IWimEs-hHQg You must start by having your kedro project either cloned down from an existing
2021-8-21
Immediately after kedro new , before you start running kedro install or your first line of code the first thing you should always do after getting a new kedro template created is to git init
2021-8-20
https://youtu.be/uqiv5LAiJe0 Kedro new is simply a wrapper around the cookiecutter templating library. The kedro team maintains a ready made template that has everything you need for a kedro project. They also maintain a few kedro star
2021-8-18
Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user a way to build pipelines that automatically take care of io through the use of abstract DataSets that the u
2021-8-17
https://youtu.be/bw5_FWDVRpU I recently switched over to using Ubuntu, it works well pretty much out of the box for me. I am using gnome with a dark theme. I am still using the built in default gnome terminal, it just works.
2021-8-16
https://youtu.be/LLk94fKpGg4 As we push the limits of tmux further and further you are bound to end up in a situation where you are mashing down a hotkey and it's just not doing what you want it to do, and you have no idea why.
2021-8-14
https://youtu.be/nT0FA1RNZEs Now your creating, jumping, and killing sessions like a boss. You are slicing through projects with ease, let me show you one more thing that can be the cream on top of this silky smooth setup we have been wo
2021-8-13
https://youtu.be/utfLA6L8o5s You've got some long running tasks, and you're trying to stay productive and knock tasks off that board, but you keep finding that your processes finish and you stay on other tasks for longer than you should.
2021-8-12
https://youtu.be/QWPyYx54JbE Now it's time to switch gears, we are onto a different part of our day and there are just too many sessions running and we need to clean up shop. One viable option is to nuke the whole dang thing.
2021-8-11
https://youtu.be/XucVVgGmesM I see you there, trying to script out your tmux layouts. Tryig to get each project setup just perfect with a script, but you keep stumbling over yourself with duplicate session error me
2021-8-9
https://youtu.be/Y1MYmL8ZolE Tmux list keys can be a useful tool to help remind you of what kebindings you have setup. You can search for them and scroll just like in tmux copy-mode. You can call list-keys from the command li
2021-8-8
https://youtu.be/mvgM8UH11_U The tmux status bar can be a handy tool to remind yourself where you are within tmux. It can also include a bunch of system information like battery status, cpu, mem, whatever you can get from the command l
2021-8-7
https://youtu.be/dDq0depPrTs So you have been tricking out that .tmux.conf , you're looking for a silky smooth workflow that lets you fly through tmux with super speed, but every time you tweak out that
2021-8-6
https://youtu.be/DkJ9rb85LC0 Quickly getting between tmux splits is critical skill for productivity. You can get by with next or prev session for awhile, but if you have more than about thre
2021-8-5
https://youtu.be/8kZnjHPYnKU Now that we are splitting up work into their own sessions, lets talk about how to navigate between them without the command line. Navigating sessions is what kept me using a too many splits and windows workf
2021-8-4
https://youtu.be/WRLRiQDjVIA So you have been working on your tmux workflow, you've dropped a too many window workflow for scoping work that belongs together into separate sessions, but you cannot remember what session your work is in. I
2021-8-3
https://youtu.be/c-a2Bnv_NJ0 Scripting tmux to open up specific applications can be intimidating your first time. It can be tricky to get it to start in the right directory. If you are trying to assign applictaions to a keybinding it c
2021-8-3
https://youtu.be/5KE7Il7SOEk This is something that I made up but use every single day, this is what keeps much of what is on my blog or my teams private work wiki going. I have a few very important directories that I have assigned dire
2021-8-2
https://youtu.be/LbQNdCAUogE This one starts a new chapter in our series that is going to open up a whole new set of workflow productivity options, understanding how the new-session command is a critical command in
2021-8-1
These are my notes from taking @nnja's FEM course git-in-depth . For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read mor
2021-7-31
https://youtu.be/JQ0yDCVu44E attach is one of the most useful features of tmux. If you have no interest in tmux for pane and window management, you should use tmux for this. It can be a life saver if you ever get disconnected from the
2021-7-31
I use kedro catalog create to boost my productivity by automatically generating yaml catalog entries for me. It will create new yaml files for each pipeline, fill in missiing catalog entries, and respect already existing catal
2021-7-30
https://youtu.be/LY41GLn_DGg tmux ls will list the sessions that you have running within the tmux server if tmux is currently running. This is handy to combine with commands such as attach . https://
2021-7-30
https://youtu.be/SNu-4IrkjAs So far we have covered a lot of tmux commands and how they map to keybindings but these same commands can be executed at the command line. Let's make a popup that displays our git status for 5s or
2021-7-29
https://youtu.be/-ypY_-VmBKk tmux copy-mode is a tmux mode that lets you scroll, search, copy, and jump your way through a pane. There are a ton of keybindings for copy-mode, the main ones you will need to know are /
2021-7-28
https://youtu.be/Vm5rRtcVXLw Join-pane allows you to join panes that you have broken away from your window, or created in a different window to the window you want it in. As far as I know there is not a default keybinding for it.
2021-7-27
https://youtu.be/ICL609F2xnc Break-pane is a handy tmux command when your layout gets too cramped and you want to just move a split into its own window. Calling break-pane does exactly that, it creates a
2021-7-26
https://youtu.be/Rn6mOarCQ-Y Zooming into the current split in tmux is a valuable tool to give yourself some screen real estate. These days I am almost always presenting, streaming, or pairing up with a co-worker over a video call. Sin
2021-7-25
https://youtu.be/YRPZBv-iYyE New window as it sounds makes new windows in tmux. Windows are kind of like tabs. They are another screen within your sessions that you can name and make new panes in. Default key bindings for cr
2021-7-24
https://youtu.be/CPZJZjN9YTY These are my MOST often used keybindings that I use in tmux. They allow me to jump between splits with ease with a vim style layout. I can hold mod and jump between panes with a familiar arrow key.
2021-7-23
https://youtu.be/06z5qf81ofo Rotate window is the main way that I navigated tmux before I learned select-pane . It allows you to change your focused pane, or rotate the position of the panes easily. Def
2021-7-22
https://youtu.be/F0mHnwTrNNc When you get many splits going in tmux sometimes its time for a new layout. There are four layout strategies that I use, main-vertical, main-horizontal, even-vertical, even-horizontal. Almost always I am usei
2021-7-21
https://youtu.be/hpFYE2LU7xc Resizing panes in tmux can be quite difficult in default tmux, I use a set of keybingings to help resize panes in the rare occasions that I do need just a bit more space. I set the keybinding to the same as
2021-7-20
https://youtu.be/79Y-kqAiMpw Choose tree is a powerful tmux utility that provides a graphical interface to preview all sessions, windows, and panes, move between them kill them, move them and much more. The default keybinding
2021-7-19
https://youtu.be/BMkpbfhbkKM The prefix key is an essential part of tmux, by default all of tmux's key-bindings sit behind a prefix. This prefix is very similar to vim's leader key. It is common for folks to change the default
2021-7-18
https://youtu.be/kzgyiHap1nQ splitting panes is a core feature of tmux. It allows us to split the terminal vertically or horizontally into new panes. 🗒️ note that '#{pane_current_path}'will keep the split in the same direct
2021-7-17
https://youtu.be/RB87EEnnMnU An ultimate productivity key-binding in tmux is one to switch to the last session. I use this to quickly get between sessions really quick. Often I am working and need to lookup a quick note, or copy someth
2021-7-16
https://youtu.be/2ZqFDsJywt8 Tmux popups are actually floating windows that you can drag around the screen. They always open in the middle (by default) when you open them, no matter where you leave them. Here are a couple of
2021-7-15
https://youtu.be/2I8fB28zfB4 In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and gene read more waylonwalker.com for mo
2021-7-14
Kedro versioned datasets can be mixed with incremental and partitioned datasets to do some timeseries analysis on how our dataset changes over time. Kedro is a very extensible and composible framework, that allows us to build solutions from the indi
2021-7-8
I recently started streaming on twitch.tv/waylonwalker and it's been a blast so far. It all started with kedro/issues/606 , Yetu called out for users of kedro to record themselves doing a walk
2021-6-12
This was my seconf time ever streaming on twitch.tv/waylonwalker , and I completely botched my mic 2x. https://youtu.be/_7MwgKu-844 For what we are creating in these posts d3 is way overkill and ver
2021-6-7
https://youtu.be/I4VenHqIEng Doing some Digital Gardening on stream After this stream all gifs on my site are converted to mp4/webm if they exist. For what we are creating in these posts d3 is way overkill and
2021-6-6
This was my first time ever streaming on twitch.tv/waylonwalker . I am excited to get going. I have been streaming early in the morning while I am still waking up, so still a bit groggy as I go. https://youtu.be/Y
2021-6-4
With the latest releases of kedro 0.17.x , it is now possible to run kedro pipelines from within scripts. While I would not start a project with this technique, it will be a good tool to keep in my back pocket when I want to s
2021-5-23
Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while trying to implement a cli extension with clean output. First, how does one silence
2021-5-20
In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and generally keep work separated into logical groups. Since making this post, I have made ~20 other posts in short form that a
2021-5-11
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2021-5-7
When a python module is called it is assigned the __name__ of __main__ otherwise if it's imported it will be assigned the __name__ of the module. Let's create a module to
2021-5-7
Creating a directory that is a union of several directories can be achieved with a few symlinks at the command line. Here is how I am creating a virtual directory of all my projects that is a combination of both work and not-work projects.
2021-5-5
I really like using conda ( miniconda ) as my python virtual environment manager of choice. It's simple and it includes its own python interpreter using the version that I specify at creation. from their
2021-4-21
I've long used neovim from within windows wsl, and for far too long, I went without a proper way to get text out of it and into windows. wsl can access clip.exe. You can do some cool things with it, such as cat a file into the clipboard,
2021-4-17
I am trying to see what an embeded replit For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2021-4-15
Testing print/log statements in pytest can be a bit tricky, capsys makes it super easy, but I often struggle to find it. capsys is a builtin pytest fixture that can be passed into any test to capture stdin/stdout. For a more comprehensive
2021-4-5
Draft Post I've really been digging @willmcgugan's rich library for creating TUI like interfaces in python. I've only recently started to take full advantage of it. I am working on a projec
2021-3-27
With the realease of kedro==0.17.2 came a new module in the project template pipeline_registry.py . Here are some notes that I learned while playing with this new module. You should now have s
2021-3-20
People who are quick to toss team members under a bus are not well trusted or highly thought of and it will lead to some toxic team dynamics. While collaborating on any project there are going to be decisions made that aren't necessarily y
2021-1-22
I create this blog with one person in mind, me. This is not completely selfish, as there are likely many others out there that think similarly to me. Everyone comes from different backgrounds and varying levels of experience. In no way d
2021-1-21
How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It's only a total of 35 lines of python, 8 in setup.py and 27 in mini_kedro_pipeline.py .
2021-1-20
This is a post that may be a work in progress for awhile, Its a collections of thoughts on managing my blog, but could be translated into anythiung that is just a collection of markdown. For what we are creating in these posts d3 is way
2021-1-20
I am making another push in 2021 to get my content out in the world and meeting users where they are. See how I plan to execute. My content is written in markdown, all markdown. I find that markdown does a really great job at getting out
2021-1-19
Recently I automated starting new posts with a python script. Today I want to work on the next part that is editing those posts quickly. One thing we all dread is mundane work of getting started, and all the hoops it takes to get going
2021-1-18
Gitui is a terminal-based git user interface (TUI) that will change the way that you work with git. I have been a long-time user of the git cli, and it's been hard to beat, mostly because there is nothing that keeps my fingers on the keyboard quite l
2021-1-17
Gitui is a terminal-based git user interface (TUI) that will change the way that you work with git. I have been a long-time user of the git cli, and it's been hard to beat, mostly because there is nothing that keeps my fingers on the keybo
2021-1-17
In python data science/engineering most of our data is in the form of some sort of table, typically a DataFrame from a library like pandas, spark, or dask. These containers for data contain many convenient methods to manipulate table like
2021-1-14
Changing conda environments is a bit verbose, I use a function with fzf that both lists environments and selects the one I want in one go. I have used conda as a virtual environment tool for years now. I started using conda for its simpli
2021-1-11
Replacing text based on whats in the current search register is a quite handy tool that I use often. I believe I picked this tip up from Nick Janetakis, check out his YouTube channel for some amazing vim tips. https://www.youtube.com/wa
2021-1-10
What does it take to create an installable python package that can be hosted on pypi? This post is somewhat inspired by the bottle framework, which is famously created as a single python module. Yes, a whole web framework is written in on
2021-1-10
I do much of my work from tmux, I love it so much that I want to setup some functionality that puts me in tmux even if I didn't ask for it. Bash function to check if the shell is in a tmux session. I often open up vim to do some
2021-1-9
If you are like me, you have created a macro or two that is pure glory, and you forget how you made it after a day or so, or you immediately want to store it away as a custom keybinding. As with most things with vim, it's easy to do once you understa
2021-1-4
Replacing text in vim can be quite frustrating especially since it doesn't have live feedback to what is changing. Today I was watching Josh Branchaud's Vim-Unalphabet series on Youtuve and realized that his vim was doing this and I had to have it.
2021-1-4
Web browsers are a black hole of productivity. I try to use them as little as possible when it is time to focus. I try to use help , ? , or ?? with ipython, or --help at the comma
2021-1-2
As projects grow patterns that worked early on break and we need to change things to make the project easier to work with, and more welcoming to new developers. Before you start mucking up a project with wild commands at the terminal check
2020-12-30
I've grown tired of the standard ipython prompt as it doesn't do much to give me any useful information. The default one gives out a line number that only seems to add anxiety as I am working on a simple problem and see that number grow to several h
2020-12-20
I use my ipython terminal daily. It's my go to way of running python most of the time. After you use it for a little bit you will probably want to setup a bit of your own configuration. Activate your virtual environment of choice and pip
2020-12-20
One thing we all dread is mundane work of getting started, and all the hoops it takes to get going. This year I want to post more often and I am taking some steps towards making it easier for myself to just get started. When I start a new
2020-12-11
In python data science we often will reach for pandas a bit more than necessary. While pandas can save us so much there are times where there are alternatives that are much simpler. The itertools and more-itertools` are full o
2020-12-10
This is episode 1 of the Waylon Walker Audio experience, posts from waylonwalker.com in audio form. So I have had this idea for awhile to add audio to my blog posts. The idea partly comes from the aws blog, if you h
2020-11-25
Inspired by discourse's link expansion I am rolling out expansions for one line links on the blog waylonwalker . I was able to find a gatsby plugin gatsby-remark-embedder that expands one line links for
2020-11-18
I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in vanilla markdown in GitHub. I have been using a snippet that puts HTML into the markdown. While this works, it's more manual/dif
2020-11-18
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-11-12
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-11-6
use this profile image Please share your professional role as a data scientist? [Also feel free to share about your personal projects, publications, etc.] I graduated with a Mechanical Engineering Degree 8 years ago. Much of
2020-11-2
There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processing DataFrames from many data sources should be considering a pipeline framework. Kedro is a great option that provides many benefi
2020-11-1
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-10-31
STOP LEAVING Browser Tabs open and save them here! https://github.com/nanotee/nvim-lua-guide https://mlingelbach.com/neovim/issue_1/ https://github.com/nvim-telescope/telescope.nvim/pull/987 For
2020-10-28
Kedro 0.16.6 is out! Let's take a look through the release notes This is really exciting to see more deployment options coming from the kedro team. It really shows the power of the framework. The power of some of the
2020-10-25
I started making stories as kind of a brain dump a few times per day and posting them to [LinkedIn]( https://www.linkedin.com/in/waylonwalker/(https://www.linkedin.com/in/waylonwalker/ ). Here are the last 11 days of stories.
2020-10-21
I was 20 commits into a hackoberfest PR when I suddenly realized they they all had my work email on them instead of my personal email 😱. This is the story of how I corrected my email address on 19 individual commits after already submitting for a PR
2020-10-17
I released a router-like plugin for kedro back in April 2020. This was not the first design, the idea actually came from one of the QB folks who taught me kedro nearly a year before. We were assembling our pipelines with something called
2020-10-8
Today I ran into an issue where we had a one-off script that just needed to work, but it was just chewing threw memory like nothing. It started with a colleague asking me How do I clear the memory in a Jupyter notebook, these are the steps
2020-10-1
A common linting error thrown by various linters is for trailing whitespace. I most often use flake8. I generally have [pre-commit]( https://waylonwalker.com/pre-commit-is-awesome ) hooks setup to strip this, but sometimes I
2020-9-30
There are many useful chrome extensions out there. I probably have way too many installed, here are four that I am currently using. This post was inspired from Chris over at daily-dev-tips Love it or hate it passw
2020-9-13
Bash is a language that is quite useful for automation no matter what language you write in. Bash can do so many powerful system-level tasks. Even if you are on windows these days you are likely to come across bash inside a cloud VM, Continuous Integ
2020-8-13
Here are three things that I see my non programming counterparts doing every single day. These really sum up so much of what folks do within an office. So many of us dabble in or become power users of spreadsheets without knowing there is an altern
2020-8-11
miniconda is a python distribution from continuum. It's a slimmed-down version of their very popular anaconda distribution. It comes with its own environment manager and has eased the install process for many that do not have a way to compile c-exten
2020-8-10
Learn Kedro in 5 days For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-8-8
This post was inspired by a comment I left on @dsteenman's post. {% post dsteenman/how-long-should-a-blogpost-be-2k6n %} Most of the time I prefer short as I am more likely to read the whole thing. If its setup as a series I a
2020-8-7
It's a really cool GitHub action that will automatically commit files changed during the action. I was using this to render a new readme based on a template. Check out the repo for git-auto-commit-action .
2020-8-3
I was particularly inspired by @chantastic episode 103 of the react podcast with @ParissAthena. They spoke about the black tech pipeline as well as Diversity, Equity, and Inclusion. Pariss is quite an inspiration. She has done so much work to create
2020-8-3
If we take a look at the release notes I see one major feature improvement on the list, auto-discovery of hooks. This one comes a bit surprising as it was just casually mentioned in
2020-8-1
As I continue to build out waylonwalker.com I sometimes run into some errors that are not caught because I do not have good testing implemented. I want to explore some integration testing options using GitHub's actions.
2020-7-27
When learning a new skill it's important to practice along the way. In order for me to show up to practice I need to make it easy to show up. An easy way to show up to practice with python is to use an online repl. With these you can try out somethi
2020-7-25
Today I setup a new machine on Digital Ocean to use with TestProject.io, Here are my installation notes. envsubst < .github/ci/docker-compose.yml > docker-compose.yml For what we are creating in these posts d3 is way overkil
2020-7-25
I am exploring a kedro catalog meta data hook, these are some notes about what I am thinking. try pandas method -> try spark -> try dict/list -> none Is there an easy way to create a nosql database in memory from a a list of dic
2020-7-24
Mypy's config parser seems to be one of the most complex. This is likely in part to it having the largest backwards compatability of all projects that I looked at. mypy/config_parser options/config.py black
2020-7-21
I recently added a button to my blog, and subsequently my posts on DEV.to . It's the best thing that I have done for it in a while. It makes it so easy to do quick edits. I refer back to my old posts quite a bit, s
2020-7-18
When first learning to code its very common to hard code everything right into the code. This happens with most folks in just about any language. Whether its HTML or markdown for front end content, or even hardcoding parameters in our backend languag
2020-7-14
I am looking into a way to replace my google reader experience that I had back in 2013 before google took it from us. I am starting by learning how to parse feeds with python, and without much previous knowledge, it proved to be much easier than ant
2020-7-13
The input will be a yaml file containing a list of Items you want to stay up to date with. Inside each item will be a url, and weight. Each url will be pulled in and parsed into a standard data scructure. Some item
2020-7-12
I ran this post on dev.to and got a great response of great examples, check it out . The GitHub profile feature just went live for a subset of users. Simply creating a repo named after your username, and clicki
2020-7-11
Can someone explain how or why twitter cards render differently from device to device? I do understand that twitter cards a built from meta tags, the full list can be found in their docs Mobile Looks fine. On D
2020-7-11
I ran a discussion on dev that collected quite a list of examples in the comment section. So many great calls to action, animations, memes, and weird tricks. Upload all of your icons to the repo in a directory such as
2020-7-10
Python *args and **kwargs are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packe
2020-7-2
While using the catalog alone will not reap all of the benefits of the framework, it does get you and your project ready for the full framework eventually. For me the full benefit of the catalog comes when you combine it with the pipeline and dont e
2020-6-29
kedro 0.16.2 just dropped last week with a long-awaited feature... catalog search ! I went as far as monkey patching this into each of my projects. I work jump between a few really big projects that have tons of datasets. Be
2020-6-22
This day 8 years ago I started my first day as a Mechanical Engineer. I am so grateful for this journey that I have been able to have. There is no way that I could have planned this journey from the beginning. My initial career plans were
2020-6-19
Passing inputs into kedro is a key concept. Understanding how it accepts a single catalog key as input is quite trivial that easily makes sense, but passing a list or dictionary of catalog entries can be a bit confusing. Check out this po
2020-6-19
I just did a quick refactoring of my JAMStack blog urls. Some didn't fit with my style, some had _ that I wanted to switch to - , and others were ridiculously long. I've been using forestry as my CMS, I
2020-6-11
It's been a long time coming. We use some very harsh language within tech so much sometimes that we become numb to it. It's time to do my very small part in this movement and purge this language from my active repos starting with this blog right he
2020-6-11
Python *args and **kwargs are super useful tools, that when used properly can make you code much simpler and easier to maintain. Large manual conversions from a dataset to function arguments can be packe
2020-6-10
Follow along the Journey as I build out kedro.dev . I have really enjoyed my own personal journey as I have started to build all of my data pipeline projects with the kedro framework. I want to start building a plac
2020-6-5
I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really it's easy. It will automatically run checks for you. In some cases, it will even automati
2020-6-5
kedro-static-viz is out with support for the newly released hooks feature. This means that you can have kedro-static-viz automatically deploy a full gatsby site before_pipeline_run keeping you
2020-5-28
There are two main ways to create kedro hooks, with modules and classes. Each one still uses the same verbiage as the function/method names. Class hooks seem a bit special as they give you a way to configure them so that they are a bit mo
2020-5-23
This post is a 🧠 branstorming work in progress. I will likely use it as a storage location/brain dump of hook ideas. If you are completely unsure what kedro is be sure to check out my what is kedro post
2020-5-22
This might be useful https://github.com/nfl/react-helmet/issues/146 https://github.com/gatsbyjs/gatsby/issues/13136 For what we are creating in these posts d3 is way overkill and very verbose, but I need to
2020-5-22
I want to incorporate some of the wonderful comments, \U0001F495, \U0001F984, and \U0001F516's that I have been getting on dev.to on my website. I have dabbled once or twice with no avail this time I am taking notes on my journey, so follow along an
2020-5-20
GitHub's actions are a new GitHub feature that will trigger GitHub to spin up a virtual machine and run some tasks with some special access to your repo. It can interact with comments/issues, it can clone your repo, You can explicitly pass in secrets
2020-5-18
stack overflow link For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-5-17
Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For the most part you just need to tell Kedro what connector to use and its filepath. When run
2020-5-15
As usual while listening to python bytes 181 I heard of a tool that I had to try out right away! This thing is 🔥 hot off the press folks, we're talking the first release only 3 weeks ago. Its something that the pytho
2020-5-15
When creating cli's I often want some nice full-width character. I find it tough to find them, and when I do half the time it is an image or something that cannot be copied 👿. I rarely get very complex with my semi-manual ASCII art. I ca
2020-5-12
Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro 0.16.0 . They allow you to hook into catalog_created , pipeline_run , and node_run
2020-5-10
This is a very rough idea for a kedro package to prevent time lost to get partway through a pipeline run only to realize that you dont have access to data or resources. For what we are creating in these posts d3 is way overkill and very
2020-5-9
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2020-5-7
find-kedro is a small library to enhance your kedro experience. It looks through your modules to find kedro pipelines, nodes, and iterables (lists, sets, tuples) of nodes. It then assembles them into a dictionary of pipelines, each modul
2020-5-4
Often when reading through javascript examples you will find some arrow functions use parentheses () while others use braces {} . This key difference is that parentheses will implicitly return the last s
2020-5-3
Inspired by Chris Achard Email course inspired by learn d3 in 5 days For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more
2020-5-1
This past week I had a really weird bug in my kedro pipeline. For some reason data running through my pipeline was coming out completely made no sense, but if I manually request raw data outside of the pipeline it matched expe
2020-4-27
Is it possible to deep watch a single python function for changes? keeping track of a python functions hash is quite simple. There is a __hash__ method attached to every python function. Calling it will return a ha
2020-4-27
If you are developing python packages and using GitHub here are four actions that you can use today to automate your release workflow. Since python tools generally have such a simple cli I have opted to use the cli for most of these, that way I know
2020-4-13
So often I see a variables type() inside of its name and it hurts me a little inside. Tell me I'm right or prove me wrong below. Pandas DataFrames are probably the worst offender that I see
2020-4-8
Here is one useful thing that you can do with GitHub actions no matter what language you use, send email. You might want to know right away when your ci passes. You might want to give your team a nice pat on the back when a new release is deployed.
2020-3-31
GitHub actions are an amazing tool that allows us to run code based on triggers inside of our repo. Their is a large and growing community of actions inside the marketplace to use with very little effort. Best of all they are free for public reposi
2020-3-16
Github actions are written in configuration files using the YAML syntax. YAML is a superset of JSON. Most YAML can be expressed inline with JSON syntax. Similar to python YAML is whitespace driven by whitespace rather than brackets tags. The argum
2020-3-16
Today I learned how to diff between two branches. Sometimes we get a little git add . && git commit -m "WIP" happy and mistakenly commit something that we just can't figure out. This is a good way to figure out what
2020-3-3
This is a quickstart to getting a new kedro pipeline up and running. After this article you should be able to understand how to get started with kedro . You can learn more about this Hello W
2020-3-2
You dont have to start out as a git wizard with the cleanest possible commit history. At first dont let yourself get too wrapped up in it, the most important part is that you make commits. You will find needs for more advanced stuff later.
2020-2-26
I am working through Wes Bos's beginnerjavascript.com/ I just hit module 18 on hoisting. It's something that I always knew was there, Its not something I typically see used or use myself. Do you have any use cases t
2020-2-25
Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user read more waylonwalker.com This is my original what-is-kedro article. There is
2020-2-24
Getting a custom scrollbar on your site makes it stand out a bit compared to the very plain stock one that are on most sites. This is how I set mine up on my gatsby site. Inspired by Wes Bos's new uses.tech I wanted
2020-2-21
"Don’t waste your time learning everything." Inspired by this linkedIn post I felt that this comment was very powerful. Here are my 2 cents. Stick to what you know, and learn a little bit of some
2020-2-19
Moving into 2020 I have been really leaning on using purple as my theme color everywhere more and more. Its time for an update to my personal site, not just because it feels plain, not just because the cover art I am using for dev.to doesn't fit my
2020-2-17
It was so easy to get a professional looking navbar with just 3 lines of code. This package seriously is so usable on mobile it is ridiculous. I found this package from day-4 of the 100 days of gatsby challenge. It is by the
2020-2-11
* https://getform.io Free to get started https://formkeep.com/?#pricing Free 7 day trial $4.99 cheapest after For what we are creating in these posts d3 is way overkill and very verbos
2020-2-10
I finally fixed my Styled-Components in gatsby.js. I am starting a redesign of my website. I have started cross posting to dev.to more regularly. With that I have been making more detailed cover images at the recommended
2020-2-8
I have happily had my personal site waylonwalker.com hosted on netlify for nearly 2 years now. In fact I have hosted about a dozen different toy projects to play with on there, 4 of which have gone far enough to get a custom d
2020-2-6
Finally after years of hand typing out a full git push --upstream my_really_long_and_descriptive_branch_name I found there is a setting to automatcally push to the current branch. More realisitically I just did a gi
2020-2-4
This morning I logged into my machine and was nearly out of space Photo by Simon Migaj on Unsplash These are the commands that I often use to reclaim space. Its so easy to fill
2020-2-1
I love using URL shorteners to easily share links without hitting character limits, but they loose their meaning. Services like bit.ly will save my links for me so that I can find them, but I would rather them to be easy to remember. http
2020-1-29
cli tools are super handy and easy to add to your python libraries to supercharge them. Even if your library is not a cli tool there are a number of things that a cli can do to your library. Things a cli can do to enhance your library.
2020-1-29
I spend a lot of my time at the terminal for my daily work, mostly in Linux or wsl. One big reason for using wsl over cmd.exe is the ease of walking through history that fzf provides. This week we had a windows bug in a cli and I was stuck in vanil
2020-1-23
Mine is the python debugger. I was a long holdout thinking that print statements were sufficient. That was untill I started having errors crop up in functions that took minutes to run. The thing that I most notably wish I would have known about is po
2020-1-10
I have been using oh-my-zsh successfully for about 2 years now. But lately my startup time has been really bothersome. It has grown to the point where it was taking about 5.5s to startup a shell! This is ok if I am going to sp
2020-1-3
Vim's (neovim in my case) location list can provide some very useful information while developing. Mine gives me information about linting and type checking errors with fairly little config. Generally, it sits nicely at the bottom of the screen and
2020-1-1
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2019-12-21
Click primarily takes two forms of inputs Options and arguments. I think of options as keyword argument and arguments as regular positional arguments. **From the Docs To get the Python argument name, the chosen na
2019-11-11
I am tweeting out most of these snippets as I add them, you can find them all here #kedrotips . Below are some quick snippets/notes for when using kedro to build data pipelines. So far I am just compiling snippets. E
2019-11-2
|-|-| |github: | https://github.com/zaxr/bulwark | I definitely want to try this out with kedro. Bulwark is a package for convenient property-based testing of pandas dataframes, supported for Python 3.5+.
2019-10-14
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2019-10-1
Pathlib is an amazing cross-platform path tool. Current Directory Users Home Directory module directory Others Let's create a path relative to our current module. recur
2019-9-26
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j read more waylonwalker.com
2019-9-25
Good for method chaining, i.e. adding more methods or filters without assigning a new variable. general purpose, this is probably the most common method you see in training/examples capable of including multiple strings to inclu
2019-9-24
I love digital ocean for it's simplicity and its commitment to open source. For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are j
2019-9-22
tqdm is one of my favorite general purpose utility libraries in python. It allows me to see progress of multipart processes as they happen. I really like this for when I am developing something that takes some amount of time and I am unsure of perf
2019-9-18
If you are a regular listener of TalkPython or PythonBytes you have hear Michael Kennedy talk about Named Tuples many times, but what are they and how do they fit into my data science workflow. As you graduate your s
2019-9-11
This post is intended as an extension/update from background tasks in python . I started using background the week that Kenneth Reitz released it. It takes away so much boilerplate from running backgrou
2019-9-10
I have used %autoreload for several years now with great success and 🔥 rapid reloads. It allows me to move super fast when developing libraries and modules. They have made some great updates this year that allows class module
2019-9-8
Bash is super powerful. Show Remaining Space on Drives show largest files in current directory Move files then symlink them edit in vim cat a file bash execute
2019-9-8
Throw that mouse Away its time to setup some keyboard shortcuts. These sortcuts were the baseline for switching from tmux/vim to vscode. Most folks posts I was able to find gave great tips on replacing vim, but very few have focused on t
2019-7-7
My git workflow based on real life. Its not always clean and simple. sometimes things get messy pull 👉 branch 👉 format 👉 work👉 add 👉 commit 👉 pull 👉 rebase 👉 push For what we are creating in these pos
2019-5-27
Testing out forestry.io I still ♥️ your product dont be forestry is simple I have been playing with the netlify cms for a while now, and it has been a decent experience, but I really str
2019-4-23
rage unstage to wipte out history of staged commit git checkout HEAD~n -- removes history --SOFT locally before push after push after push
2019-2-5
rage unstage to wipte out history of staged commit git checkout HEAD~n -- removes history --SOFT locally before push after push after push
2019-2-5
This is my story into data science. I am addicted to the process of learning and improving my skills nearly to a fault. The reason I say nearly is because my addiction is fueled with results. I crave the output of my work enhance the wor
2019-2-2
This morning I log into my VCS and check activity on my projects to find that someone else has been very active on my projects fo the last few weeks. I quicklyhover over the missing avatar to find that
2019-1-21
The year of intenionality This is a follow up to my previous post 2018-retrospective This year I will become more productive, by intensionally working on a well thought out plan, learning the right technologies,
2019-1-12
2018 was a year of many ups and downs, and learning to deal with a whole new set of problems professionally and at home. In 2018 I logged in to my first Linux system, setup my own webserver, data pipelines, database. I learned to use react and d3. S
2019-1-5
Today I want to take some time to talk about the things that make me happy in my work environment. This is completely free-flow off the cuff, but are things that I do that make me happy, not having them would definitely be a deal breaker for me .
2019-1-1
For what we are creating in these posts d3 is way overkill and very verbose, but I need to start somewhere! These are just stepping stones into real custom visualizations that cannot be done in any other tool today. I still cannot explain how excit
2018-5-8
I recently finished up the flexbox-zombies course to learn more about flexbox, and to become proficient with it. I can truly say that this course has changed the way that I create layouts. Flexbox is very intuitive now. What this course does reall
2018-4-22