-
💭 Migration guide for config loaders — kedro 0.19....
Migrating from kedro 0.18.4 to the latest version involves handling the deprecated OmegaConf loader. Switching over doe
-
Make Kedro Runs Beautiful
Kedro rich is a very new and unstable (it's good, just not ready) plugin for kedro to make the command line prettier. Install kedro rich There is no pypi p
-
Lambda Function as a Kedro Node
I keep my nodes short and sweet. They do one thing and do it well. I turn almost every DataFrame transformation into its own node. It makes it must easier to
-
Add New Dependencies to Your Kedro Project
As you work on your kedro projects you are bound to need to add more dependencies to the project eventually. Kedro uses a fantastic command `pip-compile` under
-
Practice making pipelines with kedro
I am a huge believer in practicing your craft. Professional athletes spend most of their time honing their skills and making themsleves better. In Engineering
-
Running Kedro on Ubuntu 21.10 Impish Indri
I just installed a brand new Ubuntu 21.10 Impish Indri, and wanted a kedro project to play with so I did what any good kedroid would do, I went to my command li
-
kedro catalog create
I use `kedro catalog create` to boost my productivity by automatically generating yaml catalog entries for me. It will create new yaml files for each pipeline,
-
nvim conf 2021 | IDE's are slow | Waylon Walker
[https://youtu.be/E18m4KkJUnI](https://youtu.be/E18m4KkJUnI){.youtube-embed} Slides 👇 welcome Other possible titles Using Vim as a Team Lead I 💜 Tmux Why I stop
-
Kedro-Broken-Urls
Broken Urls [ ] <https://github.com/josephhaaga>) [ ] <https://example.com/file.h5> [ ] <https://raw.githubusercontent.com/kedro-org/kedro/develo
-
Setting Parameters in kedro
Parameters are a place for you to store variables for your pipeline that can be accessed by any node that needs it, and can be easily changed by changing your e
-
Writing your first kedro Nodes
https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let's make some nodes with some vanilla data structures. import node You will need to i
-
Running your Kedro Pipeline from the command line
Running your kedro pipeline from the command line could not be any easier to get started. This is a concept that you may or may not do often depending on your
-
kedro Virtual Environment
Avoid serious version conflict issues, and use a virtual environment anytime you are running python, here are three ways you can setup a kedro virtual environme
-
Kedro Pipeline Create
Kedro pipeline create is a command that makes creating new pipelines much easier. There is much less boilerplate that you need to write yourself. https://youtu
-
Kedro Install
Kedro comes with an `install` command to install and manage all of your projects dependencies. https://youtu.be/IWimEs-hHQg cd into your project directory and a
-
Kedro Git Init
Immediately after `kedro new`, before you start running `kedro install` or your first line of code the first thing you should always do after getting a new kedr
-
Kedro New
[https://youtu.be/uqiv5LAiJe0](https://youtu.be/uqiv5LAiJe0){.youtube-embed} Kedro new is simply a wrapper around the cookiecutter templating library. The kedr
-
What is Kedro
Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user a way to build pipelines that automatica
-
How I Kedro
<https://youtu.be/bw5_FWDVRpU> Ubuntu I recently switched over to using Ubuntu, it works well pretty much out of the box for me. I am using gnome with a
-
Incremental Versioned Datasets in Kedro
Kedro versioned datasets can be mixed with incremental and partitioned datasets to do some timeseries analysis on how our dataset changes over time. Kedro is a
-
I Started Streaming on Twitch
I recently started streaming on [twitch.tv/waylonwalker](https://www.twitch.tv/waylonwalker) and it's been a blast so far. python kedro Data Science Data E
-
Upcoming Stream
!!! Caution I'm no longer streaming As much as I would really love to make streaming work, its really hard for my family situation to make large b
-
Kedro Spaceflights - part 2 | Stream replay June 7...
This was my seconf time ever streaming on [twitch.tv/waylonwalker](https://twitch.tv/waylonwalker), and I completely botched my mic 2x. [https://youtu.be/_7MwgK
-
Kedro Spaceflights - part 1 | Stream replay June 4...
This was my first time ever streaming on [twitch.tv/waylonwalker](https://twitch.tv/waylonwalker). I am excited to get going. I have been streaming early in th
-
Comprehensive guide to creating kedro nodes
The Kedro node is an essential part of the pipeline. It defines what catalog entries get passed in, what function gets ran, and the catalog entry to save the re
-
Creating pypi-list with kedro
I had an idea come to me via twitter. Short one word name packages are becoming hard to find on pypi. Short one word readable package names that are not a play
-
Using Kedro In Scripts
With the latest releases of kedro `0.17.x`, it is now possible to run kedro pipelines from within scripts. While I would not start a project with this techniqu
-
Silence Kedro Logs
Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while try
-
Vim Fugitive
Add current file and commit with diff in a split :on[ly] _C-W o_ :on[ly] will make the current buffer the only one on the screen. This is super helpful as many
-
Custom Kedro Logger
DRAFT -
-
kedro replit
I am trying to see what an embeded replit
-
Kedro pipeline_registry.py
With the realease of `kedro==0.17.2` came a new module in the project template `pipeline_registry.py`. Here are some notes that I learned while playing with th
-
Minimal Kedro Pipeline
How small can a minimum kedro pipeline ready to package be? I made one within 4 files that you can pip install. It's only a total of 35 lines of python,
-
Kedro Dependency Management
Docs https://kedro.readthedocs.io/en/stable/04_kedro_project_setup/01_dependencies.html?highlight=install pip-tools pip-compile requirements requirements.in req
-
Kedro - My Data Is Not A Table
In python data science/engineering most of our data is in the form of some sort of table, typically a DataFrame from a library like pandas, spark, or dask. Data
-
Testing Data Pipelines
Thinking through testing best practices for Data Pipelines
-
reasons-to-kedro
There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processing DataFrames from many data sources shou
-
Reasons to Kedro
Reasons to Kedro collaboration Sharable catalog small nodes over monolithic notebooks catalog easily load anything without needing to run No need to write read/
-
What's New in Kedro 0.16.6
This version of kedro releases a new set of supported deployment options and the spaceflights pipeline is officially added as a starter alias.
-
A brain dump of stories
I started making stories as kind of a brain dump a few times per day and posting them to [LinkedIn](https://www.linkedin.com/in/waylonwalker/(https://www.linked
-
Kedro Basics
In my upcoming free kedro course, you can learn how to start building pipelines in 5 days.
-
What's New in Kedro 0.16.4
If we take a look at the [release notes](https://github.com/kedro-org/kedro/blob/master/RELEASE.md) I see one **major** feature improvement on the list, auto-di
-
Kedro Catalog
I am exploring a kedro catalog meta data hook
-
Gracefully adopt kedro, the catalog
Why use kedro catalog? While using the catalog alone will not reap all of the benefits of the framework, it does get you and your project ready for the full fra
-
How to find things in your kedro catalog
kedro 0.16.2 just dropped last week with a long-awaited feature... **catalog search**! I went as far as monkey patching this into each of my projects. I work
-
How Kedro handles your inputs
Passing inputs into kedro is a key concept. Understanding how it accepts a single catalog key as input is quite trivial that easily makes sense, but passing a
-
003
🔥 #kedrotips hooks can be created using modules
-
004
🔥 #kedrotips use find-kedro to assembly your pipelines
-
002
**kedro-static-viz** 0.3.0 just launched with _hooks_ support 🎉
-
Kedro Static Viz 0.3.0 is out with Hooks Support
[kedro-static-viz](https://github.com/WaylonWalker/kedro-static-viz) is out with support for the newly released hooks feature. This means that you can have `ke
-
001
**practice** building pipelines with _#kedro_ today
-
Brainstorming Kedro Hooks
This post is a 🧠 branstorming work in progress. I will likely use it as a storage location/brain dump of hook ideas. What is Kedro 🤔 If you are completely unsu
-
Create Custom Kedro Dataset
Kedro provides an efficient way to build out data catalogs with their yaml api. It allows you to be very declaritive about loading and saving your data. For t
-
creating the kedro-preflight hook
Kedro Hooks Intro - kedro hooks are an exciting upcoming feature of kedro `0.16.0`. They allow you to hook into `catalog_created`,`pipeline_run`, and `node_run
-
📝 Kedro Preflight Notes
run checks before running the pipeline
-
📢 Announcing find-kedro
kedro is an amazing project that allows for super-fast prototyping of data pipelines, yet yielding production-ready pipelines. find-kedro enhances this experience by adding a pytest-like node discovery eliminating the need to bubble up pipelines through modules.
-
Create New Kedro Project
This is a quickstart to getting a new [kedro](https://kedro.readthedocs.io) pipeline up and running. After this article you should be able to understand how to
-
What is Kedro
[[ what-is-kedro ]] This is my original what-is-kedro article. There is a brand new one Kedro is an open source data pipeline framework. It provides guardrail
-
Kedro
My Notes about using kedro