Brainstorming Kedro Hooks

edit ✏️

This post is a 🧠 branstorming work in progress. I will likely use it as a storage location/brain dump of hook ideas.

What is Kedro 🤔

If you are completely unsure what kedro is be sure to check out my what is kedro post

after_catalog_created

before_pipeline_run

after_pipeline_run

before_node_run

after_node_run

Execution Order

hooks are executed in reverse order of the hooks list.

hooks with tryfirst will be moved to the end of the list hooks with trylast will be moved to the end of the list

  1. after_catalog_created
  2. before_pipeline_run

  3. args

  4. run_params = run_params = {'run_id': '2020-05-23T15.24.23.958Z', 'project_path': '/mnt/c/temp/kedro0160', 'env': 'local', 'kedro_version': '0.15.9', 'tags': (), 'from_nodes': [], 'to_nodes': [], 'node_names': (), 'from_inputs': [], 'load_versions': {}, 'pipeline_name': None, 'extra_params': {}, 'git_sha': None}
  5. pipeline
  6. catalog

  7. before_node_run

  8. after_node_run 3.

When does data get saved???

??Unsure??

🥾 steel toes

I was way too excited about this one and already created it

prevents pain from stepping on your teammates toes

Kedro is so amazing at promoting collaboration between team members. Each team member can check out the code, branch, and start work on their own section of the pipeline. Issues can arrise if the team members section of the pipeline happen to cross. Breaking changes happen, BREAKS during development happen and can completely kill a teammates workflow.

how

Run only nodes that have changed

Static viz hook

Before pipeline run

Before node run

After node run

On pipeline error

On node error

After pipeline run

After node run



Socials