Testing Data Pipelines
━━━━━━━━━━━━━━━━━━━━━━

Thinking through testing best practices for Data Pipelines

Date: November 16, 2020

[1m[38;2;167;192;128mLint/Format/Doc[0m
[38;2;71;82;88m━━━━━━━━━━━━━━━[0m

- black
- flake8
- interrogate
- mypy

[1m[38;2;167;192;128mPipeline Assertions[0m
[38;2;71;82;88m━━━━━━━━━━━━━━━━━━━[0m

- pipeline constructs
- pipeline as expected nodes
- pipeline has minimum nodes
- test minimum tags
- test alternate tags

[1m[38;2;167;192;128mCatalog Assertions[0m
[38;2;71;82;88m━━━━━━━━━━━━━━━━━━[0m

- test catalog follows naming structure
-

[1m[38;2;167;192;128mNode Tests[0m
[38;2;71;82;88m━━━━━━━━━━[0m

- test function does the correct operations on test data

[1m[38;2;167;192;128mGreat Expectations[0m
[38;2;71;82;88m━━━━━━━━━━━━━━━━━━[0m