I came across awesome-data-engineering from igorbarinov, and it’s packed with great features and ideas. A curated list of data engineering tools for software developers
Archive
All published posts
2605 posts
latest post 2026-10-24
Publishing rhythm
Sep 30 – Oct 6, 20197 years ago
Featured this weekLink
Link
I’m really excited about vscode-python, an amazing project by microsoft. It’s worth exploring! Python extension for Visual Studio Code
Sep 16 – Sep 22, 20197 years ago
Featured this weekWriting
Quick Progress Bars in python using TQDM
tqdm is one of my favorite general purpose utility libraries in python. It allows me to see progress of multipart processes as they happen. I really like this for when I am developing something that takes some amount of time and I am unsure of performance. It allows me to be patient when the process is going well and w…
Link
If you’re into interesting projects, don’t miss out on Recreation-of-Nature, created by Kashu7100. ALife simulation with Python: patterns, behavior, and cognition.
Sep 9 – Sep 15, 20197 years ago
Featured this weekWriting
Clean up Your Data Science with Named Tuples
If you are a regular listener of TalkPython or PythonBytes you have hear Michael Kennedy talk about Named Tuples many times, but what are they and how do they fit into my data science workflow. Example As you graduate your scripts into modules and libraries you might start to notice that you need to pass a lot of data…
Writing
Background Tasks in Python for Data Science
This post is intended as an extension/update from background tasks in python. I started using the week that Kenneth Reitz released it. It takes away so much boilerplate from running background tasks that I use it in more places than I probably should. After taking a look at that post today, I wanted to put a better dat…
Link
I’m impressed by bake from kennethreitz. Bake — the strangely familiar workflow utility.
Link
Check out terminal by microsoft. It’s a well-crafted project with great potential. The new Windows Terminal and the original Windows console host, all in the same place!
Sep 2 – Sep 8, 20197 years ago
Featured this weekWriting
Autoreload in Ipython
I have used for several years now with great success and 🔥 rapid reloads. It allows me to move super fast when developing libraries and modules. They have made some great updates this year that allows class modules to be automatically be updated. What I like about autoreload 🔥 Blazing Fast 💥 Keeps me in the comfort of…
Link
If you’re into interesting projects, don’t miss out on starship, created by starship. ☄🌌️ The minimal, blazing-fast, and infinitely customizable prompt for any shell!
Link
alttch has done a fantastic job with rapidtables. Highly recommend taking a look. Super fast list of dicts to pre-formatted tables conversion library for Python 2/3
Link
If you’re into interesting projects, don’t miss out on psutil, created by giampaolo. Cross-platform lib for process and system monitoring in Python
Link
If you’re into interesting projects, don’t miss out on promote-open-source-project, created by zenika-open-source. 📄 How to promote my open source project?
Link
Check out watchtower by kislyuk. It’s a well-crafted project with great potential. Python CloudWatch Logging: Log Analytics and Application Intelligence
Aug 26 – Sep 1, 20197 years ago
Featured this weekLink
I recently discovered arrow by apache, and it’s truly impressive. Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
Aug 5 – Aug 11, 20197 years ago
Featured this weekLink
Just starred shell-functools by sharkdp. It’s an exciting project with a lot to offer. Functional programming tools for the shell
Jul 1 – Jul 7, 20197 years ago
Featured this weekWriting
Keyboard Driven VSCode
Throw that mouse Away its time to setup some keyboard shortcuts. These sortcuts were the baseline for switching from tmux/vim to vscode. Most folks posts I was able to find gave great tips on replacing vim, but very few have focused on the hackability of tmux. tmux allows me to rapidly fire up a workspace, create new w…
Jun 24 – Jun 30, 20197 years ago
Featured this weekLink
Looking for inspiration? Jupyter-Atom-Dark-Theme by burglarbenson. A dark theme for Jupyter Lab
Jun 17 – Jun 23, 20197 years ago
Featured this weekLink
tarpas has done a fantastic job with pytest-testmon. Highly recommend taking a look. Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.
Link
If you’re into interesting projects, don’t miss out on vim-flog, created by rbong. A blazingly fast, stunningly beautiful, exceptionally powerful git branch viewer for Vim/Neovim.
May 27 – Jun 2, 20197 years ago
Featured this weekWriting
Realistic Git Workflow
My git workflow based on real life. Its not always clean and simple. sometimes things get messy The Clean Path pull 👉 branch 👉 format 👉 work👉 add 👉 commit 👉 pull 👉 rebase 👉 push Pull As complicated as that seems it is pretty straight forward. When you sit down to work the first thing you do is to pull down the teams la…
Link
I like mcfunley’s project pugsql. A HugSQL-inspired database library for Python
Link
I like ggreer’s project the_silver_searcher. A code-searching tool similar to ack, but faster.
May 20 – May 26, 20197 years ago
Featured this weekLink
Just starred kedro by kedro-org. It’s an exciting project with a lot to offer. Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
May 13 – May 19, 20197 years ago
Featured this weekLink
Check out forestryio and their project forestry.io. Forestry.io website
May 6 – May 12, 20197 years ago
Featured this weekLink
Check out maildown by chris104957. It’s a well-crafted project with great potential. A super simple CLI for sending emails
Apr 22 – Apr 28, 20197 years ago
Featured this weekWriting
Forestry.io
Testing out forestry.io Sorry Netlify CMS I still ♥️ your product dont be forestry is simple I have been playing with the netlify cms for a while now, and it has been a decent experience, but I really struggle configuring it. Forestry is so simple to setup. My favorite part is that I can code up my gatsby.js site, stor…
Mar 11 – Mar 17, 20197 years ago
Featured this weekLink
Just starred eslint-config-wesbos by wesbos. It’s an exciting project with a lot to offer. No-Sweat™ Eslint and Prettier Setup - with or without VS Code
Link
Check out ydataai and their project ydata-profiling. 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
Link
I came across ydata-profiling from Data-Centric-AI-Community, and it’s packed with great features and ideas. 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
Mar 4 – Mar 10, 20197 years ago
Featured this weekLink
mdbartos has done a fantastic job with tabview. Highly recommend taking a look. Python curses command line CSV viewer
Link
TabViewer has done a fantastic job with tabview. Highly recommend taking a look. Python curses command line CSV and tabular data viewer
Link
I came across voidrice from LukeSmithxyz, and it’s packed with great features and ideas. My dotfiles (deployed by LARBS)
Feb 4 – Feb 10, 20197 years ago
Featured this weekWriting
Rewrite History with Git
rebase git commit –amend Unstage rage unstage to wipte out history of staged commit Undo file rage quit git reset HEAD~n removes modifications keeps hitsory of changes and undoes them git checkout HEAD~n – keeps modifications removes history –SOFT –HARD –Mixed undo n commits back locally before push after push update.g…
Link
I like FormidableLabs’s project webpack-dashboard. A CLI dashboard for webpack dev server
Jan 28 – Feb 3, 20197 years ago
Featured this weekWriting
It’s not all about winning
This is my story into data science. The Journey Begins I am addicted to the process of learning and improving my skills nearly to a fault. The reason I say nearly is because my addiction is fueled with results. I crave the output of my work enhance the work of others. I jump with joy as I see users gain insights they c…
Link
Looking for inspiration? fzf-preview.vim by yuki-yano. The plugin that powerfully integrates fzf and (Neo)vim. It is also possible to integrate with coc.nvim.
Link
If you’re into interesting projects, don’t miss out on background, created by ParthS007. Runs things in the background.
Jan 21 – Jan 27, 20197 years ago
Featured this weekWriting
Update Git User
This morning I log into my VCS and check activity on my projects to find that someone else has been very active on my projects fo the last few weeks. I quicklyhover over the missing avatar to find that It’s Me. What’s wrong here, why do I look like two different people throughout the day! upon further investigation I s…
Link
The work on react-select by JedWatson. The Select Component for React.js
Link
I recently discovered vimade by TaDaa, and it’s truly impressive. Vimade let’s you dim, fade, tint, animate, and customize colors in your windows and buffers for (Neo)vim
Jan 7 – Jan 13, 20197 years ago
Featured this weekWriting
2019 goals
The year of intenionality This is a follow up to my previous post 2018-retrospective professional This year I will become more productive, by intensionally working on a well thought out plan, learning the right technologies, and leave behind a positive legacy. Productivity Last year I was able to make some great stride…
Dec 31 – Jan 6, 20197 years ago
Featured this weekWriting
2018 Retrospective
2018 was a year of many ups and downs, and learning to deal with a whole new set of problems professionally and at home. In 2018 I logged in to my first Linux system, setup my own webserver, data pipelines, database. I learned to use react and d3. Stepped up my python, javascript, and sql. At home the doctor appointmen…
Writing
Do More of What Brings You Joy
Today I want to take some time to talk about the things that make me happy in my work environment. This is completely free-flow off the cuff, but are things that I do that make me happy, not having them would definitely be a deal breaker for me. Attitude A positive work atmosphere goes a long ways. We all have enough n…
Link
Check out engineer-man and their project youtube. Code from the Engineer Man YouTube channel. Please do not submit pull requests, they will be ignored/closed. The code in the repo needs to remain as it was in the video.
Nov 26 – Dec 2, 20187 years ago
Featured this weekLink
feather by wesm is a game-changer in its space. Excited to see how it evolves. Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow
Link
Just starred NES.css by nostalgic-css. It’s an exciting project with a lot to offer. NES-style CSS Framework | ファミコン風CSSフレームワーク
Oct 8 – Oct 14, 20188 years ago
Featured this weekLink
kennethreitz has done a fantastic job with responder. Highly recommend taking a look. A familiar HTTP Service Framework for Python.
Link
cheatsheets by rstacruz is a game-changer in its space. Excited to see how it evolves. Cheatsheets for web development - devhints.io
Link
If you’re into interesting projects, don’t miss out on awesome-vscode, created by viatsko. 🎨 A curated list of delightful VS Code packages and resources.
Calendar data could not load. Try again.