Drafts

Draft and unpublished posts

0 posts

vows

Story I can still rememember the day that we we realized our new baby girl had gone days without a bowel movement. Holding her while she looked as if she was about to pop at any moment, completely nervous for what was to come. Little did we know what that strong little girl had in store for us. She was perparing us for something much bigger. Little did we know how much scary medical journeys would soon become a “normal” part of our life. We spent two solid years fighting for this girl every day. I still remember that Fall day. That day you drove my car home alone while I sat with our kids, patiently awaiting your arrival. When you arrived, no words needed to be spoken, the minute you opened the car door I knew our lives had changed forever. Scared out of our minds we held it together the best we could for those two kids we were holding. Fast forward 6 weeks, you are recovering from the biggest surgery of your life, and we notice that little girl is now feeding enough for herself...

python lsp setup

Setting up python with the native nvim>0.5 lsp was mr lsp-config # [1] https://github.com/neovim/nvim-lspconfig lua << EOF require'lspconfig'.pyright.setup{} EOF pyls#190 # [2] https://github.com/palantir/python-language-server/issues/190 lspconfig.pyls.setup { cmd = {"pyls"}, filetypes = {"python"}, settings = { pyls = { configurationSources = {"flake8"}, plugins = { jedi_completion = {enabled = true}, jedi_hover = {enabled = true}, jedi_references = {enabled = true}, jedi_signature_help = {enabled = true}, jedi_symbols = {enabled = true, all_scopes = true}, pycodestyle = {enabled = false}, flake8 = { enabled = true, ignore = {}, maxLineLength = 160 }, mypy = {enabled = false}, isort = {enabled = false}, yapf = {enabled = false}, pylint = {enabled = false}, pydocstyle = {enabled = false}, mccabe = {enabled = false}, preload = {enabled = false}, rope_completion = {enabled = false} } } }, on_attach = on_attach } mypy # [3] Getting mypy working with ...

How to use git cherry pick

~/git via 🐍 v3.8.5 ❯ mkdir git-cherry-pick-learn ~/git via 🐍 v3.8.5 ❯ cd git-cherry-pick-learn ~/git/git-cherry-pick-learn ❯ git init Initialized empty Git repository in /home/walkews/git/git-cherry-pick-learn/.git/ git-cherry-pick-learn on  main ❯ touch readme.md git-cherry-pick-learn on  main [?] ❯ git status On branch main No commits yet Untracked files: (use "git add <file>..." to include in what will be committed) readme.md nothing added to commit but untracked files present (use "git add" to track) git-cherry-pick-learn on  main [?] ❯ git add . git-cherry-pick-learn on  main [+] ❯ git commit -m "init readme" [main (root-commit) ebd1ff2] init readme 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 readme.md git-cherry-pick-learn on  main ❯ echo "Learn Cherry Pick" Learn Cherry Pick git-cherry-pick-learn on  main ❯ git add . git-cherry-pick-learn on  main ❯ git commit -m "add title git-cherry-pick-learn on  main ❯ echo "# Learn Cherry P...

Vim Fugitive

:G :G status :G commit :G add % :Gdiff :G push :Glog Add current file and commit with diff in a split # [1] function! s:GitAdd() exe "G add %" exe "G diff --staged" exe "only" exe "G commit" endfunction :command! GitAdd :call s:GitAdd() nnoremap gic :GitAdd<CR> :on[ly] # [2] C-W o :on[ly] will make the current buffer the only one on the screen. This is super helpful as many of fugitive commands will open in a split by default. C-I C-O # [3] cycle through the jumplist This one has nothing to do with fugitive, but is a native vim feature that makes fugitive glorious. Before I realized how to utilize C-i and C-o, I would get completely lost when using fugitive. Digging deep into the log, opening a file from a specific commit, then no way to get back where I was in the log. C-i jump :jump[s] # [4] show the jumplist The jumplist is sorted Oldest to newest :Telescope jumplist # [5] When navigating the jumplist with :Telescope jumplist, it will add a new entry to the jumpli...

Custom Kedro Logger

DRAFT - formatters: mine: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s - %(me)s" handlers: mine_handler: class: logging.StreamHandler level: INFO formatter: mine stream: ext://sys.stdout loggers: me: level: DEBUG handlers: [mine_handler] root: level: INFO handlers: [console, info_file_handler, error_file_handler]

📝 Docker Deep Dive - Notes

https://www.hanselminutes.com/784/doing-open-source-with-brian-douglas Play With Docker # [1] A handy way to try weird things in docker is using play-with-docker [2]. You get a four hour session for free, after four hours everything will be deleted, but you can start a new session. Installing Docker on Linux # [3] Installing on Ubuntu. wget -qO- https://get.docker.com/ | sh Running Docker commands without sudo # [4] In order to run docker commands without using sudo you need to add docker to your group. sudo usermod -aG docker ubuntu Architecture and Theory # [5] Container - Isolated area of an OS with resource usage limits applied. Namespaces and Control Groups are hard, which is why containers were unusable by mortals before docker. Namespaces # [6] Isolation Each container looks and feels like a regular OS. It has its own eth0, users, kernel. These are completely isolated from every other container running on the system. Namespaces are analogous to what Hypervisors d...

Zev Averbach Interview

Zev Averbach, Frustrated spreadsheet jockey to software developer at 36 Q: Tell me about your journey as a spreadsheet jockey into Data Engineering? A: First of all, it’s hilarious that I accidentally found your questions for this interview by Googling myself. 😊 I’ve always been a frustrated software user, and that frustration led me to be a “power user” (keyboard shortcuts etc) of my most used applications, as well as a “visual coder” using desktop automation like Alfred and Keyboard Maestro (Mac). Now that I’ve met data analysts and finance people that use Excel all day, I don’t think I’d claim to have been a true “spreadsheet jockey” in comparison to them. However, hitting up against the limitations of spreadsheets for running my transcription business [1] – specifically for bookkeeping – created a new frustration for me: As the business grew I was spending more and more time copying entries from Google Sheets to the Freshbooks web app for invoicing purposes. I tried to auto...

fix crlf for entire git repo

Final Result # [1] git checkout main git reset --hard git rm -rf --cached . echo "* text=auto" > .gitattributes git add . References: [1]: #final-result
1 min read

Automatic Conda Environments

I have automated my process to create virtual environments in my python projects, here is how I did it. I’ve really been digging my new tmux session management setup. Now I have leveled it up by adding direnv to my workflow. It will execute a shell script whenever I cd into a directory. One thing I wanted to add to this was, automatic activation of python environments whenever I cd into a directory, or create a new environment if one does not exist. https://waylonwalker.com/tmux-nav-2021/ Direnv # [1] You can learn more about direnv [2] on their website. There are several libraries that seem to do a similar concept, run a bash script when I cd into a directory direnv is the one that I chose to use, but you can likely do the same concept in other ones. Installation # [3] direnv is in many package repos, or can be installed from binary builds with a one liner. See their installation [4] instructions for more information. curl -sfL https://direnv.net/install.sh | bash .envrc # [...
3 min read

How I Review Pipeline Code

I have started doing more regular PR’s on my teams Kedro [1] pipelines. I generally take a two phase approach to the review in order to give the reviewee both quick and detailed feedback. What is Kedro [2] initial scan (Phase1) # [3] - passing ci - Variable Names - Antipatterns - No commented out code - Docsttrings generally make sense Phase1 is typically a quick scan over the PR right within the PR window in my browser. Passing CI # [4] - flake8 - black - isort - interrogate - pytest - build The very first thing that needs to happen is automated CI. We use things like flake8, black, isort, interrogate to ensure that everyone follows generic style guides like pep8. The project does a build within the PR, but no deploy. Variable Names # [5] I strugle really hard to not impose my own opinion into the PR at this point, and sometimes really want to change a lot of variable names. Typically I make sure they don’t grow longer than necessary, too short, misspelled, or inc...
2 min read

Sample

title subtitle # [1] sub-subtitle # [2] sub-sub-subtitle # [3] sub-sub-sub-subtitle sub-sub-sub-sub-subtitle Glossary # [4] There is a glossary item in vibe coding [5] here and clippy [6]-no-simpy/" class="glossary-term" title=""Clippy no Simpy" is a term coined by Louis Rossmann, when people try to stand up for companies doing scummy things like charging your for features that you...">clippy no simpy. Now you don’t have to manually link to how to create a virtual environment [7] every time you mention virtual environments in any post that needs a virtual environment. Paragraph # [8] Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor Hover me incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt...
4 min read

🐍 Pluggable Architecture with Python

pytest has open sourced their amazing plugin framework pluggy, it allows library authors to give their users a way to modify the libaries behavior without needing to submit a change that may not make sense to the entire library. Previous Experience # [1] My experience so far as a plugin user, and plugin author has been great. Building and using plugins are incredibly intuitive. I wanted to dive a bit deeper and see how they are implemented inside of a library and its a bit of a mind bend the first time you try to do it. Plugins vs. Hooks # [2] A hook is a single function that has a specific place that it is ran by the PluginManager. A Plugin is a collection of one or more hooks. Layers # [3] - library author - plugin author - end user Using a plugin # [4] For a plugin to be registered is must be registered by the PluginManager which is implemented by the library author. It is the job of the library author to determine what plugins are actively registered or disabled. Ther...
4 min read

Create Og Image Covers Using Python

There are so many tutorials out there for creating OG [1] images with nodejs or puppeteer, but I have yet to see many using python. PIL # [2] Template # [3] Text # [4] Overlays # [5] References: [1]: /og/ [2]: #pil [3]: #template [4]: #text [5]: #overlays
1 min read

⚙ How Python Tools Are Configured

There are various ways to configure python tools, config files, code, or environment variables. Let’s look at a few projects that allow users to configure them through the use of config files and how they do it. Motivation # [1] This will not include how they are implemented, I’ve looked at a few and its not simple. This will focus on where config is placed and the order in which duplicates are resolved. The motivation of this article is to serve as a bit of a reference guide for those who may want to create their own package that needs configuration. Flake8 # [2] Global # [3] User settings can exist in the users ~/.config/flake8 file to configure how flake8 runs on their machine. - ~/.config/flake8 Per-Project # [4] Only One project config file will be considered, but allows for several options. These files all use the ini format and must have a [flake8] section header to be consideered. Selection of the config file can also be overridden by the --config cli option. An e...
5 min read

Kedro Dependency Management

Docs # [1] https://kedro.readthedocs.io/en/stable/04_kedro_project_setup/01_dependencies.html?highlight=install pip-tools # [2] pip-compile # [3] requirements # [4] - requirements.in - requirements.txt References: [1]: #docs [2]: #pip-tools [3]: #pip-compile [4]: #requirements

Blog Data With Python

Generating an api for a blog is much simpler than one might expect with python. Markdown # [1] Frontmatter # [2] Fill in the blanks # [3] fix missing data Fast # [4] References: [1]: #markdown [2]: #frontmatter [3]: #fill-in-the-blanks [4]: #fast

Todo

Overrated underrated command line applications - Ship More - Ship Less - get feedback and apply feedback, build the right things kedro YouTube series # [1] Get started on daily kedro shorts, these are indended to be short clips that people can watch a playlist and learn about kedro concepts at their own pace. This is meant to be low barrier to entry for me to create. Like the tmux series, I hope to make a sub three minute video within one or two takes, no edits, all straight from obs. written # [2] - how I kedro 8/16 - what is kedro 8/17 - kedro new 8/18 - three ways to create a kedro environment 8/19 - git [3] init 8/20 - kedro install 8/21 - kedro pipeline create 8/22 staging # [4] - making your first nodes in kedro 8/23 - kedro run 8/24 - kedro catalog create 8/25 - add new dependencies 8/26 - listing things from the kedro cli 8/27 - Storing catalog entries 8/30 - Comprehensive nodes - modular piplines 8/29 - kedro pipeline package - kedro pipeline pull - ...
2 min read