Published

It happens to the best of us # [1] We can’t all remember every single function signature out there, it’s just not possible. If you want to stay productive while coding without the temptation to hit YouTube or Twitter. Use the built in help. Here are 5 ways to get help without leaving your terminal. https://youtu.be/TZrRAP-9UMk Docstrings # [2] In any python repl you can access the docstring of a function by calling for help. help(df.rolling) In Ipython we can even get some syntax highlighting with the ?. df.rolling? Source Code # [3] Sometimes the docstrings are not good enough, and don’t give us the content we need, and we just need to look at the source. Without leaving your terminal there are two ways I often use to get to the source of a function I am trying to use. import inspect inspect.getsource(df.rolling) The more common way I do it is with the ipython ??. df.rolling?? Bonus rich.inspect # [4] You thought the syntax highlighting was good with ipython, check out w...

Are you participating in Hacktoberfest 2021? I got my start with open source with Hacktoberfest 2018. I remember being so nervous for those first pr’s. I have continued to participate every year, and it feels good to give back. It made a big change on me # [1] Something that it really did for me was to lower the hurdle to contribute regularly. I am not a huge contributor to open source. I am not necessarily a regular maintainer of any large project, but I do often raise issues when I see things as a user that would make things easier or more clear. After participating it does not feel like such a big deal to go through the motions of making my own fork, making a small change, and submitting it for review. What can you do? # [2] As a user, you might be in a unique position to see a library from the perspective of other users rather than from the eyes of a maintainer. You can help bring the user perspective to your favorite library. https://twitter.com/_WaylonWalker/status/1446...

I’ve slowly adding more and more lua functions into my neovim configuration, and recently I noticed a pattern for a class of functions that reach out to run shell commands that can be abstracted away. https://youtu.be/8m5ipBuopPU Telegraph.nvim # [1] Check out the project readme [2] for the most up to date details on the plugin itself. Motivation # [3] I want a simple way to make remaps into shell commands that can open new tmux windows, popups, or just run a command with context from the editor. For example I want to make remaps to do things like open the current file in lookatme. # vim :terminal nnoremap <leader>s <cmd>Telegraph pipx run lookatme {filepath} --live-reload --style gruvbox-dark<cmd> # tmux session nnoremap <leader><leader>s <cmd>lua require'telegraph'.telegraph({cmd='pipx run lookatme {filepath} --live-reload --style gruvbox-dark', how='tmux'})<CR> # tmux popup nnoremap <leader><leader>S <cmd>lua require'telegraph'.telegraph({cmd='pipx run lookatme {filepath...

Parameters are a place for you to store variables for your pipeline that can be accessed by any node that needs it, and can be easily changed by changing your environment. Parameters are stored in the repository in yaml files. https://youtu.be/Jj5cQ5bqcjg What is Kedro [1] 👆 Unsure what kedro is? Check out this post. parameters files # [2] You can have multiple parameters files and choose which ones to load by setting your environment. By default kedro will give you a base and local parameters file. - conf/base/parameters.yml - conf/local/parameters.yml base # [3] The base environment should contain all of the default values you want to run. # /conf/base/parameters.yml test_size: 0.2 random_state: 3 features: - engines - passenger_capacity - crew - d_check_complete - moon_clearance_complete - iata_approved - company_rating - review_scores_rating NOTE base will always be loaded first. accessing parameters # [4] Parameters can be accessed through context or throug...

https://youtu.be/-gEwU-MrPuA Before we jump in with anything crazy, let’s make some nodes with some vanilla data structures. import node # [1] You will need to import node from kedro.pipeline to start creating nodes. from kedro.pipeline import node func # [2] The func is a callable that will take the inputs and create the outputs. inputs / outputs # [3] Inputs and outputs can be None, a single catalog entry as a string, mutiple catalog entries as a List of strings, or a dictionary of strings where the key is the keyword argument of the func and the value is the catalog entry to use for that keyword. our first node # [4] Sometimes in our pipelines our data is coming from an api where we already have python functions built to pull with. Thats ok, kedro supposrts that with inputs=None. def create_range(): return range(100) make_range = node( func=create_range, inputs=None, outputs='range' ) second node # [5] Now we have some data to work from, lets use that as our inpu...

Running your kedro pipeline from the command line could not be any easier to get started. This is a concept that you may or may not do often depending on your workflow, but its good to have under your belt. I personally do this half the time and run from ipython half the time. In production, I mostly use docker and that is all done with this cli. https://youtu.be/ZmccpLy-OEI What is Kedro [1] 👆 Unsure what kedro is? Check out this post. Kedro run # [2] To run the whole darn project all we need to do is fire up a terminal, activate our environment, and tell kedro to run. kedro run Specific Pipelines # [3] Running a sub pipeline that we have created is as easy as telling kedro which one we want to run. kedro run --pipeline dp Single Nodes # [4] While developing a node or a small list of nodes in a larger pipeline its handy to be able to run them one at a time. Besides the use case of developing a single node I would not reccomend leaning very heavy on running single nodes, le...