Posts tagged: python

My favorite pandas pattern I work with a lot of transactional timeseries data that includes categories. I often want to create timeseries plots with each category as its own line. This is the method that I use almost data to achieve this result. Typically the data that am working with changes very slowly and trends happen over years not days or weeks. Plotting daily/weekly data tends to be noisy and hides the trend. I use this pattern because it works well with my data and is easy to explain to my stakeholders. import pandas as pd import numpy as np % matplotlib inline Lets Fake some data # [1] Here I am trying to simulate a subset of a large transactional data set. This could be something like sales data, production data, hourly billing, anything that has a date, category, and value. Since we generated this data we know that it is clean. I am still going to assume that it contains some nulls, and an irregular date range. n = 365*5 cols = {'level_0': 'date', 'level_1': 'item', ...

I have tried most of the different methods in the past and found that copying and pasting the threadpoolexecutor example [1] or the processpoolexecutor example [2] from the standard library documentation to be the most reliable. Since this is often something that I stuff in the back of a utility module of a library it is not something that I write often enough to be familiar with, which makes it both hard to write and hard to read and debug. If you are looking for a good overview of the difference concurrency Raymond Hettinger [3] has a great talk about the difference between the various different methods, when to use them and why. Recently a new python library was released to make running tasks in the background very simple. The background [4] project by Kenneth Reitz is a high level implementation of python 3’s ThreadPoolExecutor. I have been playing around with this project over the last week and I will say that this is definitely the simplest way to run background tasks in pyth...