Skip to content

Joblib and Memoization


The memoization pattern - is a caching method. It caches the result of inputs so when the same inputs are used again the calculation/work is skipped and the result is produced.

To be remembered…

A good example of this in the python standard library is functools.lru_cache

Caveats - ideally you want to memoize functions that are deterministic


joblib is a python library to provide lightweight pipelining. Running multiple jobs in parrallel for large datasets specifically for numpy arrays.

It appears to be a tool more focused on the data and analytics processing side of things - instead of real-time transactional processing.