pytest has open sourced their amazing plugin framework
, it allows
library authors to give their users a way to modify the libaries behavior
without needing to submit a change that may not make sense to the entire
My experience so far as a plugin user, and plugin author has been great. Building and using plugins are incredibly intuitive. I wanted to dive a bit deeper and see how they are implemented inside of a library and its a bit of a mind bend the first time you try to do it.
Plugins vs. Hooks
A hook is a single function that has a specific place that it is ran by the PluginManager.
A Plugin is a collection of one or more hooks.
- library author
- plugin author
- end user
Using a plugin
For a plugin to be registered is must be registered by the PluginManager which is implemented by the library author. It is the job of the library author to determine what plugins are actively registered or disabled. There are two common ways that I have seen that plugins are registered, through entrypoints or configuration.
Using a plugin - entrypoints
Plugins that are implemented with entrypoints are the simplest for the user.
They are simply activated by
pip install plugin
or deactivated by
. The library author will show an entrypoint in their docs
which tells plugin authors how to setup entrypoints so that they will be loaded
Using a plugin - config
Another way to configure plugins is through configuration. This may come in the form of a list in a python module or listed in a text file in the config. This route requires the user to add the plugin to a list or import it into a python module.
I really stuggled to find a good example of pluggy to get started. I found the best way for me to understand was to create one myself. the pluggy repo has one simple example , but it is unclear who owns each piece from the example. The whole point of pluggy is to pass ownership of implementation from the library author to the plugin author.
Kedro is a data pipelining framekwork that includes a hooks based architecture that allows users to modify the behavior of the framework at different points through the lifecycle. There is a hooks module that implements everything, and a test_plugin that is used for testing, but also serves as a good example.
Another example is the palantir python language server. Check out their hookspec module.
- implemented by the library author
- gives a namespace for pluggy to store hooks
- created and used by libary author
- created by libary author
- used by plugin author
- implementation of plugins in the library
_empty hooks created by the library author
# hookspec.py import pluggy hookspec = pluggy.HookspecMarker("printer") class PrinterHooks: @hookspec def pre_print(msg): "pre print hook" pass @hookspec def post_print(msg): "pre print hook" pass
used by the plugin author
Implementations of plugins much match the name of the spec exactly. They can include some or all of the arguments listed in the spec, but no others. They can be implemented as a module with functions that match the name of the spec or as a class with methods that match the name of the spec.
Class Style Plugin
# plug.py # would be imported from the library authors hookspec from hookspec import hookimpl class Pre: @hookimpl def pre_print(msg): msg = msg.upper() return "BEFORE" class Post: @hookimpl def post_print(msg): print(f"\033[A\033[2Knot today")
Module Style Plugin
# plug/Pre.py from hookspec import hookimpl @hookimpl def pre_print(msg): msg = msg.upper() # plug/Post.py class Post: @hookimpl def post_print(msg): print(f"\033[A\033[2Knot today")
note These plugins only implement one hook. Each plugin may implement one or more hooks, a plugin is not required to only implement one hook.
implementing the hooks into the library
import pluggy import importlib from hookspec import PrinterHooks from plug import Pre pm = pluggy.PluginManager("printer") pm.add_hookspecs(PrinterHooks) pm.register(Pre) def printer(msg): pm.hook.pre_print(msg=msg) print(msg) pm.hook.post_print(msg=msg)
Running the library
Now if we run the printer function as a user we will see this output.
>>> printer('hello world') HELLO WORLD
Now if we register the Post plugin we will see the following output.
from plug import Pre, Post pm.register(Pre) pm.register(Post)
>>> printer('hello world') not today
plugin wipes away the last line from the console and
Plugin Manager - with dynamic imports
In a real library we might want to allow the user to configure their
plugins through a config file. If we do this we will need to reach
to handle the imports based on a string.
import pluggy import importlib # from hookspec import hookspec from hookspec import PrinterHooks # from hookspec import hookimpl plugins = ["plug.Pre", "plug.Post"] pm = pluggy.PluginManager("printer") pm.add_hookspecs(PrinterHooks) for plug in plugins: if isinstance(plug, str): # plug is a str representing a module to import try: # module style plugins plugin = importlib.import_module(plug) except ModuleNotFoundError as e: # class style plugins if "." in plug: mod = importlib.import_module(".".join(plug.split(".")[:-1])) plugin = getattr(mod, plug.split(".")[-1]) else: raise e else: # plug is a module that is already imported plugin = plug pm.register(plugin) def printer(msg): pm.hook.pre_print(msg=msg) print(msg) pm.hook.post_print(msg=msg)