Skip to content

Building Plugins for Opsbox

Creating plugins for Opsbox is a straightforward process with a few key steps. Let's walk through them!

Tip

Opsbox uses the Pluggy library for plugin management, loading, and validation. If you're familiar with Pluggy, this workflow will feel familiar.

Steps to Build Plugins

  1. Specify your hookimpl
  2. Define configurations (optional)
  3. Define activation (optional)
  4. Add plugin toml file
  5. Use results model
  6. Package it up (optional)
  7. Add more!

Specify Your Hookimpl

Pluggy operates using a system of hook specifications ("hookspecs"), which define methods to be implemented. To create a plugin, you implement methods from these hookspecs and designate these methods as hook implementations.

To specify a hook implementation, import HookimplMarker from Pluggy and set the project to "opsbox":

from pluggy import HookimplMarker

# Define a hookimpl (implementation of the contract)
hookimpl = HookimplMarker("opsbox")

Implement Basespec

Opsbox plugins can optionally: - Specify configuration through a Pydantic model. - Specify delayed activation that executes after setting plugin data.

All hook implementations should be collected from a single class, meaning all hook implementations should be within the same class.

Example Implementation

from pydantic import BaseModel, Field
from pluggy import HookimplMarker

hookimpl = HookimplMarker("opsbox")

class HelperAssistant:
    @hookimpl
    def grab_config(self) -> type[BaseModel]:
        """Return the plugin's configuration Pydantic model."""
        class AssistantConfig(BaseModel):
            aggregate: bool = Field(..., description="Whether to aggregate past results")
        return AssistantConfig

    @hookimpl
    def set_data(self, model: BaseModel) -> None:
        """Set the plugin's data."""
        self.config = model

    @hookimpl
    def activate(self) -> None:
        """Activate the plugin by initializing the OpenAI client."""
        if self.config.aggregate:
            client = client_with_aggregation
        else:
            client = client_without_aggregation

Define Configuration (Optional)

To define the expected configuration for your plugin, implement the grab_config and set_data methods. These methods should return a Pydantic model with necessary attributes and set class data, respectively. The model attributes for will be checked for upon startup in the applications configuration parameters.

Example Configuration

class HelperAssistant:
    @hookimpl
    def grab_config(self) -> type[BaseModel]:
        """Return the plugin's configuration Pydantic model."""
        class AssistantConfig(BaseModel):
            aggregate: bool = Field(..., description="Whether to aggregate past results")
        return AssistantConfig

    @hookimpl
    def set_data(self, model: BaseModel) -> None:
        """Set the plugin's data."""
        self.config = model

    ...

Define Activation (Optional)

Sometimes, you need to initialize data before processing but after setting the class' data. Use an activate function to achieve this.

Example Activation

from service_sdk import service_client

class ClientOutput:
    @hookimpl
    def grab_config(self) -> type[BaseModel]:
        """Return the plugin's configuration Pydantic model."""
        class ClientOutputConfig(BaseModel):
            service_key: str = Field(..., description="The service API key")
        return ClientOutputConfig

    @hookimpl
    def set_data(self, model: BaseModel) -> None:
        """Set the plugin's data."""
        self.config = model

    @hookimpl
    def activate(self) -> None:
        """Activate the plugin by initializing the service client."""
        self.client = service_client(api_key=self.config.service_key)

Define plugin info toml file

Each plugin should include a TOML file with essential information. The TOML file should be in the following format:

[info]
name = "Plugin Name"
module = "plugin_module"
class_name = "PluginClassInModule"
type = "assistant"
uses = ["general"]

Where: - name is what you'll refer to the plugin as (please no spaces!) - module is the name of your python module (normally found from your .py file) - class_name is the class in the module that corresponds to your plugin. - type is the type of the plugin. This allows us to dispatch plugins to the right handler. - uses is a list of used plugins, handlers, etc.

For Rego checks, include additional information:

...

uses = ["provider_name", "rego"]
[rego]
rego_file = "path/to/regofile.rego"
description = "Description of policy results"

(More info can be found in the creating rego plugins document!)

It helps to put all your plugin info and modules into a new folder in the plugin directory.

Define other hookspecs

In order to implement the various types of plugins in Opsbox, you need to implement their respective hooks.

All plugins can use hookspecs defined in BaseSpec. These include those that implement basic optional features such as delayed activation and configuration gathering.

For the two base handlers, the following hookspecs exist:

General Handler - InputSpec, Hooks that implement methods to generate formatted results - ProviderSpec, Hooks that implement methods to gather data that other plugins can rely on - OutputSpec, Hooks that implement methods to output formatted result data - AssistantSpec, Hooks that implement methods to transform formatted data

Rego Handler - RegoSpec, Hooks that implement methods to generate formatted results from Rego check outputs.

Read the documentation for rego and handler plugins! They require a bit more than other plugin types.

Check out some more of these documents for more info on how to structure various types of plugins.

Utilize the Result Models

OpsBox uses Result models to represent the outputs of plugins, especially when data needs to be passed between plugins in the pipeline.

Understanding the Result Model

The Result model is defined as follows:

from pydantic import BaseModel

class Result(BaseModel):
    """A model representing the results of a plugin's processing.

    Attributes:
        relates_to (str): The entity the result relates to.
        result_name (str): The name of the result.
        result_description (str): A description of the result.
        details (dict | list[dict]): Additional details of the result.
        formatted (str): A formatted string representation of the result.
    """
    relates_to: str
    result_name: str
    result_description: str
    details: dict | list[dict]
    formatted: str

How to Use the Result Model in Your Plugin

When your plugin processes data and produces output that needs to be passed to subsequent plugins, you should encapsulate that output in a Result object.

Example

Suppose you have an input plugin that gathers data and needs to output it for assistant plugins to process.

from pydantic import BaseModel
from opsbox import Result
from pluggy import HookimplMarker

hookimpl = HookimplMarker("opsbox")

class ExampleInputPlugin:
    @hookimpl
    def process(self, data):
        # Simulate data gathering
        gathered_data = {
            "key1": "value1",
            "key2": "value2"
        }

        # Create a Result object
        result = Result(
            relates_to="ExampleInputPlugin",
            result_name="GatheredData",
            result_description="Data gathered from the example input plugin.",
            details=gathered_data,
            formatted=str(gathered_data)
        )

        # Return the result in a list
        return [result]

In an assistant plugin, you can process the results from previous plugins:

from opsbox import Result
from pluggy import HookimplMarker

hookimpl = HookimplMarker("opsbox")

class ExampleAssistantPlugin:
    @hookimpl
    def process_input(self, input_results: list[Result]) -> list[Result]:
        processed_results = []
        for result in input_results:
            # Perform some processing on result.details
            processed_data = {k: v.upper() for k, v in result.details.items()}

            # Create a new Result object
            new_result = Result(
                relates_to=result.relates_to,
                result_name="ProcessedData",
                result_description="Data processed by the assistant plugin.",
                details=processed_data,
                formatted=str(processed_data)
            )
            processed_results.append(new_result)
        return processed_results

Finally, an output plugin can take the processed results and output them accordingly:

from opsbox import Result
from pluggy import HookimplMarker

hookimpl = HookimplMarker("opsbox")

class ExampleOutputPlugin:
    @hookimpl
    def process_results(self, results: list[Result]) -> None:
        for result in results:
            # Output the formatted result
            print(f"Output from {result.relates_to}: {result.formatted}")

Packaging Your Plugin for Distribution

Opsbox supports loading plugins in the virtual environment without specifying directories.

To use this feature, you must package your plugin in a pip-installable distribution with specific entry points.

Let's go over each step to produce a pip-installable plugin.

Directory structure

In order to package your plugin, we recommend starting out with a subfolder layout for your module.

This might look like this:

myproject/ ├── LICENSE ├── README.md ├── pyproject.toml └── plugin_name/ │ ├── init.py │ ├── module1.py | ├── manifest.toml ├── tests/ └── docs/

This allows us to define setuptools entrypoints and other things.

Setup UV

Make sure to initalize this as a uv project using uv init in the root of your new directory.

Add any 3rd party dependencies or packages needed with uv add.

If you have multiple plugins in the same repo, it might be worthwhile to investigate uv's workspaces feature.

Save manifest

Follow the prior steps in the tutorial, but save your plugin manifest as manifest.toml and put it in your module's subfolder. This is required to search for the toml file needed.

Mess around in pyproject.toml

With your plugin created, you now need to define your entrypoints so that opsbox can find the required plugins.

Step 1: Add entrypoint

We need to add a setuptools entrypoint under opsbox.plugins to allow opsbox to see our plugins.

Say for example, we have a class TestPlugin defined in module1.py according to the subfolder layout above, and this is the main class of our plugin implementing one of our hooks.

We'd want to add the following to our pyproject.toml file

[project.entry-points.'opsbox.plugins']
plugin_name = "plugin_name.module1:TestPlugin"

Step 2: Include Metadata

Next step is to include the plugin's manifest so it can be easily found.

To do this, add the following to your pyproject file:

[tool.setuptools]
include-package-data = true

[tool.setuptools.package-data]
"plugin_name" = ["manifest.toml"] # include "*.rego" if you are making a rego plugin.

As long as your manifest is packaged in that subfolder unnder manifest.toml, your plugin should now be buildable.

Step 3: Building the plugin

We will use UV's in-built building tool to build the plugin.

Go to the root of the directory and type uv build. If everything goes well, you should be able to find your distribution in the dist/ folder in your workspace's root.

Best Practices

  • Consistency: Ensure that all plugins return their outputs encapsulated in Result objects for consistency across the pipeline.
  • Information Preservation: Include meaningful information in all fields of the Result model to aid in debugging and downstream processing.
  • Avoid Data Loss: When processing results, be careful not to inadvertently discard important information in the details or other fields.

Add More!

Feel free to extend your plugins with additional functionality as needed. You can define more methods, utilize other libraries, and customize your plugin to fit your specific requirements.

Additional Notes

  • Testing Your Plugin: It's a good idea to write tests for your plugin to ensure it behaves as expected within the OpsBox framework.
  • Documentation: Document your plugin's functionality, configuration options, and any dependencies it may have.
  • Contribution: If you believe your plugin could benefit others, consider contributing it back to the OpsBox community!