corebridge

This package provides functions and classes to run wodan style processing functions in the Stactics AICore environment.

Installation

Use

pip install corebridge

to install corebrdige.

How to use

Introduction

Wodan is a proprietary backend service that applies high performance, custom analytical processing to timeseries data in the Whysor data and dashboarding environment.

Each wodan module defines one function that operates as the entry point. The parameter annotations in this function definition are used to format data and retrieve parameters from the originating call to the wodan api. This function is called with data retrieved according to a specification and with additional parameters as annotated.

A simple function might look like:

import numpy as np

def multiply(data:np.ndarray, multiplier:float=1.0):
    return data * multiplier
    

Wodan binds this function to a service endpoint and takes care of fetching data and parameters and converting the result for the caller.

AICore modules

For AICore users define a class, always named Module with a constructor __init__ and a method infer.

This package defines a baseclass to quickly construct a Module class that is able to use a wodan processor function inside the AICore system:

import numpy as np
import corebridge

def multiply(data:np.ndarray, multiplier:float=1.0):
    return data * multiplier

class Module(corebridge.aicorebridge.AICoreModule):
    def __init__(self, save_dir, assets_dir, *args, **kwargs):
        super().__init__(multiply, save_dir, assets_dir, *args, **kwargs)
    

That’s it. Well, you can add parameters to __init__ that can be used as hyperparameters in the web-interface and you could override infer for the same reason. The baseclass takes care of converting call parameters and data to the function specification and, calls the function and converts the result for the caller, similar to the original Wodan service.

You don’t have to use AICoreModule baseclas but can create your own class and use functionality within CoreBridge to convert call parameters and data to fit your needs. AICoreModule was developed to make the deployment of existing Wodan modules within AICore easier. It also helps developing functions independently of AICore while being able to deploy these functions as modules within AICore.

Development

NBDev

This library is developed with NBDev - a literate programming toolkit that supports developing code using jupyter notebooks and mix code with documentation.

Literate programming is a methodology - introduced in 1984 by Donald Knuth - that combines a programming language with a documentation language. In this approach, a program is explained in a human language (such as English) alongside code snippets. The literate source file is then processed by a preprocessor to produce both source code and formatted documentation.

This paradigm enhances program robustness, portability, and maintainability, making it a valuable tool in scientific computing and data science[^1]

Quarto

Documentation is prepared from the notebook with Quarto. Quarto too combines code with documentation but it does not extract source code into modules like nbdev.

Installation

Quarto

Quarto uses Pandoc and, for pdf format, LaTeX. These must be available on your system.

Install Quarto as you see fit, there is a VSCode extension which handles this.

NBDev

NBDev is available as PyPi package and is installed with

pip install nbdev

or if you are using conda

conda install -c fastai -y nbdev

If so desired you can let NBDev install Quarto with

nbdev_install_quarto

But this ask for the system admin password.

Local editing & testing

Setup a virtual environment, activate it and install the development package and dependencies with, on linux

    pip install -e ‘.[dev]’

or on Windows

    pip install -e .[dev]

Jupyter

The above pip install should also install jupyter but to use it the kernel needs to be installed with:

    python -m ipykernel install --user --name=corebridge.venv

nbdev cycle

• edit
• nbdev_prepare

The latter performs - nbdev_export - nbdev_test - nbdev_clean - nbdev_readme

Before committing changes to git run - nbdev_clean

Then commit to git and for a new pip release upload to Pypi with nbdev_pypi

[^1]: Wikipedia on ‘Literate Programming’