Skip to main content

Parallel parameter sweeps with joblib/MPI and tqdm progress bars.

Project description

ParameterRun

parameterrun is a lightweight utility for running parameter sweeps in Python with a simple, function-based API.

It is designed for scientific and numerical workflows where you want to evaluate a function over one or more parameter grids, optionally in parallel, while keeping progress bars and output reshaping convenient.

The package supports:

  • single-parameter scans
  • multi-parameter scans
  • grouped parameters that must vary together
  • parallel execution with joblib
  • MPI execution with mpi4py
  • automatic reshaping of outputs into arrays
  • functions returning either one output or multiple outputs

Installation

With pip

pip install parameterrun

With optional MPI support

pip install parameterrun[mpi]

Basic idea

The main entry point is:

from parameterrun import parameterrun

The function parameterrun(...) evaluates a user-defined function over a set of parameter values. The parameters to sweep are identified by their names exactly as they appear in the target function signature. Additional fixed arguments can be passed as keyword arguments.

Quick start

1. Single parameter

from parameterrun import parameterrun


def square(x):
    return x ** 2


result = parameterrun(square, param_names="x", param_values=[0, 1, 2, 3], n_workers=1, )

print(result)  # array([0, 1, 4, 9])

2. Multiple independent parameters

This creates a Cartesian product of the parameter groups.

from parameterrun import parameterrun


def f(x, y, scale=1):
    return scale * (x + y)


result = parameterrun(f, param_names=["x", "y"], param_values=[[1, 2, 3], [10, 20], ], scale=2, )

print(result.shape)  # (3, 2)

3. Grouped parameters

Parameters in the same group vary together and must have the same number of values.

from parameterrun import parameterrun


def f(x, y, z):
    return x + y + z


result = parameterrun(f, param_names=[["x", "y"], ["z"]], param_values=[[[1, 2], [10, 20]],  # (x, y) = (1,10), (2,20)
                                                                        [[100, 200, 300]],  # z varies independently
                                                                        ], n_workers=1, )

print(result.shape)  # (2, 3)

This grouped-input mode is useful when some parameters are logically linked and should not be combined independently.

Multiple outputs

If the target function returns a tuple, parameterrun returns a list of arrays, one per output channel.

from parameterrun import parameterrun


def stats(x):
    return x + 1, x ** 2


mean_like, square_like = parameterrun(stats, param_names="x", param_values=[1, 2, 3], n_workers=1, )

print(mean_like)
# array([2, 3, 4])

print(square_like)  # array([1, 4, 9])

# Ask for plain Python lists instead of numpy arrays
mean_like, square_like = parameterrun(stats, param_names="x", param_values=[1, 2, 3], n_workers=1,
                                      result_as_array=False, )

print(mean_like)  # [2, 3, 4]

If the function returns a single object, the result is returned as a single array when reshaping is possible. Set result_as_array=False to convert outputs to Python lists.

For multi-dimensional sweeps, reshaping is automatic when reshape=True (default): the first dimensions always match the parameter grid shape.

Validation behavior

parameterrun validates inputs before launching workers and raises clear ValueErrors when:

  • parameter names are duplicated
  • a swept parameter does not exist in the target function signature
  • extra keyword arguments do not match the target function signature
  • a keyword argument conflicts with a swept parameter name

Parallel backend

parameterrun supports two execution backends:

  • joblib: a simple, local parallelization library that works well for most use cases.
  • mpi: a distributed parallelization approach using MPI, suitable for running on clusters or supercomputers.

If backend=None, the package tries to detect whether MPI is available. If MPI is not available, it falls back to joblib; otherwise it uses MPI.

Joblib backend

result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="joblib",
                      n_workers=-1, )

If n_workers=1, the function runs serially without spawning joblib workers.

MPI backend

result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="mpi", )

Run the script with MPI, for example:

mpirun -n 4 python my_script.py

Under MPI, only rank 0 returns the final result; the other ranks return None.

Progress bars and verbosity

By default, progress bars are enabled:

result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], pbar_bool=True, )

Verbose logging can be enabled with:

result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], verbose=True, )

The progress-bar description is generated automatically from the function name and parameter names, unless you provide desc explicitly.

API reference

parameterrun

parameterrun(fun, param_names, param_values, n_workers=-1, pbar_bool=True, verbose=False,
             reshape=True, result_as_array=True, backend=None, desc=None, **kwargs, )
  • fun: function to evaluate
  • param_names: parameter name or grouped parameter names
  • param_values: values corresponding to param_names (any iterable, e.g. list, tuple, range, numpy array)
  • n_workers: number of workers for joblib
  • pbar_bool: enable or disable progress bars
  • verbose: print backend and timing information
  • reshape: reshape results into arrays when possible
  • result_as_array: return numpy arrays by default; if False, convert outputs to Python lists
  • backend: "joblib", "mpi", or None
  • desc: custom progress-bar label
  • **kwargs: extra fixed keyword arguments forwarded to fun

Input formats

parameterrun accepts three input styles.

At every style, value containers can be any iterable (for example lists, tuples, ranges, generators, or numpy arrays).

A. Single parameter

param_names = "x"
param_values = [1, 2, 3]

B. Multiple independent parameters

param_names = ["x", "y"]
param_values = [[1, 2, 3], [10, 20], ]

C. Multiple groups of parameters

param_names = [["x", "y"], ["z"]]
param_values = [[[1, 2], [10, 20]], [[100, 200, 300]], ]

Development checks

Before opening a PR or publishing a release, run:

pytest -m "not mpi"
ruff check .
mypy
python -m build
python -m twine check dist/*

For MPI numerical-equivalence checks (Linux/OpenMPI, 4 ranks):

uv sync --extra mpi
mpirun -n 4 uv run pytest -m mpi -o addopts="-q -m mpi"

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

parameterrun-0.4.1.tar.gz (19.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

parameterrun-0.4.1-py3-none-any.whl (10.8 kB view details)

Uploaded Python 3

File details

Details for the file parameterrun-0.4.1.tar.gz.

File metadata

  • Download URL: parameterrun-0.4.1.tar.gz
  • Upload date:
  • Size: 19.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for parameterrun-0.4.1.tar.gz
Algorithm Hash digest
SHA256 428eb0fd9a61151fd592d3b08e4f60a0a27429b0aeab6025116b5b0ddca3692e
MD5 c4ab586b11addd698cce3e8c8eca76e6
BLAKE2b-256 7f7b3b4b399e2505f0549625e8507c2ca029ed075c7a1aadab955179c174b349

See more details on using hashes here.

Provenance

The following attestation bundles were made for parameterrun-0.4.1.tar.gz:

Publisher: publish.yml on Davtax/ParameterRun

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file parameterrun-0.4.1-py3-none-any.whl.

File metadata

  • Download URL: parameterrun-0.4.1-py3-none-any.whl
  • Upload date:
  • Size: 10.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for parameterrun-0.4.1-py3-none-any.whl
Algorithm Hash digest
SHA256 3579971d0f81067fd4017ed55a8e542b1aa259fc9c769a1f38c1d0a2fd5c8a52
MD5 dc5bacdd0bdf82b0dc1e7d4db504fc04
BLAKE2b-256 2d57337c785526c7a3909d2ca6bfcfc63977ea7276b63238bf210d7dbaa6d662

See more details on using hashes here.

Provenance

The following attestation bundles were made for parameterrun-0.4.1-py3-none-any.whl:

Publisher: publish.yml on Davtax/ParameterRun

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page