Parallel parameter sweeps with joblib/MPI and tqdm progress bars.
Project description
ParameterRun
parameterrun is a lightweight utility for running parameter sweeps in Python with a simple, function-based API.
It is designed for scientific and numerical workflows where you want to evaluate a function over one or more parameter grids, optionally in parallel, while keeping progress bars and output reshaping convenient.
The package supports:
- single-parameter scans
- multi-parameter scans
- grouped parameters that must vary together
- parallel execution with
joblib - MPI execution with
mpi4py - automatic reshaping of outputs into arrays
- functions returning either one output or multiple outputs
Installation
With pip
pip install parameterrun
With optional MPI support
pip install parameterrun[mpi]
Basic idea
The main entry point is:
from parameterrun import parameterrun
The function parameterrun(...) evaluates a user-defined function over a set of parameter values. The parameters to
sweep are identified by their names exactly as they appear in the target function signature. Additional fixed arguments
can be passed as keyword arguments.
Quick start
1. Single parameter
from parameterrun import parameterrun
def square(x):
return x ** 2
result = parameterrun(square, param_names="x", param_values=[0, 1, 2, 3], n_workers=1, )
print(result) # array([0, 1, 4, 9])
2. Multiple independent parameters
This creates a Cartesian product of the parameter groups.
from parameterrun import parameterrun
def f(x, y, scale=1):
return scale * (x + y)
result = parameterrun(f, param_names=["x", "y"], param_values=[[1, 2, 3], [10, 20], ], scale=2, )
print(result.shape) # (3, 2)
3. Grouped parameters
Parameters in the same group vary together and must have the same number of values.
from parameterrun import parameterrun
def f(x, y, z):
return x + y + z
result = parameterrun(f, param_names=[["x", "y"], ["z"]], param_values=[[[1, 2], [10, 20]], # (x, y) = (1,10), (2,20)
[[100, 200, 300]], # z varies independently
], n_workers=1, )
print(result.shape) # (2, 3)
This grouped-input mode is useful when some parameters are logically linked and should not be combined independently.
Multiple outputs
If the target function returns a tuple, parameterrun returns a list of arrays, one per output channel.
from parameterrun import parameterrun
def stats(x):
return x + 1, x ** 2
mean_like, square_like = parameterrun(stats, param_names="x", param_values=[1, 2, 3], n_workers=1, )
print(mean_like)
# array([2, 3, 4])
print(square_like) # array([1, 4, 9])
# Ask for plain Python lists instead of numpy arrays
mean_like, square_like = parameterrun(stats, param_names="x", param_values=[1, 2, 3], n_workers=1,
result_as_array=False, )
print(mean_like) # [2, 3, 4]
If the function returns a single object, the result is returned as a single array when reshaping is possible.
Set result_as_array=False to convert outputs to Python lists.
For multi-dimensional sweeps, reshaping is automatic when reshape=True (default): the first dimensions always match
the parameter grid shape.
Validation behavior
parameterrun validates inputs before launching workers and raises clear ValueErrors when:
- parameter names are duplicated
- a swept parameter does not exist in the target function signature
- extra keyword arguments do not match the target function signature
- a keyword argument conflicts with a swept parameter name
Parallel backend
parameterrun supports two execution backends:
joblib: a simple, local parallelization library that works well for most use cases.mpi: a distributed parallelization approach using MPI, suitable for running on clusters or supercomputers.
If backend=None, the package tries to detect whether MPI is available. If MPI is not available, it falls back to
joblib; otherwise it uses MPI.
Joblib backend
result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="joblib",
n_workers=-1, )
If n_workers=1, the function runs serially without spawning joblib workers.
MPI backend
result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="mpi", )
Run the script with MPI, for example:
mpirun -n 4 python my_script.py
Under MPI, only rank 0 returns the final result; the other ranks return None.
Progress bars and verbosity
By default, progress bars are enabled:
result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], pbar_bool=True, )
Verbose logging can be enabled with:
result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], verbose=True, )
The progress-bar description is generated automatically from the function name and parameter names, unless you provide
desc explicitly.
API reference
parameterrun
parameterrun(fun, param_names, param_values, n_workers=-1, pbar_bool=True, verbose=False,
reshape=True, result_as_array=True, backend=None, desc=None, **kwargs, )
fun: function to evaluateparam_names: parameter name or grouped parameter namesparam_values: values corresponding toparam_names(any iterable, e.g. list, tuple, range, numpy array)n_workers: number of workers forjoblibpbar_bool: enable or disable progress barsverbose: print backend and timing informationreshape: reshape results into arrays when possibleresult_as_array: return numpy arrays by default; ifFalse, convert outputs to Python listsbackend:"joblib","mpi", orNonedesc: custom progress-bar label**kwargs: extra fixed keyword arguments forwarded tofun
Input formats
parameterrun accepts three input styles.
At every style, value containers can be any iterable (for example lists, tuples, ranges, generators, or numpy arrays).
A. Single parameter
param_names = "x"
param_values = [1, 2, 3]
B. Multiple independent parameters
param_names = ["x", "y"]
param_values = [[1, 2, 3], [10, 20], ]
C. Multiple groups of parameters
param_names = [["x", "y"], ["z"]]
param_values = [[[1, 2], [10, 20]], [[100, 200, 300]], ]
Development checks
Before opening a PR or publishing a release, run:
pytest -m "not mpi"
ruff check .
mypy
python -m build
python -m twine check dist/*
For MPI numerical-equivalence checks (Linux/OpenMPI, 4 ranks):
uv sync --extra mpi
mpirun -n 4 uv run pytest -m mpi -o addopts="-q -m mpi"
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file parameterrun-0.4.1.tar.gz.
File metadata
- Download URL: parameterrun-0.4.1.tar.gz
- Upload date:
- Size: 19.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
428eb0fd9a61151fd592d3b08e4f60a0a27429b0aeab6025116b5b0ddca3692e
|
|
| MD5 |
c4ab586b11addd698cce3e8c8eca76e6
|
|
| BLAKE2b-256 |
7f7b3b4b399e2505f0549625e8507c2ca029ed075c7a1aadab955179c174b349
|
Provenance
The following attestation bundles were made for parameterrun-0.4.1.tar.gz:
Publisher:
publish.yml on Davtax/ParameterRun
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
parameterrun-0.4.1.tar.gz -
Subject digest:
428eb0fd9a61151fd592d3b08e4f60a0a27429b0aeab6025116b5b0ddca3692e - Sigstore transparency entry: 1350468901
- Sigstore integration time:
-
Permalink:
Davtax/ParameterRun@4bf805c3e508604b4b87828011997b00f789a693 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/Davtax
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4bf805c3e508604b4b87828011997b00f789a693 -
Trigger Event:
workflow_run
-
Statement type:
File details
Details for the file parameterrun-0.4.1-py3-none-any.whl.
File metadata
- Download URL: parameterrun-0.4.1-py3-none-any.whl
- Upload date:
- Size: 10.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3579971d0f81067fd4017ed55a8e542b1aa259fc9c769a1f38c1d0a2fd5c8a52
|
|
| MD5 |
dc5bacdd0bdf82b0dc1e7d4db504fc04
|
|
| BLAKE2b-256 |
2d57337c785526c7a3909d2ca6bfcfc63977ea7276b63238bf210d7dbaa6d662
|
Provenance
The following attestation bundles were made for parameterrun-0.4.1-py3-none-any.whl:
Publisher:
publish.yml on Davtax/ParameterRun
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
parameterrun-0.4.1-py3-none-any.whl -
Subject digest:
3579971d0f81067fd4017ed55a8e542b1aa259fc9c769a1f38c1d0a2fd5c8a52 - Sigstore transparency entry: 1350468991
- Sigstore integration time:
-
Permalink:
Davtax/ParameterRun@4bf805c3e508604b4b87828011997b00f789a693 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/Davtax
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4bf805c3e508604b4b87828011997b00f789a693 -
Trigger Event:
workflow_run
-
Statement type: