Framework for large language model evaluations
Project description
Welcome to Inspect, a framework for large language model evaluations created by the UK AI Security Institute.
Inspect provides many built-in components, including facilities for prompt engineering, tool usage, multi-turn dialog, and model graded evaluations. Extensions to Inspect (e.g. to support new elicitation and scoring techniques) can be provided by other Python packages.
To get started with Inspect, please see the documentation at https://inspect.aisi.org.uk/.
Inspect also includes a collection of over 200 pre-built evaluations ready to run on any model (learn more at https://inspect.aisi.org.uk/evals/).
To work on development of Inspect, clone the repository and install with the -e flag and [dev] optional dependencies:
git clone https://github.com/UKGovernmentBEIS/inspect_ai.git
cd inspect_ai
pip install -e ".[dev]"
Optionally install pre-commit hooks via
make hooks
Run linting, formatting, and tests via
make check
make test
If you use VS Code, you should be sure to have installed the recommended extensions (Python, Ruff, and MyPy). Note that you'll be prompted to install these when you open the project in VS Code.
Frontend development (TypeScript)
The web UI lives in a git submodule at src/inspect_ai/_view/ts-mono/. These steps are only needed if you plan to work on the TypeScript/React frontend — Python-only contributors can skip this entirely.
Initialize the submodule and install dependencies — see the one-time setup guide.
Documentation
To work on the Inspect documentation, install the optional [doc] dependencies with the -e flag and build the docs:
pip install -e ".[doc]"
cd docs
quarto render # or 'quarto preview'
If you intend to work on the docs iteratively, you'll want to install the Quarto extension in VS Code.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file inspect_ai-0.3.220.tar.gz.
File metadata
- Download URL: inspect_ai-0.3.220.tar.gz
- Upload date:
- Size: 45.6 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1b67fd9aaa203c4dae9e602ce867a6cfafa0df7d4f4884b17e4e85b73bac0816
|
|
| MD5 |
08143df5cd08e415461fb438dc4bd455
|
|
| BLAKE2b-256 |
ceb75588fe9c5b58a77a162579d463fa75d5399712482e5d8dba0e0258a47306
|
File details
Details for the file inspect_ai-0.3.220-py3-none-any.whl.
File metadata
- Download URL: inspect_ai-0.3.220-py3-none-any.whl
- Upload date:
- Size: 36.1 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1d22c9de47c2f98f847650d77183ba91c02609e618a7725441e6d589d36c01c2
|
|
| MD5 |
ff6aa9077b0494bb3cdc128b3956e558
|
|
| BLAKE2b-256 |
c1f0e74a150d58d4ebcb3449a5894834899f61809e7e3d98e86da2caf8863e4f
|
