CLI tool and Python utility functions for manipulating SQLite databases
Project description
sqlite-utils
Python CLI utility and library for manipulating SQLite databases.
Some feature highlights
- Pipe JSON (or CSV or TSV) directly into a new SQLite database file, automatically creating a table with the appropriate schema
- Run in-memory SQL queries, including joins, directly against data in CSV, TSV or JSON files and view the results.
- Configure SQLite full-text search against your database tables and run search queries against them, ordered by relevance
- Run transformations against your tables to make schema changes that SQLite
ALTER TABLEdoes not directly support, such as changing the type of a column - Extract columns into separate tables to better normalize your existing data
Read more on my blog: sqlite-utils: a Python library and CLI tool for building SQLite databases and other entries tagged sqliteutils.
Installation
pip install sqlite-utils
Or if you use Homebrew for macOS:
brew install sqlite-utils
Using as a CLI tool
Now you can do things with the CLI utility like this:
$ sqlite-utils memory dogs.csv "select * from t"
[{"id": 1, "age": 4, "name": "Cleo"},
{"id": 2, "age": 2, "name": "Pancakes"}]
$ sqlite-utils insert dogs.db dogs dogs.csv --csv
[####################################] 100%
$ sqlite-utils tables dogs.db --counts
[{"table": "dogs", "count": 2}]
$ sqlite-utils dogs.db "select id, name from dogs"
[{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Pancakes"}]
$ sqlite-utils dogs.db "select * from dogs" --csv
id,age,name
1,4,Cleo
2,2,Pancakes
$ sqlite-utils dogs.db "select * from dogs" --table
id age name
---- ----- --------
1 4 Cleo
2 2 Pancakes
You can import JSON data into a new database table like this:
$ curl https://api.github.com/repos/simonw/sqlite-utils/releases \
| sqlite-utils insert releases.db releases - --pk id
Or for data in a CSV file:
$ sqlite-utils insert dogs.db dogs dogs.csv --csv
sqlite-utils memory lets you import CSV or JSON data into an in-memory database and run SQL queries against it in a single command:
$ cat dogs.csv | sqlite-utils memory - "select name, age from stdin"
See the full CLI documentation for comprehensive coverage of many more commands.
Using as a library
You can also import sqlite_utils and use it as a Python library like this:
import sqlite_utils
db = sqlite_utils.Database("demo_database.db")
# This line creates a "dogs" table if one does not already exist:
db["dogs"].insert_all([
{"id": 1, "age": 4, "name": "Cleo"},
{"id": 2, "age": 2, "name": "Pancakes"}
], pk="id")
Check out the full library documentation for everything else you can do with the Python library.
Related projects
- Datasette: A tool for exploring and publishing data
- csvs-to-sqlite: Convert CSV files into a SQLite database
- db-to-sqlite: CLI tool for exporting a MySQL or PostgreSQL database as a SQLite file
- dogsheep: A family of tools for personal analytics, built on top of
sqlite-utils
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file sqlite-utils-3.23.tar.gz.
File metadata
- Download URL: sqlite-utils-3.23.tar.gz
- Upload date:
- Size: 173.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
04d94e3ff8160c4461e11ced879ce37a0847afba33c8019e449c68784e9b6966
|
|
| MD5 |
9374987d7680c8996c54032c588cead1
|
|
| BLAKE2b-256 |
88ca721f4b48498b3b7af2127ab07be0b7ab662de1ba24121c4c32292b0a847d
|
File details
Details for the file sqlite_utils-3.23-py3-none-any.whl.
File metadata
- Download URL: sqlite_utils-3.23-py3-none-any.whl
- Upload date:
- Size: 56.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7c47c448886e78eec8a80b5087bea50a31774872a89c1e28c977e15a655b1792
|
|
| MD5 |
082fffda8390e2b69f6f8edd4b6a0973
|
|
| BLAKE2b-256 |
1526de57116dff0c5b692038e325072ef1460132d3a4670cd54d52d4b6b1bb23
|
