GeoLibs Dator - A data extractor
Project description
GeoLibs-Dator
Dator, a data extractor (ETL as a library), that uses Pandas' DataFrames as in memory temporal storage.
Features
| Source | Extract | Transform | Load |
|---|---|---|---|
| BigQuery | Y | Y | |
| CARTO | Y | Y | Y* |
| CSV | Y | Y | |
| Pandas | Y | ||
| PostgreSQL | Y | Y | Y |
* Note: We are waiting for the append feature on CARTOframes, because the one we are using is a ñapa.
Configuration
Create a config.yml file using the config.example.yml one as guide. You can find in that one all the possible ETL cases.
If you are using BigQuery in your ETL process, you need to add a GOOGLE_APPLICATION_CREDENTIALS environment variable with the path to your Google Cloud's credentials.json file.
You can test them with the example.py file.
TODOs
- Better doc.
- Tests.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file geolibs-dator-0.0.2.tar.gz.
File metadata
- Download URL: geolibs-dator-0.0.2.tar.gz
- Upload date:
- Size: 7.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/0.12.14 CPython/3.6.8 Linux/4.14.91+
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
183990cf2264db160afbd42caa763275aff6b459cea63332ae7ecdc94f612556
|
|
| MD5 |
7909e5af7aa75b9ac46f78889bb54d31
|
|
| BLAKE2b-256 |
c7c352fa06ea966ea69415032c4b552d1eb7aa0cf3792fda65cd0ca1ab875abf
|
File details
Details for the file geolibs_dator-0.0.2-py3-none-any.whl.
File metadata
- Download URL: geolibs_dator-0.0.2-py3-none-any.whl
- Upload date:
- Size: 10.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/0.12.14 CPython/3.6.8 Linux/4.14.91+
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3232c5e0e9d697b2de455e3e249818c8f0ddaebe52e5d3b3aff3f1d1553fcebf
|
|
| MD5 |
46c720f43d0c6bbd0877768224eb2303
|
|
| BLAKE2b-256 |
d173fff5c9fb850b6ac2c8841c5d6041406de9eb48a9f469df28aa9a27ac88dc
|
