Skip to main content

Library to easily interface with LLM API providers

Project description

🚅 LiteLLM

Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, TogetherAI, Azure, OpenAI, etc.]

PyPI Version Stable Version CircleCI Downloads Y Combinator W23 git commit activity

Open In Colab

100+ Supported Models | Docs |

📣1-click deploy your own LLM proxy server. Grab time, if you're interested!

LiteLLM manages

  • Translating inputs to the provider's completion and embedding endpoints
  • Guarantees consistent output, text responses will always be available at ['choices'][0]['message']['content']
  • Exception mapping - common exceptions across providers are mapped to the OpenAI exception types

Usage

Open In Colab

By default we provide a free $10 key to try all providers supported on LiteLLM. Try it now 👇

pip install litellm
from litellm import completion
import os

## We provide a free $10 key to try all providers supported on LiteLLM.
## set ENV variables 
os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key
os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)
print(response)

Don't have a key? We'll give you access 👉 https://docs.litellm.ai/docs/proxy_api

Streaming

liteLLM supports streaming the model response back, pass stream=True to get a streaming iterator in response. Streaming is supported for OpenAI, Azure, Anthropic, Huggingface models

response = completion(model="gpt-3.5-turbo", messages=messages, stream=True)
for chunk in response:
    print(chunk['choices'][0]['delta'])

# claude 2
result = completion('claude-2', messages, stream=True)
for chunk in result:
  print(chunk['choices'][0]['delta'])

Contributing

To contribute: Clone the repo locally -> Make a change -> Submit a PR with the change.

Here's how to modify the repo locally: Step 1: Clone the repo

git clone https://github.com/BerriAI/litellm.git

Step 2: Navigate into the project, and install dependencies:

cd litellm
poetry install

Step 3: Test your change:

cd litellm/tests # pwd: Documents/litellm/litellm/tests
pytest .

Step 4: Submit a PR with your changes! 🚀

  • push your fork to your github repo
  • submit a PR from there

Learn more on how to make a PR

Support / talk with founders

Why did we build this

  • Need for simplicity: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI, Cohere

Contributors

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

litellm-0.1.745.tar.gz (850.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

litellm-0.1.745-py3-none-any.whl (894.4 kB view details)

Uploaded Python 3

File details

Details for the file litellm-0.1.745.tar.gz.

File metadata

  • Download URL: litellm-0.1.745.tar.gz
  • Upload date:
  • Size: 850.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.18

File hashes

Hashes for litellm-0.1.745.tar.gz
Algorithm Hash digest
SHA256 28ba949d8589c9b92e376ffe739f1eb310d3c19b3efce56d817d0fbb99db28d5
MD5 d855aac13b87ca5b0d34351daba263b9
BLAKE2b-256 a2b088f5335d79f88f805b4ce4d2adb639dab0fe84615b8f103113321df19905

See more details on using hashes here.

File details

Details for the file litellm-0.1.745-py3-none-any.whl.

File metadata

  • Download URL: litellm-0.1.745-py3-none-any.whl
  • Upload date:
  • Size: 894.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.18

File hashes

Hashes for litellm-0.1.745-py3-none-any.whl
Algorithm Hash digest
SHA256 60051ce9510aac6982b5a8fbd0f0afe4a44bab2dac0eb1ef04e7c2e218f4d7cc
MD5 b3ba9ee6fa8c6d76d3012ef39fbb8631
BLAKE2b-256 13233e419267530c2d6e0fcef5c25c6af662a8affe7c4439323d9fc9da11f865

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page