guido-inference

Run Guido models locally. Model code with Titans memory update during inference, HuggingFace loader, cartridge support, and CLI.

Guido's memory evolves as you talk to it. Save the memory state as a .cart cartridge file and reload it later — the model picks up where it left off.

Installation

pip install guido-inference

For Flash Linear Attention kernel support (GPU):

pip install guido-inference[fla]

Quick Start

CLI

guido run northsea-ai/Guido-3B --prompt "What is the capital of the Netherlands?"
guido run northsea-ai/Guido-3B                          # Interactive mode
guido run northsea-ai/Guido-3B --effort high             # Adaptive compute
guido run northsea-ai/Guido-3B --cartridge ./john.cart   # Resume with saved memory
guido info northsea-ai/Guido-3B                          # Model info (no GPU)

Python

from guido_inference import load_model

model, tokenizer = load_model("northsea-ai/Guido-3B")
input_ids = tokenizer.encode("Hello!", return_tensors="pt").to(model.device)
output = model.generate(input_ids)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Cartridges — Create, Save, Reload

Memory updates happen automatically during inference (Titans architecture). Save and reload the evolved state:

# Conversation evolves the model's memory
for msg in ["I'm a researcher at TU Delft.", "My focus is renewable energy."]:
    ids = tokenizer.encode(msg, return_tensors="pt").to(model.device)
    model.generate(ids)

# Save evolved memory as portable cartridge (~250KB-1MB)
model.save_memory("./researcher.cart")

# Later: reload and continue where you left off
model.load_memory("./researcher.cart")
output = model.generate(tokenizer.encode("What papers should I read?", return_tensors="pt").to(model.device))

# Reset memory to blank slate
model.reset_memory()

Model Downloads

For fine-tuning, use third-party training harnesses like Axolotl or LLaMA-Factory with a LoRA/QLoRA adapter.

See the monorepo README for architecture details and full documentation.