LLM#

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

Run prompts from the command-line, store the results in SQLite, generate embeddings and more.

Here’s a YouTube video demo and accompanying detailed notes.

Background on this project:

For more check out the llm tag on my blog.

Quick start#

First, install LLM using pip or Homebrew or pipx:

pip install llm

Or with Homebrew (see warning note):

brew install llm

Or with pipx:

pipx install llm

If you have an OpenAI API key key you can run this:

# Paste your OpenAI API key into this
llm keys set openai

# Run a prompt (with the default gpt-4o-mini model)
llm "Ten fun names for a pet pelican"

# Extract text from an image
llm "extract text" -a scanned-document.jpg

# Use a system prompt against a file
cat myfile.py | llm -s "Explain this code"

Or you can install a plugin and use models that can run on your local device:

# Install the plugin
llm install llm-gpt4all

# Download and run a prompt against the Orca Mini 7B model
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'

To start an interactive chat with a model, use llm chat:

llm chat -m gpt-4o

Chatting with gpt-4o
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Tell me a joke about a pelican
Why don't pelicans like to tip waiters?

Because they always have a big bill!
>

LLM#

Quick start#

Contents#