llama-cpp-provider/README.md

Llama.cpp Provider Extension for Pi

This extension dynamically registers models from a llama.cpp llama-server instance with their context window sizes.

Installation

Option 1: Quick Test

pi -e /path/to/pi_agent_llama_provider/llama-cpp-provider.ts

Option 2: Auto-Discovery

Copy the extension to your global extensions directory:

cp llama-cpp-provider.ts ~/.pi/agent/extensions/

Then run /reload in pi to load it automatically.

Option 3: Custom Server URL

Set the LLAMA_SERVER_URL environment variable before starting pi:

export LLAMA_SERVER_URL="http://your-server:port/v1"
pi

Or modify the LLAMA_SERVER_URL constant in the extension file.

Usage

Once loaded, the extension will:

1. Fetch all models from the llama.cpp server
2. Parse the context window size from each model's preset
3. Register them as a llama-cpp provider
4. Make them available via /model or Ctrl+L

Commands

• /llama-reload - Reload and re-load models from the server
• /llama-list - List all available models with their context sizes

Known Issues

If you run this extension as your only provider for pi agent, you'll get a warning about no loaded models. My fix was to use an empty model list in the default models.ini file: https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/models.md