Pi Local Token Costs: Quantifying the Savings of Local AI

As the world of Large Language Models (LLMs) shifts toward local model usage, a new question has emerged for power users: "How much am I actually saving by running this model locally?".

While running a model on your own hardware is great for privacy and speed, there hasn't been a good way to quantify the value of that convenience compared to using that same model from something like OpenRouter. I wanted a way to see that comparison in real-time, which is why I built pi-local-token-costs, for my new favorite coding harness.

The Motivation

For me, the motivation was simple: comparison.

When I use Pi, I love the ability to plug in a local model and just go. But I'm also always wondering if it's worth spending on hardware and power compared to just using the same or better models through online providers. I found myself constantly doing mental math: "Is it worth using a Claude or a Codex right now, or should I stick to my local super-customized setup?".

I wanted a tool that would take the guesswork out of it, something that would silently track my usage and show me, in real-time, exactly how much I am saving by choosing local compute over API tokens.

Key Features

The extension is designed to be lightweight and non-intrusive, living quietly in the `Pi powerline footer`, another great extension for Pi.

• Live Savings Comparison: It calculates the estimated cost of your current session if you had used a cloud model (via OpenRouter) versus running it locally.

• OpenRouter Price Awareness: It fetches live pricing from over 30+ providers, always selecting the cheapest rate for a given model to give you a realistic "best-case" comparison.

• Seamless Local Integration: It doesn't care if you're using Ollama, vLLM, LM Studio, or a raw GGUF file in llama.cpp - it's designed to find a match.

• Cross-Session History: Beyond just the current session, you can see your long-term savings trends.

• Drop the extension in, and you get a live footer status that updates every 1.5 seconds:

1↑in: 142k ↓out: 28k $0.20000 · Qwen3.6-35B-A3B |

Technical Deep Dive

The Real Challenge: The Model Identity Problem

Local model names are messy. One minute you're running google/gemma4-31b with a nicely keyed model name from ollama , the next you're using a full file path for the GGUF on your drive with llama-server. Meanwhile, OpenRouter expects a clean string like qwen/qwen3.6-35b-a3b for comparison.

To bridge this gap, I built a seven-level priority chain that resolves any model ID to a cloud-equivalent price:

1User config override (exact model ID match)      → ~/.pi/agent/token-costs.json
2User config override (normalized model ID)       → lowercase, no prefixes
3Live OpenRouter API (cheapest provider rate)     → Refreshed every 24h
4Stripping-based matching against live prices     → Progressive strategies
5Alias rules (regex patterns for common naming)   → Handles Ollama, vLLM, etc.
6Embedded fallback DB (~60 cloud models)          → Works offline
7Local model defaults ($0/$0)                     → Unknown models assumed free

Let's walk through how this actually works.

Level 1–2: User Config Overrides

Power users can define custom pricing in ~/.pi/agent/token-costs.json:

1{
2  "my-local-model": {
3    "input": 0,
4    "output": 0,
5    "label": "My Custom Model"
6  }
7}

Level 3: Live OpenRouter Pricing

At startup, the extension hits the OpenRouter API (/api/v1/models) and caches the cheapest rate for every model across 30+ providers:

1interface ORProviderPricing {
2  prompt: string;        // e.g., "0.000003" = $3.00/M tokens
3  completion: string;    // e.g., "0.000015" = $15.00/M tokens
4  input_cache_read?: string;
5  input_cache_write?: string;
6}

The cache is refreshed every 24 hours. For a single model like claude-sonnet-4, the extension automatically picks the cheapest provider, whether that's Anthropic directly, Together AI, or DeepInfra.

Level 5: Alias Rules

For common naming patterns, regex aliases handle the heavy lifting:

1const ALIAS_RULES: AliasRule[] = [
2  { pattern: /qwen[\s./-]*2\.5[\s./-]*coder/i, target: 'qwen2.5-coder' },
3  { pattern: /llama\s*3\.3.*70b/i, target: 'llama-3.3-70b-instruct' },
4  { pattern: /^ollama\/?llama3\.3$/i, target: 'llama-3.3-70b-instruct' },
5  { pattern: /mistral[\s./-]*nemo/i, target: 'mistral-nemo' },
6  { pattern: /gemma[\s./-]*2/i, target: 'gemma2' },
7  { pattern: /^qwen[\s./-]*/i, target: 'qwen-base' },
8];

Level 6: Embedded Fallback Database

When the OpenRouter API is unreachable, the extension falls back to an embedded database of ~60 cloud models:

1const FALLBACK_PRICES: Record<string, { input: number; output: number }> = {
2  'claude-sonnet-5': { input: 2.0, output: 10.0 },
3  'claude-sonnet-4': { input: 3.0, output: 15.0 },
4  'gpt-4o': { input: 2.5, output: 10.0 },
5  'o3-pro': { input: 400.0, output: 800.0 },
6  'gemini-2.5-pro': { input: 1.25, output: 15.0 },
7  'llama-3.3-70b-instruct': { input: 0.2, output: 0.4 },
8  // ... plus 50+ more models including the full Qwen3 family
9};

The Pricing Pipeline

Once the model is resolved, the cost calculation is straightforward. The extension intercepts every message_end event from Pi's runtime, resolves the model, calculates costs, and pushes the result to the footer. Everything happens synchronously. No background tasks, no latency impact on your AI interactions.

1Data Fetching: At startup, the extension fetches the latest pricing from OpenRouter.
2Intercepting Usage: It monitors the token usage reported by Pi's runtime.
3The Pricing Pipeline: It takes the prompt_tokens and completion_tokens, identifies the model via the matching engine, and applies the cheapest provider's rate.
4Persistence: Results are stored locally, allowing for historical analysis without needing a heavy backend.

Other goodies

The extension ships with three commands for deeper analysis:

• /token-stats : Current session tokens, cost, and per-model breakdown

• /token-history: Last 50 messages across all sessions, grouped by model

• /token-price: Debug how a specific model is resolved and priced

Wrapping up

pi-local-token-costs isn't just about saving pennies; it's about visibility. It turns the abstract concept of "local vs. cloud" into a concrete metric. For anyone building workflows with a mix of local and cloud LLMs, it provides the data needed to make informed decisions about privacy, performance, and, most importantly, cost.

The extension is available at pi-local-token-costs.

If you find it useful, please consider starring the repo! And if you run into any edge cases with model matching, the model matching docs go deep into every strategy.