Pi Local Token Costs: Quantifying the Savings of Local AI

As the world of Large Language Models (LLMs) shifts toward local model usage, a new question has emerged for power users: "How much am I actually saving by running this model locally?".

While running a model on your own hardware is great for privacy and speed, there hasn't been a good way to quantify the value of that convenience compared to using that same model from something like OpenRouter. I wanted a way to see that comparison in real-time, which is why I built pi-local-token-costs, for my new favorite coding harness.

🎯 The Motivation

For me, the motivation was simple: comparison.

When I use Pi, I love the ability to plug in a local model and just go. But I'm also always wondering if it's worth spending on hardware and power compared to just using the same or better models through online providers. I found myself constantly doing mental math: "Is it worth using a Claude or a Codex right now, or should I stick to my local custom setup?".

I wanted a tool that would take the guesswork out of it—something that would silently track my usage and show me, in real-time, exactly how much I am saving by choosing local compute over API tokens.

✨ Key Features

The extension is designed to be lightweight and non-intrusive, living quietly in the Pi powerline footer, another great extension for Pi.

• Live Savings Comparison: It calculates the estimated cost of your current session if you had used a cloud model (via OpenRouter) versus running it locally.

• OpenRouter Price Awareness: It fetches live pricing from over 30+ providers, always selecting the cheapest rate for a given model to give you a realistic "best-case" comparison.

• Seamless Local Integration: It doesn't care if you're using Ollama, vLLM, LM Studio, or a raw GGUF file in llama.cpp - it's designed to find a match.

• Cross-Session History: Beyond just the current session, you can see your long-term savings trends.

🛫 Technical Deep Dive

The real magic of the extension lies in how it solves the Model Identity Problem.

The Model Matching Challenge

Local model names are messy. One minute you're running google/gemma4-31b with a nicely keyed model name from ollama , the next you're using a full file path for the GGUF on your drive with llama-server. Meanwhile, OpenRouter expects a clean string like qwen/qwen3.6-35b-a3b for comparison.

To bridge this gap, I implemented a Priority Chain for model matching. When Pi reports a model ID, the extension runs through these steps:

11. User config override (exact model ID match)      → ~/.pi/agent/token-costs.json
22. User config override (normalized model ID)        → lowercase, no prefixes
33. Live OpenRouter API (cheapest provider rate)      → Refreshed every 24h
44. Stripping-based matching against live prices     → Progressive strategies
55. Alias rules (regex patterns for common naming)    → Handles Ollama, vLLM, etc.
66. Embedded fallback DB (~60 cloud models)           → Works offline
77. Local model defaults ($0/$0)                      → Unknown models assumed free

Intelligent Stripping & Normalization

One of the more complex parts was the Stripping logic. To match a local model like Qwen3.5-35B-A3B-Instruct-GGUF-Q4_K_M to its OpenRouter equivalent, the extension performs progressive normalization:

11. Prefix Removal: Stripping ollama/ or lm-studio/.
22. Quantization Stripping: Detecting and removing GGUF-specific suffixes like -GGUF or -Q4_K_M.
33. Parameter Awareness: Using the model's parameter count to disambiguate between versions (e.g., 7B vs 13B).

This ensures that even when you are using highly specific local builds, the "savings" calculation remains accurate by mapping you to the closest cloud counterpart.

Architecture & Data Flow

The architecture is built to be highly efficient to avoid impacting the latency of your AI interactions.

11. Data Fetching: At startup, the extension fetches the latest pricing from OpenRouter.
22. Intercepting Usage: It monitors the token usage reported by Pi's runtime.
33. The Pricing Pipeline: It takes the prompt_tokens and completion_tokens, identifies the model via the matching engine, and applies the cheapest provider's rate.
44. Persistence: Results are stored locally, allowing for historical analysis without needing a heavy backend.

🚀 Conclusion

pi-local-token-costs isn't just about saving pennies; it's about visibility. It turns the abstract concept of "local vs. cloud" into a concrete metric. For anyone building workflows with a mix of local and cloud LLMs, it provides the data needed to make informed decisions about privacy, performance, and, most importantly, cost.

If you find it useful, please consider starring the repo!