Everyone quotes the price of AI in dollars per million tokens. That number almost never answers the question that actually matters: what will this workload cost to run, and is there a cheaper way to run it without giving up anything I care about? I built the AI Inference Cost Optimizer to answer that — and, more importantly, to show its work.
Not a calculator — an advisor. Most open-source cost tools are one of two things: a price library that looks up $/token, or a runtime gateway that routes live traffic. This is a third thing — an analysis tool. You describe a workload, and it recommends the cheapest viable model and provider, then hands you an auditable trace of every rule and lever that led there. The output is a decision you can defend in a review, not a single blended rate you have to take on faith.
Three objectives, no universal answer. Every inference decision balances accuracy, latency, and cost, and you cannot maximize all three. Higher-tier models cost more per token. Real-time responses lock out the batch discounts that are typically half price. The cheapest model may not clear your quality bar or your compliance rules. There is no universal formula — so instead of pretending there is one, the tool makes the trade-offs explicit and lets you navigate them on purpose.
Describe the workload, walk the rule tree. You characterize a job along seven small dimensions — task type, latency, reuse pattern, context size, quality bar, compliance need, throughput pattern — plus a monthly volume estimate. A deterministic rule tree then does the work: it applies stacking cost levers (batch, caching, reasoning effort, right-sizing), filters by compliance and data residency, and ranks all sixteen curated models across five providers by effective monthly cost. What comes back is a recommended path with a one-line "why cheaper," the savings against a frontier baseline in both dollars and percent, an itemized list of the levers used, the full decision trace in the order the rules fired, and the next five eligible candidates so you see the trade space instead of just the verdict.
How much does routing actually save? A second, independent calculator isolates a single question that vendors love to blur: how much does a model-routing layer really cut the bill? It uses list prices only — no caching, batching, or effort levers — so any saving shown is attributable to routing alone. It prices one shared workload across the native in-cloud routers (AWS Bedrock, Azure AI Foundry) and a configurable cross-provider gateway, and it is honest enough to flip its headline to "costs more" when a difficult-heavy request mix makes routing uneconomic. Routing is not free money, and the tool refuses to pretend otherwise.
The honest part: self-hosting is not a per-token price. This is the section I am proudest of, because it is the one every other calculator gets wrong. Running your own open-weight model is fixed GPU capacity, not a rate — so the honest answer is a break-even, not a flat number. The tab plots three lines side by side: paying the API, renting the GPU, and buying it (purchase price amortized over a useful life, plus power drawn at your PUE, plus hosting). Below the break-even the API wins, because your own card would sit idle. Above it, self-hosting wins. And owning only beats renting once you keep the hardware busy, because an idle owned GPU is sunk capital. It even provisions a 70-billion-parameter model as tensor-parallel across two cards, so its break-even reflects two GPUs rather than an optimistic single one.
Honesty by construction. Every price is a dated, sourced snapshot — each one mapped to a provider document with a date. Every throughput figure carries a measured-versus-estimated provenance tag, so you always know whether a number was observed or interpolated. And the math ships with a test suite — eleven advisor assertions plus fifty self-host invariants — so the engine cannot quietly drift from what it claims on screen. It is a decision assistant, not an oracle: it gives directional guidance and then tells you, in writing, to validate on your own traffic before you commit.
Why it matters. Cost discipline is a behavior, not a dashboard — which is exactly the conviction behind my write-up on the FinOps operating model: FinOps fails when ownership is unclear and succeeds when decisions are made deliberately. This tool is that discipline applied to the single most volatile line in an AI budget — the next workload — so you can route it cheaply from the start instead of diagnosing the bill after the money is already spent.
The whole thing is one self-contained HTML file with no build step and no dependencies, so the pricing, the rules, and the tests all sit in the open where anyone can read them.
Project links