Imagine a school where several people can help with homework. A classmate can answer a spelling question. A math teacher can solve an equation. A university professor may handle a problem that neither of them understands.
Sending every question to the professor might produce good answers. It would also waste time and money. The difficult part is deciding who should receive each question.
AI systems now face the same problem. They can choose among small models, large reasoning models, search tools, medical databases, and other specialists. This process is called model routing.
The cost comes not only from the model that produces the answer, but also from the work required to choose that model.
The Hidden Decision Cost
A router must estimate how well each model will answer a prompt. A quick estimate might compare the prompt with similar questions from the past. This is cheap, but it can be wrong.
A stronger estimate may ask another model to study the prompt, search documents, or examine the beginning of a possible solution. That can improve the routing decision, but it uses time, money, and computer power.
The router therefore faces two questions:
- Which model is most likely to give the best answer?
- Is it worth paying for more information before making that choice?
The second question is the focus of Pandora's AI Model Routing Box, a Google DeepMind research paper posted as a preprint in August 2026.
Pandora's Box for AI Routing
The researchers borrow an idea from economist Martin Weitzman. In the 1979 Pandora's Box problem, a person sees several closed boxes. Each box may contain a valuable prize, but opening it costs money. The person must decide which boxes to inspect and when to stop searching.
Opening every box provides more information, but the inspection costs may exceed the value of finding a better prize. Opening none saves money, but risks making a poor choice.
The solution assigns each box a reservation price. This is a cutoff point that helps determine whether the possible benefit of opening the box is worth the cost. The most promising boxes are inspected first. The search stops when another inspection is unlikely to improve the final choice enough to justify its price.
In the DeepMind paper, each AI specialist is treated as a box. A cheap estimate gives the router an early guess. Paying for a better estimate is like opening the box.
Uncertainty matters as much as the initial score. Suppose two models receive almost equal scores. That does not automatically mean both should be inspected. If one model's cheap score is usually reliable, checking it again may add little value.
A third model may appear slightly weaker but have a much less reliable score. Inspecting it could reveal that it is much better or much worse than expected. Pandora's Router considers both the estimated value and the uncertainty around it. It does not simply inspect whichever scores are closest.
The router can also select a model without inspecting it. If the cheap information already supports a clear choice, it avoids the extra work.
What the Experiments Show
The researchers tested the method in three settings. One involved 16,512 math problems and a choice between a smaller model and a more expensive reasoning model. Another compared a plain model with specialists that searched Wikipedia or PubMed. The third involved more than 100 open-weight language models.
Across the paper's results, averaged over the tested inspection-cost levels, Pandora's Router achieved the lowest combined score for routing regret plus inspection cost in all three domains, tying the Margin baseline in the Math setting. Lower is better.
Routing regret measures how much value was lost by not selecting the best specialist in hindsight.
In the RAG experiment, Pandora's Router scored 0.118. Using only cheap estimates scored 0.150, while always buying expensive estimates scored 0.141.
The large model-selection experiment makes the cost of inspecting too many candidates especially clear. Inspecting every candidate produced a combined score of 2.356 because the inspection cost became overwhelming. Pandora's Router reduced it to 0.386 by buying better information selectively.
These are experimental scores, not direct dollar savings. The important result is that the router adapted its behavior to the price of information. It inspected frequently when better estimates were cheap and rarely when they became expensive.
The paper also extends the idea to a marketplace where AI specialists compete for work. Pandora's Router assumes that one central system controls the decision. Pandora's Bidder examines a decentralized setting in which a specialist can accept a request, reject it, or pay for a better self-assessment before deciding.
In the experiment, one specialist acts strategically while the others submit estimates without changing their behavior. This setup helps the researchers isolate the effects of the strategic specialist's choices.
A self-assessment may be useful when a specialist has information the central router cannot see, such as access to a private database or internal performance results.
But it also creates a conflict. When competing estimates are weak, the strategic bidder can improve its own payoff while the overall assignment becomes less efficient. What benefits one participant may not benefit the customer or the marketplace.
The paper is not a description of any current commercial router. Instead, it provides a framework for asking whether a routing decision is worth the cost of improving the information behind it.
What Commercial Routers Actually Do
OpenRouter, Microsoft Foundry, Amazon Bedrock, and Google Cloud all use the word "routing," but they solve different problems.
| Service | How it routes | Direct decision cost |
|---|---|---|
| OpenRouter Auto Router | Task classification and recent market-spending signals | No separate Auto Router fee |
| Microsoft Foundry | A trained router using prompt analysis and selected quality and cost preferences | Router input prompts are billed |
| Amazon Bedrock | Prompt analysis across a supported two-model pair | $1 per 1,000 routing requests |
| Google Cloud API Gateway | Declared rules and the model name in the request | Normal gateway pricing applies |
OpenRouter follows market behavior. Its Auto Router places each prompt into one of roughly 30 task types. It then examines aggregate, anonymized spending patterns from the previous seven days to see which models people use for similar work. It also considers model capabilities, tool support, output requirements, account restrictions, and the selected cost tier.
Users pay the OpenRouter-listed rate of the selected model, with no separate Auto Router surcharge. OpenRouter is not entirely free, however. As of August 23, 2026, non-crypto credit purchases carry a 5.5% fee with a minimum of $0.80. Crypto purchases carry a flat 5% fee with no minimum.
OpenRouter says it passes through underlying provider prices without adding a markup to inference pricing.
Its method differs from Pandora's Router. OpenRouter uses task categories and market-spending signals. Pandora's Router asks whether paying for more information about a specific specialist is worthwhile.
This also helps explain OpenRouter's strategic position. Its value comes not from owning the models, but from simplifying model selection, access, billing, and provider management.
Microsoft Foundry uses a trained routing model. It examines each prompt and selects among eligible models. Customers can choose Balanced, Cost, or Quality mode.
Balanced mode searches for a cost-effective model within a narrow quality range. Cost mode accepts a wider quality trade-off to reduce spending. Quality mode selects the highest-rated model without prioritizing price.
Microsoft's documentation states that Model Router usage is charged for input prompts at the rate on its pricing page. The total cost therefore includes the router's input-prompt charge and the token usage of the selected model.
Organizations should confirm current router and underlying-model rates for their Azure region, deployment type, and commercial agreement.
Amazon Bedrock offers a narrower form of intelligent routing. In the currently documented configuration, a prompt router uses a supported two-model pair from the same family. Supported families include Amazon Nova, Anthropic Claude, and Meta Llama.
Bedrock analyzes the prompt and routes it to the model that it predicts can meet the requested quality level at the lowest cost. AWS charges $1 per 1,000 routing requests, in addition to the selected model's usage.
That fee is a real-world example of a routing decision having its own cost. It is analogous to the paper's decision cost, but it is not identical to Pandora-style inspection. AWS does not publicly describe its internal process as selectively paying to inspect individual models.
The same economic test still applies: the router creates value only when its choices save more money or produce better results than the routing service costs.
Google Cloud currently provides rules-based routing. Its API Gateway model-routing feature is in public preview. Developers define model names, destinations, routing rules, and a default model in a configuration file.
The gateway provides one OpenAI-compatible endpoint for configured Gemini, Claude, and OpenAI models. During the preview, routing is based on the model name supplied in the request. It does not assess prompt difficulty, expected answer quality, or the value of gathering more information.
API Gateway calls, the selected model's inference usage, data transfer, and related platform services may all contribute to the final bill, depending on usage and applicable free allowances.
The FinOps Lesson
The word "free" can be misleading. A routing product may charge per request, charge for router input, or include routing inside a broader gateway service. Even when there is no visible routing fee, the selected model still consumes resources.
A router can also increase total costs through its choices. Sending too many prompts to powerful models raises spending. Sending difficult prompts to models that are too weak can create poor answers, delays, retries, and human correction.
Total cost of a useful answer = model usage + routing and evaluation + delays + failed answers + retries + human rework
Some of these costs appear directly on a cloud invoice. Others appear as engineering time, slower user workflows, support tickets, failed automations, and manual review.
A model router should be evaluated against this complete cost, not only against the price of the selected model or whether routing has its own line on the invoice.
That is the paper's practical lesson. Do not pay for more information unless it is likely to improve the final model choice. Do not assume a routing system saves money simply because its routing layer is advertised as free.
References
Research
- Google DeepMind — Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation, preprint posted August 20, 2026
Product Documentation and Pricing
- OpenRouter — Auto Router documentation, accessed August 23, 2026
- OpenRouter — Pricing and fees, accessed August 23, 2026
- Microsoft Learn — Model router for Microsoft Foundry, accessed August 23, 2026
- Microsoft Azure — Foundry Model Router pricing, accessed August 23, 2026
- Amazon Web Services — Intelligent Prompt Routing documentation, accessed August 23, 2026
- Amazon Web Services — Amazon Bedrock pricing, accessed August 23, 2026
- Google Cloud — API Gateway model-routing overview, accessed August 23, 2026
- Google Cloud — API Gateway pricing, accessed August 23, 2026