Choosing an API supplier is much like choosing a database. A team compares a few options, discusses price and features, and may build a prototype. Once the application ships, its integration with the chosen supplier often becomes part of the architecture, even if nobody planned for it to be permanent.
The supplier’s name then spreads through configuration files, error handling, monitoring dashboards, operating procedures, and commercial agreements. Replacing it remains technically possible, but can be costly, especially when the API’s concepts have worked their way into the business logic. A competing service may be cheaper, faster, or more reliable and still fall short of justifying the engineering effort and operational risk of a migration. The incumbent does not have to be the best supplier on the market; it only has to be better than the cost of switching.
An application can structure this dependency differently. Instead of calling one supplier’s API directly, it can define the macrofunctionality it needs and make several suppliers available through adapters that implement the same application-owned contract:
With this structure, the application can choose a supplier when it handles each request, rather than locking in a choice during design. It might send one request to the cheapest qualified provider, another to the one with the lowest current latency, and a third to the only provider allowed to process that customer’s data.
The application can also defer the choice. A lower-cost service might handle routine cases, with uncertain results passed to a more capable and expensive provider. If a service slows down, raises its prices, or runs out of capacity, traffic can be redirected without changing the business logic. Regularly using the alternative implementations also helps ensure the fallback still works when needed.
The integration becomes a portfolio of suppliers, selected according to the needs of each request.
More than a fallback
Hiding an external dependency behind an interface is a familiar practice. Ports and adapters, dependency inversion, the Strategy pattern, and other approaches have used the same basic idea for decades. Systems that rely on a critical external service also often keep a second provider for emergencies.
The key question is whether that alternative stays dormant, appearing mainly in architecture diagrams and used only during serious incidents, or handles enough routine production traffic to remain reliable.
A fallback that receives no traffic is hard to trust. Authentication can expire, suppliers can change their APIs, rarely used code can accumulate defects, and operational knowledge can fade. The fallback may fail just when an outage makes it necessary. Sending a small but steady share of normal work to multiple suppliers keeps their integrations exercised and provides current evidence about cost, latency, failure rates, and, when results can be evaluated automatically, quality.
That evidence allows more flexible routing than a fixed primary and secondary. An OCR supplier might handle low-quality German invoices especially well, while another costs less and works well enough on clean English documents. A translation service might have the best average latency but poor tail latency in one region. A model might excel at one classification task and be unnecessarily expensive for another. If request characteristics help predict which supplier is best, traffic need not be split by fixed percentages.
The router can also use an initial result to decide what to do next. For licence plate recognition, a fast, inexpensive API may correctly read most clean local plates. Sending every image to the most accurate service would waste money, while dirty, damaged, or foreign plates may need a stronger model. The router can call the cheaper supplier first, inspect its output, and escalate cases that do not meet the required confidence level.
That validation step matters. A confidence value from the first supplier can inform the decision, but it may not measure correctness objectively or be calibrated like scores from other services. The application can consider character-level confidence, image quality, agreement across video frames, country recognition, and whether the result matches a known plate format. Even then, a valid-looking plate may be wrong. The acceptance threshold should reflect the consequences: a parking application that allows manual entry can tolerate a different error rate from a system that issues fines or charges tolls.
In a cascade like this, suppliers do not have to be interchangeable. One can handle economical first passes, another can process difficult cases, and a third can specialize in a language, country, or document type. The router can combine them into a service that none provides alone. The relevant calculation includes the cost of the first call, validation, occasional escalation, and expected errors. That total can then be compared with the cost of sending every request to the premium supplier.
Cohere Parse, a vision language model announced in August 2026 for processing enterprise documents, provides an example of how this market may develop. Traditional OCR remains cheap and effective when a page contains clean text, but it frequently loses the structure which makes a document comprehensible, including tables, diagrams, reading order and the relationship between visual elements. A general-purpose frontier vision model can recover much more of this information, although using it for every page can make a large ingestion workload unnecessarily expensive. Parse has been designed to occupy the space between these alternatives, handling document structure with a specialized model which Cohere prices at $1.50 per 1,000 pages through its API, compared with the approximately $10 per 1,000 pages which the company uses for a hyperscaler offering in its cost comparison. According to Cohere’s own ParseBench evaluation, Parse achieved an average score of 79.2, below the frontier models included in the test but above the specialized document parsers and conventional document-intelligence services against which it was compared. These results should be treated as supplier claims rather than as independent benchmarking, but the position which Cohere is targeting is more interesting for this argument than the exact ranking: Parse is not intended to be the cheapest possible OCR or the most capable vision model, but a niche model with a better price-performance trade-off for a defined class of enterprise documents.
A routing policy can turn those three positions into one processing pipeline. Clean, text-dominant pages can remain with conventional OCR, documents containing tables, diagrams or complex layouts can be sent directly to a specialized model, and only the exceptional cases which remain ambiguous, or whose business value justifies the cost, need to reach a frontier model. Alternatively, the decision can be made progressively, with each stage escalating the document when validation indicates that too much structure or meaning has been lost. Cohere itself describes a related optimization inside its Compass ingestion product, which routes documents through text or vision paths to reduce latency and token usage; that is internal routing within one supplier’s platform rather than the multi-supplier portfolio discussed here, but it relies on the same economic principle.
This does not mean that the supplier must be selected for every individual API call, because the correct routing unit depends on where state and accountability are. A document can usually move between OCR services as one self-contained unit, whereas a payment may need to remain with the same processor throughout authorization, capture, refund and dispute. The useful routing unit is the smallest unit of work which can move without breaking state, semantics or accountability.
The obvious objection
Before discussing the potential business consequences, an architect would object that this approach introduces another layer of software, together with all the complexity that the layer is expected to hide but also creates.
Instead of one integration, the team owns several adapters, more credentials, a larger test matrix, additional monitoring and a routing policy whose behaviour must be understood during normal operation as well as during an incident. Two operations which appear identical from the application point of view may have been handled by different suppliers, making defects harder to reproduce, while billing reconciliation, security reviews and compliance assessments multiply. Cascades add another trade-off, because validation and repeated calls consume resources and increase tail latency, while a badly calibrated acceptance threshold either escalates so often that the expected saving disappears or accepts too many plausible but incorrect results. The router also introduces its own failure modes, since it may react to noisy measurements, exhaust a quota, select a degraded provider or shift traffic so frequently that the system becomes less stable than any of its suppliers. A poorly designed supplier portfolio can then cost more, fail in more obscure ways and deliver less value than one direct integration. Progress!
The business case cannot be reduced to the difference between two rate cards, because the possible benefits include supplier savings, avoided losses during outages, improvements in quality or latency and the value of maintaining a credible option to move traffic, whereas the costs include adapter development, platform operation, testing, security and compliance work, commercial management and more complicated incident response. Several of these terms, particularly avoided outage losses and strategic optionality, are very easy to exaggerate, so that almost any platform proposal can be made to look profitable if these assumptions are allowed to do most of the heavy lifting.
For an application which has modest API expenditure and a supplier that performs adequately, capability routing is probably over-engineering, as it is when competing services have similar prices, failures have limited business consequences or results are too subjective to be compared. The case becomes more credible only when the external capability is expensive or business-critical, when suppliers differ in ways that matter, and when the application can define a stable contract behind which those differences can be contained.
Adoption can then be incremental. A team can start with an application-owned abstraction because it improves the separation between business logic and supplier details, then add a second implementation when a concrete resilience or commercial requirement justifies it, exercise that implementation with real traffic, and introduce dynamic allocation only after measurements show that it creates value. A sophisticated optimizer belongs at the end of this progression, if ever.
The application should own the contract
The abstraction should describe what the application needs, rather than reproducing the shape of the API offered by the first supplier, because otherwise the other adapters will merely translate one vendor’s assumptions into the vocabulary of its competitors.
For example:
public interface IDocumentExtractor
{
Task<ExtractionResult> ExtractAsync(
Document document,
ExtractionRequirements requirements,
CancellationToken cancellationToken);
}The interface is the simple part, while the real contract must also define what constitutes a valid extraction, how confidence is represented, which failures can be retried, what cancellation means and which requirements for retention, location and handling of data are mandatory. Unless these semantics are explicit, two adapters may satisfy the type system while returning results which the application cannot safely treat as equivalent. A messaging API, for instance, may distinguish between a message which has been accepted by the provider, one which has been accepted by the carrier and one which has been delivered to the device, but only the consuming application can decide which of these states satisfies its own definition of delivery. An adapter which maps all three states to Delivered will compile, and may even pass tests, but it will remain semantically wrong.
For this reason the development team, or an internal platform team acting on its behalf, is the natural owner of both the capability contract and the adapters which connect it to supplier APIs. Suppliers can provide SDKs, API documents, test environments and reference implementations, all of which reduce the work required, but they cannot own the meaning that their service has inside the consuming application.
This ownership model may change if the capability contract becomes an accepted industry standard, as happens with database vendors which can implement JDBC drivers against a common external specification. For many newer API services, however, the abstraction that creates value will initially remain specific to an application or organization, because the details which determine equivalence are part of its business requirements.
The router is a role, not necessarily a company
Once several suppliers implement the same capability, we can imagine a new category of independent brokers who sit between applications and every API market, maintain a catalogue of adapters and send each request to the winning provider. Such companies may emerge in some domains, especially where integrations and evaluation criteria are sufficiently standardized, but the architecture does not depend on their existence.
For a large organization, the more plausible arrangement is an internal routing platform in which application teams define the requirements of their workloads, while a platform team manages credentials, commercial agreements, qualified adapters, shared telemetry and company-wide constraints. One application may prioritize latency and another accuracy, although neither should be able to override a policy which prevents personal data from leaving the European Union or sends a regulated workload to a supplier which has not been approved. A smaller system can keep the router within the application, as a library and a set of policies, while a managed service may be attractive to teams which do not want to operate this themselves.
Keeping control of the policy inside the customer organization has a further advantage, which is that the information required to make a useful decision is often private. A public benchmark cannot tell a company which provider extracts its own invoices more accurately, performs better in its regions or makes the least expensive mistakes for its business, just as a public price page does not include negotiated discounts, committed spend, prepaid capacity or the cost of reviewing exceptional results manually. The cheapest supplier is the one which delivers the required outcome at the lowest effective cost under the constraints of that particular application.
Why this might work now?
The historical precedents are worth mentioning because the idea is not obvious, and the existence of interfaces and routers was never sufficient by itself. Database drivers separated applications from implementations, UDDI described services and bindings, enterprise service buses routed messages among endpoints, and the software industry has repeatedly attempted to make services discoverable or interchangeable.
These technologies did not create a general market in which applications selected commercial services dynamically, because the missing element was not a routing mechanism: services were rarely equivalent, integrations were long-lived, prices were negotiated through coarse contracts, and many workloads carried enough state to make movement impractical.
Some markets now present a different combination of conditions, since multiple suppliers expose similar API-first capabilities, usage is priced per token, document, message or transaction, applications collect detailed operational data, and at least part of the result can be evaluated automatically. At the same time, many of these operations are sufficiently self-contained that a request, document or job can be assigned without migrating a large body of state.
AI infrastructure is the most visible example, with OpenRouter’s provider-routing documentation describing selection according to price, throughput and latency, together with routing around unavailable providers. Payment orchestration provides another, more stateful example, as Stripe Orchestration places multiple supported processors behind one integration and can attempt a payment through a different processor. These examples do not demonstrate that the same model will spread across every category of software, but they show that, in a few important markets, the conditions which older approaches were missing now exist together.
From integration choice to continuous competition
When traffic can move between qualified suppliers without requiring a new application release, the economics of the relationship change because the incumbent benefits less from the cost of replacing its integration. A price increase can produce a lower allocation of traffic, rather than a migration proposal which waits six months for roadmap capacity, while a new supplier no longer needs to replace the incumbent completely and can instead begin with a limited share of work, demonstrate its performance on real workloads and expand if the evidence supports it.
This is the mechanism through which some software services delivered through APIs may become commodities. LLM inference, transcription, translation, OCR and image moderation are obvious candidates because they combine several suppliers, granular pricing and relatively self-contained requests, whereas databases and business systems remain at the other end of the spectrum, where accumulated state, different data models and deeply different behaviours make changes expensive even when an interface can be defined.
Commoditization would not necessarily produce a race to the absolute lowest price, since a supplier could preserve margins through higher quality, lower tail latency, scarce capacity, regulatory approval, proprietary data or unusually strong performance on a narrow category of work. The router can then divide the market into commodity, premium and specialist suppliers, selecting among them according to the requirements and economic value of each request rather than treating all implementations as identical; in a cascade, the inexpensive provider may receive most of the volume while the premium provider retains a smaller but economically valuable share composed of the hardest cases.
Also, customers might not capture all the resulting savings. If independent brokers came to dominate a capability market, suppliers could find themselves competing for inclusion in a relatively small number of routing platforms, which would control demand, performance data and, potentially, the definition of the capability against which suppliers are evaluated. At that point the list of supported adapters would start to resemble the index of a search engine, because inclusion would determine whether a new supplier can be discovered and evaluated, while ranking and qualification rules would determine how much traffic it receives. In fact, a broker which also executes the routing decision could become more powerful than a search engine, since it would not only influence the customer’s choice but send work and revenue directly to the selected supplier. This centralized outcome is possible, but it is different with the architectural proposal, where organizations maintain private supplier portfolios, obtain adapters from different sources and retain final control of the routing policy.
The adapter problem
The most immediate practical limit is still the cost of supporting several suppliers, because every adapter requires development, security review, testing, monitoring and maintenance, and no organization will integrate dozens of services only to create theoretical competition among them.
Better tooling can reduce this cost, with conventional code generation already handling client models and protocol plumbing, while GenAI can compare an application contract with supplier documentation, draft transformations, map error conditions and propose conformance tests. When a supplier changes an API, the same tools may identify affected mappings and prepare an update, useful as adapter maintenance is continuous work rather than a one-time implementation expense.
Capability routing is valuable, without any GenAI assistance, whenever resilience, cost or performance already justifies maintaining more than one supplier; but automated generation simply lowers the threshold at which the second or third adapter becomes economical.
Generated adapters must also pass the same qualification process as handwritten ones, and a safe process should begin with a compatibility report rather than with code. Semantic gaps have to be identified and resolved explicitly, after which the adapter can be subjected to contract tests, a supplier sandbox, shadow traffic and a controlled production canary before receiving an ordinary share of the workload. The difficult problems are rarely the names or formats of fields in the API calls, but questions such as whether a promise of no data retention is enforced, or whether confidence scores returned by two suppliers represent comparable results. GenAI can reveal and document these questions, but it cannot decide their business meaning on behalf of the application owner.
Some dependencies are hidden
Using three supplier names does not necessarily create three independent failure domains, since those suppliers may operate in the same cloud region, depend on the same network, use the same underlying model or obtain critical data from the same source. A routing layer which claims to manage concentration risk must understand these dependencies, otherwise it may distribute traffic across several brands while preserving exactly the risk exposure it was intended to mitigate.
The routing layer also becomes critical infrastructure, which means that its decisions must be reproducible and that logs must retain the selected supplier, adapter version, policy version, retry history and relevant routing inputs.
Wrapping up
Capability routing is not a new universal architecture for external APIs, because most applications should not build a brokerage platform and many integrations will remain fixed for solid technical or economic reasons. The proposal is that, in markets where several suppliers are genuinely comparable, usage is priced granularly, outcomes can be measured and limited state has to move, applications will increasingly maintain more than one qualified implementation, so that supplier choice becomes an operating policy instead of a permanent architectural decision.
Where switching friction currently protects undifferentiated providers, this change will place pressure on margins, while rewarding suppliers which are easy to integrate, easy to evaluate and demonstrably better for particular workloads. In a few domains it may also create powerful adapter registries or commercial brokers which control access to demand, although customer-owned routing remains more likely and, for organizations with enough scale, the strategical alternative.
The router itself is not the interesting invention, since software has routed work among implementations for a long time. The interesting change would be the market which emerges when an application can treat an external service as a portfolio of competing implementations, continuously decide where its work should go, and change that decision without first rewriting itself.
