Choosing the supplier of an API sis similar to choosing a database: a development team compares a few candidates, discusses prices and features, possibly builds a prototype, and then writes one integration which, once the application has been released, becomes part of the architecture even if nobody originally intended it to be permanent.
The name of the supplier starts appearing in configuration files, error-handling code, monitoring dashboards, operational procedures and commercial agreements, so that replacing it remains technically possible but is rarely cheap, particularly when the API exposes concepts that have gradually leaked into the business logic. As a consequence, a competing service may become less expensive, faster or even more reliable, while still not being sufficiently better to justify the engineering effort and operational risk of a migration. The incumbent, therefore, does not need to remain the best supplier available on the market; it only needs to remain better than the cost of moving away from it.
There is, however, another way to structure this dependency, in which the application does not call the API of a specific supplier directly, but defines the macrofunctionality that it needs and makes different suppliers available through adapters that implement the same application-owned contract
With this structure the supplier can be selected when the work is performed, rather than when the application is designed, which means that one request may be sent to the cheapest qualified provider, another to the provider that is currently producing the lowest latency, and a third to the only provider which is allowed to process the data of that customer. The decision does not even need to be final when the first supplier is called, since a cheap service can process the ordinary cases while the application escalates the minority of uncertain results to a more capable and expensive alternative. If one service degrades, increases its prices or exhausts its capacity, traffic can be moved without modifying the business logic and, assuming that the alternative implementations have been used regularly, without discovering during the incident that the fallback no longer works.
An API integration begins, in this way, to look less like a fixed dependency and more like a portfolio of suppliers.
More than a fallback
There is nothing new in hiding an external dependency behind an interface, since ports and adapters, dependency inversion, the Strategy pattern and many other variations of the same basic idea have been used for decades, while systems which depend on a critical external service frequently maintain a secondary provider for emergencies.
The relevant difference is whether the alternative implementation remains a dormant escape route, which exists mainly in an architecture diagram and is invoked only during a serious incident, or whether it becomes an active part of the production system and receives enough ordinary traffic to remain qualified.
A fallback which receives no traffic is difficult to trust because authentication expires, suppliers change the behaviour of their APIs, rarely executed code accumulates defects and operational knowledge disappears, with the predictable result that the problem is discovered during the outage which the fallback was supposed to mitigate. Sending a limited but continuous share of normal work to more than one supplier not only keeps the integrations exercised, but also generates current evidence about cost, latency, failure rate and, if the results can be evaluated, quality.
Once this evidence exists, routing can become more interesting than a fixed primary-secondary arrangement. An OCR supplier, for example, may perform particularly well on low-quality invoices in German, while another may be cheaper and perfectly adequate for clean documents in English; a translation service may offer the best average latency but a poor tail latency in one region, and a model which is excellent for one classification task may be unnecessarily expensive for another. The portfolio does not need to be divided through fixed percentages when the characteristics of the request can be used to make a better decision.
A second possibility is to make the initial result part of the routing decision. Consider an OCR service used to read licence plates: a fast and inexpensive API may correctly recognize the large majority of clean local plates, in which case sending every image directly to the most accurate service would waste money, while a dirty, damaged or foreign plate may require a better model. The router can first call the cheaper supplier, inspect the result and escalate only when the output is not sufficiently trustworthy.
The validation step is what makes this useful. A confidence value returned by the first supplier can contribute to the decision, but it should not be accepted as an objective measurement of correctness, particularly because confidence scores from different services are not necessarily calibrated in the same way. The application can combine character-level confidence with image quality, agreement across several video frames, recognition of the country and compatibility with a known plate format; even then, a syntactically valid plate can still be wrong, which means that the acceptance threshold must reflect the consequence of the error. A parking application, where a failed read leads to manual entry, can tolerate a different threshold from a system which issues fines or charges tolls.
This type of cascade also shows that the suppliers in a portfolio do not have to be interchangeable in the strictest sense. One can be the economical first pass, another the premium service used for difficult cases, and a third a specialist for a particular language, country or document type, while the router composes them into a service which none of them provides alone. The relevant calculation is no longer simply which API has the lowest unit price, but whether the cost of the first call, validation and occasional escalation remains below the cost of using the premium supplier for every request, after the expected cost of wrong results has also been included.
Cohere Parse, a vision language model announced in August 2026 for processing enterprise documents, provides an example of how this market may develop. Traditional OCR remains inexpensive and effective when a page contains clean text, but it frequently loses the structure which makes a document comprehensible, including tables, diagrams, reading order and the relationship between visual elements; a general-purpose frontier vision model can recover much more of this information, although using it for every page can make a large ingestion workload unnecessarily expensive. Parse has been designed to occupy the space between these alternatives, handling document structure with a specialized model which Cohere prices at $1.50 per 1,000 pages through its API, compared with the approximately $10 per 1,000 pages which the company uses for a hyperscaler offering in its cost comparison. According to Cohere’s own ParseBench evaluation, Parse achieved an average score of 79.2, below the frontier models included in the test but above the specialized document parsers and conventional document-intelligence services against which it was compared. These results should be treated as supplier claims rather than as independent benchmarking, but the position which Cohere is targeting is more interesting for this argument than the exact ranking: Parse is not intended to be the cheapest possible OCR or the most capable vision model, but a niche model with a better price-performance trade-off for a defined class of enterprise documents.
A routing policy can turn those three positions into one processing pipeline. Clean, text-dominant pages can remain with conventional OCR, documents containing tables, diagrams or complex layouts can be sent directly to a specialized model such as Parse, and only the exceptional cases which remain ambiguous, or whose business value justifies the cost, need to reach a frontier model. Alternatively, the decision can be made progressively, with each stage escalating the document when validation indicates that too much structure or meaning has been lost. Cohere itself describes a related optimization inside its Compass ingestion product, which routes documents through text or vision paths to reduce latency and token usage; that is internal routing within one supplier’s platform rather than the multi-supplier portfolio discussed here, but it relies on the same economic principle.
This does not mean that the supplier must be selected for every individual API call, because the correct routing unit depends on where state and accountability are. A document can usually move between OCR services as one self-contained unit, whereas a payment may need to remain with the same processor throughout authorization, capture, refund and dispute; messaging may require affinity at sender or campaign level because reputation is not portable, while storage will normally be selected for a dataset rather than for a single read operation. The useful routing unit is the smallest unit of work which can move without breaking state, semantics or accountability.
The obvious objection
Before discussing the potential business consequences, a sensible architect should raise the most immediate objection, which is that this approach introduces another layer of software, together with all the complexity that the layer is both expected to hide but also creates.
Instead of one integration, the team owns several adapters, more credentials, a larger test matrix, additional monitoring and a routing policy whose behaviour must be understood during normal operation as well as during an incident. Two operations which appear identical from the application point of view may have been handled by different suppliers, making defects harder to reproduce, while billing reconciliation, security reviews and compliance assessments multiply. Cascades add another trade-off, because validation and repeated calls consume resources and increase tail latency, while a badly calibrated acceptance threshold either escalates so often that the expected saving disappears or accepts too many plausible but incorrect results. The router also introduces its own failure modes, since it may react to noisy measurements, exhaust a quota, select a degraded provider or shift traffic so frequently that the system becomes less stable than any of its suppliers. A poorly designed supplier portfolio can therefore cost more, fail in more obscure ways and deliver less value than one direct integration.
The business case cannot be reduced to the difference between two public rate cards, because the possible benefits include supplier savings, avoided losses during outages, improvements in quality or latency and the value of maintaining a credible option to move traffic, whereas the costs include adapter development, platform operation, testing, security and compliance work, commercial management and more complicated incident response. Several of these terms, particularly avoided outage losses and strategic optionality, are very easy to exaggerate, so that almost any platform proposal can be made to look profitable if these assumptions are allowed to do most of the heavy lifting.
For an application which has modest API expenditure and a supplier that performs adequately, capability routing is probably overengineering, as it is when competing services have similar prices, failures have limited business consequences or results are too subjective to be compared. The case becomes more credible only when the external capability is expensive or business-critical, when suppliers differ in ways that matter, and when the application can define a stable contract behind which those differences can be contained.
Adoption should then be incremental. A team can start with an application-owned abstraction because it improves the separation between business logic and supplier details, then add a second implementation when a concrete resilience or commercial requirement justifies it, exercise that implementation with real traffic, and introduce dynamic allocation only after measurements show that it creates value. A sophisticated optimizer belongs at the end of this progression, if ever.
The application should own the contract
The abstraction should describe what the application needs, rather than reproducing the shape of the API offered by the first supplier, because otherwise the other adapters will merely translate one vendor’s assumptions into the vocabulary of its competitors.
For example:
public interface IDocumentExtractor
{
Task<ExtractionResult> ExtractAsync(
Document document,
ExtractionRequirements requirements,
CancellationToken cancellationToken);
}The interface is the simple part, while the real contract must also define what constitutes a valid extraction, how confidence is represented, which failures can be retried, what cancellation means and which requirements for retention, location and handling of data are mandatory. Unless these semantics are explicit, two adapters may satisfy the type system while returning results which the application cannot safely treat as equivalent. A messaging API, for instance, may distinguish between a message which has been accepted by the provider, one which has been accepted by the carrier and one which has been delivered to the device, but only the consuming application can decide which of these states satisfies its own definition of delivery. An adapter which maps all three states to Delivered will compile, and may even pass tests, but it will remain semantically wrong.
For this reason the development team, or an internal platform team acting on its behalf, is the natural owner of both the capability contract and the adapters which connect it to supplier APIs. Suppliers can provide SDKs, API documents, test environments and reference implementations, all of which reduce the work required, but they cannot own the meaning that their service has inside the consuming application.
This ownership model may change if the capability contract becomes an accepted industry standard, as happens with database vendors which can implement JDBC drivers against a common external specification. For many newer API services, however, the abstraction that creates value will initially remain specific to an application or organization, because the details which determine equivalence are part of its business requirements.
The router is a role, not necessarily a company
Once several suppliers implement the same capability, we can imagine a new category of independent brokers who sit between applications and every API market, maintain a catalogue of adapters and send each request to the winning provider. Such companies may emerge in some domains, especially where integrations and evaluation criteria are sufficiently standardized, but the architecture does not depend on their existence.
For a large organization, the more plausible arrangement is an internal routing platform in which application teams define the requirements of their workloads, while a platform team manages credentials, commercial agreements, qualified adapters, shared telemetry and company-wide constraints. One application may prioritize latency and another accuracy, although neither should be able to override a policy which prevents personal data from leaving the European Union or sends a regulated workload to a supplier which has not been approved. A smaller system can keep the router within the application, as a library and a set of policies, while a managed service may be attractive to teams which do not want to operate this themselves.
Keeping control of the policy inside the customer organization has a further advantage, which is that the information required to make a useful decision is often private. A public benchmark cannot tell a company which provider extracts its own invoices more accurately, performs better in its regions or makes the least expensive mistakes for its business, just as a public price page does not include negotiated discounts, committed spend, prepaid capacity or the cost of reviewing exceptional results manually. The cheapest supplier is the one which delivers the required outcome at the lowest effective cost under the constraints of that particular application.
Why this might work now?
The historical precedents are worth mentioning because the idea is not obvious, and the existence of interfaces and routers was never sufficient by itself. Database drivers separated applications from implementations, UDDI described services and bindings, enterprise service buses routed messages among endpoints, and the software industry has repeatedly attempted to make services discoverable or interchangeable.
These technologies did not create a general market in which applications selected commercial services dynamically, because the missing element was not a routing mechanism: services were rarely equivalent, integrations were long-lived, prices were negotiated through coarse contracts, and many workloads carried enough state to make movement impractical.
Some markets now present a different combination of conditions, since multiple suppliers expose similar API-first capabilities, usage is priced per token, document, message or transaction, applications collect detailed operational data, and at least part of the result can be evaluated automatically. At the same time, many of these operations are sufficiently self-contained that a request, document or job can be assigned without migrating a large body of state.
AI infrastructure is the most visible example, with OpenRouter’s provider-routing documentation describing selection according to price, throughput and latency, together with routing around unavailable providers. Payment orchestration provides another, more stateful example, as Stripe Orchestration places multiple supported processors behind one integration and can attempt a payment through a different processor. These examples do not demonstrate that the same model will spread across every category of software, but they show that, in a few important markets, the conditions which older approaches were missing now exist together.
From integration choice to continuous competition
When traffic can move between qualified suppliers without requiring a new application release, the economics of the relationship change because the incumbent benefits less from the cost of replacing its integration. A price increase can produce a lower allocation of traffic, rather than a migration proposal which waits six months for roadmap capacity, while a new supplier no longer needs to replace the incumbent completely and can instead begin with a limited share of work, demonstrate its performance on real workloads and expand if the evidence supports it.
This is the mechanism through which some software services delivered through APIs may become commodities. LLM inference, transcription, translation, OCR and image moderation are obvious candidates because they combine several suppliers, granular pricing and relatively self-contained requests, whereas databases and business systems remain at the other end of the spectrum, where accumulated state, different data models and deeply different behaviours make changes expensive even when an interface can be defined.
Commoditization would not necessarily produce a race to the absolute lowest price, since a supplier could preserve margins through higher quality, lower tail latency, scarce capacity, regulatory approval, proprietary data or unusually strong performance on a narrow category of work. The router can then divide the market into commodity, premium and specialist suppliers, selecting among them according to the requirements and economic value of each request rather than treating all implementations as identical; in a cascade, the inexpensive provider may receive most of the volume while the premium provider retains a smaller but economically valuable share composed of the hardest cases.
Also, customers might not capture all the resulting savings. If independent brokers came to dominate a capability market, suppliers could find themselves competing for inclusion in a relatively small number of routing platforms, which would control demand, performance data and, potentially, the definition of the capability against which suppliers are evaluated. At that point the list of supported adapters would start to resemble the index of a search engine, because inclusion would determine whether a new supplier can be discovered and evaluated, while ranking and qualification rules would determine how much traffic it receives. In fact, a broker which also executes the routing decision could become more powerful than a search engine, since it would not only influence the customer’s choice but send work and revenue directly to the selected supplier. This centralized outcome is possible, but it is different with the architectural proposal, where organizations maintain private supplier portfolios, obtain adapters from different sources and retain final control of the routing policy.
The adapter problem
The most immediate practical limit is still the cost of supporting several suppliers, because every adapter requires development, security review, testing, monitoring and maintenance, and no organization will integrate dozens of services only to create theoretical competition among them.
Better tooling can reduce this cost, with conventional code generation already handling client models and protocol plumbing, while GenAI can compare an application contract with supplier documentation, draft transformations, map error conditions and propose conformance tests. When a supplier changes an API, the same tools may identify affected mappings and prepare an update, useful as adapter maintenance is continuous work rather than a one-time implementation expense.
Capability routing is valuable, without any GenAI assistance, whenever resilience, cost or performance already justifies maintaining more than one supplier; but automated generation simply lowers the threshold at which the second or third adapter becomes economical.
Generated adapters must also pass the same qualification process as handwritten ones, and a safe process should begin with a compatibility report rather than with code. Semantic gaps have to be identified and resolved explicitly, after which the adapter can be subjected to contract tests, a supplier sandbox, shadow traffic and a controlled production canary before receiving an ordinary share of the workload. The difficult problems are rarely the names or formats of fields in the API calls, but questions such as whether a promise of no data retention is enforced, or whether confidence scores returned by two suppliers represent comparable results. GenAI can reveal and document these questions, but it cannot decide their business meaning on behalf of the application owner.
Some dependencies are hidden
Using three supplier names does not necessarily create three independent failure domains, since those suppliers may operate in the same cloud region, depend on the same network, use the same underlying model or obtain critical data from the same source. A routing layer which claims to manage concentration risk must understand these dependencies, otherwise it may distribute traffic across several brands while preserving exactly the risk exposure it was intended to mitigate.
The routing layer also becomes critical infrastructure, which means that its decisions must be reproducible and that logs must retain the selected supplier, adapter version, policy version, retry history and relevant routing inputs.
Conclusions
Capability routing is not a new universal architecture for external APIs, because most applications should not build a brokerage platform and many integrations will remain fixed for solid technical or economic reasons. The proposal is that, in markets where several suppliers are genuinely comparable, usage is priced granularly, outcomes can be measured and limited state has to move, applications will increasingly maintain more than one qualified implementation, so that supplier choice becomes an operating policy instead of a permanent architectural decision.
Where switching friction currently protects undifferentiated providers, this change will place pressure on margins, while rewarding suppliers which are easy to integrate, easy to evaluate and demonstrably better for particular workloads. In a few domains it may also create powerful adapter registries or commercial brokers which control access to demand, although customer-owned routing remains more likely and, for organizations with enough scale, the strategical alternative.
The router itself is not the interesting invention, since software has routed work among implementations for a long time. The interesting change would be the market which emerges when an application can treat an external service as a portfolio of competing implementations, continuously decide where its work should go, and change that decision without first rewriting itself.
