The popular LLM API options fit into four categories: direct provider APIs, unified API layers or aggregators, API gateways, and self-hosted or open-model endpoints. Direct providers are the simplest way to call one vendor’s models. Unified APIs put multiple models behind one interface. Gateways add routing and policy controls to endpoints you already use. Self-hosted endpoints give you the most control, but also the most infrastructure work.
The right choice depends on the problem you are solving. A team prototyping with several models may value a unified API. A platform team standardizing observability may need a gateway. A regulated workload or custom open-weight model may justify self-hosting. Treat this as a taxonomy first, then compare vendors within the category that fits.
The four LLM API option categories at a glance
| Category | How it works | Usually fits |
|---|---|---|
| Direct provider API | Your application calls the model creator’s hosted endpoint | A specific proprietary or frontier model is a hard requirement |
| Unified API / aggregator | One API exposes models from several providers or model families | You need model choice without maintaining many integrations |
| API gateway | Middleware routes and governs requests to endpoints you configure | You need fallbacks, observability, rate limits, or policy controls |
| Self-hosted / open-model endpoint | You deploy and serve model weights on infrastructure you control | You need data-path control, custom serving, or predictable high-volume capacity |
These categories can be combined. For example, a gateway can sit in front of direct providers and a unified API, while a self-hosted endpoint handles a sensitive workload. Novita AI spans the unified API and open-model infrastructure categories through its LLM API, Agent Sandbox, and GPU Cloud products.
Category 1: Direct provider APIs
A direct provider API is the model creator’s own hosted interface. OpenAI’s Chat Completions API, Anthropic’s Messages API, and Google’s Gemini API illustrate the pattern. The provider controls the model release, endpoint behavior, pricing, rate limits, and data-handling terms.
Choose this category when: your product depends on a particular provider’s model or features, and the provider’s terms and operational behavior meet your requirements.
| Dimension | Direct provider API |
|---|---|
| Cost | Usage pricing is set by the provider; compare input, output, cached-input, and other billed units where applicable |
| Control | You select from that provider’s models and supported parameters |
| Ops complexity | Low for one integration; higher when each provider has its own SDK, auth, and response conventions |
| Latency | Depends on the provider’s serving region, queueing, model, request size, and network path |
| Compliance | Review the provider’s retention, residency, subprocessors, and contractual terms for your workload |
The main drawback is dependency. A thin adapter around the provider client can keep application code portable if pricing, model names, or endpoint behavior changes. Do not assume that two providers’ similarly named parameters or tool-calling formats behave identically.
Category 2: Unified API layers and aggregators
A unified API layer presents one interface over multiple models. The platform may host the models, maintain provider relationships, or expose a catalog of model endpoints. Your application sends requests to one base URL and uses one account, while the platform handles model-specific access behind that interface.
Novita AI fits this category through its LLM API and OpenAI-compatible chat completions endpoint. Its model catalog is the source to check for current model availability; a model mentioned in a blog post should not be treated as a promise of current access.
Choose this category when: you are evaluating several models, want to reduce integration overhead, or prefer one API contract and billing relationship for a multi-model application.
| Dimension | Unified API / aggregator |
|---|---|
| Cost | Check the platform’s current pricing and any markup, minimums, or model-specific charges |
| Control | You choose among the platform’s supported models; underlying infrastructure controls remain limited |
| Ops complexity | Lower than maintaining every provider integration yourself, but you still own application routing and quality checks |
| Latency | Depends on the selected model, platform queueing, region, and any provider hop |
| Compliance | Evaluate the platform’s data handling and the policies of any underlying provider relationship |
OpenAI compatibility can reduce migration work, but it is not behavioral equivalence. Check context limits, structured output, tool calling, streaming, errors, and model-specific request fields before switching production traffic.
Category 3: API gateways
An API gateway is middleware between your application and one or more LLM endpoints. Tools such as LiteLLM and Portkey represent this category. A gateway can centralize credentials, route by model or workload, add fallback rules, record usage, enforce rate limits, and expose logs or traces.
A gateway does not automatically host or improve the models behind it. It governs the request path you configure. For example, the Portkey and Novita AI integration guide uses Portkey as the gateway and Novita AI as an endpoint.
Choose this category when: you already have endpoint access but need one operational control plane for routing, observability, access policy, or fallback behavior.
| Dimension | API gateway |
|---|---|
| Cost | Gateway hosting or managed-service cost plus the underlying endpoint cost |
| Control | High control over routing, retries, fallbacks, budgets, and policy; model behavior remains endpoint-dependent |
| Ops complexity | Medium; the gateway becomes another production component to secure, monitor, and update |
| Latency | Adds processing and usually another network hop; measure it on your route rather than assuming a fixed overhead |
| Compliance | Depends on the gateway deployment, logs, credentials, and every endpoint it can reach |
The common failure mode is treating fallback as a guarantee. A fallback can keep a request moving while changing quality, tool semantics, or data handling. Define which substitutions are allowed for each workload and log when a route changes.
Category 4: Self-hosted and open-model endpoints
Self-hosting means serving open model weights on infrastructure that you manage or dedicate to your workload. An inference server such as vLLM exposes the model through an API, while your team manages model files, GPUs, scaling, upgrades, networking, and observability.
Novita AI’s GPU Cloud provides an infrastructure path for deploying open-weight models. This is different from the shared LLM API: you choose the deployment shape and take responsibility for operating the endpoint.
Choose this category when: you need a model or serving configuration that a hosted API does not offer, have strict data-path requirements, or have a stable workload that makes dedicated capacity worth operating.
| Dimension | Self-hosted / open-model endpoint |
|---|---|
| Cost | GPU, storage, bandwidth, and operations costs; fixed capacity can be wasteful for bursty traffic |
| Control | Highest control over model version, serving parameters, batching, network, and deployment location |
| Ops complexity | Highest; plan for provisioning, model loading, health checks, scaling, patching, and incident response |
| Latency | Depends on hardware, batching, concurrency, model size, and where clients run |
| Compliance | More control over the data path, but compliance still depends on the infrastructure, software, and processes you operate |
Self-hosting is not automatically cheaper or more private. Compare the full cost of capacity and operations with current API usage, and verify that logs, backups, telemetry, and support paths follow the same data policy as inference traffic.
Compare the four categories
| Evaluation dimension | Direct provider | Unified API / aggregator | API gateway | Self-hosted / open-model |
|---|---|---|---|---|
| Cost structure | Provider usage pricing | Platform and model pricing | Gateway plus endpoint pricing | GPU and operations capacity |
| Model flexibility | Mostly one provider’s catalog | Broad within the platform catalog | Any reachable endpoint | Any compatible model you can serve |
| Serving control | Provider-defined | Platform-defined | Routing and policy control | Full deployment control |
| Ops burden | Low to start | Low to medium | Medium | High |
| Latency | Provider-dependent | Platform and model-dependent | Adds gateway path | Hardware and configuration-dependent |
| Data control | Provider-defined | Platform-defined | Gateway and endpoint both matter | Infrastructure and process-defined |
| Strongest reason to choose it | A specific model or feature | Fast multi-model access | Centralized operations | Customization or data-path control |
How Novita AI fits the taxonomy
Novita AI is an AI and agent cloud rather than an API gateway. It fits two categories in this map:
- Unified API / aggregator: The Novita AI LLM API offers a single API entry point and an OpenAI-compatible interface. Review the model catalog for current availability before selecting a model.
- Open-model infrastructure: GPU Cloud supports teams that want to deploy and operate open-weight models on dedicated capacity.
- Agent execution: Novita Agent Sandbox addresses the runtime layer for agents that need code, browser, files, or other tools alongside an LLM endpoint.
That combination is useful when the decision is not only “which model API?” but also “where will the agent execute its tools?” It does not remove the need to evaluate model behavior, data handling, cost, or operational fit for a specific workload.
How to choose an LLM API option
Use these questions before comparing vendor lists:
- Is one model or provider non-negotiable? Start with a direct provider when a proprietary model or provider-specific feature is essential. If substitution is acceptable, a unified API can make exploration easier.
- Where may request data and logs go? Map the full path, including gateway logs, provider retention, backups, telemetry, and support access. Do not infer compliance from the word “gateway” or “private.”
- Is traffic bursty or predictable? Hosted APIs avoid paying for idle GPUs. Dedicated capacity can make sense for stable, high-throughput workloads, but calculate using your actual prompts, concurrency, utilization, and operations time.
- Which controls are operational requirements? Choose a gateway or platform controls when you need centralized routing, budgets, observability, or fallback rules. Keep quality and substitution policies explicit.
- How much infrastructure can the team operate? Direct APIs usually minimize initial operations. Self-hosting gives more control at the cost of deployment and ongoing reliability work.
FAQ
What is the difference between an LLM API and an API gateway?
An LLM API serves model responses. An API gateway sits between your application and one or more LLM APIs to add routing, access control, observability, rate limiting, caching, or fallback logic. They are complementary layers, not interchangeable vendor categories.
What does OpenAI-compatible mean?
It means an endpoint follows enough of the OpenAI request and response shape for compatible client code to connect, often by changing the base URL and API key. It does not guarantee the same model quality, context length, tool behavior, streaming details, or error semantics. Test the exact features your application uses.
Can an application use more than one category?
Yes. A production architecture can use a unified API for general traffic, a gateway for routing and observability, direct access for a provider-specific feature, and a self-hosted endpoint for a sensitive or custom workload. The categories describe layers and operating models, not mutually exclusive products.
Is self-hosting always the most private or cheapest option?
No. Self-hosting can improve control over the inference data path, but your logs, backups, infrastructure provider, and operators still matter. It can also carry idle-capacity and maintenance costs. Compare the complete workload cost and data flow against hosted alternatives.
Is Novita AI a direct provider, aggregator, or gateway?
Novita AI fits the unified API and open-model infrastructure categories. Its LLM API provides a shared entry point to models in its catalog, while GPU Cloud supports dedicated deployment workflows. It is not an API gateway whose primary role is to sit in front of external providers.
Sources and availability links checked September 3, 2026: Novita LLM API, Novita API documentation, Novita model catalog, OpenAI API reference, Anthropic API reference, Google Gemini API documentation, LiteLLM documentation, Portkey documentation, and vLLM documentation.
