Vercel AI SDK: Complete Developer Guide for Building AI Applications

The Vercel AI SDK is a TypeScript toolkit for building AI applications. It handles text generation, streaming, structured outputs, tool calls, and multi-step agent loops through a single unified API — and it works with any major LLM provider. If you’ve tried wiring OpenAI or Anthropic directly into a Next.js app and ended up maintaining three different streaming implementations, this SDK solves that problem.

This guide covers the SDK’s core capabilities, how to connect it to Novita AI’s OpenAI-compatible LLM API, and where agent workflows fit in for developers building more than a simple chatbot.

What is the Vercel AI SDK?

The Vercel AI SDK (package: ai on npm) is an open-source library that abstracts over the differences between LLM providers. Instead of learning separate streaming protocols for OpenAI, Anthropic, and Google, you call the same generateText, streamText, or generateObject functions and swap providers by changing a single import.

The SDK ships in two layers:

• AI SDK Core handles model interactions: text generation, streaming, structured objects, tool calls, embeddings, and agent loops.
• AI SDK UI provides React hooks (useChat, useCompletion, useObject, useAssistant) plus adapters for Next.js, SvelteKit, and Nuxt that manage streaming state in the browser.

There’s also an @ai-sdk/openai-compatible package for connecting to any provider that implements the OpenAI Chat Completions format — which is how Novita AI integrates.

The SDK works in Node.js, Deno, edge runtimes (Cloudflare Workers, Vercel Edge), and browser environments.

AI SDK Core: generateText, streamText, and generateObject

Install the SDK and a provider package:

npm install ai @ai-sdk/openai

generateText

For non-streaming completions — batch processing, classification, one-shot generation:

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: 'Explain how LLM embeddings work in two sentences.',
});

console.log(text);

streamText

For chat interfaces where latency matters:

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-4o-mini'),
  messages: [
    { role: 'user', content: 'Walk me through setting up a Next.js app.' },
  ],
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

generateObject

When you need structured JSON output instead of raw text — schema validation is handled automatically using Zod:

import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4o-mini'),
  schema: z.object({
    name: z.string(),
    skills: z.array(z.string()),
    experienceLevel: z.enum(['junior', 'mid', 'senior']),
  }),
  prompt: 'Generate a fictional software engineer profile.',
});

console.log(object.name, object.skills);

The SDK handles the system prompt injection needed to coerce the model into valid JSON and retries on malformed output.

AI SDK Tools: Function Calling and Tool Use

Tools let models call external functions — search APIs, database queries, calculators — during a generation. The SDK’s tools parameter takes an object where each key becomes a callable function:

import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const { text, toolCalls } = await generateText({
  model: openai('gpt-4o-mini'),
  tools: {
    getWeather: tool({
      description: 'Get current weather for a location',
      parameters: z.object({
        city: z.string().describe('City name'),
        unit: z.enum(['celsius', 'fahrenheit']).default('celsius'),
      }),
      execute: async ({ city, unit }) => {
        // Replace with a real weather API call
        return { city, temperature: 22, unit, condition: 'sunny' };
      },
    }),
  },
  prompt: 'What is the weather in Tokyo right now?',
});

console.log(text);

The tool() helper provides type inference from the Zod schema into the execute function parameters. No manual JSON parsing needed.

Multi-step Tool Calls

By default, generateText stops after one round of tool calls. Set maxSteps to let the model use tool results in follow-up reasoning:

const { text } = await generateText({
  model: openai('gpt-4o'),
  maxSteps: 5,
  tools: { getWeather, searchWeb, lookupCalendar },
  prompt: 'Plan my outdoor activities for this weekend in Berlin.',
});

The SDK handles the tool call → result → continuation loop automatically. Each step is exposed via onStepFinish if you need visibility into intermediate reasoning.

Building AI Agent Loops with the Vercel AI SDK

An agent in the AI SDK is a model running in a loop with tools until it decides it has enough information to answer. The pattern is the same as multi-step tool calling, but with more tools and a longer maxSteps:

import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const result = await generateText({
  model: openai('gpt-4o'),
  maxSteps: 10,
  system: 'You are a research assistant. Use available tools to answer thoroughly.',
  prompt: 'What are the main differences between Llama 3.1 and Qwen3?',
  tools: {
    search: tool({
      description: 'Search the web for current information',
      parameters: z.object({ query: z.string() }),
      execute: async ({ query }) => searchWeb(query),
    }),
    summarize: tool({
      description: 'Summarize a URL',
      parameters: z.object({ url: z.string() }),
      execute: async ({ url }) => fetchAndSummarize(url),
    }),
  },
  onStepFinish({ stepType, toolCalls, toolResults }) {
    console.log('Step:', stepType, toolCalls?.map(t => t.toolName));
  },
});

For production agent pipelines, connect Novita AI as the model backend to control cost and throughput at scale — covered in the next section.

AI SDK React: useChat Hook and Next.js Integration

The useChat hook manages streaming chat state without boilerplate:

// app/chat/page.tsx
'use client';
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat',
  });

  return (
    <div>
      {messages.map(m => (
        <div key={m.id} className={m.role === 'user' ? 'user' : 'assistant'}>
          {m.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask something..."
          disabled={isLoading}
        />
        <button type="submit" disabled={isLoading}>Send</button>
      </form>
    </div>
  );
}

The corresponding route handler:

// app/api/chat/route.ts
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });

  return result.toDataStreamResponse();
}

toDataStreamResponse() handles the Vercel AI stream protocol, which useChat on the client understands natively. For streaming structured objects from an API, swap useChat for useObject and streamText for streamObject.

How to Use Novita AI with the Vercel AI SDK

Novita AI provides an OpenAI-compatible API at https://api.novita.ai/v3/openai, which makes it a drop-in replacement for any SDK that uses the OpenAI Chat Completions format — including the Vercel AI SDK.

Why Use Novita AI with the AI SDK

Novita AI hosts 70+ open-source models — Llama 3.3 70B, Qwen3, DeepSeek V3, Mistral, and Gemma 3 — through a single API endpoint. No GPU infrastructure to manage. For agent workflows, the serverless API scales from small 7B instruction models up to large reasoning models without configuration changes.

Setup

Install the OpenAI-compatible provider package:

npm install ai @ai-sdk/openai-compatible

Configure the provider:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

const novita = createOpenAICompatible({
  name: 'novita',
  baseURL: 'https://api.novita.ai/v3/openai',
  apiKey: process.env.NOVITA_API_KEY,
});

That’s the full setup. Now use it exactly like any other AI SDK provider:

import { generateText } from 'ai';

const { text } = await generateText({
  model: novita('meta-llama/llama-3.3-70b-instruct'),
  prompt: 'What are the tradeoffs between RAG and fine-tuning for a customer support bot?',
});

Streaming works the same way:

import { streamText } from 'ai';

const result = streamText({
  model: novita('qwen/qwen3-235b-a22b-instruct-2507'),
  messages: [
    { role: 'system', content: 'You are a helpful coding assistant.' },
    { role: 'user', content: 'Show me how to build a REST API with FastAPI.' },
  ],
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Tool Calls with Novita AI

Models that support function calling work with the AI SDK’s tool interface without any extra configuration. Llama 3.3 70B Instruct and Qwen3 support tool calls through Novita’s API:

import { generateText, tool } from 'ai';
import { z } from 'zod';

const { text } = await generateText({
  model: novita('meta-llama/llama-3.3-70b-instruct'),
  maxSteps: 5,
  tools: {
    calculator: tool({
      description: 'Evaluate a mathematical expression',
      parameters: z.object({ expression: z.string() }),
      execute: async ({ expression }) => {
        return { result: eval(expression) }; // use a safe math parser in production
      },
    }),
  },
  prompt: 'What is 12% of 847, then divide by 3.14?',
});

Environment Setup

Add your Novita API key to .env.local:

NOVITA_API_KEY=your_novita_api_key_here

Get a key at novita.ai — new accounts include free credits to test the API.

Agent Sandbox for Longer-Running Workloads

For agent tasks that run code, access filesystems, or execute multi-minute workflows, Novita AI’s Agent Sandbox provides isolated execution environments on top of the LLM API. The AI SDK handles the model reasoning layer. The sandbox handles stateful execution that can’t run inside an edge function timeout.

Choosing an LLM Provider for the Vercel AI SDK

The AI SDK makes switching providers easy, which is useful — but it’s worth understanding the tradeoffs before picking one for production.

Latency: Time-to-first-token matters more than total generation speed for streaming chat. Smaller models (8B–14B) produce first tokens faster. Novita AI’s serverless endpoints are optimized for low cold-start latency on open models.

Cost at scale: GPT-4o is excellent, but at high query volumes the price gap between a frontier model and a well-tuned open model (e.g., Llama 3.3 70B) becomes significant. The AI SDK lets you experiment with both without rewriting application logic.

Tool call support: Not all models support function calling reliably. Test your specific model against your tool schemas — behavior varies across providers even for the same model family.

Context window: For document-heavy RAG or long conversation histories, check context limits. Many open models support 128K tokens, which covers most practical use cases.

Vendor lock-in: The AI SDK’s provider abstraction with Novita AI’s OpenAI-compatible endpoint means you can switch models or add a fallback provider without touching application code.

Conclusion

The Vercel AI SDK removes the boilerplate from building AI applications — one API for text generation, streaming, tool calls, and agent loops across any LLM provider. Whether you’re building a streaming chat interface with useChat, extracting structured data with generateObject, or running a multi-step agent that calls external tools, the SDK handles the plumbing so you can focus on application logic.

For open-model inference, Novita AI’s OpenAI-compatible API slots directly into the SDK via @ai-sdk/openai-compatible. You get access to 70+ models — Llama, Qwen3, DeepSeek, Mistral — without managing GPU infrastructure, and you can swap models or add provider fallbacks without touching your application code.

Get started at novita.ai — new accounts include free credits.

FAQ

What is ai sdk react?

The AI SDK’s React integration lives in the ai/react package. It provides hooks — useChat, useCompletion, useObject, useAssistant — that connect a React frontend to a server-side streaming route. The hooks handle streaming state, message history, loading indicators, and error handling so you don’t have to manage ReadableStream in component state.

What does ai sdk 5 add?

AI SDK 5 (released as a beta in mid-2025) redesigned the provider specification for better type safety, separated UI state management from model interaction logic, and introduced a revised message format that makes agentic UI state easier to serialize. If you’re starting a new project in 2026, check ai-sdk.dev for the latest stable version — the SDK has continued to evolve through versions 6 and 7.

Is there ai sdk docs I can reference?

The canonical documentation is at ai-sdk.dev. It includes provider setup guides, API references for all core functions, framework integration walkthroughs (Next.js, Nuxt, SvelteKit), and cookbook examples for common patterns like RAG, agent loops, and structured extraction.

Can I use the ai sdk tools with non-OpenAI models?

Yes. Any provider that supports function calling works with the AI SDK’s tool() interface. Novita AI hosts several open models with function calling support. Behavior quality varies — Llama 3.3 70B and Qwen3 are the most reliable for multi-step tool use among open models available through Novita’s API.

What’s the difference between vercel ai sdk docs and the ai-sdk.dev documentation?

They’re the same product. The documentation previously lived at sdk.vercel.ai and redirects to ai-sdk.dev. The SDK is maintained by the Vercel team but is open-source and not tied to deploying on Vercel infrastructure.

How do artificial intelligence SDKs compare to calling the LLM API directly?

Direct API calls are fine for simple one-off requests. An SDK becomes worthwhile once you need multiple things at once: streaming integrated into a UI framework, multi-step tool call loops, structured output validation, unified provider switching, and consistent retry/error handling across providers. The AI SDK handles all of that at the library level so you’re not rebuilding it per project.

Recommended Articles

• Which Inference Provider Is Right for AI Agents
• Build a Multi-Agent System with Novita and CrewAI
• Best LLM API Providers in 2026