Vercel AI SDK

The Vercel AI SDK is the TypeScript toolkit behind most Next.js chat apps: generateText, streamText, embed, and the useChat React hook. deeprelay plugs into it through the official @ai-sdk/openai-compatible provider, because the deeprelay API speaks the OpenAI chat and embeddings protocol. No deeprelay-specific package, no adapter code: create a provider once, then pass deeprelay model ids anywhere the SDK takes a model.

This guide covers text generation, streaming, a Next.js chat route with useChat, tool calling, structured output, and embeddings. Everything runs on Node 20+, Bun, and Vercel's Node and Edge runtimes.

Vercel AI Gateway. If your app already routes through Vercel's AI Gateway, keep doing that for the models it lists and use the provider below for deeprelay models. The two coexist: both are plain AI SDK providers, and the SDK picks whichever one you pass as model.

Install

npm i ai @ai-sdk/openai-compatible

Put your key in the environment. The SDK never sees it in source and Vercel injects project environment variables at runtime.

export DEEPRELAY_API_KEY=deeprelay_live_...

Create the provider

Create the provider once and import it wherever you need a model. The name becomes the key under providerOptions. includeUsage asks deeprelay to append token counts to the final streaming chunk so the SDK's usage field is populated on streamed calls too, and supportsStructuredOutputs lets structured output send a JSON schema instead of falling back to prompted JSON.

// lib/deeprelay.ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

export const deeprelay = createOpenAICompatible({
  name: "deeprelay",
  baseURL: "https://api.deeprelay.ai/v1",
  apiKey: process.env.DEEPRELAY_API_KEY,
  includeUsage: true,
  supportsStructuredOutputs: true,
});

Base URL	https://api.deeprelay.ai/v1
Auth	Authorization: Bearer $DEEPRELAY_API_KEY (the provider sets it from apiKey)
Chat models	deeprelay("deeprelay/")
Embedding models	deeprelay.embeddingModel("deeprelay/")

Model ids are the ones GET /v1/models returns, for example deeprelay/deepseek-v4.1-flash or deeprelay/kimi-k3. See the Inference API for the catalog and tiers.

Generate text

import { generateText } from "ai";
import { deeprelay } from "./lib/deeprelay";

const { text, usage } = await generateText({
  model: deeprelay("deeprelay/deepseek-v4.1-flash"),
  system: "You answer in one sentence.",
  prompt: "What does a serverless inference relay do?",
});

console.log(text);
console.log(usage); // { inputTokens, outputTokens, totalTokens }

temperature, topP, maxOutputTokens, stopSequences and seed map straight onto the deeprelay request. Leave the rest at the SDK defaults.

Stream text

streamText returns the tokens as they arrive. deeprelay streams server-sent events in the OpenAI format, which is what the provider expects.

import { streamText } from "ai";
import { deeprelay } from "./lib/deeprelay";

const result = streamText({
  model: deeprelay("deeprelay/kimi-k3"),
  prompt: "Write a haiku about cold starts.",
});

for await (const delta of result.textStream) {
  process.stdout.write(delta);
}

const usage = await result.usage; // populated because includeUsage is on

Next.js chat route with useChat

The same shape as any AI SDK chat app: a route handler that streams, and a client component that renders the stream. Swap the model id for any chat model in the catalog.

// app/api/chat/route.ts
import { streamText, convertToModelMessages, type UIMessage } from "ai";
import { deeprelay } from "@/lib/deeprelay";

export const maxDuration = 60;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: deeprelay("deeprelay/deepseek-v4.1-flash"),
    messages: convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

// app/page.tsx
"use client";

import { useChat } from "@ai-sdk/react";
import { useState } from "react";

export default function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage, status } = useChat();

  return (
    <main>
      {messages.map((m) => (
        <p key={m.id}>
          <strong>{m.role}:</strong>{" "}
          {m.parts.map((p, i) => (p.type === "text" ? <span key={i}>{p.text}</span> : null))}
        </p>
      ))}

      <form
        onSubmit={(e) => {
          e.preventDefault();
          if (!input.trim()) return;
          sendMessage({ text: input });
          setInput("");
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} placeholder="Say something" />
        <button type="submit" disabled={status !== "ready"}>Send</button>
      </form>
    </main>
  );
}

useChat keeps the message history and parses the stream, so the client needs no manual SSE handling. Install @ai-sdk/react if your project does not already have it.

Tool calling

Define tools with tool() and a Zod schema. The SDK sends them as OpenAI tools, deeprelay returns the model's tool_calls, and the SDK runs execute and feeds the result back. stopWhen: stepCountIs(n) lets the loop take several round trips before it has to answer.

import { generateText, tool, stepCountIs } from "ai";
import { z } from "zod";
import { deeprelay } from "./lib/deeprelay";

const { text, toolCalls, steps } = await generateText({
  model: deeprelay("deeprelay/deepseek-v4.1-flash"),
  tools: {
    getWeather: tool({
      description: "Get the current weather for a city",
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, tempC: 18, sky: "overcast" }),
    }),
  },
  stopWhen: stepCountIs(3),
  prompt: "What is the weather in Paris right now? Use the tool, then answer in one sentence.",
});

console.log(toolCalls.map((c) => c.toolName)); // ["getWeather"]
console.log(steps.length, text);               // 2 "The weather in Paris is overcast at 18°C."

streamText takes the same tools and stopWhen, and streams the tool calls and the final text as parts. Every chat model in the catalog with tools in its supported_parameters accepts this; the Serverless Inference guide has the wire-level detail and the tool_choice modes each model allows.

Structured output

Pass output: Output.object({ schema }) to ask for JSON that matches a Zod schema. With supportsStructuredOutputs on the provider, the SDK sends the schema as response_format: { type: "json_schema", strict: true } and deeprelay enforces it on the model side, so the result parses without a retry loop.

import { generateText, Output } from "ai";
import { z } from "zod";
import { deeprelay } from "./lib/deeprelay";

const { output } = await generateText({
  model: deeprelay("deeprelay/deepseek-v4.1-flash"),
  output: Output.object({
    schema: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
    }),
  }),
  prompt: "A recipe name and three ingredients for pancakes.",
});

console.log(output); // { name: "Classic Pancakes", ingredients: ["Flour", "Eggs", "Milk"] }

The same output option on streamText exposes partialOutputStream, the object filling in as tokens arrive, and await result.output for the final value. Models that list structured_outputs in supported_parameters take a strict schema natively. For a model that has response_format but not structured_outputs, drop supportsStructuredOutputs from the provider and the SDK falls back to prompting for JSON and parsing it.

Embeddings

Embedding models go through embeddingModel. embedMany batches a list in one request; deeprelay bills embeddings per input token only.

import { embed, embedMany, cosineSimilarity } from "ai";
import { deeprelay } from "./lib/deeprelay";

const model = deeprelay.embeddingModel("deeprelay/qwen3-embedding-8b");

const { embedding } = await embed({
  model,
  value: "sunny day at the beach",
});

const { embeddings } = await embedMany({
  model,
  values: ["sunny day at the beach", "rainy afternoon in the city"],
});

console.log(embedding.length, cosineSimilarity(embeddings[0], embeddings[1]));

Pick a model at runtime

The catalog is live, so list it rather than hard-coding a long table. The plain OpenAI SDK, or fetch, reads it with the same key:

const res = await fetch("https://api.deeprelay.ai/v1/models?modality=chat", {
  headers: { Authorization: `Bearer ${process.env.DEEPRELAY_API_KEY}` },
});
const { data } = await res.json();
console.log(data.map((m: { id: string }) => m.id));

Models the flat subscription covers carry plan_covered: true; a subscriber's requests to those draw on the monthly allowance before pay-as-you-go credit.

Errors

deeprelay returns OpenAI-shaped error envelopes, so the SDK surfaces them as APICallError with the HTTP status and the parsed body. The ones worth handling:

Status	code	Meaning
401	invalid_api_key	Wrong or expired key.
402	insufficient_balance	Out of credit; top up or subscribe.
404	model_not_found	The model left the live catalog; pick another.
429	rate_limit_exceeded	Back off and retry; the SDK's maxRetries handles this.

The full list is in Conventions.

Next steps

• LiteLLM if you want the same models in Python or behind a proxy.
• Portkey for a gateway with fallbacks, caching and observability in front of deeprelay.
• Serverless Inference guide for the request and response shapes the provider is sending, tool calling, JSON mode, and reasoning hints.
• Coding agents if what you want is deeprelay inside an editor or agent rather than your own app.