Inference API.

§ 05.1Base URL & authentication¶

Everything lives under https://api.deeprelay.ai/v1 and authenticates with Authorization: Bearer deeprelay_live_.... Get a key with deeprelay login or mint one in the dashboard. Scopes: serverless:read lists models, serverless:write runs chat, image, and video generation, and billing:read reads usage; a full_access key covers all three.

Every key you mint belongs to this production host. Any other deeprelay host you may see in older examples is internal staging with its own key store, so a production key gets a 401 there. The /v1 in the path is the API version and the only one there is; the OpenAPI document's 1.0.0 is the revision of that /v1 contract, not a separate API.

Because the API is OpenAI-compatible, the official OpenAI SDKs work unchanged for chat; just override the base URL and API key. The video endpoints follow the same submit → poll → download layout as OpenAI's video API.

§ 05.2Models¶

The catalog is live: built from the models actually being served warm right now, refreshed roughly hourly and probe-verified. Each entry carries a modality (chat, image, or video), pricing, and the request parameters the model accepts. Filter with ?modality=video.

A chat model can have two tiers: serverless (the default: low latency, no cold start; use the bare model id) and economy (cheaper, compute-priced, may cold-start ~30–60s; opt in with the :economy suffix). Economy exists only where GET /v1/models lists the suffixed id as its own entry, such as deeprelay/<model>:economy. Check the live catalog before relying on it: a :economy id the catalog does not list returns model_not_found, and there are times when no model has an economy tier at all.

§ 05.3Chat completions¶

Standard OpenAI chat completions, including stream: true for server-sent events.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.deeprelay.ai/v1",
    api_key="deeprelay_live_...",
)

stream = client.chat.completions.create(
    model="deeprelay/deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Explain recursion in one sentence"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

§05.3.1OpenAI compatibility¶

The request and response shapes, streaming, tool calling and the error envelope match OpenAI's, so the official SDKs and most OpenAI-compatible clients work once you change the base URL. Not every OpenAI chat parameter is carried, though. Each one below is forwarded to the model, rejected with a 400 before anything is billed, or ignored (accepted and dropped).

§ 05.4Image generation¶

Image generation is synchronous and OpenAI-compatible. v1 returns images as base64 only: data[].b64_json carries the bytes, and response_format: "url" is rejected. The response adds usage.image_count, the metered unit.

curl https://api.deeprelay.ai/v1/images/generations \
  -H "Authorization: Bearer deeprelay_live_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"deeprelay/seedream-4.0","prompt":"a watercolor fox","n":1,"size":"1024x1024"}'

{ "created": 1733800000, "data": [{ "b64_json": "iVBORw0KGgo..." }], "usage": { "image_count": 1 } }

§ 05.5Video generation¶

Video generation is asynchronous: POST /v1/videos returns a job in the queued state immediately. Poll GET /v1/videos/{id} until the status reaches a terminal value, then download the mp4 from GET /v1/videos/{id}/content.

# 1. Submit: returns a queued job immediately
curl https://api.deeprelay.ai/v1/videos \
  -H "Authorization: Bearer deeprelay_live_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "model": "deeprelay/wan2.7-t2v",
    "prompt": "ocean waves at sunset",
    "seconds": 5
  }'

# 2. Poll until status is terminal
curl https://api.deeprelay.ai/v1/videos/vid_... \
  -H "Authorization: Bearer deeprelay_live_..."

# 3. Download the mp4
curl https://api.deeprelay.ai/v1/videos/vid_.../content \
  -H "Authorization: Bearer deeprelay_live_..." -o out.mp4

{
  "id":           "vid_...",
  "object":       "video",
  "model":        "deeprelay/wan2.7-t2v",
  "status":       "completed",
  "progress":     100,
  "seconds":      5,
  "cost_cents":   10,
  "created_at":   1733800000,
  "completed_at": 1733800240,
  "expires_at":   1733886640
}

§05.5.1Job lifecycle¶

§05.5.2Parameters¶

Each model declares the request parameters it accepts in its catalog entry (parameters on GET /v1/models), typically seconds, the clip length, whose bounds and default come from the model. Parameters a model does not declare are rejected with an invalid_request_error rather than silently ignored: live video models generate at their default output resolution, so most do not accept size. Omitting seconds uses the model's default clip length.

§05.5.3Billing, idempotency & retention¶

You're charged by seconds of output video, measured when the job completes. Jobs that never complete (failed, cancelled, or expired before finishing) are never billed. The authoritative charge is the cost_cents field on the job object; it stays on the job even after the artifact expires.

Completed videos are retained for 24 hours; the job carries an expires_at (unix seconds). After that, the artifact is gone and GET /v1/videos/{id}/content returns 410 Gone with an expired error code. Download promptly and store the file on your side.

Stop an in-flight job with POST /v1/videos/{id}/cancel (a cancel verb, not DELETE). Cancellation is idempotent: a terminal job is returned unchanged, and a job whose output is already committed can no longer be cancelled.

§ 05.6Embeddings¶

Embeddings are synchronous and OpenAI-compatible. Pass input as a single string or an array of up to 2048 strings, bounded so the response stays under 64 MiB (about 680 inputs at 4096 dimensions; over-limit requests are rejected with a 400 before any processing); you get one vector per input, in request order, with data[].index matching the input's position. List the served models with GET /v1/models?modality=embedding.

curl https://api.deeprelay.ai/v1/embeddings \
  -H "Authorization: Bearer deeprelay_live_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"deeprelay/qwen3-embedding-8b","input":["hello","world"]}'

{ "object": "list",
  "data": [{ "object": "embedding", "index": 0, "embedding": [-0.019, 0.0071, ...] }],
  "model": "deeprelay/qwen3-embedding-8b",
  "usage": { "prompt_tokens": 4, "total_tokens": 4 } }

Billed on input tokens only, at the model's listed input rate. Usage is metered in micro-cents and settled against your credit balance once a minute: each settlement charges whole cents rounded down and carries the sub-cent remainder to the next one, so you are never billed above metered usage and there is no per-request minimum. An embeddings call emits no completion tokens, so usage carries only prompt_tokens and total_tokens. encoding_format supports float only; base64 is rejected with unsupported_parameter. A chat model id here returns 404 model_not_found.

§ 05.7Audio generation¶

§ 05.8Tool calling¶

tools and tool_choice are supported on the chat models that advertise them. The model never executes anything: it returns the call it wants made, and running the function and feeding the result back is your loop to close.

{
  "model": "deeprelay/deepseek-v4-pro",
  "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather for a city",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }],
  "tool_choice": "auto"
}

The reply carries finish_reason of tool_calls and a null content; branch on that to tell a tool call from an ordinary reply.

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_07db14bq3p5fkdbnv7fbsppj",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

To close the loop, replay the assistant turn that made the call and append a role: "tool" message carrying the tool_call_id it answers; omitting that id is a 400 from us, before the request costs you anything. When streaming, tool calls arrive as fragments: accumulate them by the index field and never by position in the array, because parallel calls interleave and position-keying splices two calls' arguments into JSON that parses cleanly and is wrong. The serverless inference guide carries the full round trip, the streaming chunk shapes, and every tool_choice mode.

Web search is the exception to “the model never executes anything”. On /v1/messages and /v1/responses, the server-side web_search tool runs on our side and feeds the results back to the model within the same request. It works on models with web_search: true on GET /v1/models. Each search is billed per search, and the result text counts as input tokens. See Web search in the serverless inference guide.

§ 05.9JSON mode & structured outputs¶

{ "error": { "message": "deeprelay/qwen3.6-plus does not support structured_outputs. Models that support it: deeprelay/deepseek-v4-pro, deeprelay/deepseek-v4.1-flash, deeprelay/glm-5.3, deeprelay/kimi-k3, …", "type": "invalid_request_error", "param": "response_format", "code": "unsupported_parameter" } }

A json_schema with strict: true is served only where the upstream enforces the schema. Without strict, a model that supports json_object but not schemas serves the request in JSON mode instead, and reports the downgrade as response_format:downgraded in x-deeprelay-ignored-params. The relay never emulates a schema with a hidden prompt, so validate the reply against your schema on a downgraded response. Schemas are capped at 64 KiB and a nesting depth of 32. When several upstreams serve a model, the request is routed only to those that passed the probe, and a 503 is returned if none is healthy rather than a silent downgrade.

§ 05.10Errors¶

Errors use the OpenAI envelope, so SDK error handling works as-is:

{ "error": { "message": "...", "type": "invalid_request_error", "code": "model_not_found" } }

• model_not_found (404): the live catalog no longer serves that model id. Re-fetch GET /v1/models and pick another. On streaming this arrives as the final SSE error event before [DONE].
• stream_limit_exceeded (429): too many inference requests open at once on this API key. Close streams you're no longer reading or retry shortly; this is separate from the per-second rate limit (rate_limit_exceeded).
• upstream_unavailable (503): a model's serving capacity is temporarily degraded. Returned promptly; retry after a short interval, since the platform routes around degraded capacity automatically.