Skip to content
OpenPond
Join community
Cloud/Stream a model response

Stream a model response

Read incremental output from a hosted model and keep partial responses distinct from completed answers.

Streaming delivers a response in chunks. It does not change model weights or create a saved Model version. Choose a model from the actual catalog and use its supported options.

Example

Install openai. Set OPENPOND_API_KEY and OPENPOND_MODEL to an authorized key and a model ID from Inference.

js
import OpenAI from "openai"; const apiKey = process.env.OPENPOND_API_KEY; const model = process.env.OPENPOND_MODEL; if (!apiKey || !model) throw new Error("Set OPENPOND_API_KEY and OPENPOND_MODEL"); const client = new OpenAI({ apiKey, baseURL: "https://api.openpond.ai/opchat/v1" }); const controller = new AbortController(); const stream = await client.chat.completions.create({ model, stream: true, messages: [{ role: "user", content: "Describe a quiet harbor in one sentence." }], }, { signal: controller.signal }); let answer = ""; for await (const chunk of stream) { const text = chunk.choices[0]?.delta?.content ?? ""; answer += text; process.stdout.write(text); } process.stdout.write("\n"); // Your application's Stop action can call controller.abort().

Illustrative final text; actual output varies:

text
Small boats rest in the harbor beneath a clear evening sky.
  • Append text deltas; a chunk is not necessarily a whole word or sentence.
  • Keep an interrupted answer marked partial. Do not report cancellation or a transport failure as successful completion.
  • Retain the same request's usage/receipt evidence; receiving some text does not establish final billing or resource cleanup.
  • Tool-call deltas need their own assembly; see Tool calling.

Sources

View as Markdown