Stream a model response
Read incremental output from a hosted model and keep partial responses distinct from completed answers.
Streaming delivers a response in chunks. It does not change model weights or create a saved Model version. Choose a model from the actual catalog and use its supported options.
Example
Install openai. Set OPENPOND_API_KEY and OPENPOND_MODEL to an authorized
key and a model ID from Inference.
js
import OpenAI from "openai";
const apiKey = process.env.OPENPOND_API_KEY;
const model = process.env.OPENPOND_MODEL;
if (!apiKey || !model) throw new Error("Set OPENPOND_API_KEY and OPENPOND_MODEL");
const client = new OpenAI({ apiKey, baseURL: "https://api.openpond.ai/opchat/v1" });
const controller = new AbortController();
const stream = await client.chat.completions.create({
model,
stream: true,
messages: [{ role: "user", content: "Describe a quiet harbor in one sentence." }],
}, { signal: controller.signal });
let answer = "";
for await (const chunk of stream) {
const text = chunk.choices[0]?.delta?.content ?? "";
answer += text;
process.stdout.write(text);
}
process.stdout.write("\n");
// Your application's Stop action can call controller.abort().Illustrative final text; actual output varies:
text
Small boats rest in the harbor beneath a clear evening sky.- Append text deltas; a chunk is not necessarily a whole word or sentence.
- Keep an interrupted answer marked partial. Do not report cancellation or a transport failure as successful completion.
- Retain the same request's usage/receipt evidence; receiving some text does not establish final billing or resource cleanup.
- Tool-call deltas need their own assembly; see Tool calling.
Sources
- OpenAI JavaScript SDK: client streaming and cancellation interface.
- OpenPond Inference: endpoint, catalog and authorization contract.