Advanced FeaturesLow Latency
Server-Sent Events (SSE) Streaming Guide
Stream tokens in real time using Server-Sent Events over HTTP/2 with sub-50ms Time to First Token.
Streaming delivers tokens to users as they are decoded by the inference cluster, eliminating waiting time for large completions.
Set "stream": true in your request body. The API100 Gateway relays chunk deltas without buffer delay using the SSE text/event-stream format.
Python Streamingpython
from openai import OpenAI
client = OpenAI(
base_url="https://api.apihundred.com/v1",
api_key="your_api100_key"
)
stream = client.chat.completions.create(
model="gemini-1-5-flash",
messages=[{"role": "user", "content": "Write an essay on planetary geology."}],
stream=True
)
for chunk in stream:
token = chunk.choices[0].delta.content or ""
print(token, end="", flush=True)