Streaming

OpenDoor supports Server-Sent Events (SSE) for real-time chat completions.

How It Works

When stream: true is set in the request, the gateway returns an SSE stream with partial content as it's generated.

Request

bash
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Tell me a joke"}],
    "stream": true
  }'

SSE Format

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"Why"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":" did"},"finish_reason":null}]}

data: [DONE]

Client Example (JavaScript)

javascript
const response = await fetch('http://localhost:3001/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
    stream: true,
  }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n');

  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = line.slice(6);
      if (data === '[DONE]') continue;

      const parsed = JSON.parse(data);
      const content = parsed.choices[0]?.delta?.content;
      if (content) {
        process.stdout.write(content);
      }
    }
  }
}

Client Example (Python)

python
import requests

response = requests.post(
    'http://localhost:3001/v1/chat/completions',
    headers={'Authorization': 'Bearer YOUR_API_KEY'},
    json={
        'model': 'gpt-4o',
        'messages': [{'role': 'user', 'content': 'Hello!'}],
        'stream': True,
    },
    stream=True,
)

for line in response.iter_lines():
    if line.startswith(b'data: '):
        data = line[6:].decode()
        if data == '[DONE]':
            continue
        import json
        chunk = json.loads(data)
        content = chunk['choices'][0].get('delta', {}).get('content')
        if content:
            print(content, end='', flush=True)

Fallback During Streaming

If the primary provider fails mid-stream, OpenDoor attempts to fallback to the next provider in the chain. The client receives a new stream from the fallback provider.

Vision

Completions