GPT-Live 1 & OpenAI Realtime Voice APIs: Sub-300ms WebRTC Speech-to-Speech

Overview #

A developer guide to GPT-Live 1 and OpenAI Realtime models, examining bidirectional audio streaming over WebRTC, acoustic turn-taking, function calling mid-dialogue, and production voice agent architectures.

WebRTC vs. WebSocket Transport for Live Audio #

Traditional voice bots stitched together WebSockets, Whisper transcription, an LLM, and ElevenLabs TTS, creating 1,200ms+ latency. GPT-Live 1 uses native WebRTC UDP streaming with direct audio-in / audio-out tokens. Round-trip conversational latency drops below 300ms, creating human-grade conversational dynamics.

Natural Interruption & Acoustic Turn-Taking #

GPT-Live 1 listens while speaking. If the human interjects, the server truncates audio output immediately and updates conversation state without crashing the active session.

In-Dialogue Tool Execution #

Voice agents can trigger backend APIs (such as checking order status or booking calendar slots) while maintaining vocal filler sounds ('Checking that now...') to prevent awkward dead air.

Code Example: Initializing GPT-Live 1 Session via Ephemeral Token #

async function initRealtimeVoiceSession() {
  // Step 1: Request ephemeral session token from your backend
  const tokenRes = await fetch("/api/session/realtime");
  const data = await tokenRes.json();
  const ephemeralKey = data.client_secret.value;

  // Step 2: Establish WebRTC peer connection
  const pc = new RTCPeerConnection();
  const audioEl = document.createElement("audio");
  audioEl.autoplay = true;
  pc.ontrack = (e) => audioEl.srcObject = e.streams[0];

  // Capture local microphone stream
  const localStream = await navigator.mediaDevices.getUserMedia({ audio: true });
  localStream.getTracks().forEach((track) => pc.addTrack(track, localStream));

  // Step 3: Negotiate SDP with OpenAI Realtime Gateway
  const offer = await pc.createOffer();
  await pc.setLocalDescription(offer);

  const sdpResponse = await fetch("https://api.apihundred.com/v1/realtime?model=gpt-live-1", {
    method: "POST",
    body: offer.sdp,
    headers: {
      Authorization: `Bearer ${ephemeralKey}`,
      "Content-Type": "application/sdp"
    }
  });

  const answer = { type: "answer", sdp: await sdpResponse.text() };
  await pc.setRemoteDescription(answer);
  console.log("GPT-Live 1 Realtime Session Connected");
}

Frequently Asked Questions #

Q: How is GPT-Live 1 priced?

Realtime voice APIs bill for text tokens and audio input/output minutes (approximately $5.00/1M input tokens and $20.00/1M output tokens).

Q: Can GPT-Live 1 handle multiple accents?

Yes, trained natively on diverse international speech corpora, it handles dozens of languages and regional accents without phonetic transcription loss.

Q: Does GPT-Live 1 support custom system prompts?

Yes, developers configure tone, personality, cadence, and allowed tools through standard session configuration objects.