Integrate Currai with OpenAI voice

This workflow connects an OpenAI voice application to Currai. The main path uses the OpenAI Realtime API for a two-way browser conversation over WebRTC. A separate section explains the smaller change needed for one-way text-to-speech (TTS).

OpenAI continues to own speech recognition, model execution, and audio generation. Currai receives a best-effort, server-side record that connects the conversation the user experienced to the OpenAI model call that produced it.

To run the complete implementation first, use the public OpenAI voice example on GitHub:

git clone git@github.com:curraiapp/examples.git
cd examples/openai
cp .env.example .env.local
pnpm install
pnpm dev

Choose the correct OpenAI voice path

The Realtime path is not the same as calling the Speech endpoint. Realtime is audio-in and audio-out with an ongoing session. TTS is text-in and audio-out for one generation.

What you need

• an OpenAI project with access to the Realtime or Speech API;
• a server-only OpenAI API key;
• a working browser voice flow or TTS request;
• a Currai workspace and ingestion key pair;
• a trusted server route in the application;
• HTTPS or localhost for browser microphone access.

You do not need an OpenAI SDK or a Currai SDK. The examples below use browser WebRTC APIs and native fetch.

Recommended: connect OpenAI with the Currai Skill

Install the published Currai Skill from the root of your application repository:

npx skills add https://github.com/curraiapp/skills --skill currai

Configure the credentials from step 1 before asking your coding agent to make changes. For a Realtime browser application, paste this prompt:

Use $currai to connect this OpenAI Realtime browser voice application to Currai
using OPENAI_API_KEY, CURRAI_PUBLIC_KEY, CURRAI_SECRET_KEY, and CURRAI_BASE_URL
already configured in the server environment. Find the component that owns the
live RTCPeerConnection and Realtime data channel, not only an assistant settings
or SDP route. Retain completed user transcriptions and finalized assistant audio
transcripts without duplicates. Create one UUID session per complete call and emit
openai.conversation.turn agent events with nested openai.realtime.model events.
Keep capture best-effort and server-side. Restart or deploy the application,
complete one real voice conversation, and verify its readable User Story and
nested trace in Currai. Never print or copy credentials into source code or chat.

For one-way TTS, ask the skill to instrument the server route that calls POST /v1/audio/speech, retain the input text as model evidence, record safe output metadata rather than audio bytes, and nest the speech event beneath the agent turn that decided what to say.

The remaining sections show the manual workflow so you can review or customize the integration.

1. Configure server-only credentials

Add these values to your server environment:

OPENAI_API_KEY=<your-openai-api-key>
CURRAI_PUBLIC_KEY=<your-currai-public-key>
CURRAI_SECRET_KEY=<your-currai-secret-key>
CURRAI_BASE_URL=https://www.currai.app

Never prefix these values with NEXT_PUBLIC_, VITE_, or another browser-public prefix. The browser should receive an SDP answer or generated audio, never the OpenAI key or Currai credentials.

Restart or redeploy the application after changing its environment.

2. Prove the OpenAI flow works first

For Realtime, complete a short call before adding capture. Confirm that:

1. the browser grants microphone permission;
2. the WebRTC peer connection reaches the connected state;
3. the Realtime data channel opens;
4. remote model audio plays;
5. completed user and assistant transcripts arrive.

For TTS, send one short text input and confirm the server returns playable audio with the expected format.

Start from a known-working OpenAI request. Currai capture should observe the product path, not become part of establishing the provider connection.

3. Create the Realtime call on the server

The browser creates the WebRTC offer, but the server authenticates the unified Realtime call. Forward the SDP without trimming or rewriting it:

export async function POST(request: Request) {
  const { sdp, session } = await request.json();

  if (typeof sdp !== "string" || !sdp.trim().startsWith("v=0")) {
    return Response.json({ error: "A valid SDP offer is required." }, { status: 400 });
  }

  const form = new FormData();
  form.set("sdp", sdp); // Preserve the terminal CRLF from the browser.
  form.set("session", JSON.stringify(session));

  const upstream = await fetch("https://api.openai.com/v1/realtime/calls", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    },
    body: form,
    cache: "no-store",
  });

  const answerSdp = await upstream.text();
  if (!upstream.ok) {
    return Response.json(
      { error: "OpenAI could not create the Realtime call." },
      { status: 502 },
    );
  }

  return Response.json({ sdp: answerSdp });
}

Validate the session configuration on the server. Do not let the browser select arbitrary models, tools, instructions, or unbounded prompt content.

4. Start WebRTC in the browser

Create the peer connection, attach microphone audio, and open the Realtime event channel:

const peer = new RTCPeerConnection();
const events = peer.createDataChannel("oai-events");
const microphone = await navigator.mediaDevices.getUserMedia({ audio: true });

for (const track of microphone.getTracks()) {
  peer.addTrack(track, microphone);
}

const audio = document.createElement("audio");
audio.autoplay = true;
peer.ontrack = (event) => {
  audio.srcObject = event.streams[0];
};

const offer = await peer.createOffer();
await peer.setLocalDescription(offer);

const response = await fetch("/api/calls", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ sdp: offer.sdp, session: validatedSession }),
});

const answer = await response.json();
await peer.setRemoteDescription({ type: "answer", sdp: answer.sdp });

Keep OPENAI_API_KEY out of this browser code.

5. Create one capture context per call

Create stable identifiers before requesting microphone access. This lets you capture failures that happen before WebRTC connects.

type TranscriptEntry = {
  role: "user" | "assistant";
  text: string;
};

type OpenAIVoiceCapture = {
  sessionId: string;
  userId: string;
  boundaryEventId: string;
  assistantId: string;
  voiceId: string;
  callId?: string;
  startedAt: number;
  endedAt?: number;
  success: boolean;
  error?: string;
  transcript: TranscriptEntry[];
};

const capture: OpenAIVoiceCapture = {
  sessionId: crypto.randomUUID(),
  userId: getCurrentOrAnonymousUserId(),
  boundaryEventId: crypto.randomUUID(),
  assistantId,
  voiceId,
  startedAt: Date.now(),
  success: true,
  transcript: [],
};

Use a privacy-reviewed product user ID when one exists. For an anonymous application, persist a random UUID. Do not derive identity from a transcript, IP address, API key, or SDP payload.

6. Retain finalized Realtime transcripts

OpenAI sends incremental assistant transcript deltas and completed transcript events. Update the active UI row for deltas, but persist finalized evidence once.

Useful event boundaries include:

conversation.item.input_audio_transcription.completed
response.output_audio_transcript.delta
response.output_audio_transcript.done
response.done

A simple normalized handler looks like this:

events.addEventListener("message", (message) => {
  const event = JSON.parse(message.data);

  if (event.type === "conversation.item.input_audio_transcription.completed") {
    retainFinalTurn("user", event.item_id, event.transcript);
    return;
  }

  if (event.type === "response.output_audio_transcript.delta") {
    updateActiveAssistantTurn(event.item_id, event.delta);
    return;
  }

  if (event.type === "response.output_audio_transcript.done") {
    retainFinalTurn("assistant", event.item_id, event.transcript);
  }
});

Index turns by the OpenAI item ID. Replace an active delta with the completed transcript and suppress exact duplicates. Do not append every delta as a separate turn.

7. Upload the completed call without blocking audio

When the call ends or fails, freeze the payload once and post it to your own capture route:

async function uploadCapture(capture: OpenAIVoiceCapture) {
  capture.endedAt ??= Date.now();
  const payload = structuredClone(capture);

  for (const delay of [0, 750, 2_000]) {
    if (delay) await new Promise((resolve) => setTimeout(resolve, delay));

    try {
      const response = await fetch("/api/currai/voice", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify(payload),
        keepalive: true,
      });

      if (response.ok) return true;
      if (response.status < 500 && response.status !== 429) return false;
    } catch {
      // Currai capture must never interrupt the OpenAI voice experience.
    }
  }

  return false;
}

Reuse the same payload and event IDs on retries. New IDs would turn a retry into duplicate trace evidence.

8. Validate and redact on the server

Reject malformed UUIDs, oversized transcripts, unknown roles, invalid provider IDs, and sensitive keys. Allow only user and assistant transcript roles and cap text length before sending anything to Currai.

Never accept or forward fields named like:

authorization
cookie
password
secret
apiKey
accessToken
refreshToken
bearer

Do not store SDP offers, raw audio, bearer tokens, or API keys in Currai metadata.

9. Create the Currai session before events

Send native HTTP capture from the trusted server:

const accepted = await sendToCurrai("/api/v1/capture-session", {
  session_id: capture.sessionId,
  user_data: {
    user_id: capture.userId,
    properties: { source: "openai-realtime-web" },
  },
  metadata: {
    route: "browser-voice-call",
    environment: process.env.NODE_ENV ?? "development",
  },
  timestamp: capture.startedAt,
  client_config: "currai-native-http",
});

if (!accepted) {
  return Response.json({ error: "Capture service unavailable." }, { status: 503 });
}

Await this request before sending the first event. An event that arrives before its session loses the context Currai needs to group the call correctly.

10. Emit readable turns with nested OpenAI execution

The expected trace is:

event: openai.realtime_call

agent: openai.conversation.turn
└── model: openai.realtime.model

Each top-level agent event must use recognized conversation fields:

const agentEvent = {
  event_id: agentEventId,
  session_id: capture.sessionId,
  parent_id: null,
  kind: "agent",
  primitive_name: "openai.conversation.turn",
  args: JSON.stringify({ input: userText }),
  result: JSON.stringify({ output: assistantText }),
  success: capture.success,
  metadata: {
    provider: "openai",
    model: "gpt-realtime-2.1",
    voiceId: capture.voiceId,
    callId: capture.callId,
  },
};

Nest the model event beneath it:

const modelEvent = {
  event_id: modelEventId,
  session_id: capture.sessionId,
  parent_id: agentEventId,
  kind: "model",
  primitive_name: "openai.realtime.model",
  args: JSON.stringify({ input: userText }),
  result: JSON.stringify({ output: assistantText }),
  success: capture.success,
  metadata: {
    provider: "openai",
    model: "gpt-realtime-2.1",
  },
};

Do not put the complete conversation only in a custom transcript property. Currai uses fields such as args.input and result.output to render readable User Stories.

11. Adapt the workflow for OpenAI TTS

For one-way speech generation, call the Speech endpoint from your server and keep the OpenAI key private:

const response = await fetch("https://api.openai.com/v1/audio/speech", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-4o-mini-tts",
    voice: "coral",
    input: textToSpeak,
    response_format: "mp3",
  }),
});

Do not upload the returned audio bytes to Currai. Capture the text and safe generation metadata:

{
  kind: "model",
  primitive_name: "openai.tts",
  parent_id: agentEventId,
  args: JSON.stringify({ input: textToSpeak }),
  result: JSON.stringify({ audioGenerated: response.ok, format: "mp3" }),
  success: response.ok,
  latency: Date.now() - startedAt,
  metadata: {
    provider: "openai",
    model: "gpt-4o-mini-tts",
    voice: "coral",
  },
}

If another model decided what to say, keep that decision as its own model event. Nest TTS as the speech-rendering step instead of presenting it as the reasoning model.

12. Verify the real experience

Restart or deploy the application and complete one real interaction.

For Realtime, say:

I need to move my appointment from Tuesday afternoon to Friday morning.

Then verify:

1. the OpenAI call connected and returned audible speech;
2. Observe → Events contains one call session and openai.realtime_call boundary;
3. openai.realtime.model is nested beneath the correct openai.conversation.turn;
4. Analyse → User Stories shows readable user and assistant messages;
5. a failed call records success: false with a safe error;
6. Currai being unavailable does not interrupt the OpenAI call.

For TTS, verify that the user-facing audio plays and the trace contains the input text, model, voice, output format, success state, and latency—but not the API key or audio bytes.

Troubleshooting

OpenAI reports failed to unmarshal SDP: EOF

Forward the browser SDP unchanged. Calling .trim() removes its final CRLF and can make the offer incomplete.

The voice call works but Currai receives nothing

Confirm the Currai credentials exist in the server process, the capture route runs on call end and failure, and Currai HTTP requests are not being made from the browser.

Events exist but the User Story is empty

Put user text in args.input and assistant text in result.output on the top-level agent event. Nested model evidence alone does not replace readable agent evidence.

Assistant text appears multiple times

Use OpenAI item IDs to reconcile deltas and final transcript events. Replace the active row when the final transcript arrives and suppress exact duplicates.

TTS audio works but the trace looks like a conversation

Represent TTS as a nested model event. Only emit a top-level conversation turn when the application actually has a user-to-assistant interaction.

For the broader architecture, see Voice agent integrations. For more context, read How to use Currai with OpenAI Realtime voice and TTS.