railway-backend-index-FIXED

Overview

This server integrates several technologies to establish a zero-latency real-time voice assistant:

• Telephony streaming accepts raw PCM audio streams from Twilio call lines over WebSockets and forwards voice output deltas back to the caller
• Real-time AI logic connects to OpenAI Realtime API using WebSockets to provide conversational audio with Voice Activity Detection (VAD) and interrupt handling
• Asynchronous logging persists transcripts, call details, and AI-generated summaries into Supabase without blocking active call streams
• Multi-user isolation isolates configurations and call histories using custom headers or query parameters for dashboard APIs

How it works

The lifecycle of an incoming call flows through the following stages:

1. Twilio webhook receives an inbound call at /incoming-call/:agentId? and queries Supabase for the assigned user and agent configuration
2. TwiML response instructions tell Twilio to open a WebSocket connection containing metadata to the /media-stream/:agentId/:userId? server route
3. OpenAI session handshake initializes a WebSocket connection to wss://api.openai.com/v1/realtime with custom settings (voice model, VAD threshold, prompts, and tool configurations)
4. Bidirectional relay pipes audio payloads between the Twilio stream and the OpenAI socket. If the user begins speaking during assistant output, the server issues a truncate command and clears the Twilio output buffer
5. Function calling processes realtime tool outputs like end_call or save_contact
6. Graceful cleanup triggers on connection close, prompting Fastify to update the call record and call extractContactFromTranscript() asynchronously to extract structured database entries and generate AI summary details

API

HTTP and WebSocket Endpoints

Helper Functions

Configuration

The server relies on the following environment variables:

Where skills live

This server configuration maps across the following workspace files:

Usage

Start the Fastify backend server locally:

Copynode railway-backend-index-FIXED.js

Example request to sync a prompt configuration:

Copyconst response = await fetch('http://localhost:3000/api/sync-prompt', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    userId: 'user_12345',
    agentId: 'default',
    prompt: 'You are a friendly agent helping with appointment scheduling.',
    speaksFirst: true,
    voice: 'marin'
  })
});
const data = await response.json();
console.log(data.success);

Dependencies

• fastify — core framework used to listen to HTTP calls and manage API routing
• ws — creates connections to OpenAI WebSocket endpoints and handles standard socket actions
• dotenv — parses local file properties into node environment context variables
• @fastify/formbody — middleware translating form urlencoded inputs from Twilio
• @fastify/websocket — adds WebSocket routing structures inside Fastify
• @fastify/cors — controls resource permissions across differing origin requests
• @supabase/supabase-js — client used to interface with Supabase databases
• twilio — telephony client utilized to trigger call state closures

Best practices

• Use safe database wrappers always encapsulate Supabase queries with safeSupabaseOperation retry logic to ensure server errors do not disconnect live voice streams
• Asynchronous workflow processing defer tasks like transcript database syncs and OpenAI contact parsing until after connections close or run them in parallel without awaiting
• Resilient socket management handle potential OpenAI WebSocket dropouts by checking states and initiating reconnectConversationWs while preserving state indices
• Enforce isolated user scopes require authorization headers or query parameters to ensure caller data and configuration details do not overlap