Adam BlansettSenior Full-Stack & AI Engineer
Featured WorkActive Open-Source Development (Apache 2.0)

AI Customer Service Platform

Open-source, self-hosted AI contact center with sub-500ms voice barge-in and multi-tenant security.

Engineering RoleLead Systems & Telephony Architect

01. Problem & Context

Traditional call center SaaS platforms charge exorbitant per-minute telephony markups, lock customer voice data into closed silos, and struggle with high latency during live phone conversations. Businesses need self-hosted voice AI that answers inbound calls, identifies itself transparently, and cleanly escalates to human agents.

02. Engineering Solution

Architected an open-source, multi-tenant contact center platform combining Twilio/SIP telephony adapters, real-time bidirectional WebSocket media streaming, explicit conversation state machines, and hours-aware human escalation with callback scheduling.

03. System Architecture

The system decouples real-time audio transport from LLM inference using a MediaStreamGateway. A Next.js 14 web console manages multi-tenant configuration, knowledge bases, and live agent takeovers, while an Express API handles telephony webhooks, speech barge-in cancellation, and CRM dispatches.

Core Architectural Components

Provider-Neutral Telephony Adapter

Standardizes inbound voice handling across Twilio webhooks and custom FreeSWITCH/Asterisk SIP trunks with compliant TwiML generation.

Sub-500ms Voice Barge-In Engine

Bidirectional WebSocket audio streaming detects caller speech interruptions and immediately sends audio cancellation frames to silence assistant speech.

Multi-Tenant BYOK Credential Vault

AES-256-GCM authenticated encryption at rest ensures tenant API keys and telephony credentials remain zero-knowledge in PostgreSQL.

Hours-Aware Escalation & Callback Engine

Evaluates timezone-specific operating hours, routes to live human agents, and captures structured callback tickets when lines are busy or closed.

External CRM & Helpdesk Webhook Dispatcher

Dispatches signed HMAC-SHA256 webhook payloads to HubSpot Tickets, Zendesk, Salesforce, and Zapier workflows.

04. Notable Engineering Decisions

Sub-500ms Audio Interruption over WebSockets

Conversations feel unnatural when callers cannot interrupt. Bidirectional streaming enables instant cancellation frames rather than waiting for HTTP audio chunks to finish playing.

Drizzle ORM with PGlite Embedded Testing

Type-safe schema definitions across all 70 automated tests run against embedded in-memory WebAssembly PostgreSQL without requiring external Docker dependencies for CI.

Apache 2.0 Permissive Licensing

Enables enterprise deployment with proprietary PBXs and on-premise hardware without triggering viral copyleft restrictions.

05. Engineering Challenges & Solutions

Challenge: Handling network jitter and telephony packet drops during live audio streaming.

Engineering Solution: Implemented stateful stream sessions with chunk buffering, jitter compensation, and graceful fallback to synchronous TwiML dialogs.

Challenge: Preventing AI hallucinations from making inaccurate commitments to customers.

Engineering Solution: Engineered strict BM25 semantic knowledge retrieval with confidence thresholding, triggering automatic agent escalation when answers are not grounded in approved docs.

06. Implementation Highlights

  • Inbound voice answering with Twilio Voice webhooks and FreeSWITCH SIP integration
  • Real-time bidirectional WebSocket media gateway with sub-500ms speech barge-in
  • Full Next.js 14 App Router administration console and live conversation monitor
  • Drizzle ORM schema with 70 automated tests running on embedded PGlite
  • BYOK AES-256-GCM credential vaulting and tenant boundary isolation
  • Complete Docker Compose deployment stack with multi-stage unprivileged containers