External system
Caller / voice provider
Conversation state and structured tool requests
Back to work
AI/SaaS / 2026 / Maintained
Voice Operations Agent
A runnable voice-operations backend and console for grounded FAQ responses, CRM updates, conflict-safe appointment booking, provider-event deduplication, human handoff, failed-transfer recovery, transcripts, tool history, and operational telemetry.
Role
Designer and engineer (solo)
Disciplines
AI Integration / Backend Systems / Full-Stack
Status
Maintained
What this system solves
Voice bots often demonstrate conversation but omit the difficult operational work: grounded answers, conflict-safe bookings, CRM updates, duplicate provider callbacks, low-confidence escalation, failed human transfer, traceability, and clear authority over external tools.
System overview
The voice system accepts provider call state, evaluates grounded FAQ confidence, validates structured tool requests, upserts CRM contacts, books appointments through a protected calendar boundary, and escalates emergency, ambiguous, or weakly supported requests. Duplicate provider events are ignored and failed human transfers fall back to a recorded take-message path instead of silently failing.
Architecture
Evidence-led public architecture showing the consequential system boundaries, trust decisions, and reliability controls without exposing confidential routes, credentials, or operating procedures.
Architecture evidence
Grounded voice operations and booking flow
A voice provider submits validated call turns to a FastAPI tool service that answers grounded FAQs, updates CRM state, books conflict-safe appointments, deduplicates callbacks, and escalates uncertain or urgent requests to a human.
Service
Voice tool service
Tenant validation, intent boundary, call state
Worker / workflow
Grounding and CRM
FAQ confidence, contact lookup, controlled updates
Control boundary
Booking boundary
Database conflict protection and calendar adapter
State / data
Handoff and audit
Human transfer, take-message fallback, call metrics
Trust boundaries
- The voice provider and model request tools but cannot write directly to CRM or calendar state.
- Provider callbacks are registered by event ID before downstream processing.
- Emergency, unsupported, and low-confidence requests escalate instead of generating an unsupported answer.
Reliability controls
- Database uniqueness constraint for concurrent booking conflicts
- Duplicate-safe provider lifecycle events
- Explicit transferred and handoff-failed states
- Bounded take-message fallback with transcript and tool history
The deterministic call path is runnable and tested; a production phone line, consent policy, retention controls, and provider acceptance testing are not claimed.
- A Vapi, Retell, Twilio, or equivalent provider creates a platform call and submits validated conversation turns to the FastAPI tool service.
- FAQ requests are answered only when retrieval confidence meets the configured threshold; weak or unsupported evidence escalates.
- CRM and appointment writes are application-owned tools that validate tenant, contact, time, and current database state.
- Appointment slots are protected by a database uniqueness constraint before an external calendar event is treated as confirmed.
- Provider lifecycle callbacks pass through duplicate-event registration so repeated delivery cannot repeat downstream actions.
- Human transfer and failed-transfer outcomes remain explicit call states with recorded reason, transcript, tools, latency, and cost.
Engineering decisions
Booking conflicts belong in the database
Conversational memory cannot guarantee that concurrent callers do not choose the same slot. A unique tenant-and-start-time constraint rejects the second write deterministically.
The model never receives calendar authority
The voice provider or model can request a booking, but a validated service adapter owns calendar credentials, idempotency, and the external write.
Escalation is a successful outcome
Emergency language, unsupported intent, and weak evidence produce explicit transfer or take-message states rather than speculative answers or silent workflow failure.
Provider callbacks are untrusted repeats
Every lifecycle callback is registered by provider event ID before processing so retries and duplicated webhook delivery remain safe.
What I built
- Designed the call, contact, appointment, provider-event, transcript, tool-history, and outcome models.
- Implemented grounded FAQ retrieval, CRM contact upsert, booking, provider-event deduplication, call completion, and summary state.
- Created the calendar adapter so validated services—not a model—own external booking writes.
- Built emergency, unsupported-intent, and low-confidence human escalation with an explicit failed-transfer fallback.
- Integrated the voice workflow into the shared operations console, n8n event handling, CI suite, runbook, and architecture pack.
Validation and results
- Created two calls that requested the same appointment slot and verified the second request was rejected by the database constraint.
- Repeated one provider callback ID and verified the second delivery was labelled as a duplicate without a repeated action.
- Tested emergency language, successful transfer state, simulated transfer failure, and the bounded take-message fallback.
- Included the voice paths in the same CI run that passed backend tests, Python compilation, frontend build, and workflow JSON validation.
Limitations
- The current portfolio build simulates calls and provides live-provider integration boundaries; it does not claim a deployed production phone line.
- Real telephony, calendar, CRM, recording consent, retention, privacy, and provider acceptance require environment-specific implementation and review.
What would come next
- Connect and validate one live Vapi or Retell path with Twilio or SIP while retaining deterministic offline demonstrations.
- Add authenticated operator access, signed provider webhooks, configurable knowledge sources, and structured call-quality evaluation.
- Perform concurrent booking, provider timeout, transfer, recording, retention, and recovery tests in the selected deployment environment.
Security and privacy
- The public project contains no recordings, real callers, customer data, telephony credentials, calendar credentials, or private transcripts.
- Emergency escalation is an engineering control, not a medical or emergency-service certification.
- The project explicitly avoids HIPAA, GDPR, SOC 2, or other unsupported compliance claims.