Sophiie is a voice-first AI platform that automates the end-to-end workflow for trade-service businesses — from customer intake to job confirmation. A customer calls a tradie's business number. Instead of hitting voicemail, an AI receptionist picks up, understands the problem, collects details, and logs a structured lead in real-time.

The system then enriches that lead automatically: it calculates travel distance, estimates pricing using the tradie's own rates and live parts lookup, and can even analyse photos of the job (e.g. a leaky tap) using vision AI. The tradie sees the fully enriched lead appear live on their dashboard and can approve the quote with one tap. Once approved, the AI calls the customer back, confirms the price and time, and books the job.

Everything flows through two portals: Sophiie Orbit (the tradie's mobile-first workspace) and Sophiie Space (the admin control plane for onboarding and monitoring).

The primary interaction is voice. A customer dials a phone number — the AI receptionist answers immediately, has a natural conversation, and extracts structured data (name, address, issue, urgency). After the call, the system sends an SMS with a photo-upload link for visual assessment.

On the tradie side, the VoiceFab button provides a hands-free assistant. Tradies can ask "What jobs do I have tomorrow?" or say "Text the customer I'm running 15 minutes late" — the AI handles it through natural speech, pushing UI updates to the dashboard in real-time (SDUI).

Both portals show live connection status, real-time WebSocket updates, and push-banner alerts when new leads arrive. The entire system stays synchronised through persistent WebSocket connections.

1. Multimodal pipeline — voice, text, photo, and vision AI all feed into the same lead enrichment flow.
2. Dual-worker architecture — two specialised LiveKit voice agents (receptionist + tradie copilot) dispatch automatically based on caller identity.
3. SDUI (Server-Driven UI) — the AI assistant can push UI components to the dashboard in real-time, not just respond with text.
4. Full loop automation — from intake → enrichment → pricing → quote → confirmation → booking, with human-in-the-loop only at the approval step.
5. Production-grade UX — not a chatbot demo. Proper empty states, error handling, connection indicators, Snackbar feedback, and responsive layouts.

The platform is a monorepo (pnpm workspaces + NX) with three main packages and a shared backend:

hackathon/
├── apps/
│   ├── customer/    → Sophiie Orbit (Next.js 16, port 3000)
│   ├── admin/       → Sophiie Space  (Next.js 16, port 3001)
│   └── api/         → FastAPI backend (port 8000)
│       ├── routers/         → REST + WebSocket endpoints
│       ├── services/
│       │   ├── voice/       → LiveKit agent workers
│       │   ├── integrations/→ Vision, Distance, Pricing, SMS
│       │   └── realtime/    → WebSocket connection manager
│       ├── models/          → SQLAlchemy ORM models
│       └── schemas/         → Pydantic request/response schemas
├── docs/diagrams/           → Architecture & sequence diagrams
└── nx.json                  → NX workspace config

When a customer calls:

1. Twilio receives the call and forwards to /api/voice/incoming
2. The API creates a LiveKit room and connects the Customer Worker agent
3. The agent conducts a natural conversation, extracting structured lead data
4. On call end, the lead is persisted and the Lead Orchestrator enriches it:
   • Distance calculation (Google Maps API)
   • Photo analysis (Google Vision → part identification)
   • Price estimation (business rates + live SerpAPI hardware-store lookup)
5. The enriched lead is broadcast via WebSocket to all connected Orbit dashboards
6. The tradie reviews, approves, and the system triggers an outbound confirmation call

This flow demonstrates the full multimodal journey from a voice call → SMS photo request → vision analysis → automated pricing → tradie approval → customer confirmation.

The platform uses a dual-worker architecture for voice AI, powered by LiveKit Agents:

• VoiceFab — Microphone button in Orbit connects to the Tradie Worker for hands-free assistant interaction. Includes retry-on-failure, multi-state labels (Connecting/Listening/Speaking), and live transcript display.
• Inbound Calls — Customer calls are handled by the Customer Worker as a virtual receptionist (Twilio → LiveKit).
• SDUI — The assistant can push UI components to the dashboard in real-time (e.g. "Show me jobs for tomorrow").
• Smart Actions — Tools for SMS notifications ("I'm running 15 mins late"), outbound calls, and lead logging.
• Live Call Status — Real-time "Active Call" banner on the dashboard via WebSocket events.

To enable inbound phone calls, expose your local server via ngrok and configure Twilio:

1. Start ngrok:

   ngrok http 8000

2. Copy the Forwarding URL (e.g., https://1234-abcd.ngrok-free.app).
3. Configure Twilio:
   • Go to Twilio Console > Phone Numbers.
   • Select your tracking number.
   • Under Voice & Fax, set the "A Call Comes In" Webhook to: YOUR_NGROK_URL/api/voice/incoming
   • Ensure the method is POST.
4. Save and call the number to test.

Mobile-first workspace for Trade Professionals to manage daily operations.

• Enquiry Management: Real-time leads with inline location, distance, and estimate data. Snackbar feedback on call actions.
• Hands-free AI: Voice Assistant (VoiceFab) with retry-on-failure, multi-state labels, and live transcripts.
• Quote Approvals: Review and dispatch AI-generated quotes via push-banner modal.
• Smart States: Empty-state illustrations, relative timestamps, and live connection indicators (Live/Offline chip in app bar).
• SMS Photo Request: Toggle in the user menu to enable/disable automatic SMS photo requests after lead capture. Persisted per-profile via the API.
• Appointment Tracking: Today's schedule with status-coded cards, connection badge, and friendly empty-day messaging.

Internal control plane for Platform Administrators.

• Dashboard Overview: Stat cards with colored icons, live agent stage chip (pulse animation), and AI pipeline status.
• User Onboarding: Configure tradie business profiles, rates, and service areas.
• Customer Management: Searchable customer table with service-type chips, status badges, and empty-state guidance.
• Voice Orchestration: Map Twilio numbers to specific AI personas with validated inbound setup.
• System Monitoring: Status-dot indicators (green/red) for Deepgram, ElevenLabs, Twilio. Voice event timeline feed. Token testing tools.

• Node.js 20+ and pnpm (required by package.json)
• Python 3.11+ and uv (recommended) or pip
• ngrok (only needed for Twilio inbound call testing from public internet)

# Monorepo deps (from project root)
pnpm install

# Backend deps (choose one)
cd apps/api && uv sync
# OR
cd apps/api && pip install -r requirements.txt

cp apps/api/.env.example apps/api/.env
# Edit apps/api/.env (full variable matrix is below)

Optional frontend env files:

# apps/customer/.env.local
NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_WS_URL=ws://localhost:8000

# apps/admin/.env.local
NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_WS_URL=ws://localhost:8000

Run everything via NX:

# API + customer + admin in parallel
npx nx run-many --target=serve --projects=api,customer,admin --parallel=3 --output-style=stream

Or run individually:

# Terminal 1 — Backend API (NX)
npx nx serve api

# Terminal 2 — Customer Portal (NX)
npx nx serve customer

# Terminal 3 — Admin Portal (NX)
npx nx serve admin

Or without NX:

# Terminal 1 — Backend API
cd apps/api && uv run uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Terminal 2 — Customer Portal
cd apps/customer && npx next dev -p 3000

# Terminal 3 — Admin Portal
cd apps/admin && npx next dev -p 3001

# Terminal 4 — Tradie Voice Worker (Assistant)
pnpm worker:tradie

# Terminal 5 — Customer Voice Worker (Receptionist)
pnpm worker:customer

# Terminal 6 — Lead Analysis Worker (Redis queue consumer)
pnpm worker:lead

Shortcut: pnpm dev starts API + both portals via NX.

npx nx show projects                  # List all projects
npx nx run-many --target=serve --projects=api,customer,admin --parallel=3
npx nx serve customer                 # Start customer portal
npx nx serve admin                    # Start admin portal
npx nx serve api                      # Start backend
npx nx build customer                 # Build customer portal
npx nx build admin                    # Build admin portal
npx nx run-many --target=build        # Build all

If NX fails with EADDRINUSE, stop existing processes first:

lsof -nP -iTCP:8000 -sTCP:LISTEN
lsof -nP -iTCP:3000 -sTCP:LISTEN
lsof -nP -iTCP:3001 -sTCP:LISTEN

Quick health checks:

curl http://localhost:8000/health
curl -I http://localhost:3000
curl -I http://localhost:3001

The bootstrap super-admin is created automatically at API startup.

Override via environment variables:

BOOTSTRAP_ADMIN_ENABLED=true
BOOTSTRAP_ADMIN_EMAIL=superadmin@sophiie.ai
BOOTSTRAP_ADMIN_PASSWORD=d3m0-p@s5
BOOTSTRAP_ADMIN_NAME="Demo Super Admin"

Set in apps/api/.env:

Complete copy-paste templates and validation steps: docs/env.checklist.md

Frontend environment (apps/customer/.env.local and apps/admin/.env.local):

NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_WS_URL=ws://localhost:8000

1. Configure apps/api/.env with: TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_PHONE_NUMBER, LIVEKIT_API_KEY, LIVEKIT_API_SECRET, LIVEKIT_URL, DEEPGRAM_API_KEY, ELEVENLABS_API_KEY, OPENROUTER_API_KEY.
2. Enable Redis queue (REDIS_ENABLED=true) and set REDIS_URL.
3. Start API, portals, and workers (pnpm worker:tradie, pnpm worker:customer, pnpm worker:lead).
4. Expose API to Twilio:

ngrok http 8000

5. In Twilio number config: A Call Comes In -> POST https://<your-ngrok-domain>/api/voice/incoming.
6. Test by calling your Twilio number.
7. For SMS photo upload flow, keep FRONTEND_URL correct and ensure customer SMS notifications are enabled in Customer Portal.

Proprietary. All rights reserved.