Stabilizing an AI Concierge for SpeakerDrive to Boost Activation and Enable One-Click Outreach
Engineered a production-ready AI chat concierge with reliable RAG help-doc answers, durable client memory, and Gmail sending—turning onboarding conversations into actionable outreach inside the product.
"From fragile agent workflows to a stable, tool-using AI concierge with secure Gmail sending"
Frontend
Backend
AI Models
Infrastructure
12
AI retries observed in a single execution (pre-fix)
Diagnosed auto-output-fixer loops that created 24 duplicate DB rows and slowed chat
0
Blocking workflow failures after fallbacks and prompt/tool fixes
Added retry + fallback paths for empty/invalid agent outputs and implemented error monitoring workflow
The Challenge
Problem Statement
SpeakerDrive’s MVP relied on multi-agent n8n workflows that frequently broke in production: duplicated token-heavy memory, inconsistent JSON outputs (duplicate keys/empty outputs), phase confusion causing the bot to loop onboarding messages, and slow chat latency. On top of that, the client needed retrieval over help docs (RAG) and an in-app Gmail integration for cold outreach—without exposing OAuth tokens to the frontend.
The Solution
Our Approach
Re-architected the n8n agent system around a leaner, phase-aware flow backed by Supabase: moved critical state (message counter, client memories, phase signals) into the profiles table; created dedicated tables for agent history and secure OAuth tokens; added robust fallbacks and retries; rebuilt the docs ingestion + retrieval pipeline for consistent embeddings; and delivered a polished UI/UX for outreach composition and sending with production Netlify deployment.
AI Implementation
Help-Docs RAG + Tool-Driven Client Memory
Technical Details
Implemented a retrieval-augmented generation loop over vectorized SpeakerDrive help/strategy docs (Supabase pgvector-style storage) with ingestion safeguards (delete/re-ingest to eliminate duplicates; later rebuilt ingestion and retrieval for better chunking and relevance). Engineered tool usage for reading/writing client memories and profile context: stored memories as arrays, included timestamps, and added an analysis layer to capture sentiment and escalation signals. Resolved model/tooling incompatibilities by standardizing on OpenAI (GPT-4o mini) after Gemini embeddings/tool usage caused vector-size mismatches and unreliable tool calls.
Business Value
Users got accurate, instant answers grounded in internal documentation while the system quietly retained key preferences, upcoming events, and context—improving perceived personalization and reducing support load. The client also gained structured sentiment + escalation signals for proactive follow-up.
Technical Deep Dives
Challenges We Solved
n8n Postgres memory + primary key collisions
Session IDs were duplicated and n8n’s Postgres node supported inserts but not updates, causing failures when session_id was used as a primary key.
Removed the primary key constraint on session_id (and redesigned storage expectations), restoring reliable inserts per interaction without breaking user sessions.
n8nSupabase Postgres
Agent output instability (duplicate keys / empty output) causing workflow breaks
Agents intermittently returned malformed JSON (e.g., double \"output\" keys) or empty outputs, breaking downstream switches/parsers and triggering repeated auto-fixer executions that spammed history tables.
Reduced unnecessary structured-output enforcement: unlinked output parsers where not required, tightened system prompts, and added a fallback to rerun preprocessing when the auto-fixer returned empty. Proposed (and evaluated) a two-step approach (agent reasoning first, formatter chain second) as a controlled last-resort stabilization pattern.
n8nStructured Output ParserLLM prompt engineering
Phase confusion and repeated onboarding loops
The bot sometimes ignored current user messages and continued onboarding sequences because phase detection relied on scanning long conversation history and multi-item Supabase reads that caused nodes to execute multiple times.
Moved phase/state tracking to user profile state: introduced a message counter and persisted phase-related fields in profiles, removing dependency on ai_conversation history retrieval and eliminating redundant Supabase reads before the main agent.
Supabasen8nState machine design
RAG retrieval failure due to embedding/model mismatch and ingestion duplication
Switching to Gemini introduced embedding dimension mismatches and poor tool usage; docs ingestion also duplicated rows on every run, degrading retrieval quality and costs.
Standardized on GPT-4o mini for tool + RAG reliability; rebuilt ingestion to wipe/re-ingest cleanly (delete rows where id > 0) and later delivered a new ingestion + retrieval implementation that improved page-level understanding and answer quality.
Supabase Vector StoreOpenAI embeddings/LLMn8n ingestion workflow
Secure Gmail OAuth implementation with RLS and server-side token storage
OAuth tokens were being stored in user profiles and risked exposure to the frontend; authentication errors (invalid state parameter) and deployment environment quirks complicated rollout.
Created a dedicated oauth_tokens table with Row Level Security restricting access to server-side operations, updated backend APIs to store tokens securely and never return them to the client, adjusted frontend flows to function without token access, and wrote a migration to move existing tokens out of profiles.
Supabase RLSGoogle OAuth (Gmail)Backend APINetlify
Process
Project Timeline
1
Discovery
Reviewed the existing SpeakerDrive onboarding/chat workflow, identified failure modes (JSON parsing, token bloat, phase confusion), and aligned on priorities: stability, fallbacks, faster responses, and production-ready monitoring.
2
Build
Iteratively hardened n8n workflows and Supabase schema: fixed Postgres constraints, reduced parser brittleness, introduced stateful profiles (message_counter, memories arrays), implemented sentiment + escalation capture, rebuilt RAG ingestion/retrieval, standardized models for reliable tool calling, and developed secure Gmail OAuth with server-side token storage.
3
Launch
Shipped UI/UX improvements (dashboard animations, message rendering fixes, outreach composer UI), merged changes via PRs, deployed to Netlify, updated Google OAuth redirect URIs, published Terms/Privacy for verification, and submitted the Google app for review.
Ready to Build Something Similar?
Let's discuss how we can help transform your business with AI.
Start Your Project
