AI agents are easy to demo and hard to keep running. An agent that works perfectly in a test chat can loop, return broken output, forget context or slow to a crawl once real users arrive.
We learned this in detail rebuilding the AI concierge for SpeakerDrive, a platform for professional speakers. Its MVP ran on multi-agent n8n workflows that kept breaking in production: in one execution alone we counted 12 AI retries. After the rebuild there were zero blocking workflow failures.
The short answer: agents break in production because of state, output format, memory and tool integration, rarely because the model is not smart enough. Below are the failures we found and what fixed each one. They apply whether you build on n8n, LangChain, the OpenAI Agents SDK or your own code.
What happened: the concierge sometimes ignored what the user had just said and restarted parts of onboarding. To decide which phase the conversation was in, it scanned the long conversation history on every message, and multi-row database reads made some steps execute more than once.
The fix: explicit state instead of inferred state. We moved phase tracking into the user's profile: a message counter and persisted phase fields, updated on every turn. The agent reads one small, reliable record instead of reasoning over the whole history to work out where it is.
The lesson: never make an LLM infer something you can store. Conversation phase, the user's plan, what has already been sent: if it matters for control flow, put it in a database field.
What happened: agents intermittently returned broken JSON (for example, two output keys) or nothing at all. Downstream switches and parsers failed, and the automatic "fixer" step re-ran repeatedly, spamming the history tables.
The fix: enforce structure only where it is needed, and plan for failure. We removed structured-output parsers from steps that did not need them, tightened system prompts, and added a fallback that reruns preprocessing when the fixer returns empty. For the hardest cases we evaluated a two-step pattern: the agent reasons freely first, and a separate formatter step produces the structured output.
The lesson: structured output is a contract, and models break contracts. Validate every output, keep a fallback path, and do not ask for JSON when plain text will do.
What happened: memory was duplicated across workflows and grew token-heavy, so every call carried more context than it needed, slowing responses and raising costs. Session IDs used as primary keys also collided, because the n8n Postgres node could insert rows but not update them.
The fix: design memory like a data model. We moved the important memories (preferences, upcoming events, key context) into structured profile fields, stored as arrays with timestamps, created dedicated tables for agent history, and removed the primary-key constraint that caused the session collisions.
The lesson: "memory" is a database design problem. Decide what is worth remembering, store it in a structured way, and load only what the current step needs.
What happened: the concierge answered questions from SpeakerDrive's help docs through RAG. Two things degraded it. Switching models mid-project introduced an embedding-dimension mismatch and unreliable tool calls. And every ingestion run duplicated rows in the vector store, so retrieval returned the same chunks several times and costs rose.
The fix: one consistent stack and clean ingestion. We standardized on one provider (GPT-4o mini) for both tool calling and RAG, rebuilt ingestion to wipe and re-ingest cleanly, and then delivered a new ingestion and retrieval implementation with better chunking and page-level understanding.
The lesson: embeddings from different models are not interchangeable, and ingestion must be idempotent. If the same document can be ingested twice, it eventually will be. (Choosing the embedding model in the first place? See our comparison of 15 embedding models for RAG, and RAG vs fine-tuning if you are deciding how the agent should know things.)
What happened: the product needed to send outreach emails from the user's own Gmail account. The OAuth tokens were being stored in user profiles, where the frontend could reach them.
The fix: tokens live server-side only. We created a dedicated oauth_tokens table protected by Row Level Security so only server-side operations can read it, changed the backend so tokens are never returned to the client, and wrote a migration to move existing tokens out of the profiles table.
The lesson: every tool you give an agent is a security boundary. Credentials stay on the server, the agent asks the backend to act, and the backend decides whether it may.
RAG assistants on your documents, AI agents that act in your tools, and LLM features inside your product.
Our cutting-edge features simplify collaboration and creativity, making your workflow intuitive and efficient. Transform your vision into reality effortlessly with Hadidiz Flow.
