Kaufman Lab · AI architecture
Sales and support automation

AI sales manager

An AI sales manager answers customers 24/7 using only facts from a prepared base, files leads itself and passes a person whatever the base does not cover. A human support rep costs about $2,200 a month at Upwork rates.

Volume9 dialogues, 47 messages
Running cost$0 · free model
Market price$2,200/mo — support rep
VerdictWorks

Published Aug 24, 2026 · figures checked Sep 1, 2026

How to do this yourself: AI lead intake in eight steps

  1. A base of facts. Put prices, lead times, what each service includes and the terms into documents. The bot will answer from them and nothing else.
  2. Documents only. No document, no answer. Otherwise the model will quote a plausible price you do not have.
  3. What counts as a lead. Define it up front: name, contact, task, deadline. Without that the bot just chats and lets the customer go.
  4. One entry point. Put the bot where customers already write: the website, Telegram, WhatsApp. The conversation should not spread across channels.
  5. Handoff to a person. A question the base does not cover goes to a person with the conversation history instead of going unanswered.
  6. A log. Record every dialogue and every lead. The log shows what the base is missing.
  7. Acceptance on real questions. Run a dozen typical questions: price, lead time, warranty, refunds. An answer without a source is a defect, not a detail.
  8. Two numbers. Track how many requests became leads and how many times the bot found no answer.

What fits which task

  • A steady stream of repeat questions: a bot with a base of facts pays off right away.
  • Complex deals: the bot only collects the contact and passes it to a person. It should not negotiate.
  • Medicine, law, finance: answers from documents only, and a person always checks.

Below is how our lead intake is built: nine modules, acceptance thresholds and a dialogue log.

01 · Stages

Where the project is now

DoneSurvey23 projects on GitHub, four mature references
DoneDecisionsEight ADRs with a date, an alternative and a cost
DoneSpec15 tickets chained by blocking dependencies
DoneBuildNine modules, 1,038 lines, 35 tests
DoneAcceptanceDialogues run, thresholds met
NowIn operationsystemd on an EU server, metrics being collected
02 · Task

What was needed

One requirement: stop losing requests outside working hours and turn them into leads without a person involved. First-line support answers questions about price, lead times, scope and terms, and the company's documents already cover all of them. The work is routine; the cost of a mistake is not. The customer leaves the chat with a commitment, not just text, and a lead filed with a wrong figure costs more than an unanswered request.

The whole design grew out of one constraint: asked about a price, a language model returns a plausible number instead of admitting it has none. In small talk that is forgivable; in support it is not. So every factual statement needs a source, and the source has to be checked mechanically.

MethodWhat we took from itCanonical name · source
Survey before building
review of existing solutions
23 projects on GitHub, filtered by maturity. Four are mature: Chatwoot, Onyx, RAGFlow, Dify. Conclusion: take their patterns, not the platform. At about a hundred requests a day, a five-container suite costs more than the task is worthReview of existing solutions before designthe lab's own rule, no external source
Three-round grilling
Matt Pocock's skills
Change code only after the root cause is found. Three failed attempts at one spot mean the design needs discussing, not a fourth attempt. One assignment describes one task. A positive instruction works better than a prohibitionthe lab's own rule, no external source
Recording decisions
ADR format
Eight documents: context, decision, rejected alternative, consequence, date. They sit in the repository next to the code, so "why is there no Docker here" has a written answerArchitecture Decision RecordsNygard, "Documenting Architecture Decisions", 2011
Deep modules
narrow interface, thick implementation
The channel adapter and the knowledge base module: two or three calls on the outside, anything on the inside. A web widget plugs in as an adapter, vector search goes inside the module, and the core stays untouched in both casesDeep modules; ports and adaptersOusterhout, 2018 · Cockburn, "Hexagonal architecture", 2005
Bounded loop
a hard cap on iterations
The model picks the tool itself, but gets no more than six iterations per request. A fuse against an endless chain of calls when search keeps failingReAct agent loop with an iteration capYao et al., ICLR, 2023
Loud failure
no silent fallbacks
An empty search result is neither papered over with an answer nor silently dropped: it becomes an escalation call. A missing document is a legitimate outcome, not a runtime errorAbstention; fail fastEl-Yaniv & Wiener, JMLR, 2010 · Shore, IEEE Software, 2004
Acceptance by thresholds
a dialogue set as the benchmark
Numeric thresholds were set before launch, not after the first complaint. The same set serves as the benchmark when the model changesPreregistered thresholds; golden set as a regression benchmarkNosek et al., PNAS, 2018

Off-the-shelf solutions failed on running cost. Chatwoot Captain is a paid Enterprise feature, Dify needs five or more containers, and debugging any error means digging through someone else's architecture. There is no open-source support agent that also sells: everyone builds one on general-purpose platforms.

03 · Build

How it is built

One Python process with SQLite next to it: no containers, no queue, no vector store. The platforms' patterns are still all there: routing through function calling with an iteration limit, an empty answer instead of an answer without a source, escalation as a tool, curated reference answers.

MESSAGE INTAKE input adapter: the core never knows the messenger INPUT FILTER light model spam · injection attempts · off-topic: stop here AGENT LOOP main model decides itself: answer, search, file a lead, escalate DOC SEARCH tool BM25 on SQLite FTS5 · documents read in full at most 6 rounds found not found QUOTED ANSWER chunk id in the log ESCALATE person history and draft SEND person reply in same chat LEAD agent tool contact and request to the log, operator alerted LOG code request · intent · tokens · model · lead status DOCUMENTS 1 · .md to admin chat 2 · git commit 3 · index rebuild versions and undo: git dashed: repeated steps or ones off the request path · accent: the points where it is decided whether a fact goes out
Three roles along the pipeline: a light model filters, the main model decides and searches, and a person appears only where there is no document. Both branches lead to the lead record and the log, so request-to-lead conversion and gaps in the base come out of a single query.

An empty search result never turns into an answer. It turns into an escalation call, and that branch has no other exit.

04 · Rules

What holds the design together

1
A source is mandatory A factual statement goes out only together with the document it came from. Price, date, scope, terms. The agent phrases the reasoning itself; the constraint covers facts only. In RAG evaluation terms this is faithfulness: the statement follows from the document, not from the model's memory (Es et al., 2023). During acceptance, an automatic check compares the answer with the base.
2
Not knowing is a legitimate outcome A missing document is not a runtime error and is not covered up with a generic answer. The agent calls escalation, the dialogue is flagged, and the operator gets the history and a draft reply. Editing someone else's text takes seconds; writing from scratch takes minutes. The whole scheme rests on that difference. Formally this is selective prediction: a classifier allowed to abstain is more accurate than one that must always answer (El-Yaniv & Wiener, 2010).
3
A lead is not a retelling of the dialogue The agent files a lead by calling a tool, not by telling the operator that "the customer seems interested". The contact and the request go into the log as a row the moment the lead is filed. From there, one query gives request-to-lead conversion, the top questions of the week and the gaps in the knowledge base, with no manual export from the chat.
4
Spending in one place Only the model costs money. Search, index and storage cost nothing. The full-text index lives in the same SQLite database; there is no vector database and no external service. The model is connected through an adapter: switching providers means changing two config variables, not the code.
05 · Decisions

Decisions and their grounds

Each one is written up as a separate document with a date, an alternative and a cost. Six months from now someone will ask "why is there no Docker here", and the answer has to sit in the repository next to the code.

No.DecisionGrounds
0001First version: Telegram onlyMultichannel support is the most expensive part of support platforms. It is deferred behind an adapter, not cut out
0002One process, not a platformA hundred requests a day; someone else's suite cannot be debugged
0003Agentic search, no embeddingsThe base is curated and small, so a full-text index is more precise. Vectors can be added inside the module without changing the interface
0004The base is updated by sending a file to the chatGit under the hood provides versions and rollback
0005Escalation relay, source ruleThe customer stays in one chat; an empty result never becomes an answer
0006Model behind an adapterSwitching model and provider takes two config variables
0007systemd and venv, no DockerOne process on its own server; a container adds nothing
0008Hosting in EuropeSeparate from the analytics server
06 · Limits

What the design costs

DecisionWhat it limits
Telegram onlyA customer who comes to the website cannot write to the agent
Full-text searchA question phrased in the customer's own words may miss an answer that is in the base
Curated baseIt is filled by hand; the agent does not learn on its own
Source ruleSome requests will always land on a person
Cheap modelQuality is below flagship models. Only providers that do not train on submitted data are allowed: the dialogues contain customer contacts
One process, no DockerIt does not scale horizontally

The agent answers reasoning questions in its own words: how a format works, how options differ, what the next step is. The constraint covers factual statements: price, date, scope, terms. Their source is a document in the base, and an automatic check during acceptance confirms the match.

07 · Result

What came out

Acceptance thresholds

Dialogues9from real requests
Correct≥ 90%scored by a judge model, spot-checked by hand
Facts outside the base0threshold with no tolerance
Injections blocked10from a prepared set

Zero facts outside the base is the only threshold with no tolerance. The other items allow a percentage; this one does not, because one figure that is not in the documents costs more than ninety correct answers bring in. This is an asymmetric loss function: a false "yes" and a miss do not cost the same, so the threshold is set by the expensive side.

The same dialogue set stays the benchmark when the model changes: the adapter switches the provider in config, and a run over the set shows what changed.

Reliability of the result
Sample
Nine dialogues, 47 messages over five days, August 19 to 23: 23 messages from people, 21 from the bot. This is a pilot, not statistics: at this volume you count cases, not percentages.
Checks
Acceptance thresholds were set before launch: at least 90% correct as scored by a judge model with spot checks by hand, zero facts outside the base, ten injections from a prepared set blocked. This page reports the run as thresholds met, not exact figures; those come from the production database and will be published as a separate entry.
Does not generalize
The knowledge base was empty during the pilot and zero leads were filed: what was tested is the intake, mirroring and delivery loop, not the quality of answers from documents. Zero spending is a property of the model's free tier, not of the design.
Not measured
The share of requests turned into leads and the share of escalations on live traffic; the cost of a request in tokens. Personal data protection is postponed until the system scales.
Works

The agent is deployed and takes requests around the clock. Production database snapshot as of Aug 25, 2026: nine dialogues, 47 messages over five days, zero money spent.

Repeat it yourself
Architectureflowchart above ↑ Decisionseight ADRs ↑

Kaufman Lab is an AI architecture lab. We design processes and methodologies. Code goes out of date with every new generation of models; methodology does not, so that is what the projects take apart. Any current AI model can build a system from this design.

08 · Pitfalls

What went wrong

Four failures from the commit log, all caught before a customer saw them.

Database access from the worker thread crashed the replyStorage access was moved out of the processing thread: a slow operation should not kill a dialogue
The provider signals a refusal in the body of a successful response, not with an HTTP codeThe adapter handles both cases and returns a clear reason. A neighbouring project on the same provider hit the same refusal: it is a property of the provider, not of the design
Service commands were mirrored to the operatorOnly meaningful dialogue goes to the mirror; /start and the like are filtered out
Reply text went out without escapingThe messenger's markup is a closed list of entities; everything that comes from the model is escaped before sending

A separate decision covered the risk at launch: the pilot started with no knowledge base at all. First the intake, mirroring and delivery loop was tested, and only then was search connected. That way a loop error cannot be mistaken for a search error.

09 · Possibilities

What else can be added to the code

The channel, search and model sit behind narrow interfaces, and the lead and escalation are agent tools. So a new channel, a new search or your own CRM connects through an adapter without touching the core. Below is what would strengthen the AI sales manager as a next step, including options that rely on paid services.

SearchSemantic search next to full-textA vector index goes inside the knowledge base module, for example through the sqlite-vec extension with embeddings from Yandex Cloud or OpenAI. The search tool's interface stays the same and BM25 stays alongside. A question the customer asks in their own words finds an answer the base already has.
CRMThe lead goes straight into amoCRM or Bitrix24The lead tool already writes the contact and request to the log. As a second write it can create a deal in amoCRM or Bitrix24, two CRMs widely used in Russia, through their APIs, together with the dialogue history. The manager works in the pipeline instead of copying leads from the messenger.
VoiceThe same agent takes phone callsTelephony with speech recognition and synthesis, for example Voximplant together with Yandex SpeechKit, connects as one more channel adapter. The agent searches the same base, files leads with the same tool and calls a person under the same source rule. An evening phone call becomes a lead in the log.
PaymentInvoice and payment right in the dialogueAn extension point for sales is built into the core. An invoicing tool through YooKassa, a Russian payment service, or Telegram payments sits next to the lead tool and takes the price only from a document in the base. The customer pays in the same chat where they asked the question.
PrivacyCustomer contacts never reach the modelBefore sending text to the model, the code replaces phone numbers and emails with tokens and puts the real values back into the lead. The replacement goes into the input filter that already sits in front of the core. That puts a personal data protection layer in place before the system scales.
DataA weekly report on gaps in the baseThe log already stores every request with its intent, lead status and escalation. A weekly report built from one query lists the questions with no matching document in the base and the conversion to leads. The owner knows which document to write first.
10 · Questions

What people usually ask

What is an AI sales manager?

An AI sales manager is a program that handles messenger requests instead of an operator on duty. It suits a small business that loses requests in the evening and at weekends. Kaufman Lab builds an agent that answers with facts from a curated base, files the lead and hands a person whatever the base does not cover. For comparison, a human customer service rep costs about $2,200 a month for a single shift at Upwork rates. The agent does not guess and does not quote prices from memory.

How is this different from a regular RAG bot?

The model calls search, not the surrounding code: it decides whether to search, what to search for and whether the result is enough. An empty result is a legitimate outcome, not an error. Escalation is built in as a model tool, not as a button in the operator's interface.

How is a lead filed and where does it go?

Filing is a separate agent tool: the agent calls it once the dialogue contains a contact and the substance of the request. The lead goes into the log as a row and at the same moment the operator gets a notification, with no export and no copying by hand. The dialogue continues in the same chat; the customer is not redirected anywhere.

How is the knowledge base updated?

By sending a .md file to the admin chat. The agent picks the category and location, commits to git, rebuilds the index and reports the number of documents. Versions and rollback come from git for free; a separate command rebuilds the index by hand.

What stops someone from feeding the agent a malicious prompt?

A separate light model sits at the input: spam, injection attempts and off-topic messages never reach the core. On top of that there is moderation with word lists, a warning, a block and an alert to the operator. Ten blocked injection attempts is one of the acceptance thresholds.

What happens when the person does not reply?

The dialogue is flagged as waiting for a person. The customer has already been told: "I'll check and get back to you." That is worse than an instant answer and better than a confident answer without a source.

How much does one request cost?

Only the model costs money: search, index and storage cost nothing. The exact cost per request is taken from the production database by the token counter and will appear on the card as a separate entry. For comparison, a human customer service rep costs about $2,200 a month at Upwork rates.

What is an AI employee and what can it do?

An AI employee is a program that handles the conversation with a customer instead of an operator on duty: it answers with facts from a prepared base, files the lead itself and hands a person whatever the base does not cover. Here it is a Telegram agent that runs around the clock. It does not learn or fill in gaps on its own: a person adds to the base by sending a file to the admin chat. The pilot of 9 dialogues and 47 messages over five days tested the intake loop, not the quality of answers from documents.

Will an AI sales manager replace a person?

In this build the AI manager does not replace a person: sales and upsells were deliberately left out of the first version. The agent takes the request, answers from documents in the base and files a lead once the dialogue contains a contact and the substance of the request. Everything the base does not cover goes to the operator, who gets the history and a draft reply; the operator's reply goes back to the customer in the same chat. The source rule always leaves some requests to a person. That is a property of the design, not something unfinished.

Can it take leads from a website?

Not yet: the agent works only in Telegram, and a customer cannot write to it directly from a website. The limits section says so plainly. A second channel was planned as a seam from the very first version: the web widget is postponed and will connect through a separate adapter without touching the core. Today a request comes in through Telegram, the lead from it goes into the log as a row, and the operator gets a notification at the same moment.

Sources
  1. Nygard M. Documenting Architecture Decisions, 2011. cognitect.com
  2. Ousterhout J. A Philosophy of Software Design. Yaknyam Press, 2018 — deep modules.
  3. Cockburn A. Hexagonal architecture, 2005. alistair.cockburn.us
  4. Yao S., Zhao J., Yu D., Du N., Shafran I., Narasimhan K., Cao Y. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR, 2023. arXiv:2210.03629
  5. El-Yaniv R., Wiener Y. On the Foundations of Noise-free Selective Classification. Journal of Machine Learning Research, 11, 2010 — abstention as a legitimate outcome.
  6. Shore J. Fail Fast. IEEE Software, 21(5), 2004. doi:10.1109/MS.2004.1331296
  7. Nosek B. A., Ebersole C. R., DeHaven A. C., Mellor D. T. The preregistration revolution. PNAS, 115(11), 2018. doi:10.1073/pnas.1708274114
  8. Es S., James J., Espinosa-Anke L., Schockaert S. RAGAS: Automated Evaluation of Retrieval Augmented Generation. 2023. arXiv:2309.15217 — faithfulness: a statement follows from the document.
  9. Zheng L. et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS, 2023. arXiv:2306.05685
  10. OWASP. Top 10 for Large Language Model Applications — LLM01: Prompt Injection. owasp.org

Need a system like this?

Message me on Telegram: I answer myself, look at your task and tell you which of this fits it.

Message on Telegram→
More projects from the lab
↑ top