Website FAQ Chatbot

Demo · All data is synthetic

I built this demo because most support chats on fintech sites are the same dozen questions about fees, payouts, refunds and disputes, and a bot that makes up an answer about someone's money does more harm than good. The company, Kestrel Pay, is made up, and I wrote its 24-entry FAQ for the demo. The widget is one script tag on any site. Each question is matched against the FAQ with a simple search, and the bot only answers when the match is strong enough, quoting the FAQ entry and showing which one it used. When it isn't confident it says so and asks for an email so a person can follow up, and if someone pastes a card number it tells them not to and keeps the number out of the logs. Every question is logged with what it matched, so the gaps in the FAQ show up on their own. It runs without any API key, and there is an optional hook for an OpenAI or Claude key that phrases the answer from the matched FAQ entries only. I did not have a key while building it, so that path is untested and the demo runs on search alone. I also wrote a small test set of in-scope, out-of-scope and edge-case questions to check it behaves the way I intended. This is a demo, not a live deployment.

How it works

Widget → POST /api/chat → small-talk / card-number check → BM25 search over the FAQ → score ≥ threshold? → answer + source : hand-off form → POST /api/handoff

Behaviour

Retrieval

Each FAQ's question and tags are weighted above its answer text, with a short synonym map. Plain BM25, no dependencies.

Confidence threshold

The top match must score at least 3.5. Below that, the bot doesn't answer and offers a hand-off instead.

Card-number guard

Anything that looks like a 13-19 digit card number gets a "please don't share card numbers" reply, and the log stores [redacted].

Logs

Each question is logged with the answer type, matched FAQ and score, so unanswered questions show what to add to the FAQ.

Optional LLM mode

With an OpenAI or Anthropic key the bot phrases answers from the matched FAQ entries only. No key was available while building it, so this path is untested.

Test set

32 test questions: 25 in scope, 5 out of scope, one card number and one greeting. I wrote them alongside the FAQ, so this checks the bot behaves as designed, not accuracy on real questions.

Demo

A short screen recording of the demo site and widget, running in search-only mode.

Screenshots

Answers with the FAQ entry they came from

Stack

Node 20 with no npm dependencies (built-in http server, vanilla JS widget) and a JSON knowledge base.

This is a demo. Kestrel Pay is a made-up company, the FAQ was written for the demo, and no real customer data, API key or paid service is used.