A RAG chatbot is only as good as the content behind it. You can pick the best model and the fastest vector database, and the bot will still give wrong answers if the knowledge base is old, duplicated or half empty.

That was my problem with the BytezTech website assistant. The site changed every week, and nobody had time to update the chatbot's knowledge by hand. So I built a pipeline that does it automatically, once a week, with no one touching it.

This post explains how it works, and more importantly the four bugs that made it unreliable until I fixed them.

The idea in one minute

Every week the workflow does this:

  1. Reads the website's sitemap to get the list of pages
  2. Scrapes the text of each page
  3. Asks GPT-4o to turn that text into clear question-and-answer pairs (FAQs)
  4. Converts each FAQ into an embedding with text-embedding-3-small
  5. Saves the vectors in Pinecone, where the chatbot searches them

I built it in n8n, and later published the workflow to the n8n Creator Portal.

Why FAQs and not raw page text

My first attempt stored raw page chunks. It worked, but answers felt vague, because a visitor asks "Do you build mobile apps?" while the page says something like "Our engineering services span cross-platform development."

Turning each page into question-and-answer pairs bridges that gap. The question side looks like what real visitors type, so the search finds the right piece more often. It also forces the model to pull out the facts that matter instead of keeping every marketing sentence.

One rule I follow: the prompt tells GPT-4o to use only the page text and not to add anything. If a page does not answer something, there should be no FAQ for it. A confident wrong answer is worse than no answer.

The four bugs that cost me the most time

1. Namespace mismatch

The pipeline wrote vectors into one Pinecone namespace, and the chatbot searched a slightly different one. Everything looked fine. The upsert succeeded, the index showed vectors, and yet the bot kept saying it had no information.

The fix was boring but important: one namespace value defined in a single place and used by both the ingestion workflow and the chatbot. If two systems need to agree on a string, never type it twice.

2. Duplicate vectors piling up

Each weekly run added the same FAQs again with new random IDs. After a few weeks the same answer existed many times, search results were full of copies, and the real variety of answers got pushed out.

The cure is a deterministic ID. Instead of a random ID, I build the ID from the page URL and the question, so the same FAQ always gets the same ID, and an upsert overwrites instead of adding a copy.

const crypto = require('crypto');

function faqId(url, question) {
  return crypto
    .createHash('sha1')
    .update(`${url}::${question.trim().toLowerCase()}`)
    .digest('hex');
}

Note that on some n8n setups you need to allow the crypto built-in module for Code nodes. If a page is removed from the site, its old vectors also need deleting, which I treat as a separate cleanup step.

3. Metadata disappearing

The first versions saved the vector but lost the extra information: the source URL, the page title and the date. Without the source URL, the chatbot could not tell the visitor where an answer came from, and I could not debug a bad answer.

The cause was a node in the middle of the flow that rebuilt the items and dropped fields. Now every vector is saved with its metadata, and I check one sample record in Pinecone after each run to confirm the fields are still there.

4. Python in an n8n world

I prototype ideas in Python because it is quicker for me. But n8n Code nodes run JavaScript, so everything I had tested had to be rewritten. Small differences caused real bugs: how dates are handled, how arrays are copied, and how undefined values behave.

My rule now: prototype the logic in Python if I like, but test the final version inside n8n with real data, not just sample input.

What a healthy weekly run looks like

I do not just trust that it ran. After every run I look at three things:

  • How many pages were scraped compared with last week. A sudden drop usually means the sitemap or a page layout changed.
  • How many FAQs were created per page. Zero means the scrape returned empty text.
  • Total vectors in the namespace. It should stay steady, not grow every week.

If any of the three looks strange, the workflow sends me an alert instead of quietly continuing.

What I would do differently

  • Use deterministic IDs from the first day. Cleaning duplicates later is painful.
  • Keep one config file for names. Namespace, index name and model names in one place.
  • Log what was skipped. Pages that fail to scrape should show up in a report, not disappear.
  • Add a review step for sensitive pages. For pricing and legal pages, I would rather have a human approve the FAQs.

Final thoughts

RAG quality is mostly a data pipeline problem. The model gets the credit when answers are good, but the unglamorous work, clean input, stable IDs, correct metadata and monitoring, is what makes the answers trustworthy week after week.

If you are building a RAG system for your own website or business and want a second pair of eyes on the pipeline, reach out through the contact page.

✳ ✳ ✳