Key takeaways
- An LLM fallback in n8n means your workflow tries a second model when the first one fails, so a customer never sees silence.
- Fall back on timeouts, rate limits (429) and server errors (5xx). Do not fall back on bad requests (400), because the second model will fail the same way and you pay twice.
- Retry the primary model briefly first. Many failures clear in seconds.
- Always check the fallback answer before it reaches a user, because a different model answers differently.
- Test the fallback on purpose, before a real outage tests it for you.
Why you need an LLM fallback in n8n
On 13 November 2026, Anthropic reported elevated errors and sign-in failures across claude.ai, the Console and Claude Code, while API traffic was reportedly unaffected. The same news roundup counted it as the fourth reliability incident that month. Whatever provider you use, this pattern repeats: a model that works at 10 AM can fail at 10:05 AM.
If your n8n agent answers WhatsApp leads, books appointments or triages support mail, a provider problem becomes your problem. The customer does not know or care which model you picked. They only see that your bot stopped answering.
An LLM fallback in n8n is a simple idea: when the primary model fails in a way a retry cannot fix, the workflow sends the same request to a second model, ideally from a different provider. This guide shows what to retry, what to switch on, how to build it, and how to test it. If you are new to building agents in n8n, start with the n8n AI agent tutorial first.
What actually fails when an AI model goes down
Not every failure looks like an outage. These are the ones you will meet in production:
- Timeouts. The provider is slow, and your workflow waits until its own limit.
- Rate limits (HTTP 429). You sent too many requests, or the provider is busy.
- Server errors (HTTP 500, 502, 503). The provider has a problem on its side.
- Authentication errors (HTTP 401, 403). A key expired, was revoked or hit a billing limit.
- Bad requests (HTTP 400). Your input is wrong, too long, or blocked by a content filter.
The first three are temporary and the fallback should handle them. The last two need a human, and the next section explains why.
Which errors should trigger a fallback
This is the part most tutorials skip, and it is where money is lost. A fallback should switch on problems that a different provider can fix, and stop on problems it cannot.
| Error | Meaning | Retry first? | Switch to fallback? |
|---|---|---|---|
| Timeout | Provider too slow | Yes, once or twice | Yes |
| 429 | Rate limit hit | Yes, with a wait | Yes |
| 500, 502, 503 | Provider problem | Yes, short wait | Yes |
| 401, 403 | Key or billing problem | No | Yes, but alert a human right away |
| 400 | Bad or blocked input | No | No, fix the input |
| Context too long | Input exceeds the model limit | No | Only if the fallback has a bigger limit |
Check the exact status codes against your provider's own error documentation, because each provider groups errors a little differently.
Step 1: Pick a primary and a fallback model
Choose the fallback from a different provider than the primary. If both models come from the same company, one outage takes out both. Keep the fallback model good enough for your task, but it does not need to be the most expensive one.
Three rules that save trouble later:
- Use the same type of model. A chat model should fall back to a chat model, not to something with a different tool-calling format.
- Test your system prompt on both. A prompt tuned for one model can behave differently on another.
- Keep both API keys in n8n credentials and note which workflows use them, so a key rotation does not break the fallback silently.
If you worry about getting locked into one vendor, read how to keep your AI workflows safe from vendor lock-in.
Step 2: Retry before you switch
Most short failures clear on their own. Before switching models, let the primary model try again. In n8n, each node has a Settings tab with retry options: Retry On Fail, Max Tries and Wait Between Tries. Two or three tries with a short wait is a sensible starting point for a chatbot, because a customer is waiting.
Do not retry for a long time. A user who waits 40 seconds for a reply has already left, so a fast switch is better than a patient retry. If you want a full pattern for retries and alerts, see self-healing n8n workflows.
Step 3: Build the fallback in n8n
There are three practical ways to do it. Pick the simplest one that fits your setup.
Option A: the fallback option on the AI Agent node. Recent n8n versions let you attach a second chat model to the AI Agent node as a fallback. The limit to know about: the n8n community reports that the agent supports a single fallback, not an ordered list. For many small bots, one backup is enough. Check that your n8n version shows this option before you plan around it.
Option B: a failover loop. The public n8n template called "Build resilient AI workflows with automatic GPT and Gemini failover chain" uses the agent's error output. When the agent fails, the flow loops back, increases a failure counter, and a router picks the next model in the list. If every model fails, the workflow stops with an error. The template uses community nodes, so it works on self-hosted n8n only.
Option C: a gateway in front of the models. A proxy such as LiteLLM sits between n8n and the providers. Your workflow talks to one endpoint and the proxy handles the routing and fallback. This adds one more service to run, but it keeps your workflows simple and works for many workflows at once.
For one WhatsApp bot, Option A is usually enough. For several workflows with many models, Option C scales better.
Step 4: Check the answer from the fallback model
A fallback model gives a different answer than the primary. It may be longer, shorter, in a different tone, or wrong in a new way. Before the reply goes to a customer, add a quick check:
- Format check. If your flow expects JSON or a fixed structure, confirm the output has it.
- Length check. Block empty replies and extremely long ones.
- Language check. Confirm the reply is in the customer's language.
- Forbidden content check. Block phrases your brand never uses, such as promises about price or delivery.
- Log the model used. Store which model answered each message, so you can compare quality later.
If the answer fails the check, send a safe short message instead. Which brings us to the last layer.
Step 5: Fail safely and tell yourself
When both models fail, do not leave the customer in silence. Send a short honest message such as "We are busy right now, a team member will reply soon," and put the request in a queue for a human or a later retry.
Then alert yourself. A fallback that works quietly is dangerous, because you will run on the backup model for weeks without knowing. Send a message to Slack, email or your phone every time the fallback is used, with the error code and the time. For risky actions you should also keep a person in the approval path, as explained in human-in-the-loop AI.
My failure drill: what I broke and what happened
A fallback you have never tested is only a hope. The safest test is to break the primary model on purpose in a test copy of your workflow. Two easy ways: put a wrong API key on the primary model, or point it at a model name that does not exist.
I ran this drill on [TODO: name the bot or workflow]. Here is what I measured:
| What I checked | Result |
|---|---|
| How I broke the primary model | [TODO] |
| Time from failure to the fallback answer | [TODO] seconds |
| Did the fallback answer pass my format check | [TODO] |
| Where the fallback answer was worse | [TODO] |
| Did my alert arrive, and how fast | [TODO] |
| Cost of the same 100 test messages on each model | [TODO] |
[TODO: add one screenshot of the workflow execution and one sentence on what surprised you.]
What a fallback costs
A fallback usually costs little, because it only runs when the primary fails. The real costs are elsewhere:
- A different price per token. The backup model can be cheaper or more expensive. Work it out for your average message and multiply by your daily volume.
- Double billing on failed calls. Some failures still use tokens. Retrying a bad request or a too-long input pays twice for nothing, which is why the table above says to stop on 400 errors.
- Surprise traffic. During a long outage, all your traffic lands on the backup model. Set a spending limit on that key.
For a wider view of controlling spend, read how to control AI costs in chatbot, RAG and automation projects and the guide on setting a token budget.
Common mistakes with LLM fallback
- Falling back on every error. Bad requests fail on every model. Stop on those.
- Using two models from the same provider. One outage hits both.
- No answer check. The backup model sends something wrong and nobody notices.
- No alert. You run on the backup for weeks.
- Never testing it. The first real test is a real outage.
- Forgetting limits. The agent loops or the backup key has no spending cap.
- Long retries. The customer leaves before the switch happens.
Test checklist before you go live
- Primary and fallback models are from different providers.
- Retry has a small number of tries and a short wait.
- Only timeouts, 429 and 5xx trigger the switch, and 400 errors stop with a clear message.
- A wrong key on the primary model switches to the fallback in a test copy.
- The fallback answer passes the format, length and language checks.
- If both models fail, the customer gets a safe message and the request is queued.
- You receive an alert each time the fallback is used.
- Both keys have spending limits.
- You logged which model answered each message.
Where to go next
If you run an AI agent for customers, a fallback is one of several safety layers. An off switch for runaway agents is the next one to add, and the guide on building an AI agent kill switch shows how. If your bot answers on WhatsApp, the story of moving a WhatsApp AI chatbot from n8n to FastAPI covers what broke in a larger production setup.
FAQ
What is an LLM fallback in n8n?
It is a setup where your workflow sends a request to a second AI model when the first model fails with a timeout, rate limit or server error, so the user still gets an answer.
Should the fallback model come from a different provider?
Yes. If both models come from the same provider, one outage can stop both. A different provider gives you real protection.
Which errors should trigger the switch?
Timeouts, rate limits (429) and server errors (5xx). Do not switch on bad requests (400), because the second model will usually fail the same way.
Does a fallback model change my answers and cost?
Yes, both can change. Answers differ in tone, length and accuracy, so add a quick check before sending them. Price per token differs by model, so calculate the cost for your own message volume.
How do I test a fallback without waiting for a real outage?
Break the primary model on purpose in a test copy of your workflow, for example with a wrong API key or a model name that does not exist, and confirm the fallback answers and your alert arrives.
n8n AI Agent Tutorial: Add Tools, Memory and Human Approval
Build an n8n AI agent step by step. Add tools and memory, put a human approval step before risky actions, set limits, and avoid common mistakes before you go live.