The first time I looked at the bill for an AI chatbot I had built, I was surprised. The bot worked well. Customers were happy. But the cost per conversation was higher than I had planned, and I had no idea which part of the system was responsible.
If you build chatbots, RAG systems, voice agents or automations, sooner or later you will meet the same problem. This guide covers seven habits that help me keep AI costs under control without making the product worse. They apply whether you use n8n, Python or any other stack.
First, understand what you are paying for
Most AI providers charge by tokens, which are small pieces of text. You pay for the tokens you send (input) and the tokens the model writes back (output). Output is usually priced higher than input.
The basic formula for one request is:
cost = (input_tokens / 1,000,000 × input_price)
+ (output_tokens / 1,000,000 × output_price)
For example, imagine a model priced at $1 per million input tokens and $4 per million output tokens (these are made-up prices, only to show the math). A request with 2,000 input tokens and 500 output tokens would cost:
- Input: 2,000 / 1,000,000 × $1 = $0.002
- Output: 500 / 1,000,000 × $4 = $0.002
- Total: $0.004
That looks tiny, but multiply it by thousands of conversations, each with several turns, and it adds up. Always check your provider's current price page, because prices change.
Habit 1: Measure cost per task, not per month
A monthly bill tells you nothing about where the money goes. I log the token counts for every model call, together with a label for what the call was for, such as "answer question", "summarise call" or "classify intent".
import time
def log_usage(feature, model, input_tokens, output_tokens):
record = {
"time": time.time(),
"feature": feature,
"model": model,
"input_tokens": input_tokens,
"output_tokens": output_tokens,
}
# Save this to a database, a file or a spreadsheet
print(record)
Most APIs return the token counts in the response, so this costs you almost nothing. After a week you can see which feature is the most expensive, and the answer is often not the one you expected.
Habit 2: Use different models for different jobs
Not every task needs the biggest model. Deciding whether a message is a question, an order or a complaint is a simple job. Writing a careful reply to an upset customer is harder.
I split work into two groups:
- Small and cheap model: classification, extracting fields, simple formatting, routing
- Large model: complex reasoning, long answers, tasks where mistakes are costly
Test this carefully. Run the same sample messages through both models and compare the results before you switch anything in production.
Habit 3: Do not send more text than needed
Input tokens add up fast, and the biggest culprit is usually the prompt itself. Long instructions, repeated examples and a full conversation history sent every single turn all increase the cost.
Things that helped me:
- Remove instructions that no longer matter
- Keep only the last few turns of a conversation, plus a short summary of the earlier part
- Send only the fields the model needs, not the entire database record
Habit 4: Limit what retrieval returns
In a RAG system, every document chunk you retrieve is added to the prompt. Retrieving ten chunks when three would answer the question means paying for seven chunks of unused text on every request.
Start with a small number of chunks and increase it only if answer quality drops. Also check the size of your chunks. Very large chunks carry a lot of unrelated text along with the useful part.
Habit 5: Cache answers you have already paid for
Many users ask the same things: opening hours, prices, how to reset a password. If a question has appeared before and the answer has not changed, there is no reason to call the model again.
A simple approach is to store the question (after cleaning it up) with its answer, and check that store first. Be careful with anything personal or time-sensitive, and set an expiry so old answers do not live forever.
Habit 6: Keep outputs short on purpose
Because output tokens cost more, long answers are expensive. They are also tiring to read. In chat and voice agents, I tell the model to answer in a few sentences and to offer more detail only if the user asks.
This is one of the rare cases where the cheaper option is also the better user experience.
Habit 7: Set limits and alerts before you need them
Bugs happen. A loop that calls the model repeatedly, or a workflow that triggers itself, can burn through a budget overnight. Protect yourself:
- Set a monthly spending limit with your provider if they offer one
- Add a maximum number of model calls per conversation
- Set an alert when daily usage goes far above normal
- Add a timeout and a retry limit to every call
A quick checklist
Before you launch an AI feature, ask:
- Do I know the average cost per task?
- Is the cheapest model that works being used for each job?
- Is the prompt as short as it can be?
- Does retrieval return only what is needed?
- Are repeated questions cached?
- Are replies kept short?
- Do I have limits and alerts?
Keep your options open
Prices and models change often. Keep your model name and settings in a single configuration file, not scattered through your code. That way, switching to a cheaper or better model is a small change instead of a rewrite.
Final thoughts
Controlling AI cost is not about being cheap. It is about knowing where the money goes so you can spend it where it improves the product. Start by measuring, fix the biggest item first and repeat.
If you are planning an AI project and want help estimating the running costs before you build, you can reach me through the contact page.