Why the totals change
An LLM is great at reading, writing and explaining. It is not a calculator you can audit.
Ask it to total a big file twice and you may get two different answers. Even when the chat writes code for you, it can write different code each run. Sometimes it only reads part of the file. Sometimes it guesses what a column means, and guesses wrong.
The line
For a quick one-time look, prompt-only is fine. For a number someone will make a decision on, it isn't.
The fix is old-fashioned: a pipeline
- Load the file the same way every time.
- Validate it. Missing columns, duplicates and wrong types get caught before anything else runs.
- Store the clean data, then compute the numbers in code. Same input, same output, every run.
- Serve the results through an API. Then the AI explains: it summarizes, charts and answers questions on numbers that are already checked.
- Monitor it. The same input should always give the same answer. If it doesn't, you hear about it before anyone uses the number.
Side by side
Prompt-only
file → AI chat → answer Fast, but the answer can change.Pipeline + AI
file → load → validate → store → compute → API → AI explains A bit more setup, and the answer is the same tomorrow.The AI still does the part it's great at. It just isn't the thing doing the math.
My rule: code computes, AI explains. Where do you draw that line in your work? Tell me on LinkedIn.
Further reading
The post is an explainer in my own words. These go deeper on the two halves of it.