What we tell clients before they add an LLM to their product

A growing share of the enquiries we receive open with a technology instead of a problem. "We want to add AI to our platform" is a reasonable instinct and a difficult brief, because it does not say what should be true afterwards that is not true now.

Start from the failure case

The most productive early question is not what the model does when it works. It is what happens when it is wrong and sounds certain.

If the answer is that a user sees an odd suggestion and ignores it, the risk is low and you can move fast. If the answer is that an incorrect figure reaches a financial report, or a patient reads inaccurate guidance, then you are building something with review workflows, audit trails, and a human decision point in the path. That is a materially different project with a materially different budget.

Retrieval beats fine-tuning for most business cases

Teams often arrive convinced they need a custom-trained model. In practice most business problems we see are better served by retrieval over your own documents with a general model on top. It costs less, the knowledge stays current because the documents are the knowledge, and you can point at the source of any answer. That last property matters enormously the first time somebody challenges an output in a meeting.

Cost behaves differently to normal infrastructure

Traditional infrastructure cost scales with the number of users. Token cost scales with usage per user, and one heavy user can cost many times what a light one does. That distribution needs modelling before launch, rather than after the first invoice arrives and somebody asks who signed off on it.

Where it genuinely earns its place

The cases that hold up are unglamorous: summarising long documents, pulling structure out of messy input, drafting a first pass that a person then edits, and routing enquiries to the right queue. Each replaces work people find tedious rather than work they find meaningful. That is usually the tell for a feature still switched on a year later.