Every classification system starts as a tidy list of categories written by someone smart on day one. By month six, the largest category is "other" — and "other" is where the insight goes to die.
The service classified market data into a hardcoded category list. The list was reasonable when it was written and wrong within a season, because markets do not hold still and the things people sell stop matching the boxes someone drew for them.
That produced the standard failure in two directions at once. Items got forced into categories that were nearly right, quietly corrupting every aggregate built on top of them. Everything else fell into "other", which grew until it was the largest bucket and therefore the least informative.
Maintaining the list by hand did not work either. Adding a category is easy; knowing whether the category you are adding is genuinely new — or the same concept someone else already added under a different word — is the actual problem, and it needs context nobody has in their head.
A taxonomy is not a schema decision made once. It is a living structure, and anything that treats it as fixed will be wrong in a predictable direction.
An analysis service where the taxonomy is derived rather than declared. Each classification decision is made with the current category tree supplied as context, so a new item either joins an existing branch or has to justify creating one — and the justification is visible.
When two categories turn out to be the same concept under different words, they merge to a canonical form. That merge is deliberate, not a bug: a tree that sprawls is as useless as a tree that is too coarse, and the whole value is in staying legible.
The existing taxonomy is passed in with every decision. The model is not inventing categories in a vacuum — it is placing an item into a structure it can see, which is the difference between a taxonomy and a pile of labels.
Categories that are the same concept under different words collapse to one canonical tag. Deliberate consolidation keeps the tree navigable as the data grows.
FastAPI over Postgres. The interesting part is the classification logic; the storage and serving layers are conventional on purpose, so anyone can query the data directly without going through the service.
The service is containerised with an explicit rebuild step after changes, so what runs is always what was committed — no drift between the code you read and the code that is serving.
Classification behaviour is a design property here rather than a benchmark figure.
"Other" stopped being a dumping ground, which means the aggregates built on top of the categories became worth trusting.
The taxonomy tracks the market rather than the assumptions held when the schema was written — new segments appear because they exist, not because someone remembered to add them.
Maintenance stopped being a recurring chore with no owner.
Any dataset where categories are contested, drifting, or arriving faster than a human can curate: product catalogues, market and competitor tracking, support ticket triage, spend classification, content tagging.
It is unnecessary where the taxonomy is genuinely fixed by an external standard. If a regulator defines your categories, use theirs — the fixed list is the requirement, not the problem.
Tell us what's not working. The first conversation is free — whether it's a build, a long-term partnership, or just questions, you'll get an honest read and a clear next step.
Start the conversation →