Context for AI, context by AI: metadata needs a confidence score
Every AI pitch in the data space leans on one word, context, and quietly uses it to mean two different things. Context for AI is what people write so the model understands the business. Context by AI is what the model derives on its own and hands back as if it were fact.
The first kind of context is a foundation. The second is a guess, and a guess should tell you how good a guess it is.
What can’t AI learn about your business from your data?
John Giles has built conceptual data models for organizations ranging from not-for-profits to clients so secretive he never learned what they did. He recently argued that every organization has corners AI cannot see, and that those corners come in distinct kinds. Some are hidden on purpose, like acquisition plans that are never documented or are kept well out of sight. Others were never written down at all: He describes a room of social workers who discovered their own shared way of working only when a modeling workshop put their separate services side by side. The last kind does not exist yet, because the business is inventing something new and there is no data for AI to read.
From this, he concludes that AI should assist the modeler rather than take the driver’s seat. We agree, and we would add one point to his argument: the corners AI cannot see are exactly where it starts to guess, and most tools give you no way to tell a guess from a fact.
Context for AI is written by people
A model can only extend the context it was given. Reading your schemas teaches it how an application chose to store something, not what your business means by customer or revenue. So the foundation of any conceptual data model has to come from people. It’s built from your concept categories, the glossary definitions your team argued its way to, your naming conventions, and your requirements.
In DeltaVault, that business context is a governed setting, not a prompt. Your organization’s categories and conventions are injected as a context layer into every AI skill that depends on them. Glossaries and requirements carry more weight still, because they’re must-honor types, not background reading.
Context by AI extends the foundation, and it compounds
AI earns its keep by extending that foundation. It drafts descriptions, proposes relationships between tables, and suggests in Map & Match which business entity a source table represents. That’s context by AI. It’s valuable because it’s cheap to produce.
So what goes wrong? A drafted description reads as authoritatively as one a steward wrote. Nothing in it tells the next reader it was inferred, and that reader is often another AI run. AI-generated metadata lands in the catalog. The next skill reads it as context. A weak guess becomes the foundation for the next one, and nobody ever decided it was a fact.
Giving AI-generated metadata a confidence score
The fix isn’t to stop AI from deriving context. It’s to make every derived fact arrive with its confidence attached, then let a person decide where the bar sits.
- Suggestions carry a score and a reason. In Map & Match, every entity suggestion and every proposed table relationship comes with a confidence score and a written rationale.
- Weak guesses are flagged, not hidden. When the knowledge base proposes splitting a mixed document, a warning marks the low-confidence parts. Only parts scoring 60% or above arrive pre-selected.
- The label stays until a person owns it. A document type the AI chose carries an AI badge that shows its confidence. Once you change the type yourself, it counts as your choice.
Names, descriptions, mappings, and relationships accepted from an AI proposal keep that badge too. So the next person who opens them knows they were inferred.
Nothing writes until someone accepts a candidate. Map & Match auto-accepts a match only when it meets or exceeds a confidence floor an administrator sets and locks. So the floor is a person’s acceptance, given in advance. To pick it, review a batch of suggestions by hand. Then set it at the score above which you’d have accepted every match without correction.
Can you trust a confidence score an LLM gives itself?
The model’s own score is context by AI too, so it never travels alone. You check the rationale, because a reason can be wrong in a way you can see, and a number can’t. The National Institute of Standards and Technology’s AI Risk Management Framework makes the same case more broadly: design trustworthiness in rather than assume it.
Would a local LLM see the corners?
Why not run a local large language model inside your walls? It can read material you’d never send to a public service. That closes one gap, permission to look, but leaves the others open.
John’s most secretive client sealed its inner room off from the rest of its own building, while a catalog is built to be read by everyone. A model on your own hardware reads the same documents a hosted one does, and the social workers’ new way of working appeared in none of them. A regulator standing up under a new statute leaves nothing for any model to read, wherever that model runs.
A local model changes where the AI runs, not what exists for it to read.
People set the context and the bar
None of this argues for keeping AI away from your metadata. It argues for being deliberate about who starts the work. People write context for AI. AI proposes new context with its confidence attached. And a person decides what’s good enough to publish.
Here’s where to start.
- Write your foundation down first, so the categories, the glossary, and the naming conventions exist before any suggestion arrives.
- Point Map & Match at one ugly source, and before you accept anything, read the rationale behind its lowest-scored suggestions.
- Decide on your confidence floor before you delegate a single acceptance.