Governance and Ontology

What it is

Most governance tooling asks you to govern your data twice: once in the place where you design it, and again in a separate suite where you tag columns, assign stewards, and maintain a glossary that drifts the moment anyone renames a table. DeltaVault refuses the split. Governance lives inside the business model. You describe your business once (entities like Customer and Order, and the attributes that define them), and classification, ownership, definitions, and quality expectations all attach right there. Physical tables and columns are never tagged directly; they link to the model through accepted mappings, so governance is authored in exactly one place.

The model is fed by a knowledge base: upload glossaries, requirements documents, vendor data dictionaries, and domain notes, and DeltaVault classifies them, indexes them, and puts them in front of the AI whenever it works on your metadata. Describe what you want researched and DeltaVault puts the question to a lineup of AI research vendors, translates what comes back, and proposes a curated set of knowledge documents for your review.

Standards get teeth through a validation rules engine. Conventions that would otherwise live in a wiki (naming patterns, Data Vault modeling invariants, structural integrity) run as checks when you commit metadata, with each rule tunable per organization and the strict ones able to block a commit outright.

In the free edition

Included: governing the business model. Entities, attributes and subjects, the glossary, classification settings, the category methodology, naming conventions, validation rules on the model, the knowledge base and the governance review pages are all part of the free edition. Governance review lives on the knowledge base surface rather than a surface of its own, so it comes with it.

Not included: governance of the physical catalog. Classification chips on catalog grids, derived classification down a column, Map and Match between source metadata and the business model, and the cascade impact view over physical tables all read and write real tables and columns, which the free edition has no catalog to hold. The gates that stop a build or a scheduled run are outside it too, because builds and schedules are.

The governance model itself is not thinned for the free edition. What narrows is the estate it governs: the business model rather than the business model plus the warehouse.

How it works

Govern the model, not the plumbing

Business entities and attributes carry the full governance load. Three classification schemes ship as defaults: Sensitivity (Public, Internal, Confidential, Restricted), Personal Data Categories (Contact, Identifier, Financial, Health, Biometric, Behavioral), and Regulatory (GDPR, HIPAA, CCPA, Sarbanes-Oxley). Your organization can rename, recolor, extend, or disable any of them. Deleting a classification value that’s still in use is blocked, with a usage count shown, so references never dangle.

Each entity gets an accountable owner and a day-to-day steward, picked from your organization’s users; attributes inherit their entity’s accountability. The attribute’s description doubles as its glossary definition, with a real approval workflow: definitions move from Draft to Approved with an audit stamp of who approved and when, and any later edit reverts the status so stale approvals can’t linger. Entities also carry retention policies (“7 years”) and freshness targets (“daily by 06:00”). Quality expectations are authored on attributes in the same shape as column-level expectations, ready to seed downstream.

Every one of these edits lands on your working branch, gets reviewed and committed like any other metadata change, and is versioned in Git as readable YAML. A governance pass is a reviewable unit of work, not a pile of silent database updates.

Map and match: governance without mandatory modeling

The bridge between source metadata and the business model is a two-level Map & Match workbench. Level one suggests an entity match for the tables you select in a chosen scope (a run covers exactly the rows you ticked, never a whole scope by accident); level two drills into a table and suggests an attribute match for every column. Candidates follow the launch surface: launched inside a workspace, a run matches against that workspace’s own models, with an opt-in to widen the pool to the owning project’s models too; launched inside a project, a run matches against the project’s models; launched from a connection, a run matches against every model in the organization. A workspace or project launch shows its active pool, and refuses to run against an empty one, offering a one-click way to widen it instead of leaving you stuck. The same widening ladder answers the far more common case where the pool is populated but nothing in it matched: a single-table run that comes back with no existing entity says so and offers the next scope out, this workspace, plus its project’s models, then the whole organization, one rung at a time, without discarding the suggestion it did produce. Matching is reachable from wherever you are looking at a table, not only from the workbench: right-clicking a table on any canvas that draws source tables, a project or workspace canvas, the catalog’s relationship canvas, or a table’s lineage graph, opens that table’s detail with the matching panel already on it, neighbors included. Each suggestion comes with a confidence score and a written rationale. Matching is family-aware: a derived table (a landing, staging, or persistent-staging copy) inherits the entity already mapped elsewhere in its chain, or is matched against the clean source name rather than being dismissed, and accepting one member offers to apply that entity across the rest of the family in one step. Matching is also group-aware: an entity group (a container like Order with member entities like Order Header and Order Lines) is treated as one unit, so table suggestions see the whole group and prefer whichever member’s grain matches the table, column suggestions draw on every member’s attributes and name the entity that owns each match, and a new attribute lands on the member the suggestion names. Matching also runs in reverse: standing on a business entity (its detail header, its row in the entities tree, or its node on a model canvas), one action finds the unmapped source tables that resemble it by name and by meaning and matches them against exactly that entity and its group, a bounded lookup over a capped candidate set rather than a scope scan, so the cost stays fixed as the catalog grows. In a model’s own workbench tab, the source picker starts with the projects and workspaces that carry that model, and the grid sorts the tables related to it first and starts them selected, so the deliberate path is also the short one.

Every match run, whether it is matching tables or matching one table’s columns, shows its own progress live: a candidate preparation step, then each AI batch in turn, then a finalize step, each carrying its own status as it runs, succeeds, or fails. A batch that fails stays visible on the timeline rather than being swallowed, and the run carries on to the rest instead of stopping cold; a completion summary tallies what it actually produced once it finishes. Bulk-accept handles the easy ones; an “accept all high-confidence” control with a visible threshold speeds up large scopes; re-running a match never overwrites a mapping someone already accepted. For teams that want to move one suggestion at a time, a guided review mode steps through each candidate individually, showing the rationale and confidence for each before asking for a decision. The same column-level actions travel outside the workbench too: every columns grid carries a quick action to suggest a match for one column, and another to add it as a new attribute on the mapped entity or one of its group members, named through your conventions, without a trip through Map & Match at all.

Acceptance discipline is an organization-level setting with three levels governed by an administrator lock: review each (every suggestion requires explicit approval), allow bulk (the default, which enables the bulk-accept controls), and auto-accept (the organization delegates acceptance of suggestions at or above an administrator-set confidence floor, applied as soon as a match run completes, with the rest left for human review). Auto-accept is not an ungoverned bypass: the organization sets the floor, the administrator locks the setting, and every accepted mapping is recorded under the same audit trail as a manual accept. Nothing below the floor changes without a person approving it.

The part that matters for teams who don’t want to model first: accepting a match for an unmodeled table auto-creates a skeleton entity, marked as derived, that carries governance immediately and can be deepened into a full model later. Or build the entity from the source itself: pick the columns that describe it and Map & Match creates the entity and its attributes in one pass, named through your conventions. Modeling depth is optional, and the structure is shared either way. Where a target attribute carries quality expectations, accepting a column match offers to seed those expectations onto the column. Derived skeletons show up on the modeling board like any other entity, with a subtle marker and a one-click “adopt into model” action.

One shared model to govern and bind, with live coverage read off the work

Business models, Business entities, Glossary, and Map & Match are each their own page, all reading and writing one shared model. The binding between model and sources reads both directions. From a business entity, the Overview tab shows the source tables and columns mapped to it, lets you bind one more in place, or jumps to a workbench already seeded to that entity. From the catalog you do the mirror: map a source table to an entity, or a column to an attribute, in context, with a one-click jump to the matcher when you’d rather let it suggest. Every one of these is the same write, so a bind made on the catalog shows on the entity and the reverse.

Coverage is computed straight from that same work. How far the loop has closed (sources verified, tables mapped, columns mapped, definitions approved) is read off the mappings and definitions you already authored, so the score is never a separate spreadsheet to maintain. Each open gap surfaces in context with the action that closes it: unmapped tables in the matcher, draft definitions in the glossary, stewardless entities in the model. Where an AI pipeline exists, it closes the gap through the same review surfaces it always has: unmapped tables open the Map & Match workbench, where you select the tables and run the matcher; definition drafting runs in the Ask AI pane, with results landing in the same review surfaces those pipelines already use.

Completion gates: from tracked to enforced

Tracking a gap is not the same as closing it, so governance can move from advisory to enforced without leaving the same screen. An organization administrator turns on governance requirements from a settings page under Business, off by default; turning it back off never removes the governance editors themselves, only the requirement chrome layered on top. The requirements matrix sets each asset type’s applicable properties to Off, Recommended, or Required: business entities carry all five (definition, steward, owner, category, and classification), business attributes carry two (definition and classification), and source tables and columns each carry two (being mapped or carrying an inline definition, plus classification). Recommended only nudges. Required counts toward completeness, and completeness can be wired to an actual stop.

Four gates turn that stop on: Generate build, Approve definition, Accept mapping, and Complete import, each configured Off or Block until complete, scoped to the asset types the gate checks, on the same settings page. Blocking is enforced on the server action itself, not just painted into the interface: attempting a blocked action throws a “Blocked by governance” notice listing every blocked asset and its missing properties by name (or “governance status not yet computed” when no status exists yet). The four checkpoints are real: approving an entity’s definition, a user accepting a table or column mapping draft, whether one at a time or through Map & Match’s bulk “Accept all” (the scheduled auto-accept run stays ungated, since that path is meant to run unattended), generating a build, and completing an import, though import gating only ever covers assets that already existed in the catalog before the import ran, since a table or column the import is about to create cannot be governed before it exists. Imports that update tables or columns already in the catalog are checked against the Complete import gate; brand-new assets created by the import are not, because they cannot be governed before they exist. Governance never blocks a branch commit. Wherever a governance panel already shows an asset, an incomplete one covered by an active gate carries its own “Blocked: {gate} needs {properties}” line, so the same verdict is visible before anyone attempts the action. Changing a requirement backfills every affected asset’s completion status in the background, so a property flipped to Required does not leave stale badges behind.

None of this demands modeling first. An unmapped source table or column can carry its own inline definition and classification directly on the row, the same lightweight draft described above, and a single Promote to attribute step turns that draft into a real, governed business attribute (and a derived skeleton entity if none exists yet), which then becomes the authoritative source. Every definition field, mapped or inline, carries a Draft with AI action: a preview-then-apply proposal a human reviews and applies or discards, launched with the project scope of the screen it was opened from.

Classification goes deeper, and travels further

A mapped source column or table shows its classification as derived and read-only, labeled with where it came from (“From attribute Email Address,” “From entity Customer”), because governance is still authored once in the model. But the value on the ground is not always the value the model assumes, so an “Override classification” control opens the same scheme-chip editor the unmapped path uses and takes precedence over the derived value until cleared; the underlying definition stays owned by the attribute either way, so only classification, never the glossary text, can diverge locally. Propagation runs the opposite direction: from an entity’s or an attribute’s governance panel, a “Propagate classification” action launches an AI skill that walks every source column mapped to it and proposes writing the same classification onto each one, one approvable change per column, reviewed with the same preview-then-apply discipline as every other AI action here.

A governance lint pass adds an on-demand, organization-wide sweep alongside the model’s live coverage: launched like any AI skill, it gathers the facts server-side (attributes carrying no definition, unmapped columns carrying no governance, names that break the organization’s naming convention, entities marked deprecated that are still bound to live tables) and hands back a report on its own review page, one section per finding type, each row linking straight to the fix. And classification now leaves the model in a second, physical sense: generating a Databricks table emits native Unity Catalog column tags for every governed column, a tag for each classification scheme applied, a “governance_certified” tag once the mapped attribute’s definition is Approved, and a tag naming the governing attribute, so what a steward classified in the model is queryable straight out of Unity Catalog.

The ontology at scale

The modeling board shows governance at a glance: a sensitivity chip, the steward’s avatar, and a definition-status dot on each entity card, plus filters for missing stewards and draft definitions, so the gaps are visible where the work happens. The ontology itself has structure: every entity carries a type (Person, Place, Thing, Event, and so on), domains group related models, and large models split into submodels for their subject areas. For cross-cutting sweeps there’s a flat entity directory: search by name, synonym, or technical name across every domain, filter by domain, model, category, status, or tag, and run bulk operations (set status, reassign default domain, add or remove tags, archive) over the selection.

A knowledge base the AI actually reads

Knowledge documents come in three types (glossary, business requirements, domain knowledge), and the type is a contract, not a label. Glossaries and requirements are given to the AI in full whenever they fit the context budget; domain knowledge is searched in pieces, with only the relevant sections retrieved. Every document is scoped at upload: organization-wide, or limited to one project, and retrieval honors that scope. Documents created by the Data Steward persona are organization-wide; the Data Engineer persona can scope either way.

Don’t know the type? Pick “Let AI decide” and DeltaVault reads the file after upload and chooses, showing the choice with a confidence figure you can overrule. When a file genuinely mixes content kinds (half data dictionary, half business rules), the AI doesn’t guess. It proposes a split into typed parts, parks the original, and waits for your review: accept, edit-then-accept, or reject each part, with high-confidence parts (0.60 and up) pre-selected. The split preserves your original wording; it reorganizes, never rewrites.

Every organization document also gets an import-time ontology pass: the business entities and attributes it mentions are extracted with verbatim quotes, linked to your existing catalog when the match is confident, shown on the document page for accept or reject review, and fed as a compact digest into the modeling workshop’s discovery prompts, so a requirements document contributes structure, not just searchable text.

Documents have a full lifecycle. Edit any document in place and it re-indexes automatically. Run AI-suggested consolidation over redundant documents and review each proposed merge with written reasoning and a side-by-side, word-level diff before anything changes; merged sources are archived with provenance, not deleted. When the organization has defined its business categories, consolidation can also draft a second candidate, a glossary whose terms are grouped under those category lanes, reviewed and accepted on the same screen as the merged document. Select documents and ask the AI to improve, expand, or summarise them, or ask anything free-form, with every proposed change shown as a preview you apply or discard, plus an undo window after applying. And research turns an open question into knowledge: describe what you want to know and DeltaVault puts the request to a lineup of three AI research vendors at once, each asked for its most relevant sources and held to five, so a full lineup surfaces at most fifteen distinct citations rather than dozens. At least two vendors must come back with something for the run to count, because two is the floor at which a judging pass has anything to compare and the word merge stays honest. The judge then consolidates what returns into at most one draft per document type, three in all, each carrying its citations for review before a word of it enters retrieval.

Rules with teeth

The validation engine unifies structural correctness and governance conventions under one findings model. Rules ship built-in across six packs: structural integrity (broken references, missing configuration), Data Vault modeling invariants, naming and layer-placement conventions, business-model hygiene (every entity has an identifier, every relationship resolves), industry fit (a model named for a storage layer or a technology rather than a business function, or carrying a generic domain name the organization’s industry record has not claimed), and governance completeness (personal data left unclassified or not marked restricted, a description that only repeats the name of the thing it describes). Each rule carries a severity and an enforcement level your organization can override per rule: advisory rules surface as findings in the workspace Validation tab; blocking rules stop a commit cold unless someone explicitly acknowledges them, and the acknowledgment is audit-logged with the rule identifiers. The defaults are deliberately conservative: only unambiguous breakage blocks out of the box; everything subjective starts advisory and tunable.

Naming conventions are checked against the methodology your organization has chosen. Functional layers (landing, staging, persistent staging, data vault, data mart, and so on) are canonical; methodologies group and rename them. Medallion ships built in, in Data Vault and data-warehouse flavors, with layer labels renameable per organization.

AI under the same governance

AI control follows the same lock-and-control posture as conventions and rules. Organizations govern model quality and cost through the Effort dial (Quick, Standard, Deep, or Expert) and the model each level resolves to, bring their own provider key to remove platform spend caps, set monthly credit allowances and concurrency limits by member allowance tier where they run on platform credentials, and lock the entire AI settings surface so that only administrators can change it. AI configuration is not a separate system: it is governed alongside conventions, roles, and budgets, under the same administrator discipline.

Roles without walls: one platform, two personas

The same govern-the-model flow looks different depending on who is doing the work, so DeltaVault skins itself to two personas without splitting into two products. A Data Steward sees a focused governance surface: the business model, the catalog, and the knowledge base, where definitions, classifications, and mappings stay healthy. A Data Engineer sees the full surface (modeling, the catalog, templates, pipelines) where the governed model becomes physical. Each persona reskins navigation and the landing page; neither of them changes what a person is allowed to do, because persona is orthogonal to role.

Assignment is an administrator decision: each member gets a home persona and an optional set of personas to switch between, set from the same member grid that manages roles. Platform administrators can preview the app as any persona, read-only and client-side, to see exactly what that persona experiences before assigning it. The preview is faithful: previewing a content persona hides the administration and settings areas a real holder would never see, and two administrator targets let a platform administrator preview at the right fidelity. View as Organization Admin shows exactly the chrome a regular organization administrator sees, with the platform-level tools hidden; View as Platform Admin restores the complete view in one click. Every persona can manage knowledge documents; documents from the Data Steward persona are organization-scoped. The result is that the simplification is structural, not cosmetic: stewards govern, and engineers ship, each from a surface shaped to their work.

Why it matters

Hand-run governance is plumbing. Somebody maintains a spreadsheet of which columns hold personal data. Somebody else owns a wiki glossary that was accurate eighteen months ago. Standards live in a document nobody opens, enforced by whoever happens to look at the pull request. Every one of those artifacts decays independently, and reconciling them is a recurring project with no end state.

DeltaVault collapses that into one managed structure. Classify the Email Address attribute once and every column mapped to it carries the classification, across however many landing tables, staging tables, and vault satellites that attribute touches. Approve a definition once and it’s the definition, with an audit stamp, until the text changes. Naming and modeling standards stop being prose and become checks that run at commit, where a violation is cheapest to fix. The team’s job shifts from maintaining governance artifacts to making governance decisions: who stewards Customer, whether birth date is Restricted, which rules block. The propagation, the enforcement, and the audit trail are the platform’s problem.

And because the model, its governance, and its mappings share one store with live coverage read straight off the work, “is the business governed yet” stops being a question you answer by hand. Completion is computed from the model itself, and binding a source to the model can start from whichever side you happen to be looking at, so closing the loop is the same motion whether you came from the catalog or the model.

Category contrast

Standalone governance suites govern after the fact: they crawl your platforms, build a parallel object model, and ask stewards to tag physical columns one by one, then re-tag them when the pipeline changes. The governance layer and the design layer are different products, often owned by different teams, connected by sync jobs. DeltaVault authors governance where the design happens and derives the physical view, so there is no second model to reconcile.

Catalog-first tools that added a chat panel store classification on the column. At ten thousand columns that’s ten thousand tagging decisions, and a renamed column is an orphaned tag. Deriving classification from a mapped attribute makes the per-column work a one-time match (assisted, scored, and bulk-acceptable) instead of a permanent maintenance burden.

Code-first transformation frameworks put quality tests next to the code, which is genuinely good, but they have no business ontology: no entity, no steward, no approved definition, no classification scheme. Their governance story ends at “the test passed.” Hand-built lakehouse pipelines are the same, minus the tests.