A trust harness for a semantic layer

The problem

Most silently wrong BI numbers come from one schema shape: two independent one-to-many branches off the same parent table.

This harness audits every measure, dimension and filter combination in a star schema (90 queries), executing each one and comparing it to a ground truth computed without the join logic being tested.

Headline numbers

• 40 of 90 queries (44%) are chasm queries with no unique correct answer until someone defines an allocation rule.
• Revenue by carrier returns $149,370,462 (flat join, +39.5%), $136,329,809 (+27.3%) or $107,085,715 (the true total), depending on the implementation.
• Fan-out inflates an order-level freight quote by up to 38.6%. The flat compiler gets 6 of 12 fan-out queries wrong; the grain-aware compiler gets 0 wrong.
• The grain-aware compiler refuses all 40 chasm queries instead of answering them.

What it means

A semantic layer, or an LLM writing SQL on top of one, should refuse questions it cannot answer correctly instead of silently inventing an allocation rule. The check can run in CI, where a schema change that turns a many-to-one into a many-to-many would otherwise pass unnoticed.