The Modern Data Engineer Part 15: The Question Anyone Can Ask

The button everybody asks for

Every BI project I have worked on has ended up with the same feature request: can we just export this to Excel? People want it on the report, on the dashboard, on the paginated report nobody looks at, and they usually want every row while they’re at it. It’s easy to dismiss that as a failure to embrace BI, but I think it tells us something more useful. The spreadsheet is already there, it opens without a ticket, and people know how to work with it. The BI platform asks them to learn its filter pane, its field list and somebody else’s idea of what a measure is, and then makes them work out how to get the answer they actually need. When the spreadsheet is easier, exporting is the rational choice. Everything that follows the click lands on the organization, and the reflex is contagious enough to have earned a name: exportitis.

What the export actually costs

The first cost is the easy one to describe. In Part 14 I argued for putting every consumer through a single enforcement point, so that a row filter or a column mask gets applied by the engine and there’s no route around it. An exported file has left that engine. Whatever the policy said about who could see which rows, the spreadsheet sitting on somebody’s laptop says nothing at all, and the moment it gets forwarded to a colleague who was never entitled to those rows, the control has been bypassed by somebody who was only trying to be helpful. Nobody involved did anything wrong, which is part of why training doesn’t fix it.

Then the file gets used for something. Somebody builds a weekly summary off it, somebody else builds their own summary off that one, and within a few months you have a small unmanaged pipeline running on whoever remembers to press the button. That one is familiar enough that I don’t think it needs much elaboration.

The cost I care about more is harder to see, because nothing breaks. In Part 12 I conceded that self-service BI quietly assumes a capable analyst, and that the assumption stays invisible until you serve a population whose skills run from very capable to barely present. The adoption numbers say roughly the same thing: BARC and Eckerson Group put active use of BI and analytics tools at 25% of employees, with minimal growth across the seven years they had been tracking it [1]. Part 14 dealt with the availability half of that problem. A dataset is there the moment somebody’s role entitles them to it. The language half doesn’t get closed by anything you do to the data layer, and that has held true through every generation of tooling I’ve worked with, each of which promised otherwise.

What changes when anyone can ask

Asking a question in your own words is the first interface I’ve seen that reaches past the people who already know how to query. Think about the person who knows a domain well enough to look at a number and tell you it’s wrong. They can’t write the query that would prove it. So they raise a ticket, wait for an analyst who knows SQL and doesn’t know the domain, and eventually read back an answer to a question that shifted slightly somewhere in the handover. That round trip costs a few days and usually one misunderstanding. They learn to spend it on the most defensible version of their question, and the two follow-ups that were where they were actually headed get quietly dropped.

The bigger effect is one nobody measures, because it consists of the questions that never got asked. If a request costs a ticket, a queue and two weeks, then it has to be worth a ticket, a queue and two weeks, and plenty of reasonable questions aren’t. How last month’s churn splits by tenure band, or whether the volume drop in one region is just seasonal, are exactly the kind of thing that fails that test. The decision gets made on instinct. No system anywhere records that a question was skipped, so nobody can tell you afterwards how many of these an organisation loses in a year.

Follow-up questions go unasked for a slightly different reason. Part 12 made the point that a fixed dashboard answers the question somebody had at the time it was built, and a good answer generally shows you where to look next, so anyone who gets one immediately wants several more. Under the ticket model each of those rejoins the same backlog at the back and pays the full round trip again, and two weeks isn’t an unusual turnaround for any single one of them. They also have to be asked in order, because you can’t frame the second question until you’ve seen the answer to the first, so getting three layers deep into something takes months.

Decisions don’t wait that long. They get made on the first answer, or on somebody’s recollection of a similar situation, or on the spreadsheet that happened to be open at the time.

The export button was buying familiarity. A question in plain language supplies the same thing without the data ever leaving the platform. That doesn’t make the export redundant, and in plenty of cases it can’t be. Fabric data agents, for example, are built for conversational answers and cap a response at 25 rows and 25 columns [2], which rules out most of what anyone wanted a full extract for. The agent takes away one of the reasons to press the button. Once a question can be answered where the governed data already lives, the spreadsheet has no job left as a query interface, and exportitis subsides from there. It’ll still get used for plenty of other things, which is fine.

A lower skill floor changes who’s in a position to ask, which is the point of the exercise and also the risk in it. Behaviour change is the harder half of any rollout like this and it’s usually where they fail. There’s a second risk sitting underneath that one. Point these agents at unconformed data and anybody at all can produce a confident wrong number in plain English, faster than a shared semantic model ever allowed. So it’s worth asking what has to hold underneath before the answer coming back can be acted on.

Why the instruction box is the wrong place to fix things

When you set up a Fabric data agent, the first thing you’re handed is an instruction field that takes up to 15,000 characters of plain English [3], and it’s the obvious place to start because it’s the fastest route to something that works. It does real work, too. The agent uses those instructions to pick which source to query, “using schema information and AI instructions you provide” [4], and the same pane is where you’d define your business acronyms or set the tone of the responses.

The problem is that the instructions don’t reach the part of the system that writes the query. Against a Power BI semantic model, the DAX generation tool “relies solely on the semantic model’s metadata and Prep for AI configurations” and “ignores any instructions you add at the data agent level for DAX query generation” [4]. It’s an easy line to skim past. If you type “revenue means net revenue” into that instruction pane, you’ve put your definition in the one place the DAX generator won’t look. Microsoft says so directly: “don’t add semantic model specific instructions at the data agent level. Instead, keep all semantic model instructions in Prep for AI where the DAX generation tool can use them” [4].

Databricks ends up somewhere similar from the other direction. Genie has been generally available since 2025 [5], and its guidance has you build an agent out of SQL expressions that define metrics and filters, plus Unity Catalog metric views that pre-define metrics, dimensions and aggregations. Free text comes last in that ordering, recommended “only as a last resort when SQL expressions and examples cannot address the need” [6]. Both vendors, working from different products, are pointing you past the agent’s own settings and towards the data model it reads. Microsoft is blunter about what that model should look like, since the absence of a star schema is first on its list of pitfalls, because “DAX is optimized for star schema with clear fact and dimension tables” [4].

What happens when two measures disagree

Say your model has three measures that all claim to be revenue, which isn’t unusual in a model that’s been around a few years. The agent picks one. It won’t tell you which one it picked, because from its side nothing happened that was worth reporting, and the answer comes back as a sentence with a number in it and no indication that a choice was made at all. So nobody reading it has any reason to go and check. Microsoft lists this under common pitfalls in its guidance on building semantic models for data agents: “duplicate or overlapping measures: multiple measures that calculate similar metrics (for example, Total Sales, Sales Amount, Revenue) create ambiguity” [4].

Part 13’s resolver takes that choice away from the agent entirely, and the useful side effect is that the number it returns reconciles with the number on the dashboard, because both of them came out of the same definition file. Part 4A does a version of the same job a layer further down, committing to one meaning for customer and one for order, so the agent is never choosing between three tables that disagree about grain. None of this is new work if you’ve been reading along. It’s the same modelling discipline as before, with a new reason to care about it.

Why column names matter more than they used to

Non-descriptive naming sits on that same list of pitfalls. Microsoft’s examples are object names like TR_AMT, F_SLS and DIM_GEO_01, which it says “provide no context for the DAX generation tool” [4]. A human colleague decodes names like that from a year of sitting near the person who wrote them, and most of us have worked somewhere that ran on exactly that kind of shared knowledge. An agent has the column names, the data types and whatever went into the description field. Faced with TR_AMT, it has to guess. It might mean transaction amount, transferred amount or trailing amount, and while the surrounding columns might give it a clue, a neighbouring column isn’t a definition.

That’s the point where descriptions, business-friendly names and a stated grain stop being documentation hygiene and start being an input to whether the answer is correct. As a documentation project it never finishes. It isn’t affordable either, which is why Part 5 argued for treating governance metadata as a byproduct of delivery instead. The description ships with the model, in the same file as the classification tags Part 14’s policy reads. A catalog entry carrying a description, an owner and a grain lets the agent find what the user is entitled to and, more usefully, know what it has found.

Somebody has to do that work, and who it is matters more than it used to. The analyst who took the ticket was doing two jobs at once, answering the question and quietly deciding which of the three revenue measures the question meant. Only the first of those was visible. When an agent takes over the answering, the deciding still has to happen, and it now has to happen once, in the model, in writing. Whoever builds the gold layer, whether that’s a data analyst or a BI engineer, carries more of the outcome than they did when a colleague was patching over the ambiguity one query at a time.

Lineage belongs in this section too, though it earns its place for the reader more than for the agent. Somebody about to act on a generated sentence needs a route back to where the number came from, and the four-link chain from Part 5 gives them one.

Row filters and the answer with a hole in it

The access side of this is table stakes. An agent that queries with its own service identity instead of the user’s is a data leak from the moment you switch it on, because it will cheerfully answer questions about rows the person asking was never entitled to see, and no amount of prompting fixes that. Genie evaluates data access “using each end user’s own Unity Catalog permissions” [7], and the Fabric data agent enforces least-privilege access on the requesting user’s credentials [2]. Both are generally available. An agent pointed at the warehouse chokepoint can’t query what the policy disallows, and that guarantee doesn’t depend on the agent choosing to behave itself, which is the right way round.

When an object is refused outright, these products behave more or less like the ticket queue did. Genie returns an empty response to any question about data the user can’t access [7], Fabric blocks the answer, and either way the person gets a visible event with an owner attached to it. The case that isn’t handled sits in between those two. Unity Catalog row filters are “enforced per user automatically” [7], so the query doesn’t fail, it just succeeds against a narrower set of rows, and the agent then writes the same fluent paragraph it would have written over the whole table. Microsoft names the outcome directly, noting that assets marked sensitive by Purview policies “might be inaccessible to the agent, which can result in incomplete answers” [2]. Neither vendor’s documentation describes a mechanism for telling the reader that it happened, and I did go looking for one.

Genie will at least show you the SQL it generated, which is the only mechanism on offer for checking an answer. It doesn’t help here. A row filter is attached to the table and applied by the engine, so it never appears in the query text, which means the SQL you’re shown reads as complete while the result behind it is a subset. The security boundary holds all the way through and does exactly what it was built to do. The failure is an epistemic one instead: somebody reading a fluent answer has no way to tell whether it covers the whole question or only the slice the policy let through. Because each answer tends to seed the next few questions, a gap in one propagates quietly into all of them.

How you will know whether it worked

Part 1 of this series closed on communication, on stakeholders who speak in domain logic. “Active customer” and “qualified lead” are precise in a meeting and hopelessly ambiguous in a data model. The engineer’s job is to turn that ambiguity into something queryable and agreed upon. An agent performs that translation at the moment somebody asks, and it can only do that because the ambiguity was already resolved in code. Part 1’s own example term, active customer, is the metric that Part 4B and Part 13 spent their time pinning down.

Which gives you a test that costs nothing to run. Six months after the agent lands, watch what people ask for. If export to Excel is still the top request, then people are still going elsewhere to find a language they can work in, and all the agent changed was the door they leave through. It’s a crude signal, but it’s a cheap one, and I’d trust it further than an adoption dashboard.

Part 16 goes after that metadata and asks where it is supposed to come from. Frameworks, parameterization and processing metadata turn out to be one architecture wearing three faces, and building it that way makes the metadata a product of the pipeline instead of something somebody maintains by hand.

References

[1] BARC and Eckerson Group, Strategies for Driving Adoption and Usage with BI and Analytics, Mar. 2022. [Online]. Available: https://barc.com/news/new-study-identifies-drivers-of-bi-and-analytics-adoption-in-companies-today/

[2] Microsoft, “Fabric data agent concept”, Microsoft Fabric documentation, updated 23 Jun. 2026. [Online]. Available: https://learn.microsoft.com/en-us/fabric/data-science/concept-data-agent

[3] Microsoft, “Create a Fabric data agent”, Microsoft Fabric documentation. [Online]. Available: https://learn.microsoft.com/en-us/fabric/data-science/how-to-create-data-agent

[4] Microsoft, “Semantic model best practices for data agent”, Microsoft Fabric documentation, updated 11 Aug. 2026. [Online]. Available: https://learn.microsoft.com/en-us/fabric/data-science/semantic-model-best-practices

[5] Databricks, “AI/BI Genie is now Generally Available”, Databricks blog, 12 Jun. 2025. [Online]. Available: https://www.databricks.com/blog/aibi-genie-now-generally-available

[6] Databricks, “Genie best practices”, Databricks documentation. [Online]. Available: https://docs.databricks.com/aws/en/genie/best-practices

[7] Databricks, “Set up a Genie Agent”, Databricks documentation. [Online]. Available: https://docs.databricks.com/aws/en/genie-agents/set-up