AI should amplify what is already great about how we work - not add noise.
I have spent the better part of the last two years experimenting with how AI can meaningfully interact with data platforms. From Slack-based prototypes to MCP integrations, from ChatGPT connectors to Claude plugins, the experimentation has been relentless - mostly because the landscape moves so fast that what was cutting-edge six months ago is now table stakes.
This post captures what I have learned, what I believe, and where I think the industry is heading.
The Promise and the Trap
The promise of AI in data is seductive: ask a question in plain English, get an answer from your data warehouse. No SQL. No waiting for an analyst. Instant insight.
The trap is that this only works when the foundation is strong. Without good metadata, clear business semantics, and proper access controls, AI becomes a confident hallucination machine. It will join tables that should never be joined. It will invent metrics that do not exist. And because it speaks with authority, people will trust the wrong answer more readily than they would trust a blank screen.
This is the central tension I keep coming back to: AI analytics only works well when the data foundation is strong. The unsexy work of cataloguing datasets, writing descriptions, defining semantic layers, and enforcing contracts - that is the work that makes AI useful. Without it, you are building a sports car on a dirt road.
Platform Over Chatbot
Early on, I built bespoke chatbots that could answer data questions. They worked, impressively at times, but they required constant prompt engineering and broke every time models were updated. The maintenance burden was unsustainable for a small team.
The shift in my thinking came when I realised that connecting AI to a well-structured data platform gives better ROI with far less maintenance. Instead of teaching the AI about our data, we structure our data so that any AI can understand it. A semantic layer through Looker means consistent business metrics. Metadata coverage means the AI knows what columns mean. Access controls mean the AI cannot leak sensitive data.
This is platform thinking applied to AI: invest in the foundation, and the applications build themselves.
Agentic Engineering
The concept of agentic engineering - where AI agents run tools in a loop to achieve a goal - is where things get genuinely exciting. An agent does not just answer a question. It decomposes a problem, calls tools, evaluates results, and iterates. It is the difference between a calculator and an engineer.
I recently read Péter Szász's piece on Agentic Engineering Management and it crystallised something I had been feeling. He maps three pillars - execution, team dynamics, and personal development - against autonomy fitness and trust gradients. The insight that resonated most: "The purely operational EM role is the most at risk of becoming obsolete. Those who center their role on judgment, relationships, and organisational influence become more valuable."
I think the same applies to data engineers. The purely operational data engineer - the one who writes pipelines and monitors DAGs - faces the same shift. AI agents will handle routine pipeline creation, schema migration, and incident triage. What remains, and what becomes more valuable, is the judgment: which data to collect, how to model it for the business, where to invest in quality, and how to build trust across teams.
What I Have Built
Over the past year, my team and I have built several things that reflect this philosophy:
AI-assisted analytics: Connecting AI tools to our Looker semantic layer so that anyone in the company can ask data questions in natural language and get answers grounded in our governed metrics.
MCP integrations: Building Model Context Protocol servers that let AI agents interact with our BigQuery warehouse, Looker, and internal tools. This is the plumbing that makes agentic workflows possible.
Claude plugins: Creating custom plugins that encode domain knowledge - our specific metrics, data models, and business context - so that AI assistants understand our world without needing to be re-prompted every time.
Incident management tooling: Using AI to help with data incident communication - summarising impact, generating stakeholder updates, and suggesting root causes based on pipeline metadata.
The Human Element
For all my enthusiasm about AI, I want to be clear about something: the human element is not going away. It is becoming more important. When AI handles the how, humans shift to the what and the why. When agents can execute, the value moves to judgment, taste, and relationships.
This is why I believe the best data engineers of the next decade will not be the ones who write the most SQL or deploy the most pipelines. They will be the ones who understand the business deeply, who can translate ambiguous stakeholder needs into precise technical requirements, and who can build trust across organisational boundaries.
The agentic engineer is not someone replaced by agents. It is someone who orchestrates agents to amplify their own judgment and impact.
What Comes Next
I am optimistic. Not in a naive way - I am well aware of the hype cycle, the hallucination risks, and the cultural resistance to change. But I have seen enough working prototypes, enough genuine time savings, and enough delighted stakeholders to believe that AI-augmented data work is not a fad. It is a fundamental shift in how we operate.
The question is not whether to adopt AI in your data practice. The question is whether your data foundation is ready for it. If it is not, that is where to start. The AI will meet you there.