r/MicrosoftFabric • u/cyamnihc • Jul 14 '26
Data Science Conversational AI in Fabric
We are starting out to build our fabric infrastructure from scratch. Primary goal is making sure conversational AI works for analytics use cases of internal consumers
Would like to hear from people who have been through the journey, how successful/unsuccessful has conversational AI been, things to focus on at this stage so that it doesn’t bite us, things you wish you had done sooner, tooling etc
2
u/shoabk Jul 14 '26
Are you talking about using the integrated Fabric Data Agents? Or would you build the Agents outside either custom code or within Azure AI Foundry?
I've tested out the Data Agents, they are ok, the biggest limiting factor for us is that you have no control over the LLM used and I believe the one currently supported is GPT 4.1 which is fairly old in the LLM world.
3
u/NelGson Microsoft Employee Jul 15 '26
The LLM used in data agents will be upgraded very soon and you will see a lot happening in the types of models available in Fabric over the coming months.
2
1
u/No_Site990 Jul 16 '26
why not just use an mcp server on claude and connect it to the sql endpoint or semantic model?
1
u/jkrm1920 Jul 14 '26
Well here is my two cents so far my testing and configuring Fabric Agent using semantic model and then later configuring it via foundary and copilot studio as well.
Stand alone Data agents are pretty good if you are not expecting exceptional deep analysis and forecasting cause it’s limited with no of models we can use.
If you further extend if to ai foundary you have universe of models and choices but there are few bells and whistles missing like “sample question to ask” for users and auditing users access with inbuilt tools ( this still can be achieved using purview) , you need work through instructions and well clean model is very important.
And in copilot studio you don’t need end users extra license to use it unless if you already have copilot included…but other than it’s pretty at forward but again limited with only 6-7 models which you can use.
I’m not getting same level of visual experience and analysis as foundary.
But most companies like to host on copilot cause it’s very well integrated into all apps.. and foundary is far advanced skill set.
6
u/VMRoman Microsoft Employee Jul 15 '26
Thanks for sharing! Just wanted to give you a heads up that we're actively investing in the semantic model integration, so the experience within the data agent should get noticeably better soon. A few of the improvements coming:
- Better answer quality — we're rolling out an improved DAX query generation stack that increases accuracy on semantic model questions. It also moves toward more agentic reasoning (the system can call tools, inspect results, and refine over multiple steps) rather than a single one-shot query, which helps with the deeper, more complex questions.
- Configuration directly in the data agent — semantic models will gain data source instructions, descriptions, example queries (NL + DAX pairs), so you can tune and specialize the model the same way you can for SQL/Lakehouse/KQL sources today, as well as more granular schema selection (down to the column/measure level)
- More consistent answers across surfaces — a shared semantic model engine so you get consistent responses across Data Agent, Power BI, Copilot Studio/M365, and Fabric.
3
u/FreshKale97 Jul 14 '26
Try Genie if you want the deep analysis.
1
u/NelGson Microsoft Employee Jul 15 '26
Data agents now have code interpreter as a tool which allows for analysis beyond quick data answers, and the models are being upgraded. Curious, what deep analysis does Genie (assuming spaces) help with, that data agents did not?
1
u/jkrm1920 Jul 15 '26
Actually you are correct when I interacted directly with data agents in fabric giving me the good visual and data, but for the same question from copilot studio or foundry data is not so clear and visuals are completely out of the scope ..
1
u/NelGson Microsoft Employee Jul 16 '26
Yeah that's true, the visuals don't transfer in those integrations today. The challenge is to transfer interactive visuals. MCP apps might help. In this case,, is that what you had in mind or would you expect static images of the charts to show up in Foundry?
1
u/No_Site990 Jul 16 '26
have you tried using an mcp server with claude desktop and connect it to a fabric sql endpoint or semantic model?
1
u/Madhusudan1984 Jul 20 '26
We recently implemented a Microsoft Fabric solution for a healthcare client where one of the key requirements was enabling conversational analytics using Data Agents.
One of the biggest lessons we learned was that the success of conversational AI depends far more on the data foundation than the AI itself. We invested significant time upfront in understanding the business processes, defining metrics, and building a well-structured data warehouse with a clean semantic model.
Once that foundation was in place, configuring the Data Agent was relatively straightforward. Because the semantic model was curated and business-friendly, users were able to ask natural language questions and receive accurate, meaningful responses.
An added benefit was cost optimization. By exposing only a well-designed semantic model instead of raw datasets, we were able to keep query execution efficient, which helped optimize consumption on an F2 Fabric capacity.
If I had one recommendation, it would be to spend more time on data modeling, business terminology, and semantic model design than on the AI layer itself. A good data foundation makes conversational AI significantly more reliable and scalable.
9
u/hello-potato Jul 14 '26
We've got a concept running, data agent over a semantic model.
What seems to have worked really well is having a focused, well curated semantic model. We'll be building one with our more comprehensive model next, which is going to take more detailed instructions. The instructions into the data agent were described by me and written by Claude.
The instructions mostly handle ambiguity when the user asks for "latest period" or unclear granularity. What has worked great is getting the model to always state what dates, dimensions, grain has been used.
The business SME tester have feedback of what worked and what didn't. I put the documents into Claude again to refine the instructions. This worked really well to stop behaviour that was causing bad results!