How to add a dynamic code that can run when user has some question to agent

Question

How to add a dynamic code that can run when user has some question to agent

Noopur Nanda 40

I have a foundry agent that connects with knowledge base and get me correct response from kb.But i want some way to add a code so that within my index the custom fields which i have for document name and type ofdocument i can ask agent how many total number it can run some python code and search through the search service and give me total count of it . Instead of going to knowledgebase a simple api endpoint or code interpreter tool addition. Mcp i dont want to add as its asking for audience and my apis are hosted in an web app

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-25T20:03:32.5633333+00:00

Hello @Noopur Nanda

I hope this helps you get back on track! If you're still facing issues, could you share more details?
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-28T06:51:11.2066667+00:00

Hello @Noopur Nanda

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thankyou!
Noopur Nanda 40 Reputation points

2026-06-29T06:42:09.5266667+00:00

@Anshika Varshney can you tell me how to perform this. Lets say i have my endpoint that is doing the count and hosted in web app. Now i have a foundey agent which always gets triggered when user asks question how the agent can be able to see the endpoint response and send in prompt
Noopur Nanda 40 Reputation points

2026-06-29T08:59:06.55+00:00

@Anshika Varshney i created the openapi tool but agent with gpt-5-mini is not supported in the openapi tool. Any other way to do this ?
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-01T16:41:06.6333333+00:00
Hello @Noopur Nanda

Sorry for the delayed response.

Since GPT-5-mini does not currently support OpenAPI tools, you can use Function Calling or an Azure Function as a tool instead. The agent can invoke your endpoint, retrieve the count from your Web App or Azure AI Search, and include that result in its response. Azure AI Foundry supports tool-based integrations for these dynamic scenarios.

For troubleshooting, please verify:

Confirm that your endpoint is reachable and returns the expected response when called directly.

Check whether the endpoint requires authentication and ensure the agent can access it.

Test the endpoint independently (Postman/cURL) to validate that the count is returned correctly.

Review agent traces/logs to see whether the tool invocation is triggered when the user asks the question.

If using Function Calling, ensure the function schema is defined correctly so the model can select and invoke it.

Hope this helps. Please let us know if you have any further questions.

Thankyou!

Answer accepted by question author

Alex Burlachenko 25,120 MVP Volunteer Moderator

Noopur Nanda hi, thx for sharing urs issue here at Q&A portal,

For this use case, don’t use the knowledge base path only. KB/RAG is good for answering from docs, but it’s not great for exact counts like ‘how many docs by type’. Best fit is a custom tool that calls Azure AI Search directly and returns the count. The agent can decide to call that tool when the user asks things like ‘total documents’, ‘count by document type’, etc.

U don’t need MCP for this if u already have an API in a Web App. Expose a small endpoint like /count-documents?type=invoice or /stats/documents, then add it to the agent as an OpenAPI/function tool. The tool should query Search with filters/facets, not ask the model to count text chunks.

For Azure AI Search, exact counting can be done with $count=true, and grouped counts are better done with facets if ur field is marked facetable https://learn.microsoft.com/en-us/azure/search/search-faceted-navigation

So flow would be somthing like user asks count -> agent calls ur API tool -> API queries Azure AI Search -> returns JSON count -> agent formats the answer.

Example API response could be

{

'documentType': 'Policy',

'count': 42

}

Code interpreter isn’t the best option here bc it won’t automatically have secure access to ur Search service and it’s more for ad-hoc file/data work. For live index queries, a tool/API is cleaner and easier to secure.

At the end )))) make sure documentName and documentType fields in ur search index are marked as filterable / facetable depending on how u wanna query them. If they aren’t, the agent/tool won’t be able to count them properly without reindexing.

rgds,

Alex

&

If my answer was helpful pls mark it and additional thx if u follow me at Q&A portal

Noopur Nanda 40 Reputation points

2026-06-28T07:25:19.3966667+00:00

Hi Alex, yes i am not planning to do via kb search. I want agent should understand when an iser asking how many available and with each type. I have a code to count just want to expose that via agent. That part i am unsure. So. For that, after having api in web app what process to perform on agent side basically that i want to perform

2 additional answers

Your answer

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-25T20:03:32.5633333+00:00

Hello @Noopur Nanda

I hope this helps you get back on track! If you're still facing issues, could you share more details?
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-28T06:51:11.2066667+00:00

Hello @Noopur Nanda

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thankyou!
Noopur Nanda 40 Reputation points

2026-06-29T06:42:09.5266667+00:00

@Anshika Varshney can you tell me how to perform this. Lets say i have my endpoint that is doing the count and hosted in web app. Now i have a foundey agent which always gets triggered when user asks question how the agent can be able to see the endpoint response and send in prompt
Noopur Nanda 40 Reputation points

2026-06-29T08:59:06.55+00:00

@Anshika Varshney i created the openapi tool but agent with gpt-5-mini is not supported in the openapi tool. Any other way to do this ?
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-01T16:41:06.6333333+00:00

Hello @Noopur Nanda

Sorry for the delayed response.

Since GPT-5-mini does not currently support OpenAPI tools, you can use Function Calling or an Azure Function as a tool instead. The agent can invoke your endpoint, retrieve the count from your Web App or Azure AI Search, and include that result in its response. Azure AI Foundry supports tool-based integrations for these dynamic scenarios.

For troubleshooting, please verify:

Confirm that your endpoint is reachable and returns the expected response when called directly.

Check whether the endpoint requires authentication and ensure the agent can access it.

Test the endpoint independently (Postman/cURL) to validate that the count is returned correctly.

Review agent traces/logs to see whether the tool invocation is triggered when the user asks the question.

If using Function Calling, ensure the function schema is defined correctly so the model can select and invoke it.

Hope this helps. Please let us know if you have any further questions.

Thankyou!
Noopur Nanda 40 Reputation points

2026-06-28T07:25:19.3966667+00:00

Hi Alex, yes i am not planning to do via kb search. I want agent should understand when an iser asking how many available and with each type. I have a code to count just want to expose that via agent. That part i am unsure. So. For that, after having api in web app what process to perform on agent side basically that i want to perform

Answer 1

Hi @Noopur Nanda

I would also recommend against using the KB/RAG path for counting, it's not designed for aggregation. Instead, expose a small endpoint on your existing Web App (e.g. /count-documents?type=invoice) and register it as an OpenAPI tool in your Foundry agent.

The agent will call it automatically when the user asks count-related questions. Your API queries Azure AI Search with include_total_count=True and top=0, then returns a simple JSON like {"documentType": "Policy", "count": 42}. Just make sure your documentType field is marked filterable and facetable in the index, or the counts won't work.

Please Upvote and accept the answer if it helps!!

Answer 2

Hello @Noopur Nanda .

Yes, this is possible without using MCP.

A common approach is to add a custom tool that calls your own API hosted in Azure App Service. Azure AI Foundry agents support calling external APIs through OpenAPI-specified tools, allowing the agent to invoke your endpoint when it determines a count or aggregation query is being asked. The API can then query Azure AI Search and return the document count or other metadata statistics. [Deep Dive...0_20250801 | PowerPoint], [Azure AI Foundry | PowerPoint]

You can also use function/tool calling patterns where the agent invokes custom functions that perform Azure AI Search queries and return the results back to the agent. Foundry agents support external API integration and custom tools for these scenarios.

The built-in Code Interpreter is primarily intended for running Python code against provided files and data analysis tasks. If you need live queries against your Azure AI Search index, a custom API tool is usually a better fit than Code Interpreter. [learn.microsoft.com],

Since your APIs are already hosted in a Web App, exposing them as an OpenAPI tool is likely the simplest solution. The agent can then decide when to call the API and return responses such as:

Total document count
Count by document type
Count by custom metadata fields
Other aggregations from Azure AI Search

This avoids the need to configure an MCP server while still enabling dynamic execution against your search index.

Hope this helps. Do let me know if you have any further queries.

Thankyou!