How to make use of Azure AI Services with Microsoft Entra ID and Also how to activate it if we need to use it as a SDK

Hello @Yogesh Dhavale (LTIMindtree Limited) ,

Thank you for your patience while we were working on this.

Azure AI Foundry provides a unified environment for building AI applications, managing models, creating agents, integrating AI services, securing workloads and monitoring deployments from a single workspace. It is designed to simplify AI development by bringing models, agents, tools, governance, and observability together in one platform. The capabilities being seen under the AI Services tab is a part of this.

Understanding the Difference:

AI Foundry is the central platform that helps build, manage, secure and monitor AI applications .It acts as a unified platform where models, agents, tools, and AI services can be managed together

Key capabilities include:

• AI project management
• Agent development
• Access to model catalog
• AI service integrations
• Security and governance controls
• Monitoring, tracing, and observability
• SDK and API support
• Enterprise-ready deployment capabilities

While Azure AI Language, Speech and Translator resources are dedicated Azure AI services that provide specific capabilities.

1. Azure AI Language Used for:
   • Language Detection
   • Sentiment Analysis
   • PII Detection
   • Text Classification
   • Conversational Language Understanding
2. Azure AI Speech Used for:
   • Speech-to-Text
   • Text-to-Speech
   • Speech Translation
   • Voice capabilities
3. Azure AI Translator Used for:
   • Text Translation
   • Document Translation
   • Custom Translation
   These services can operate independently or be connected to Azure AI Foundry projects when additional capabilities are required. AI Foundry supports integration with AI Language, AI Speech, AI Translator, Azure OpenAI, Vision, Search, and other services.

Please use Azure AI Foundry when

1. Building new AI applications.
2. Building AI agents.
3. Using Azure OpenAI alongside other AI services.
4. Managing multiple AI services from one location.
5. Requiring monitoring, tracing, evaluation, and governance.
6. Wanting the simplest onboarding experience.

Consider creating dedicated Azure AI Language Resources When

• Language Detection is required.
• Sentiment Analysis is required.
• PII Detection is required.
• Custom language models are required.

Consider Creating Dedicated Azure AI Speech Resources When

• Speech-to-Text is required.
• Text-to-Speech is required.
• Speech Translation is required.
• Voice experiences are required.

Consider creating dedicated Azure AI Translator Resources When

• Text Translation is required.
• Document Translation is required.
• Custom Translation models are required.

For a brand-new deployment, the recommended sequence is:

1. Create a Resource Group.
2. Create a Virtual Network.
3. Create Azure AI Foundry.
4. Create a Foundry Project.
5. Configure Private Networking.
6. Configure Microsoft Entra ID authentication.
7. Test connectivity through SDK.
8. Add Language, Speech or Translator resources only if specific business requirements need them.

Please check if the following step-by-step setup guide helps

1. Creating a Resource Group
   1. Sign in to Azure Portal.
   2. Select Resource Groups.
   3. Select Create.
   4. Enter:
      • Subscription
      • Resource Group Name
      • Region
   5. Select Review + Create. Use the Azure portal and Azure Resource Manager to Manage Resource Groups - Azure Resource Manager | Microsoft Learn
2. Creating a Virtual Network Since the deployment is VNet-secured:
   1. Navigate to Virtual Networks.
   2. Select Create.
   3. Enter:
      • Name
      • Region
      • Address Space
   4. Create the Virtual Network.
   Quickstart: Create an Azure Virtual Network | Microsoft Learn
3. Creating an Azure AI Foundry Resource
   1. Select Create Resource.
   2. Search for Azure AI Foundry.
   3. Select Create.
   4. Provide:
      • Subscription
      • Resource Group
      • Region
      • Resource Name
   5. Complete deployment. Azure AI Foundry serves as the primary platform for AI projects, models, agents, and AI service integrations Create a Foundry resource - Foundry Tools | Microsoft Learn
4. Creating a Foundry Project
   1. Open Azure AI Foundry.
   2. Select Create Project.
   3. Choose the Foundry Resource.
   4. Enter a Project Name.
   5. Create the project.
   A Foundry Project becomes the workspace that centrally manages models, agents, tools, and connected resources. How to use Foundry Tools in Microsoft Foundry portal - Foundry Tools | Microsoft Learn
5. Networking Configuration - Configureing private endpoint Since the environment uses private networking, additional configuration is required.
   1. Open the Azure AI Foundry resource.
   2. Navigate to Networking.
   3. Select Private Endpoint Connections.
   4. Create a Private Endpoint.
   5. Select the Virtual Network and subnet.
   6. Approve the connection.
   Configure Virtual Networks for Foundry Tools - Foundry Tools | Microsoft Learn
6. Configuring Private DNS After Private Endpoint creation:
   1. Create the appropriate Private DNS Zone.
   2. Link it to the Virtual Network.
   3. Verify DNS resolution from systems inside the VNet.
   Validate:
   • Resource FQDN resolves to a private IP.
   • Applications can reach the resource from the VNet.
   • DNS resolution is working correctly.
   This is one of the most common issues encountered in private endpoint deployments. What is a private endpoint? - Azure Private Link | Microsoft Learn
7. Enabling Microsoft Entra ID Authentication Microsoft Entra ID is Microsoft's recommended authentication method because it eliminates the need to manage API keys and provides centralized access control. Step 1 – Assign RBAC Permissions
   1. Open the Azure AI Foundry resource.
   2. Navigate to Access Control (IAM).
   3. Select Add Role Assignment.
   4. Assign: Cognitive Services User to users or applications that need access. Azure permissions for AI + machine learning - Azure RBAC | Microsoft Learn Agent identity concepts in Microsoft Foundry - Microsoft Foundry | Microsoft Learn
   Step 2 – Enable Managed Identity For Azure-hosted applications:
   1. Open:
      • App Service
      • Function App
      • Virtual Machine
      • AKS
   2. Navigate to Identity.
   3. Enable System Assigned Managed Identity.
   4. Save changes.
   Step 3 – Granting the Managed Identity Access
   1. Open the Azure AI resource.
   2. Go to Access Control (IAM).
   3. Assign Cognitive Services User to the Managed Identity. The application can now authenticate without storing API keys.
   Managed identities for Azure resources - Managed identities for Azure resources | Microsoft Learn
8. Using Azure AI Services from SDK Step 1 – Install SDK Packages Install Azure Identity first: install azure-identity Install service-specific SDKs as required. Azure AI Language -pip install azure-ai-textanalytics Azure AI Translator - pip install azure-ai-translation-text Azure AI Speech - pip install azure-cognitiveservices-speech Azure Text Analytics client library for Python | Microsoft Learn Azure Text Translation client library for Python | Microsoft Learn Azure Speech Documentation - Tutorials, API Reference - Foundry Tools | Microsoft Learn Step 2 – Authenticate Using Microsoft Entra ID Use:

      from azure.identity import DefaultAzureCredential
       
      credential = DefaultAzureCredential()
   

   DefaultAzureCredential automatically uses:
   • Developer login when running locally.
   • Managed Identity when deployed in Azure.
   azure.identity.DefaultAzureCredential class | Microsoft Learn
9. Adding Language, Speech or Translator Later One of the benefits of Azure AI Foundry is that Language, Speech, Translator, OpenAI, and other AI resources can be connected later as project requirements evolve. For example:
   • Need Sentiment Analysis - Create and connect Azure AI Language.
   • Need Speech-to-Text - Create and connect Azure AI Speech.
   • Need Translation - Create and connect Azure AI Translator.
   This allows starting simple and expanding only when additional capabilities are needed. Azure AI Foundry supports integration of these Azure AI services through connected resources.

Please let us know if the response was helpful

Thank you

Hello @Yogesh Dhavale (LTIMindtree Limited) ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

 Thank you for sharing the detailed implementation requirements.

The multilingual implementation scenario using Azure AI Language (Language Detection), Azure Translator (Text Translation), Azure AI Speech (Speech-to-Text), and Speech Translation is supported with Microsoft Entra ID authentication. Subscription keys do not need to be enabled for this implementation. When local authentication is disabled, access can be securely managed through Microsoft Entra ID, Azure RBAC permissions and token-based authentication using Azure Identity libraries.

The recommended implementation approach is to:

1. Provision the required Azure AI resources.
2. Connect those resources to the Azure AI Foundry project.
3. Configure Microsoft Entra ID authentication and Azure RBAC permissions.
4. Consume the services through SDKs or REST APIs using access tokens instead of subscription keys.

This provides a fully keyless and centrally managed authentication model.

Azure AI Foundry does not require separate activation of Language, Translator, or Speech capabilities. The required Azure AI resources must first be created and then connected to the Azure AI Foundry project.

Required resources:

• Azure AI Language
• Azure Translator
• Azure AI Speech

Configuration steps:

1. Open the Azure AI Foundry project.
2. Navigate to Project Settings → Connected Resources.
3. Add the required Azure AI resources.
4. Select Microsoft Entra ID as the authentication method instead of API Key authentication.Azure AI Foundry does not require separate activation of Language, Translator, or Speech capabilities. The required Azure AI resources must first be created and then connected to the Azure AI Foundry project. Required resources:
   • Azure AI Language
   • Azure Translator
   • Azure AI Speech
   Configuration steps:
   1. Open the Azure AI Foundry project.
   2. Then navigate to Project Settings > Connected Resources.
   3. Add the required Azure AI resources.
   4. Select Microsoft Entra ID as the authentication method instead of API Key authentication.

Microsoft Entra ID Authentication Flow

When API key authentication is disabled, authentication follows this flow:

1. An application identity is granted Azure RBAC permissions.
2. Azure Identity acquires an OAuth access token from Microsoft Entra ID.
3. The SDK or REST API uses the token to access Azure AI services.

Supported identity options include:

• Managed Identity (recommended for Azure-hosted applications)
• Service Principal
• DefaultAzureCredential (recommended for development and testing)When API key authentication is disabled, authentication follows this flow:
  1. An application identity is granted Azure RBAC permissions.
  2. Azure Identity acquires an OAuth access token from Microsoft Entra ID.
  3. The SDK or REST API uses the token to access Azure AI services.
  Supported identity options include:
  • Managed Identity (recommended for Azure-hosted applications)
  • Service Principal
  • DefaultAzureCredential (recommended for development and testing)
  Azure AI Services token scope: https://cognitiveservices.azure.com/.default

Assign Required Azure RBAC Permissions

Permissions should be assigned at the Azure AI resource level:

Azure Portal > Azure AI Resource > Access Control (IAM) > Add Role Assignment

1. Azure AI Language - Cognitive Services User
2. Azure Translator - Cognitive Services User
3. Azure AI Speech - Cognitive Services Speech User or Cognitive Services Speech Contributor

The Contributor role should only be used when resource management permissions are required and is generally not recommended for application runtime access.

After assigning permissions:

• Allow several minutes for RBAC propagation.
• Verify the role is assigned to the correct managed identity or service principal.

Verifying Resource Endpoint Configuration

For Microsoft Entra ID authentication, service-specific endpoint requirements should be verified.

For Azure Speech SDK authentication, a custom-domain endpoint is required:

https://<resource-name>.cognitiveservices.azure.com/

The endpoint can be verified from:

Azure Portal > Azure AI Resource > Keys and Endpoint

SDK Authentication Without Subscription Keys subscriptionKey represent key-based authentication and can be replaced with Azure Identity credentials.

DefaultAzureCredential >Microsoft Entra ID >OAuth Access Token >Azure AI SDK Client

Azure AI Language – Language Detection

Azure AI Language SDK supports token-based authentication.

Implementation approach:

• Use DefaultAzureCredential or another Azure Identity credential.
• Provide the Azure AI Language endpoint.
• Create the Language SDK client using the token credential.

Azure Translator – Text Translation

Azure Translator supports Microsoft Entra ID authentication.

Implementation approach:

1. Configure the Translator resource following the documented Entra ID authentication guidance.
2. Assign the required Azure RBAC permissions.
3. Replace subscription key authentication with bearer-token authentication.

For REST scenarios:

• Acquire an OAuth access token.
• Send the token as a Bearer token.
• Include Translator resource authorization information where required.Azure Translator supports Microsoft Entra ID authentication. Implementation approach:
  1. Configure the Translator resource following the documented Entra ID authentication guidance.
  2. Assign the required Azure RBAC permissions.
  3. Replace subscription key authentication with bearer-token authentication.
  For REST scenarios:
  • Acquire an OAuth access token.
  • Send the token as a Bearer token.
  • Include Translator resource authorization information where required.

Azure AI Speech – Speech-to-Text and Speech Translation

Azure Speech supports Microsoft Entra ID authentication through the Speech SDK.

Implementation approach:

1. Acquire an Entra ID access token using Azure Identity.
2. Configure Speech SDK authentication using a supported token-based authentication method.

Depending on the SDK language and version, authentication can be implemented using:

• TokenCredential-based authentication
• Speech authorization token format: aad#<resourceId>#<access-token>

Perform validation in the following order:

1. Confirm Azure AI resources are connected to Azure AI Foundry.
2. Confirm local authentication remains disabled.
3. Verify Azure RBAC role assignments.
4. Verify token acquisition using https://cognitiveservices.azure.com/.default
5. Test each capability independently - Successful responses confirm that Microsoft Entra ID authentication has been configured correctly.

The following references might be helpful , please check them out

• Authentication in Foundry Tools - Foundry Tools | Microsoft Learn
• Authentication and authorization in Azure Translator in Foundry Tools service - Foundry Tools | Microsoft Learn
• Enable Microsoft Entra ID authentication - Foundry Tools | Microsoft Learn
• How to configure Microsoft Entra authentication - Foundry Tools | Microsoft Learn
• Disable local authentication in Foundry Tools - Foundry Tools | Microsoft Learn
• Microsoft Foundry documentation | Microsoft Learn

Please let us know if the response was helpful

Thank you

First provision a unified Azure AI services multi-service resource in the Azure Portal, which bundles Language, Translator, and Speech into a single instance. Next, in the Azure AI Foundry Portal, use the Operate management dashboard to add a connection to your multi-service resource, and explicitly select Microsoft Entra ID or Managed Identity as your authentication type to bond these tools to your project workspace.

Because API keys are disabled on your resource, your application needs to rely on token-based authentication via the Azure.Identity SDK. To authorize these data-plane actions, modify RBAC settings of your Azure AI Services resource by assigning the Cognitive Services User role to your application's identity. This should ensure that your identity has the permissions needed to handle websocket streaming and authorization tokens for Speech-to-Text and Speech Translation operations.

In your application code, you can use the New Azure AI Foundry Project SDK alongside your token credentials to access these services . By initializing the AIProjectClient with a DefaultAzureCredential and your project's unique connection string, you can dynamically reference your connected services. For Language Detection and Text Translation, you pass the token credential directly into their respective client builders along with your custom endpoint URL. For the Azure Speech SDK, you will use your identity credential to fetch an OAuth 2.0 access token scoped to the cognitive services environment, then pass that string into your speech configuration using the authorization token setter method.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin