SpringTutorials

    Azure OpenAI Service

    Enterprise-grade AI with Azure's security, compliance, and regional deployment options. Same powerful models, your infrastructure.

    Azure OpenAI Service provides access to OpenAI's powerful models—GPT-4, GPT-4 Turbo, DALL-E, Whisper—through Azure's enterprise infrastructure. For organizations with strict compliance requirements, data residency needs, or existing Azure investments, this is the path to production AI.

    The key difference from direct OpenAI: your data stays within your Azure tenant, protected by your security policies and compliance certifications. There's no data sharing with OpenAI for model training. Combined with Azure Private Link, you can ensure AI traffic never traverses the public internet.

    Why Azure OpenAI?

    Enterprise Security

    SOC 2, HIPAA, GDPR, FedRAMP compliance. Data encrypted at rest and in transit. Azure Active Directory integration for access control.

    Data Residency

    Deploy in specific Azure regions to meet data sovereignty requirements. Your data stays in your chosen geography.

    Azure Ecosystem

    Seamless integration with Azure Cognitive Search, Cosmos DB, Azure Functions, and the entire Azure platform.

    Private Networking

    Use Azure Private Link to access OpenAI through private endpoints. Traffic stays on Microsoft's backbone network, never touching the public internet. Essential for regulated industries.

    Enterprise Support

    Leverage your existing Azure support agreement. Get SLAs, dedicated support channels, and integration with Azure Monitor for observability.

    Core Concepts

    Deployments vs Models

    Unlike direct OpenAI where you specify model: "gpt-4", Azure uses deployments. You create a deployment (e.g., "my-gpt4-prod") that references a specific model and version. This enables:

    • Multiple deployments of the same model (dev/staging/prod)
    • Controlled updates to specific model versions
    • Separate quotas and access policies per deployment

    Content Filters

    Azure includes built-in content safety filters that cannot be disabled. They detect and block:

    • Hate speech and discriminatory content
    • Self-harm related content
    • Sexual and violent content
    • Jailbreak attempts

    Your app must handle content_filter errors gracefully.

    Key difference from OpenAI: Your prompts and completions are not used for training OpenAI models. Data stays within your Azure tenant and is subject to your data retention policies.

    Configuration

    Maven Dependencies

    Add the Azure OpenAI Spring Boot starter

    pom.xml
    <!-- Azure OpenAI Starter --><dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId></dependency><!-- Add Spring AI BOM for version management --><dependencyManagement><dependencies><dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-bom</artifactId><version>1.0.0-M4</version><type>pom</type><scope>import</scope></dependency></dependencies></dependencyManagement>

    Application Properties

    Configure your Azure OpenAI connection

    application.properties
    # Azure OpenAI Endpoint (from Azure Portal)spring.ai.azure.openai.endpoint=https://YOUR-RESOURCE.openai.azure.com/# Authentication - Option 1: API Keyspring.ai.azure.openai.api-key=${AZURE_OPENAI_API_KEY}# The deployment name you created in Azure Portal (NOT the model name!)spring.ai.azure.openai.chat.options.deployment-name=gpt-4-deployment# Model parametersspring.ai.azure.openai.chat.options.temperature=0.7spring.ai.azure.openai.chat.options.max-tokens=2000# Optional: Embedding deploymentspring.ai.azure.openai.embedding.options.deployment-name=text-embedding-ada-002

    Deployment name ≠ Model name! Don't use gpt-4 directly. Use the deployment name you created in Azure Portal (e.g., my-gpt4-prod).

    Passwordless Authentication (Managed Identity)

    For production deployments, avoid API keys entirely. Use Azure Managed Identity for secure, automatic authentication. Your app's identity is granted access to the OpenAI resource—no secrets to manage.

    Managed Identity Configuration
    # Enable Azure Identity (DefaultAzureCredential)spring.ai.azure.openai.endpoint=https://YOUR-RESOURCE.openai.azure.com/# No api-key property needed! Uses Managed Identity automatically# Required: Azure SDK identity dependency# <dependency>#     <groupId>com.azure</groupId>#     <artifactId>azure-identity</artifactId># </dependency>

    Setup Steps

    1. Enable Managed Identity on your App Service/AKS
    2. Go to Azure OpenAI resource → Access Control (IAM)
    3. Add role: "Cognitive Services OpenAI User"
    4. Assign to your app's managed identity

    Benefits

    • No secrets to rotate or store
    • Credentials managed by Azure
    • Automatic token refresh
    • Works with Azure Key Vault

    Implementation Example

    Azure OpenAI Chat Service

    AzureOpenAIService.java
    @ServicepublicclassAzureOpenAIService{privatefinalChatClient chatClient;publicAzureOpenAIService(ChatClient.Builder builder){this.chatClient = builder
.defaultSystem("""
You are a helpful enterprise assistant for Contoso Inc.
Always be professional, accurate, and compliant with company policies.
Never share confidential information in responses.
""").build();}publicStringchat(String userMessage){return chatClient.prompt().user(userMessage).call().content();}publicFlux<String>streamChat(String userMessage){return chatClient.prompt().user(userMessage).stream().content();}// Override deployment per-request (e.g., for A/B testing)publicStringchatWithDeployment(String message,String deploymentName){return chatClient.prompt().user(message).options(AzureOpenAiChatOptions.builder().deploymentName(deploymentName).temperature(0.5).build()).call().content();}}

    The code is identical to OpenAI integration—that's Spring AI's portable API in action. The only difference is configuration. If you need to switch between direct OpenAI and Azure OpenAI, just change yourapplication.properties and dependencies.

    Handling Content Filters

    Azure's content filters are always active. When triggered, they throw exceptions that your app must handle gracefully.

    Content Filter Handling
    @ServicepublicclassSafeAzureAIService{privatefinalChatClient chatClient;publicChatResultsafChat(String message){try{String response = chatClient.prompt().user(message).call().content();returnChatResult.success(response);}catch(AzureOpenAiContentFilterException e){// Content filter triggered
log.warn("Content filter triggered: {}", e.getContentFilterResult());returnChatResult.blocked("I can't process that request due to content policy. "+"Please rephrase your question.");}catch(AzureOpenAiApiException e){if(e.getStatusCode()==429){
log.warn("Rate limit exceeded");returnChatResult.rateLimited("Please try again in a moment.");}throw e;}}}publicrecordChatResult(boolean success,String message,boolean blocked){publicstaticChatResultsuccess(String msg){returnnewChatResult(true, msg,false);}publicstaticChatResultblocked(String msg){returnnewChatResult(false, msg,true);}publicstaticChatResultrateLimited(String msg){returnnewChatResult(false, msg,false);}}

    Regional Deployment & Failover

    Available Regions

    • 🇺🇸 East US, East US 2
      GPT-4o
    • 🇺🇸 South Central US
      GPT-4
    • 🇪🇺 West Europe
      GPT-4
    • 🇯🇵 Japan East
      GPT-4
    • 🇦🇺 Australia East
      GPT-4

    Model availability varies by region. Check Azure docs for current availability.

    Multi-Region Failover

    For high availability, deploy resources in multiple regions and implement failover:

    Multi-Region Setup
    @BeanpublicChatClientprimaryClient(@Value("${azure.ai.primary.endpoint}")String primary,@Value("${azure.ai.secondary.endpoint}")String secondary){returnChatClient.builder().withRetryPolicy(newRegionalFailover(primary, secondary)).build();}

    Pricing Comparison

    Azure OpenAI pricing is comparable to direct OpenAI, with some enterprise benefits:

    Azure Advantages

    • Use Azure credits & Enterprise Agreements
    • Consolidated billing with other Azure services
    • Reserved capacity discounts available
    • No data egress charges within Azure

    Considerations

    • Slightly higher latency than direct OpenAI
    • New models available later than OpenAI
    • Model availability varies by region
    • Quota limits per subscription

    Enterprise Best Practices

    🔒 Security

    • Use Managed Identity, not API keys
    • Enable Private Link for sensitive workloads
    • Configure RBAC with least-privilege access
    • Enable Azure Defender for AI

    📊 Monitoring

    • Enable Azure Monitor and Log Analytics
    • Track token usage and costs
    • Set up alerts for quota limits
    • Monitor content filter triggers

    🚀 Performance

    • Co-locate app and AI in same region
    • Use streaming for better UX
    • Implement caching for common queries
    • Request quota increases proactively

    🏗️ Architecture

    • Separate deployments for dev/staging/prod
    • Version-pin model deployments
    • Use Azure API Management for governance
    • Implement circuit breakers for resilience

    Ready for Enterprise AI?

    Azure OpenAI gives you the power of GPT models with enterprise security. Start building intelligent applications that meet your compliance requirements.