Microsoft Azure — Azure Policy & Compliance

Azure is Microsoft's cloud computing platform offering over 200 services. This article covers azure policy & compliance with production-ready configurations, best practices, and hands-on examples.

Why Azure Policy Matters in Production

Azure Policy is a service in Azure that you use to create, assign, and manage policies. These policies enforce different rules and effects over your resources, so those resources stay compliant with your corporate standards and service level agreements. In production, ungoverned resources lead to security breaches, cost overruns, and compliance failures. Azure Policy helps you avoid these by applying guardrails at scale. For example, you can prevent deployment of VMs in unsupported regions, enforce tagging for cost tracking, or require HTTPS on storage accounts. Without policies, your environment drifts into chaos. This article walks through the core concepts, writing custom policies, assignment strategies, and remediation — all from a production perspective.

Core Concepts: Policy Definition, Initiative, Assignment

A policy definition expresses a rule using JSON. It includes a condition (e.g., resource type equals 'Microsoft.Compute/virtualMachines') and an effect (e.g., deny, audit, append). An initiative is a group of policy definitions, often used to group related policies (like 'ISO 27001'). An assignment applies a policy or initiative to a specific scope: management group, subscription, or resource group. Effects determine what happens when a resource is non-compliant: Deny blocks creation, Audit logs a warning, Append adds tags, DeployIfNotExists triggers remediation. Understanding these is critical because misconfigured effects can break deployments. For instance, a Deny effect on a tag that is missing will block all new resources until the tag is added. Always test with Audit first in production.

{
  "properties": {
    "displayName": "Require tag: CostCenter",
    "policyType": "Custom",
    "mode": "Indexed",
    "description": "Deny deployment if CostCenter tag is missing",
    "metadata": {
      "category": "Tags"
    },
    "parameters": {
      "tagName": {
        "type": "String",
        "defaultValue": "CostCenter"
      }
    },
    "policyRule": {
      "if": {
        "field": "tags[parameters('tagName')]",
        "exists": "false"
      },
      "then": {
        "effect": "deny"
      }
    }
  }
}

Writing Your First Custom Policy

Custom policies let you enforce rules specific to your organization. Start by defining the policy rule in JSON. The rule has an 'if' condition and a 'then' effect. Conditions can check resource properties (field), tags, locations, SKUs, etc. Use the Azure Policy extension for VS Code or the portal editor. A common pattern is to enforce mandatory tags. The example below denies any resource that doesn't have a 'CostCenter' tag. After writing the definition, create it with New-AzPolicyDefinition (PowerShell) or az policy definition create (CLI). Then assign it to a scope. Test in a non-production subscription first. Remember: custom policies are powerful but can cause friction if too restrictive. Involve your engineering teams in the design.

# Create custom policy definition
$definition = New-AzPolicyDefinition -Name 'require-costcenter-tag' `
  -DisplayName 'Require CostCenter tag' `
  -Description 'Deny if CostCenter tag is missing' `
  -Policy '.\.\policy-definition.json' `
  -Mode Indexed

# Assign to a subscription
$subscriptionId = '00000000-0000-0000-0000-000000000000'
$assignment = New-AzPolicyAssignment -Name 'require-costcenter-tag-assignment' `
  -DisplayName 'Require CostCenter tag assignment' `
  -Scope "/subscriptions/$subscriptionId" `
  -PolicyDefinition $definition

Write-Output "Policy assigned to subscription $subscriptionId"

Assigning Policies at Scale with Management Groups

Management groups let you organize subscriptions hierarchically. Assign policies at the management group level to apply to all child subscriptions. This is essential for large enterprises. For example, assign a policy that requires encryption at rest at the root management group. All subscriptions inherit it. Inheritance is additive: a policy assigned at a higher scope applies to lower scopes unless overridden by an exclusion. Exclusions are done by assigning a policy with 'Exclude' parameter on specific child scopes. Be careful: too many exclusions defeat the purpose. Use initiatives to bundle related policies. Also, use policy parameters to make assignments flexible (e.g., allowed locations list).

#!/bin/bash
# Assign policy at management group scope
MG_ID="my-root-group"
POLICY_DEF="/providers/Microsoft.Authorization/policyDefinitions/require-costcenter-tag"

az policy assignment create \
  --name 'require-costcenter-mg' \
  --display-name 'Require CostCenter at MG' \
  --scope "/providers/Microsoft.Management/managementGroups/$MG_ID" \
  --policy "$POLICY_DEF" \
  --params '{"tagName":{"value":"CostCenter"}}'

Remediation: Fixing Non-Compliant Resources

Policies with 'DeployIfNotExists' or 'Modify' effects can automatically remediate non-compliant resources. For example, a policy that deploys a network watcher if missing. Remediation tasks run on existing resources. To set up, create a managed identity for the policy assignment, grant it permissions (e.g., Contributor on the scope), and define the deployment template in the policy definition. The remediation task can be triggered manually or on a schedule. Be cautious: remediation can modify many resources at once. Test on a small scope first. Also, monitor remediation logs to catch failures. Common issues: missing permissions, template errors, or resource conflicts.

{
  "properties": {
    "displayName": "Deploy Network Watcher if missing",
    "policyType": "Custom",
    "mode": "Indexed",
    "description": "Deploy Network Watcher in each region where VMs exist",
    "metadata": {
      "category": "Network"
    },
    "parameters": {},
    "policyRule": {
      "if": {
        "field": "type",
        "equals": "Microsoft.Network/virtualNetworks"
      },
      "then": {
        "effect": "deployIfNotExists",
        "details": {
          "type": "Microsoft.Network/networkWatchers",
          "resourceGroupName": "NetworkWatcherRG",
          "existenceCondition": {
            "field": "location",
            "equals": "[resourceGroup().location]"
          },
          "deployment": {
            "properties": {
              "mode": "incremental",
              "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {},
                "resources": [
                  {
                    "type": "Microsoft.Network/networkWatchers",
                    "name": "[concat('NetworkWatcher_', resourceGroup().location)]",
                    "location": "[resourceGroup().location]",
                    "apiVersion": "2020-05-01",
                    "properties": {}
                  }
                ]
              },
              "parameters": {}
            }
          }
        }
      }
    }
  }
}

Compliance Dashboard and Reporting

Azure Policy provides a compliance dashboard in the portal. It shows overall compliance percentage, non-compliant resources, and policy details. You can export compliance data to Log Analytics or Event Hubs for custom reporting. Use Azure Resource Graph to query compliance state across subscriptions. For example, find all non-compliant VMs. Set up alerts for compliance changes using Azure Monitor. In production, regularly review compliance reports to catch drift. A common pitfall: compliance data is not real-time; it refreshes every few hours. For critical policies, use 'Audit' effect and monitor logs. Also, use policy exemptions for temporary exceptions, but track them with expiration dates.

// Query non-compliant resources using Azure Resource Graph
policyresources
| where type =~ 'microsoft.authorization/policyassignments'
| extend complianceState = tostring(properties.complianceState)
| where complianceState == 'NonCompliant'
| project subscriptionId, resourceGroup, resourceName = properties.resourceId, policyAssignmentName = name

Policy as Code: CI/CD for Azure Policy

Treat policies as code: store definitions in Git, validate with tests, and deploy via CI/CD. Use Azure DevOps or GitHub Actions. Steps: 1) Export existing policies to JSON. 2) Store in a repo. 3) Use az policy definition create in pipeline. 4) Validate with az policy definition list and custom tests. 5) Assign policies via ARM templates or Bicep. Benefits: version control, peer review, automated testing. A common failure: deploying a policy that breaks production because it wasn't tested. Use separate environments (dev, test, prod) with different scopes. Also, use policy parameters to make definitions reusable across environments.

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: AzureCLI@2
  displayName: 'Create or update policy definitions'
  inputs:
    azureSubscription: 'my-service-connection'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      for file in policies/*.json; do
        name=$(basename $file .json)
        az policy definition create --name $name --rules $file --mode Indexed --display-name $name
      done

- task: AzureCLI@2
  displayName: 'Assign policies'
  inputs:
    azureSubscription: 'my-service-connection'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az policy assignment create --name 'require-tags' --policy /providers/Microsoft.Authorization/policyDefinitions/require-costcenter-tag --scope '/subscriptions/00000000-0000-0000-0000-000000000000'

Common Pitfalls and How to Avoid Them

1. Overly restrictive policies: Deny effects without audit first block deployments. Always start with Audit. 2) Ignoring inheritance: Policies at higher scopes affect all children. Use exclusions sparingly. 3) Not using parameters: Hardcoded values make policies inflexible. Use parameters for allowed locations, tag names, etc. 4) Neglecting remediation permissions: Managed identities need proper RBAC. 5) Forgetting to monitor: Compliance dashboards are not real-time. Set up alerts. 6) Policy conflicts: Two policies can conflict (e.g., one denies a location, another requires it). Test in a sandbox. 7) Not planning for exceptions: Use exemptions with expiration dates for temporary waivers. Avoid permanent exemptions.

Integrating with Azure Blueprints and Landing Zones

Azure Blueprints (deprecated) and Enterprise-Scale Landing Zones use Azure Policy to enforce governance. Blueprints package policies, RBAC, and resource templates. Landing zones (via Terraform or Bicep) include policy assignments as part of the foundation. For example, the Azure Landing Zone accelerator includes policies for encryption, networking, and monitoring. When adopting landing zones, you inherit a set of policies. Customize them by adding your own. Be careful: modifying landing zone policies can break upgrades. Use policy exemptions for deviations. Also, consider using Azure Policy's 'Initiative' to group your custom policies alongside landing zone policies.

Production Monitoring and Alerting for Policy Compliance

Set up Azure Monitor alerts for policy compliance changes. Use Activity Log alerts when a policy assignment is created, modified, or deleted. Also, alert on compliance state changes using Azure Resource Graph scheduled queries. For example, alert if more than 10 resources become non-compliant in an hour. Integrate with ITSM tools like ServiceNow. In production, compliance drift often happens after deployments or configuration changes. Automate remediation where possible, but always have a manual approval for critical changes. Also, use Azure Policy's 'Guest Configuration' for in-VM compliance (e.g., installed software, security settings).

# Create an alert rule for policy compliance changes
$actionGroup = New-AzActionGroup -Name 'PolicyAlertGroup' -ShortName 'Policy' -EmailReceiver @{Name='admin';EmailAddress='admin@contoso.com'}

$condition = New-AzMetricAlertRuleV2Criteria -MetricName 'NonCompliantResources' -TimeAggregation 'Total' -Operator 'GreaterThan' -Threshold 10

Add-AzMetricAlertRuleV2 -Name 'HighNonCompliance' -ResourceGroupName 'monitoring-rg' -WindowSize 01:00:00 -Frequency 00:05:00 -TargetResourceId '/subscriptions/00000000-0000-0000-0000-000000000000' -Condition $condition -ActionGroupId $actionGroup.Id -Severity 2

Cost Management with Azure Policy

Azure Policy can help control costs by enforcing resource limits. For example, deny VMs above a certain size, require auto-shutdown schedules, or enforce resource group naming conventions for cost tracking. Use 'Audit' effect to report on expensive resources. Combine with Azure Cost Management budgets. A common pattern: require a 'CostCenter' tag on all resources, then use tag-based cost reports. Also, use policy to block creation of expensive services (e.g., Azure NetApp Files) unless approved. Be careful: overly restrictive cost policies can block innovation. Balance with exemptions for approved projects.

{
  "properties": {
    "displayName": "Deny VMs larger than Standard_D4s_v3",
    "policyType": "Custom",
    "mode": "All",
    "description": "Deny creation of VMs with SKU larger than D4s v3",
    "metadata": {
      "category": "Compute"
    },
    "parameters": {
      "allowedSizes": {
        "type": "Array",
        "defaultValue": ["Standard_D2s_v3", "Standard_D4s_v3"]
      }
    },
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Compute/virtualMachines"
          },
          {
            "field": "Microsoft.Compute/virtualMachines/sku.name",
            "notIn": "[parameters('allowedSizes')]"
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    }
  }
}

Next Steps: Advanced Policy Patterns

After mastering basics, explore advanced patterns: 1) Policy exemptions with expiration dates for temporary waivers. 2) Using 'Modify' effect to add tags automatically. 3) Cross-subscription policies using management groups. 4) Policy for Azure Kubernetes Service (AKS) to enforce pod security. 5) Custom policy aliases for new resource properties. 6) Using Azure Policy with Terraform via azurerm_policy_definition resource. 7) Policy-driven governance for Azure DevOps pipelines (e.g., require approval for production deployments). Always stay updated with Azure Policy documentation as new features are added regularly.

Regulatory Compliance Built-in Initiatives

Azure Policy includes built-in regulatory compliance initiatives mapped to major frameworks: SOC 2, ISO 27001, HIPAA/HITRUST, FedRAMP Moderate/High, PCI DSS, and the Microsoft Cloud Security Benchmark (MCSB). These initiatives bundle 50-200+ policy definitions scoped to specific compliance controls. Assign them at the root management group to evaluate compliance across the entire organization. Use the Compliance dashboard in Microsoft Defender for Cloud to see a unified view of regulatory compliance posture, with drill-down to specific controls and resources. For new regulations, Microsoft publishes initiatives in preview that you can test in audit-only mode. Customize built-in initiatives by adding your own policies via 'custom initiative' definitions. In production, always review the specific control mappings — some built-in policies may not apply to your architecture. Export compliance reports to Excel, CSV, or integrate with GRC tools via Power BI.

az policy set-definition show \
  --name "SOC2Type1" \
  --query "{displayName:displayName, description:description, policyType:policyType}" \
  -o table

# Assign SOC 2 initiative at subscription scope
az policy assignment create \
  --name "soc2-compliance" \
  --display-name "SOC 2 Type 1 Compliance" \
  --policy-set-definition "SOC2Type1" \
  --scope /subscriptions/00000000-0000-0000-0000-000000000000 \
  --enforcement-mode Default

Microsoft Cloud Security Benchmark and Defender for Cloud Integration

The Microsoft Cloud Security Benchmark (MCSB) is the successor to the Azure Security Benchmark, providing prescriptive best practices for security across Azure, multicloud, and on-premises. It maps to CIS, NIST, and PCI frameworks. MCSB recommendations are available in Microsoft Defender for Cloud, which provides a unified security posture management platform. Defender for Cloud has two tiers: (1) Free tier — continuous assessment, secure score, and Azure Policy integration; (2) Paid (Defender plans) — advanced threat detection, vulnerability scanning, and just-in-time VM access. Defender for Cloud automatically assigns the MCSB initiative to your subscriptions. Use its secure score to track progress: each recommendation has a potential score increase, and you can prioritize by impact. Integrate Defender for Cloud with Azure Policy: enabling Defender plans can be governed via policy. For production, enable at least Defender for Cloud foundational CSPM (free) and consider Defender for Servers, Defender for Storage, and Defender for SQL on critical workloads.

az security pricing create --name "CloudPosture" --tier "Standard"
az security pricing create --name "VirtualMachines" --tier "Standard"
az security pricing create --name "StorageAccounts" --tier "Standard"
az security pricing create --name "SqlServers" --tier "Standard"

Azure Policy Effects Deep Dive and Evaluation Logic

Azure Policy supports nine effects: Deny, Audit, Append, AuditIfNotExists, DeployIfNotExists, Modify, Disabled, Manual, and DenyAction. Each effect has specific behavior and performance implications. Deny blocks create/update operations — use for hard guardrails. Audit logs non-compliant resources without blocking — use for discovery and soft governance. Append adds fields (e.g., tags) during create/update — use for enforcing required tags. DeployIfNotExists triggers a deployment to fix non-compliant resources — use with a managed identity. Modify is faster than Append for adding/altering tags and works on existing resources. DenyAction (Preview) blocks specific actions (e.g., delete) on resources. Policy evaluation happens at resource create/update, when a policy is assigned/updated, and during the 24-hour compliance cycle. Multiple policies at the same scope are evaluated independently; if any have Deny effect on a condition, the resource is denied. Use policy exemptions with expiration dates for temporary waivers instead of disabling policies.

{
  "effects": {
    "Deny": { "use": "Hard block on non-compliant resources", "cost": "Low", "existing": "No impact" },
    "Audit": { "use": "Discover non-compliant resources", "cost": "Low", "existing": "Marked non-compliant" },
    "Append": { "use": "Add missing fields during create", "cost": "Low", "existing": "No impact" },
    "Modify": { "use": "Add/alter tags on existing resources", "cost": "Medium", "existing": "Remediation task needed" },
    "DeployIfNotExists": { "use": "Deploy missing resource (e.g., diagnostic settings)", "cost": "High", "existing": "Remediation task needed" },
    "DenyAction": { "use": "Block delete action on critical resources", "cost": "Low", "existing": "Action blocked at runtime" }
  }
}