Microsoft Azure — Azure Sentinel (SIEM)

Azure is Microsoft's cloud computing platform offering over 200 services. This article covers azure sentinel (siem) with production-ready configurations, best practices, and hands-on examples.

Why Azure Sentinel?

Azure Sentinel is Microsoft's cloud-native SIEM (Security Information and Event Management) and SOAR (Security Orchestration Automation and Response) solution. Unlike traditional on-prem SIEMs like Splunk or QRadar, Sentinel is built on Azure Monitor and Log Analytics, offering near-infinite scalability with no upfront infrastructure. It ingests data from Microsoft 365, Azure Activity, Windows Events, and third-party sources via connectors. For DevOps teams, Sentinel's key advantage is its native integration with Azure DevOps and GitHub for automated incident response. However, beware of cost: data ingestion and retention can balloon if not governed properly. Always set daily ingestion caps and use Basic Logs for low-fidelity data.

{
  "location": "eastus",
  "properties": {
    "name": "DailyIngestionCapAlert",
    "description": "Alert when daily ingestion exceeds 50 GB",
    "severity": 2,
    "enabled": true,
    "evaluationFrequency": "PT5M",
    "windowSize": "PT24H",
    "targetResourceType": "Microsoft.OperationalInsights/workspaces",
    "criteria": {
      "allOf": [
        {
          "metricName": "DailyIngestionGB",
          "operator": "GreaterThan",
          "threshold": 50
        }
      ]
    }
  }
}

Deploying Sentinel with Terraform

Deploy Sentinel via Terraform for repeatable infrastructure. Start with a Log Analytics workspace, then enable Sentinel (which is just a feature toggle). Use the azurerm_log_analytics_workspace and azurerm_sentinel_log_analytics_workspace_onboarding resources. For production, separate workspaces per environment (dev, staging, prod) to isolate data and costs. Use data sources to reference existing workspaces. Always tag resources for cost tracking. Here's a minimal Terraform configuration that creates a workspace and enables Sentinel.

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "sentinel" {
  name     = "rg-sentinel-prod"
  location = "East US"
}

resource "azurerm_log_analytics_workspace" "sentinel" {
  name                = "log-sentinel-prod"
  location            = azurerm_resource_group.sentinel.location
  resource_group_name = azurerm_resource_group.sentinel.name
  sku                 = "PerGB2018"
  retention_in_days   = 90
}

resource "azurerm_sentinel_log_analytics_workspace_onboarding" "sentinel" {
  workspace_id = azurerm_log_analytics_workspace.sentinel.id
}

Ingesting Data with Connectors

Sentinel ingests data via connectors. The most common are Azure Activity, Azure AD, Office 365, and Windows Security Events. For custom sources, use the Log Analytics Agent (legacy) or Azure Monitor Agent (AMA). AMA is the future: it supports data collection rules (DCRs) for granular filtering. For example, to ingest only Security Events with Event ID 4625 (failed logon), create a DCR that filters events. This reduces ingestion costs. For third-party sources like AWS or GCP, use the Sentinel connector or a syslog forwarder. Always test connectors in a staging workspace first.

{
  "properties": {
    "dataSources": [
      {
        "kind": "WindowsEventLogs",
        "streams": ["Microsoft-WindowsEventLogs"],
        "dataSources": [
          {
            "eventLogName": "Security",
            "eventIds": [4625],
            "providerGuids": []
          }
        ]
      }
    ]
  }
}

Writing KQL Queries for Detection

Kusto Query Language (KQL) is the backbone of Sentinel analytics. Write queries to detect threats like brute force attacks, malware, or data exfiltration. For example, to detect multiple failed logons from a single IP, use the following query. It aggregates failed logon events (EventID 4625) over 5 minutes and alerts if count > 10. Use the summarize operator with bin for time windows. Always test queries in Log Analytics before creating analytics rules. Optimize queries by filtering early and using materialized views for high-volume data.

SecurityEvent
| where TimeGenerated > ago(5m)
| where EventID == 4625
| where AccountType == "User"
| summarize FailedLogons = count() by IpAddress = IpAddress, Account = TargetAccount
| where FailedLogons > 10
| project IpAddress, Account, FailedLogons, TimeGenerated

Creating Analytics Rules

Analytics rules in Sentinel automatically run KQL queries and generate incidents. There are two types: Scheduled (run every N minutes) and NRT (near-real-time, run every minute). For production, use scheduled rules with a frequency of 5-15 minutes to balance cost and latency. Configure entity mapping (Account, IP, Host) to enrich incidents. Set alert threshold and suppression duration to reduce noise. For example, create a rule for the brute force query above: run every 5 minutes, look at last 5 minutes, generate incident if >10 failed logons. Use the Azure CLI or Terraform to deploy rules as code.

resource "azurerm_sentinel_alert_rule_scheduled" "brute_force" {
  name                       = "Brute Force Detection"
  log_analytics_workspace_id = azurerm_log_analytics_workspace.sentinel.id
  display_name               = "Brute Force Attack"
  severity                   = "High"
  query                      = <<-QUERY
SecurityEvent
| where TimeGenerated > ago(5m)
| where EventID == 4625
| summarize FailedLogons = count() by IpAddress, Account
| where FailedLogons > 10
QUERY
  query_frequency            = "PT5M"
  query_period               = "PT5M"
  trigger_operator           = "GreaterThan"
  trigger_threshold          = 0
  suppression_duration       = "PT5M"
  enabled                    = true
}

Incident Management and Automation

Incidents in Sentinel are groups of related alerts. Use automation rules to assign, classify, or run playbooks. For example, automatically assign high-severity incidents to the SOC team and run a playbook to block the offending IP via Azure Firewall. Playbooks are Azure Logic Apps triggered by Sentinel. They can query external threat intel, create tickets in ServiceNow, or send Teams messages. For DevOps, integrate with Azure DevOps to create work items for investigation. Always test playbooks in a non-production environment first.

{
  "properties": {
    "displayName": "Block IP on High Severity",
    "order": 1,
    "triggeringLogic": {
      "conditions": [
        {
          "property": "Severity",
          "operator": "Equals",
          "value": "High"
        }
      ]
    },
    "actions": [
      {
        "order": 1,
        "actionType": "RunPlaybook",
        "playbookId": "/subscriptions/.../providers/Microsoft.Logic/workflows/block-ip"
      }
    ]
  }
}

Threat Intelligence Integration

Sentinel can ingest threat intelligence feeds (STIX/TAXII) to enrich alerts. Use the Threat Intelligence - TAXII connector to pull indicators from sources like AlienVault OTX or Microsoft Defender. Indicators are stored in the ThreatIntelligenceIndicator table. In KQL queries, join with this table to detect matches. For example, check if an IP in a log matches a known malicious indicator. This reduces false positives and prioritizes incidents. For production, set up automatic feed updates every hour. Be mindful of performance: use materialized views for indicator lookups.

let MaliciousIPs = ThreatIntelligenceIndicator
| where TimeGenerated > ago(1d)
| where IndicatorType == "ipv4-addr"
| project IndicatorValue = tostring(IndicatorValue);
SecurityEvent
| where TimeGenerated > ago(5m)
| where EventID == 4625
| where IpAddress in (MaliciousIPs)
| project TimeGenerated, IpAddress, Account

Monitoring and Alerting on Sentinel Health

Sentinel itself needs monitoring. Use the SentinelHealth table to track connector status, query failures, and ingestion delays. Create analytics rules to alert on connector disconnections or high latency. For example, alert if any connector has been disconnected for more than 30 minutes. Also monitor workspace usage with the Usage table to track ingestion trends. Set up a dashboard for SOC managers. In production, we had a connector silently fail for 2 days because no one monitored health. Now we have a rule that sends a Teams message on any connector failure.

SentinelHealth
| where TimeGenerated > ago(1h)
| where Status == "Failure"
| summarize FailureCount = count() by ConnectorName, StatusDescription
| where FailureCount > 0
| project ConnectorName, FailureCount, StatusDescription

Cost Optimization Strategies

Sentinel costs are driven by data ingestion and retention. To optimize: use Basic Logs for low-fidelity data (e.g., firewall logs), set retention policies per table (e.g., 30 days for SecurityEvent, 90 days for AzureActivity), and use data collection rules to filter out noise. Enable Commitment Tiers (e.g., 100 GB/day) for predictable pricing. Use Azure Cost Management to set budgets and alerts. For DevOps, tag workspaces and use Terraform to enforce policies. In production, we saved 40% by moving verbose DNS logs to Basic Logs and reducing retention from 365 to 90 days.

resource "azurerm_log_analytics_linked_storage_account" "sentinel" {
  data_source_type      = "CustomLogs"
  resource_group_name   = azurerm_resource_group.sentinel.name
  workspace_id          = azurerm_log_analytics_workspace.sentinel.id
  storage_account_id    = azurerm_storage_account.sentinel.id
}

resource "azurerm_log_analytics_workspace_table" "security_event" {
  workspace_id     = azurerm_log_analytics_workspace.sentinel.id
  name             = "SecurityEvent"
  retention_in_days = 30
  total_retention_in_days = 90
}

Disaster Recovery and Multi-Region

For high availability, deploy Sentinel workspaces in paired regions (e.g., East US and West US). Use Azure Site Recovery or manual failover for connectors. Export analytics rules and playbooks as ARM templates or Terraform. For data redundancy, enable cross-region replication in Log Analytics (preview). In a disaster, redeploy the workspace from templates and reattach connectors. Test failover annually. In production, we had a regional outage that took down our primary workspace. We failed over to a secondary workspace in 2 hours using Terraform.

resource "azurerm_resource_group" "sentinel_secondary" {
  name     = "rg-sentinel-secondary"
  location = "West US"
}

resource "azurerm_log_analytics_workspace" "sentinel_secondary" {
  name                = "log-sentinel-secondary"
  location            = azurerm_resource_group.sentinel_secondary.location
  resource_group_name = azurerm_resource_group.sentinel_secondary.name
  sku                 = "PerGB2018"
  retention_in_days   = 90
}

resource "azurerm_sentinel_log_analytics_workspace_onboarding" "sentinel_secondary" {
  workspace_id = azurerm_log_analytics_workspace.sentinel_secondary.id
}

Cost Limit Enforcement for KQL Queries and Notebooks

Sentinel data lake costs can spike from runaway KQL queries or heavy notebook workloads. The cost limit enforcement feature (preview) lets you set hard limits on query and notebook costs. Once a threshold is exceeded, new queries are blocked, and running queries finish normally. This eliminates surprise bills. You can set limits per workspace, per user, or per query type. For production, set a daily cost limit of 50 GB for ad-hoc queries and 200 GB for scheduled analytics rules. Monitor enforcement events in the SentinelHealth table and alert when limits are hit. This is critical for organizations with many analysts running exploratory queries.

# Set cost limit for KQL queries
$limitParams = @{
    workspaceId = 'log-sentinel-prod'
    limitType = 'Query'
    dailyLimitGB = 50
    action = 'Block'
}
Set-AzSentinelCostLimit @limitParams

Write-Host 'Cost limit enforced for KQL queries at 50 GB/day'

Sentinel Data Federation: Query Across Workspaces

Enterprises often have multiple Sentinel workspaces per region or environment. Data federation (preview) lets you query across workspaces without moving data. It uses a federation target workspace that maps to source workspaces. You write KQL queries as if data is local, and Sentinel distributes the query to source workspaces, aggregating results. This eliminates data duplication and reduces egress costs. For production, use data federation for cross-workspace hunting, global threat detection, and compliance reporting. Federation supports up to 10 source workspaces per target. Query performance depends on source workspace latency.

// Query across federated workspaces
union workspace('law-sentinel-eus').SecurityEvent,
      workspace('law-sentinel-wus').SecurityEvent
| where TimeGenerated > ago(1d)
| where EventID == 4625
| summarize FailedLogons = count() by Account, bin(TimeGenerated, 1h)
| where FailedLogons > 5

OSINT Reports in Threat Analytics for Actionable Intelligence

Sentinel's Threat Analytics now ingests curated open-source intelligence (OSINT) alongside Microsoft-authored reports. These OSINT articles come from trusted sources like AlienVault, Recorded Future, and academic research. Each article includes extracted IOCs, mapped MITRE ATT&CK techniques, and recommended actions. For production, use OSINT reports to enrich alerts and prioritize incidents. Configure automation rules to cross-reference IOCs from OSINT with your data. This reduces the time from vulnerability disclosure to detection. Monitor the Threat Analytics dashboard weekly for new OSINT reports relevant to your industry.

// Match OSINT IOCs against your data
let OSINT_IOCs = ThreatIntelligenceIndicator
| where SourceSystem contains 'OSINT'
| where TimeGenerated > ago(1d)
| where IndicatorType == 'ipv4-addr'
| project IOC = tostring(IndicatorValue);
CommonSecurityLog
| where TimeGenerated > ago(1h)
| where DestinationIP in (OSINT_IOCs)
| project TimeGenerated, SourceIP, DestinationIP, Message