GCP Audit Logs: Admin Activity, Data Access, and Log Exports

You just discovered that a production database was deleted at 3 AM. No one on your team admits to doing it. Without audit logs, you're blind. GCP Audit Logs are the immutable record of every API call, every IAM change, every configuration mutation. They are not optional—they are your first line of defense against insider threats, misconfigurations, and compliance failures. In this article, I'll show you how to read them, export them, and build alerting around them. By the end, you'll know exactly how to answer 'who deleted the database' in under 30 seconds.

Audit Logs Overview: The Three Types

GCP Audit Logs are divided into three categories: Admin Activity, Data Access, and System Events. Admin Activity logs capture all API calls that modify configuration or metadata—creating a VM, changing IAM policies, deleting a bucket. These are always enabled and free. Data Access logs capture reads and writes to user data—reading from BigQuery tables, downloading objects from Cloud Storage. These are opt-in and incur costs. System Events logs capture GCP infrastructure actions like automatic failovers or maintenance events. Understanding the distinction is critical: Admin Activity logs are your 'who changed what', Data Access logs are your 'who read what'.

gcloud logging read "logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity" --limit=5 --format=json

Admin Activity Logs: The Default Security Net

Admin Activity logs are enabled by default for all GCP projects. They record every API call that modifies a resource's configuration or metadata. This includes creating, deleting, or modifying Compute Engine instances, IAM policies, Cloud Storage buckets, and more. The logs are retained for 400 days and cannot be disabled. They are the first place to look when investigating unauthorized changes. The log entries include the method name, the principal (user or service account), the resource affected, and the request metadata. Use them to build a baseline of normal administrative activity.

gcloud logging read "logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity AND protoPayload.methodName=google.cloud.compute.v1.Instances.Delete" --limit=10 --format="value(protoPayload.authenticationInfo.principalEmail, protoPayload.resourceName)"

Data Access Logs: Tracking Data Reads and Writes

Data Access logs capture API calls that read or write user-provided data. This includes reading rows from BigQuery tables, downloading objects from Cloud Storage, or writing to Firestore. Unlike Admin Activity logs, these are not enabled by default. You must enable them per service and per resource. They are critical for compliance (e.g., HIPAA, SOC 2) and for detecting data exfiltration. However, they generate high volume and cost. Best practice is to enable them only for sensitive resources and to use exclusion filters to reduce noise.

# Enable Data Access logs for a Cloud Storage bucket
gcloud storage buckets update gs://my-sensitive-bucket --audit-log-configs=read=true,write=true

# Verify with:
gcloud storage buckets describe gs://my-sensitive-bucket --format="value(iamConfiguration)"

System Event Logs: Infrastructure Actions You Didn't Initiate

System Event logs record actions taken by Google Cloud infrastructure on your behalf. Examples include automatic failover of a Cloud SQL instance, maintenance events on Compute Engine hosts, or zone outages. These logs are always enabled and free. They are essential for understanding why your application behaved unexpectedly—e.g., a VM was live migrated without your knowledge. System Event logs are written to the log name cloudaudit.googleapis.com/system_event. They are less commonly used but invaluable for root cause analysis of infrastructure-level incidents.

gcloud logging read "logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fsystem_event" --limit=5 --format="value(protoPayload.methodName, timestamp)"

Log Exports: Sending Audit Logs to Durable Storage

By default, audit logs are retained for 400 days. For compliance or long-term analysis, you must export them to durable storage like Cloud Storage, BigQuery, or Pub/Sub. Log exports are configured via log sinks. A sink defines a destination and a filter to select which logs to export. You can create sinks at the project, folder, or organization level. Organization-level sinks are recommended for centralized audit log management. Exports to Cloud Storage are cost-effective for archival; exports to BigQuery enable SQL-based analysis; exports to Pub/Sub enable real-time streaming to SIEM systems.

# Create a log sink to export all Admin Activity logs to BigQuery
gcloud logging sinks create admin-activity-bq \
  bigquery.googleapis.com/projects/PROJECT_ID/datasets/audit_logs \
  --log-filter="logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"

# Create a sink to export all audit logs to Cloud Storage
gcloud logging sinks create all-audit-logs-storage \
  storage.googleapis.com/audit-logs-bucket \
  --log-filter="logName:cloudaudit.googleapis.com"

Filtering Audit Logs with Logs Explorer

Logs Explorer is the primary UI for querying audit logs. It supports a powerful filtering language. Common filters include: by log name (activity, data_access, system_event), by method name, by principal email, by resource type, and by timestamp. You can also use regular expressions. For complex queries, use the advanced filter mode. Always filter by time range to avoid scanning the entire log history. Save frequently used queries as saved searches for team reuse.

# Find all IAM policy changes by a specific user in the last 7 days
gcloud logging read "
  logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity AND
  protoPayload.methodName=google.iam.admin.v1.SetIamPolicy AND
  protoPayload.authenticationInfo.principalEmail=admin@example.com AND
  timestamp >= '2026-07-05T00:00:00Z'
" --limit=20 --format=json

Alerting on Audit Logs with Log-Based Metrics

You can create log-based metrics from audit logs and set up alerts. For example, alert on any deletion of a Cloud Storage bucket, any IAM policy change, or any Data Access log from an unexpected IP. Log-based metrics count log entries matching a filter. You can then create alerting policies based on the metric's value. This is the foundation of real-time security monitoring. Be careful with Data Access logs—they can trigger false positives if not scoped properly.

# Create a log-based metric for bucket deletions
gcloud logging metrics create bucket-deletion-alert \
  --description="Alert on Cloud Storage bucket deletions" \
  --log-filter="
    logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity AND
    protoPayload.methodName=storage.buckets.delete
  "

# Create an alert policy (via gcloud or console)
gcloud alpha monitoring policies create \
  --display-name="Bucket Deletion Alert" \
  --condition-filter="metric.type=logging.googleapis.com/user/bucket-deletion-alert" \
  --condition-threshold-value=1 \
  --condition-duration=0s \
  --notification-channels="projects/PROJECT_ID/notificationChannels/CHANNEL_ID"

Audit Logs for Compliance: HIPAA, SOC 2, and PCI DSS

Audit logs are a cornerstone of compliance frameworks. HIPAA requires logging of all access to ePHI. SOC 2 requires monitoring of system changes. PCI DSS requires logging of all access to cardholder data. GCP Audit Logs, when properly exported and retained, can satisfy these requirements. Key practices: enable Data Access logs for sensitive data stores, export logs to immutable Cloud Storage buckets (with object lock), retain logs for the required period (e.g., 7 years for PCI), and restrict access to log destinations.

# Create a Cloud Storage bucket with retention policy for compliance
gcloud storage buckets create gs://compliance-audit-logs \
  --location=US \
  --retention-period=2555d \
  --uniform-bucket-level-access

# Enable object lock
gcloud storage buckets update gs://compliance-audit-logs --object-lock-mode=Enabled

Cost Management for Audit Logs

Audit logs can become expensive, especially Data Access logs. Costs come from log ingestion, storage in Cloud Logging, and export destinations. To manage costs: use exclusion filters to drop noisy logs (e.g., health checks), set log bucket retention to minimum required, export to Cloud Storage (cheaper than BigQuery for archival), and use BigQuery partitioning and clustering for query efficiency. Monitor your logging costs in the Billing console.

# Create an exclusion filter to drop health check logs
gcloud logging exclusions create drop-health-checks \
  --description="Exclude health check requests from Data Access logs" \
  --log-filter="
    logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access AND
    protoPayload.methodName=storage.objects.get AND
    protoPayload.resourceName=projects/_/buckets/health-check-bucket/objects/*
  "

Troubleshooting Missing Audit Logs

Sometimes audit logs are missing. Common causes: logs are not yet ingested (up to a few minutes delay), the log sink filter is too restrictive, the log bucket retention period has expired, or the resource is in a different project. Also, some services do not generate audit logs for certain operations (e.g., internal Google APIs). To troubleshoot: check the Logs Explorer with a broad time range, verify sink configuration, and ensure the log bucket exists and is writable.

# Verify sink configuration
gcloud logging sinks describe admin-activity-bq

# Check if logs are being written to the destination
gcloud logging read "logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity" --limit=1 --format="value(timestamp)"

Best Practices for Audit Log Management

1. Enable Data Access logs only for sensitive resources. 2. Export all audit logs to a centralized BigQuery dataset for analysis. 3. Set up alerts on critical events (IAM changes, resource deletions). 4. Use exclusion filters to reduce noise and cost. 5. Retain logs according to compliance requirements. 6. Restrict access to audit logs to security and compliance teams. 7. Regularly review audit logs for anomalies. 8. Automate log analysis with scheduled queries in BigQuery.

-- Find top 10 users by number of IAM changes in the last 30 days
SELECT
  protoPayload.authenticationInfo.principalEmail AS user,
  COUNT(*) AS changes
FROM
  `PROJECT_ID.audit_logs.cloudaudit_googleapis_com_activity_*`
WHERE
  protoPayload.methodName = 'google.iam.admin.v1.SetIamPolicy'
  AND _TABLE_SUFFIX >= FORMAT_TIMESTAMP('%Y%m%d', TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY))
GROUP BY
  user
ORDER BY
  changes DESC
LIMIT 10;

ADMIN_READ Audit Logs: The Third Category You Might Be Missing

Beyond Admin Activity, Data Access, and System Events, there is a fourth subcategory often overlooked: ADMIN_READ logs. These capture API calls that read metadata or configuration—like listing buckets, getting IAM policies, or describing instances. ADMIN_READ is a type of Data Access log (since it requires an IAM permission with type ADMIN_READ), but it is disabled by default for most services. Enabling ADMIN_READ logs gives you visibility into who is enumerating your resources, which can indicate reconnaissance activity. For example, repeated storage.buckets.list calls from an unfamiliar IP might precede an attack. ADMIN_READ logs are cheaper than full DATA_READ/DATA_WRITE because metadata reads are less frequent than data reads. Best practice: enable ADMIN_READ for critical services (IAM, Cloud Storage, Compute Engine) to monitor for reconnaissance without the cost of full Data Access logs. Configure them via the IAM Audit Logs page or the auditConfigs API with logType: ADMIN_READ.

# Enable ADMIN_READ audit logs for Compute Engine via API
cat > audit_config.json <<EOF
{
  "auditConfigs": [
    {
      "service": "compute.googleapis.com",
      "auditLogConfigs": [
        { "logType": "ADMIN_READ" }
      ]
    }
  ]
}
EOF

# Read current IAM policy, merge audit config, set
gcloud projects get-iam-policy my-project --format=json > policy.json
jq '.auditConfigs += [{"service": "compute.googleapis.com", "auditLogConfigs": [{"logType": "ADMIN_READ"}]}]' policy.json > updated_policy.json
gcloud projects set-iam-policy my-project updated_policy.json

# Query ADMIN_READ logs
gcloud logging read "logName=projects/my-project/logs/cloudaudit.googleapis.com%2Fdata_access AND protoPayload.methodName=compute.instances.list" --limit=5

Building an Audit Log Analysis Pipeline with BigQuery and Looker Studio

Raw audit logs in Cloud Logging are hard to query at scale, expire after 400 days, and lack visualization. The solution: export all audit logs to BigQuery and build dashboards in Looker Studio. Create organization-level log sinks that route Admin Activity, Data Access, and System Events to a centralized BigQuery dataset. Use the --use-partitioned-tables flag to create date-partitioned tables for cost-efficient queries. Then build Looker Studio dashboards showing: activity heatmaps (hour x day), top actors, IAM changes timeline, failed operations, and resource deletion tracking. Scheduled queries in BigQuery can run daily compliance reports. For example, a query that finds all IAM policy changes in the last 24 hours can be emailed to the security team. This pipeline turns audit logs from a reactive debugging tool into a proactive security monitoring platform.

# Step 1: Create organization-level sink to BigQuery
gcloud logging sinks create org-admin-activity-sink \
  bigquery.googleapis.com/projects/audit-project/datasets/org_audit_logs \
  --organization=ORG_ID \
  --log-filter='logName:"cloudaudit.googleapis.com/activity"' \
  --include-children \
  --use-partitioned-tables

# Grant sink writer access to BigQuery
SINK_SA=$(gcloud logging sinks describe org-admin-activity-sink \
  --organization=ORG_ID --format="value(writerIdentity)")
bq query --use_legacy_sql=false \
  "GRANT \`roles/bigquery.dataEditor\` ON SCHEMA \`audit-project\`.org_audit_logs TO \"$SINK_SA\""

# Step 2: Sample query for user activity summary
bq query --use_legacy_sql=false '
SELECT
  timestamp,
  protopayload_auditlog.authenticationInfo.principalEmail AS principal_email,
  protopayload_auditlog.methodName AS method,
  resource.type AS resource_type,
  resource.labels.project_id AS project_id
FROM `audit-project.org_audit_logs.cloudaudit_googleapis_com_activity`
WHERE timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
ORDER BY timestamp DESC
LIMIT 100'

SQL Queries for Security Insights: Detecting Privilege Escalation and Anomalies

Once audit logs are in BigQuery, you can run SQL queries for security analysis. Key query patterns include: 1) Detecting privilege escalation—find all SetIamPolicy calls where a role like roles/iam.serviceAccountAdmin or roles/owner was granted to a new principal. 2) Identifying failed authentication attempts—group by source IP to detect brute force. 3) Finding changes to Logging settings—an attacker might disable audit logging to cover tracks. 4) Detecting service account impersonation—look for SetIamPolicy on service accounts. 5) High API usage by a single principal—compute average daily calls and standard deviation, flag anything beyond 3 sigma. These queries turn audit logs into an active detection system. Schedule them to run via BigQuery scheduled queries and email results to your security team.

-- Detect privilege escalation: new IAM role grants
SELECT
  timestamp,
  protopayload_auditlog.authenticationInfo.principalEmail AS grantor,
  JSON_VALUE(bindingDelta.member) AS grantee,
  JSON_VALUE(bindingDelta.role) AS role,
  protopayload_auditlog.resourceName AS resource
FROM `audit-project.org_audit_logs.cloudaudit_googleapis_com_activity`,
UNNEST(JSON_QUERY_ARRAY(protopayload_auditlog.serviceData.policyDelta.bindingDeltas)) AS bindingDelta
WHERE
  protopayload_auditlog.methodName = "google.iam.admin.v1.SetIamPolicy"
  AND JSON_VALUE(bindingDelta.action) = "ADD"
  AND JSON_VALUE(bindingDelta.role) IN ("roles/owner", "roles/editor", "roles/iam.securityAdmin")
  AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
ORDER BY timestamp DESC;

-- Detect anomalous API usage (3 sigma)
WITH daily_stats AS (
  SELECT
    protopayload_auditlog.authenticationInfo.principalEmail AS user,
    DATE(timestamp) AS day,
    COUNT(*) AS calls
  FROM `audit-project.org_audit_logs.cloudaudit_googleapis_com_activity`
  GROUP BY user, day
)
SELECT
  user,
  AVG(calls) AS avg_calls,
  STDDEV(calls) AS stddev_calls,
  MAX(calls) AS max_calls
FROM daily_stats
GROUP BY user
HAVING MAX(calls) > AVG(calls) + 3 * STDDEV(calls)
ORDER BY max_calls DESC;