Cloud Build: CI/CD Pipelines, Build Triggers, and Artifacts

Your production deployment just failed because the build pipeline silently skipped a failing test. The root cause? A misconfigured trigger that only ran on pushes to main, not on pull requests. Cloud Build promises simplicity, but without understanding its internals, you'll ship broken code. This isn't a tutorial—it's a battle-tested guide to building CI/CD pipelines that don't lie. We'll cover triggers, artifact management, and the gotchas that cost teams hours of debugging. By the end, you'll know exactly how to set up a pipeline that catches failures before they hit production.

What Is Cloud Build?

Cloud Build is Google Cloud's managed CI/CD platform. It executes build steps in a containerized environment, pulling source from Cloud Source Repositories, GitHub, Bitbucket, or GitLab. Each step runs in a Docker container, and you define the pipeline in a cloudbuild.yaml file. Cloud Build is serverless—no VMs to manage, no cluster to scale. It integrates with Artifact Registry, Cloud Run, GKE, and other GCP services. Use it when you need a simple, event-driven pipeline that scales to zero when idle. But don't mistake simplicity for lack of depth: misconfigured triggers or missing artifacts can silently break your deployment.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA']
- name: 'gcr.io/cloud-builders/kubectl'
  args: ['set', 'image', 'deployment/my-app', 'my-app=us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA']
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster'
artifacts:
  objects:
    location: 'gs://my-bucket/build-artifacts/'
    paths: ['build-output/*']

Build Triggers: Automating Your Pipeline

Build triggers automate pipeline execution based on events like pushes, pull requests, or tag creation. You can configure triggers via the Cloud Console, gcloud CLI, or Terraform. Each trigger points to a source repository and a cloudbuild.yaml file. Triggers support regex-based branch filtering, so you can run different pipelines for main vs. feature branches. A common mistake is forgetting to enable trigger on PRs—without it, you won't catch failures until code is merged. Also, triggers can be paused or disabled, which is useful during maintenance but dangerous if forgotten.

gcloud builds triggers create github \
  --name="my-app-trigger" \
  --repo-owner="my-org" \
  --repo-name="my-app" \
  --branch-pattern="^main$" \
  --build-config="cloudbuild.yaml" \
  --substitutions="_ENV=prod"

Artifact Management: Storing Build Outputs

Artifacts are the outputs of your build process—Docker images, compiled binaries, test reports, etc. Cloud Build can store artifacts in Cloud Storage or push images to Artifact Registry. Use the artifacts block in cloudbuild.yaml to specify which files to persist. Artifacts are essential for reproducibility: you can redeploy any past build by referencing its artifact. A common pitfall is not storing artifacts at all, making rollbacks impossible. Also, set lifecycle policies on your bucket to avoid runaway storage costs.

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', '-o', 'build-output/myapp', '.']
artifacts:
  objects:
    location: 'gs://my-artifacts-bucket/$BUILD_ID/'
    paths: ['build-output/*']
  images:
  - 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA'

Building Docker Images Efficiently

Docker image builds are the most common Cloud Build step. To speed them up, use Docker layer caching. Cloud Build automatically caches layers from previous builds if you use the same builder image. However, the cache is per-host, and if your build runs on different hosts, you lose the cache. To mitigate, use Kaniko or build with --cache-from pointing to a previous image. Also, order your Dockerfile layers from least to most frequently changing to maximize cache hits. A typical mistake is installing dependencies before copying source, invalidating the cache on every change.

FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/index.js"]

Running Tests in the Pipeline

Tests should run as early as possible in your pipeline. Cloud Build supports running tests in parallel using step groups or by splitting steps across multiple containers. Use the waitFor field to control execution order. For unit tests, run them in a container that has your test framework. For integration tests, spin up a service (e.g., a test database) using a sidecar container. A common failure is not failing the build when tests fail—ensure your test command exits with a non-zero code. Also, store test reports as artifacts for debugging.

steps:
- name: 'node:18'
  entrypoint: 'npm'
  args: ['ci']
- name: 'node:18'
  entrypoint: 'npm'
  args: ['test']
  id: 'unit-tests'
- name: 'gcr.io/cloud-builders/docker'
  args: ['run', '-d', '--name=test-db', 'postgres:13']
  id: 'start-db'
- name: 'node:18'
  entrypoint: 'npm'
  args: ['run', 'test:integration']
  waitFor: ['unit-tests', 'start-db']
  env:
  - 'DATABASE_URL=postgres://user:pass@localhost:5432/test'
artifacts:
  objects:
    location: 'gs://my-artifacts-bucket/$BUILD_ID/'
    paths: ['test-results/*']

Deploying to Cloud Run

Cloud Run is a common deployment target for Cloud Build. After building and pushing a Docker image, use the gcloud builder to deploy. The deploy step requires the image URL and service name. Use substitutions to parameterize the service name and region. A common mistake is not specifying the region, causing the deployment to fail or deploy to the wrong region. Also, set concurrency and timeout flags to match your application needs. For zero-downtime deployments, Cloud Run handles traffic shifting automatically, but ensure your health checks pass.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: 'gcloud'
  args: ['run', 'deploy', 'my-service',
         '--image=us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA',
         '--region=us-central1',
         '--platform=managed',
         '--allow-unauthenticated',
         '--concurrency=80',
         '--timeout=300']

Deploying to Google Kubernetes Engine (GKE)

For GKE deployments, Cloud Build can update a Kubernetes deployment using kubectl. You need to authenticate to the cluster using the gcloud container clusters get-credentials command. Then use kubectl set image to update the deployment. A common pitfall is not having the correct IAM permissions—the Cloud Build service account needs permissions to access the cluster. Also, consider using Helm or Kustomize for more complex deployments. For production, use a rolling update strategy to minimize downtime.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA']
- name: 'gcr.io/cloud-builders/kubectl'
  args:
  - 'set'
  - 'image'
  - 'deployment/my-app'
  - 'my-app=us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA'
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-central1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster'

Secrets and Environment Variables

Never hardcode secrets in cloudbuild.yaml. Use Cloud Secret Manager to store sensitive data like API keys or database passwords. Access secrets via the availableSecrets block, which injects them as environment variables or volumes. For environment variables that are not secrets, use substitutions or the env field. A common mistake is using substitutions for secrets—substitutions are visible in build logs. Always use Secret Manager for anything sensitive. Also, limit the scope of secrets to only the steps that need them.

steps:
- name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    echo "$$DB_PASSWORD" > /tmp/db_password
  secretEnv: ['DB_PASSWORD']
availableSecrets:
  secretManager:
  - versionName: projects/$PROJECT_ID/secrets/DB_PASSWORD/versions/latest
    env: 'DB_PASSWORD'

Parallelism and Step Dependencies

Cloud Build runs steps sequentially by default. To speed up pipelines, run independent steps in parallel using the waitFor field. Set waitFor: ['step-name'] to make a step wait for another. Use '-' to indicate no dependency (runs immediately). You can also use step groups (not natively supported, but you can simulate with multiple steps). A common mistake is not setting waitFor correctly, causing steps to run out of order. For example, tests should wait for the build step. Also, consider using build timeouts to prevent runaway builds.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'my-image', '.']
  id: 'build'
- name: 'node:18'
  entrypoint: 'npm'
  args: ['test']
  waitFor: ['build']
  id: 'unit-tests'
- name: 'gcr.io/cloud-builders/docker'
  args: ['run', 'my-image', 'npm', 'run', 'test:integration']
  waitFor: ['build']
  id: 'integration-tests'
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['run', 'deploy', 'my-service', '--image=my-image']
  waitFor: ['unit-tests', 'integration-tests']

Monitoring and Logging Builds

Cloud Build logs are automatically sent to Cloud Logging. You can view them in the Console or export to BigQuery for analysis. Set up alerts for build failures using Cloud Monitoring. A common mistake is not monitoring build duration—if builds start taking longer, it could indicate a problem (e.g., cache invalidation). Also, use build status notifications via Pub/Sub to trigger downstream actions (e.g., notify Slack). For debugging, enable verbose logging by setting _CLOUD_BUILD_LOGGING=verbose substitution.

gcloud alpha monitoring policies create \
  --policy-from-file=build-failure-policy.yaml

Cost Optimization

Cloud Build charges per build minute, with a free tier of 120 build-minutes per day. To optimize costs, use smaller machine types (e.g., e2-standard-2 instead of e2-standard-8) if your build doesn't need many cores. Also, cache aggressively to reduce build time. Use Kaniko for faster image builds. Set lifecycle policies on artifact buckets to delete old artifacts. A common mistake is leaving unused triggers active—they still incur costs if triggered. Disable or delete triggers for archived repositories.

steps:
- name: 'gcr.io/kaniko-project/executor:latest'
  args:
  - --destination=us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-image:$SHORT_SHA
  - --cache=true
  - --cache-ttl=24h
options:
  machineType: 'E2_HIGHCPU_8'

Cloud Deploy Integration: From CI to CD Without the Gap

Cloud Build handles CI (build, test, package). Cloud Deploy handles CD (progressive delivery, promotion, rollback). Together they form a complete CI/CD pipeline. After a successful build, Cloud Build creates a Cloud Deploy release, specifying the image digest. Cloud Deploy then progresses the release through targets (dev -> staging -> prod) with promotion gates, approval requirements, and canary strategies. Cloud Deploy uses Skaffold under the hood to render Kubernetes manifests or Cloud Run YAML with the correct image digest. You define targets with optional approval requirements and deployment strategies (standard, canary with phased percentage rollouts, or blue-green). Promotion can be automated or require manual approval. A key advantage: the same artifact built once in Cloud Build is deployed to every environment, eliminating 'works on my machine' issues. For production, combine Cloud Deploy's canary deployment with Binary Authorization to ensure only attested images are deployed. In production, we use Cloud Build + Cloud Deploy for a 4-environment pipeline: dev (auto-promote), staging (auto-promote after tests), pre-prod (manual approval), prod (canary 25% -> 50% -> 100% with auto-rollback on failure).

apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: app-pipeline
serialPipeline:
  stages:
  - targetId: dev
    profiles: ["dev"]
    strategy:
      standard:
        verify: false
  - targetId: staging
    profiles: ["staging"]
    strategy:
      standard:
        verify: true
  - targetId: prod
    profiles: ["prod"]
    strategy:
      canary:
        canaryDeployment:
          percentages: [25, 50, 100]
          verify: true
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: prod
description: production cluster
requireApproval: true
gke:
  cluster: projects/my-project/locations/us-central1/clusters/prod-cluster

Private Pools and Approval Gates: Security for Enterprise Pipelines

By default, Cloud Build runs on shared Google-managed infrastructure. For compliance requirements (accessing private VPC resources, no public internet exposure, static IP for egress), use private pools. A private pool runs in your VPC, with worker VMs that can reach Cloud SQL, GKE clusters, or internal services without traversing the internet. Private pools also support static egress IPs for integrations that require IP allowlisting. Create a private pool with gcloud builds worker-pools create specifying the VPC network and subnet. Builds using the pool are billed at the pool's machine type rates. For additional security, Cloud Build supports approval gates on triggers: when enabled, triggered builds enter a PENDING_APPROVAL state until a user with cloudbuild.approver role explicitly approves or rejects them. This is useful for production deployments where you want a human in the loop without giving everyone trigger-edit permissions. In production, we use private pools for all builds that access private Artifact Registry or internal test databases, and approval gates on production deployment triggers.

# Create a private pool in your VPC
gcloud builds worker-pools create my-private-pool \
  --region=us-central1 \
  --peered-network=projects/my-project/global/networks/default \
  --worker-machine-type=e2-standard-4 \
  --worker-disk-size=100

# Create a trigger that uses the private pool
gcloud builds triggers create github \
  --name="private-ci" \
  --repo-owner="my-org" \
  --repo-name="my-app" \
  --branch-pattern="^main$" \
  --build-config="cloudbuild.yaml" \
  --worker-pool=projects/my-project/locations/us-central1/workerPools/my-private-pool