Cloud Source Repositories: Git Hosting, Triggers, and Code Search

Your CI/CD pipeline just failed. Again. The build triggered on a push to develop, but the test suite ran against the wrong branch because your Git hooks were misconfigured. Meanwhile, your security team is demanding audit logs for every code push, and your junior dev just accidentally force-pushed to main. Sound familiar? Cloud Source Repositories (CSR) is Google Cloud's answer to these headaches—a managed Git hosting service that doesn't just store code but enforces policies, triggers builds, and indexes your entire codebase for search. Unlike GitHub or GitLab, CSR is built from the ground up to integrate with Cloud Build, IAM, and Secret Manager, giving you a single pane of glass for code, CI/CD, and access control. In this guide, we'll walk through setting up CSR, configuring triggers that actually work in production, and using Code Search to find that one line of code you wrote six months ago. No fluff, just production patterns.

Setting Up a Cloud Source Repository

Creating a CSR repository is straightforward, but the real value comes from how you configure it. Start by enabling the Cloud Source Repositories API and creating a repository via the console or gcloud. Use gcloud source repos create my-app --project=my-project. This creates an empty Git repo hosted at https://source.developers.google.com/p/my-project/r/my-app. You can then clone it and push your code. However, in production, you'll want to mirror an existing GitHub or Bitbucket repo to centralize access. Use the 'Mirror a repository' option in the console, which sets up a webhook to keep CSR in sync. This is critical for teams that want to keep their existing workflow but need GCP-native triggers. One common mistake: forgetting to set up IAM permissions. By default, only project editors and owners can push. Grant source.writer role to developers and source.reader to CI/CD service accounts. Also, enable 'Require signed commits' in repository settings to enforce commit signing—this prevents the 'who pushed this?' blame game.

#!/bin/bash
# Create a Cloud Source Repository
PROJECT_ID="my-gcp-project"
REPO_NAME="my-app"

gcloud source repos create $REPO_NAME --project=$PROJECT_ID

# Clone the empty repo
gcloud source repos clone $REPO_NAME --project=$PROJECT_ID
cd $REPO_NAME

# Add your code and push
git add .
git commit -m "Initial commit"
git push origin main

# Set IAM permissions for developers
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="user:dev@example.com" \
  --role="roles/source.writer"

Configuring Cloud Build Triggers

Triggers in CSR are the backbone of your CI/CD pipeline. They automatically start a Cloud Build run when code is pushed to a branch or a tag is created. To create a trigger, go to Cloud Build > Triggers > Create Trigger. Choose 'Cloud Source Repository' as the event source. You can filter by branch (e.g., ^main$), tag (^v.), or file paths (/.yaml). In production, use branch-specific triggers: one for main that deploys to production, one for develop that deploys to staging, and one for feature branches that runs tests only. Avoid using 'any branch' triggers—they waste build minutes and can accidentally deploy to production. Also, set up a trigger for tags to handle releases. Use a cloudbuild.yaml that includes steps for linting, testing, building, and deploying. One critical setting: 'Invert regex' for file path filters. Use this to skip builds when only documentation changes. For example, filter by !docs/** to avoid triggering on doc updates.

steps:
- name: 'gcr.io/cloud-builders/git'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    git fetch --unshallow
    git diff --name-only HEAD~1 | grep -q '^docs/' && exit 0 || true
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-app:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'gcr.io/$PROJECT_ID/my-app:$SHORT_SHA']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: 'gcloud'
  args: ['run', 'deploy', 'my-app', '--image=gcr.io/$PROJECT_ID/my-app:$SHORT_SHA', '--region=us-central1']
substitutions:
  _DEPLOY_REGION: us-central1
options:
  machineType: 'E2_HIGHCPU_8'
  substitution_option: 'ALLOW_LOOSE'

Advanced Trigger Patterns: Manual Approvals and Pub/Sub

For production deployments, you need more than just automatic triggers. Cloud Build supports manual approval gates that pause a build until a reviewer approves it. To set this up, create a trigger with 'Require approval' enabled. The build will stop at a designated step (e.g., 'deploy') and wait for approval via the console or gcloud builds approve. Another pattern is using Pub/Sub to trigger builds from external events, like a new issue in Jira or a Slack command. Create a Cloud Function that listens to a Pub/Sub topic and calls the Cloud Build API. This decouples your CI/CD from Git events. For example, you can trigger a rebuild of the production environment when a config change is approved in an external system. Also, consider using 'Build triggers with substitutions' to parameterize builds. For instance, pass the branch name as a substitution variable to dynamically set the deployment target.

import base64
import json
from google.cloud import cloudbuild_v1

def trigger_build(event, context):
    """Trigger a Cloud Build from a Pub/Sub message."""
    pubsub_message = base64.b64decode(event['data']).decode('utf-8')
    data = json.loads(pubsub_message)
    
    client = cloudbuild_v1.CloudBuildClient()
    build = cloudbuild_v1.Build()
    build.steps = [
        {"name": "gcr.io/cloud-builders/gcloud", "args": ["builds", "submit", "--config=cloudbuild.yaml"]}
    ]
    build.source = {"repo_source": {"repo_name": "my-app", "branch_name": data.get("branch", "main")}}
    
    operation = client.create_build(project_id="my-project", build=build)
    operation.result()
    print(f"Build triggered: {operation.metadata.build.id}")

Code Search: Finding Needles in the Haystack

CSR's Code Search indexes your entire repository and allows you to search across all repos in your project using regular expressions. This is a lifesaver when you need to find where a deprecated API is used or track down a hardcoded secret. Access it via the console under 'Source Repositories > Code Search'. You can search by filename, content, or symbol. For example, repo:my-app path:src/*/.py 'def handle_' finds all Python functions starting with handle_. In production, use Code Search to enforce coding standards. For instance, search for password or secret in your codebase to find potential leaks. Combine with IAM to restrict search access—only users with source.reader can search. One limitation: Code Search only indexes the default branch (usually main). If you need to search other branches, you must manually trigger a reindex. Also, the search index can be up to 10 minutes stale, so don't rely on it for real-time auditing.

# Search for all TODO comments in the repo
gcloud source repos list --project=my-project | while read repo; do
  echo "Searching $repo for TODOs..."
  gcloud source repos search "repo:$repo path:**/*.py 'TODO'" --project=my-project
done

# Search for hardcoded secrets
gcloud source repos search "repo:my-app 'password='" --project=my-project

IAM and Access Control Best Practices

CSR integrates with Cloud IAM, allowing you to grant fine-grained permissions at the project or repository level. The key roles are source.reader (read-only), source.writer (read/write), and source.admin (full control). In production, never use primitive roles (owner, editor, viewer) for CSR—they grant too much access. Instead, create custom roles that combine source.writer with cloudbuild.builds.submit for developers who need to trigger builds. Use IAM conditions to restrict access by IP address or time. For example, allow pushes only from corporate VPN IPs. Also, set up VPC Service Controls to prevent data exfiltration. One common mistake: granting source.admin to a service account that only needs to read code. This can lead to accidental deletion. Always follow the principle of least privilege. For audit logging, enable Data Access audit logs for CSR to track who read or wrote to repos.

#!/bin/bash
PROJECT_ID="my-gcp-project"
REPO_NAME="my-app"

# Create a custom role with minimal permissions
gcloud iam roles create csr_developer --project=$PROJECT_ID \
  --title="CSR Developer" \
  --permissions="source.repos.get,source.repos.list,source.repos.update,source.repos.push,cloudbuild.builds.create" \
  --stage=GA

# Grant the role to a group
gcloud source repos add-iam-policy-binding $REPO_NAME \
  --member="group:dev-team@example.com" \
  --role="projects/$PROJECT_ID/roles/csr_developer" \
  --project=$PROJECT_ID

# Add IP condition (requires gcloud beta)
gcloud beta source repos add-iam-policy-binding $REPO_NAME \
  --member="user:remote@example.com" \
  --role="roles/source.reader" \
  --condition="expression=request.headers['x-forwarded-for'].startsWith('10.0.0.'),title=CorporateIP" \
  --project=$PROJECT_ID

Mirroring External Repositories

If your team uses GitHub or Bitbucket, you can mirror those repos to CSR. This gives you GCP-native triggers and Code Search while keeping your existing workflow. To set up a mirror, go to 'Source Repositories > Add Repository > Mirror external repository'. You'll need to authenticate with your external provider. CSR will create a webhook that syncs changes in near real-time. In production, use this pattern to centralize all code in GCP for compliance. However, be aware of limitations: mirroring is one-way (external -> CSR). You cannot push to CSR and have it sync back. Also, the mirror only syncs the default branch and tags. If you need other branches, you must manually trigger a sync. Another gotcha: if your external repo is deleted, the mirror becomes orphaned. To avoid this, set up a Cloud Function that periodically checks the health of the mirror.

# Add a GitHub mirror
gcloud source repos create my-app-mirror --project=my-project

gcloud source repos update my-app-mirror \
  --add-mirror=https://github.com/myorg/my-app.git \
  --project=my-project

# Verify the mirror
gcloud source repos describe my-app-mirror --project=my-project

Using CSR with Cloud Functions and App Engine

CSR integrates directly with Cloud Functions and App Engine for source-based deployments. Instead of uploading a zip file, you can deploy from a CSR repo. For Cloud Functions, use gcloud functions deploy with --source=. For App Engine, use gcloud app deploy with --source=. This ties your deployment to a specific commit, making rollbacks easy. In production, always specify a branch or tag in the source URL to avoid deploying the latest commit unintentionally. For example, --source=https://source.developers.google.com/p/my-project/r/my-app/refs/heads/main. One common issue: if your function has dependencies, you must include a package.json or requirements.txt in the repo. Also, Cloud Functions deployment from CSR can be slow because it clones the entire repo. To speed it up, use a shallow clone by setting --source with a commit hash instead of a branch.

# Deploy a Cloud Function from CSR
gcloud functions deploy my-function \
  --source=https://source.developers.google.com/p/my-project/r/my-app/refs/heads/main \
  --entry-point=helloWorld \
  --runtime=nodejs18 \
  --trigger-http \
  --allow-unauthenticated \
  --region=us-central1

# Deploy App Engine from CSR
gcloud app deploy \
  --source=https://source.developers.google.com/p/my-project/r/my-app/refs/tags/v1.0.0 \
  --version=v1 \
  --quiet

Troubleshooting Common CSR Issues

Even with careful setup, things go wrong. Here are common issues and fixes. Push rejected: Check IAM permissions—you need source.writer. Also, if 'Require signed commits' is enabled, your commit must be signed with a GPG key. Trigger not firing: Verify the trigger's branch filter regex. Use ^main$ not main. Also, check that the Cloud Build service account has source.reader on the repo. Code Search not showing results: The index may be stale. Wait up to 10 minutes or trigger a reindex by pushing a dummy commit. Mirror sync failing: The webhook may be blocked by a firewall. Ensure your external repo can reach Google's webhook endpoint. Also, check that the mirror's service account has access. Slow clone times: CSR repos can be large. Use shallow clones with git clone --depth 1. For CI/CD, use gcloud source repos clone with --depth=1.

# Check IAM permissions
gcloud source repos get-iam-policy my-app --project=my-project

# Test trigger manually
gcloud builds submit --config=cloudbuild.yaml --substitutions=_DEPLOY_REGION=us-central1

# Force reindex Code Search
git commit --allow-empty -m "Force reindex"
git push origin main

# Shallow clone for CI
gcloud source repos clone my-app --project=my-project --depth=1

Integrating with Secret Manager

CSR doesn't have built-in secret management, but you can integrate with Secret Manager to avoid hardcoding secrets in your codebase. Use Cloud Build to access secrets during builds. For example, to pull a Docker image from a private registry, store the credentials in Secret Manager and access them in your cloudbuild.yaml using the availableSecrets field. This keeps secrets out of your Git history. In production, never commit .env files or service account keys. Instead, use Secret Manager and pass secrets as environment variables. One pattern: use a build step that writes secrets to a file only during the build, then deletes them. Also, set up IAM on Secret Manager to restrict which service accounts can access which secrets. This prevents a compromised build from leaking all secrets.

steps:
- name: 'gcr.io/cloud-builders/docker'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    docker login -u _json_key --password-stdin https://us-central1-docker.pkg.dev < /workspace/key.json
  secretEnv: ['DOCKER_PASSWORD']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$SHORT_SHA']
availableSecrets:
  secretManager:
  - versionName: projects/$PROJECT_ID/secrets/docker-password/versions/latest
    env: 'DOCKER_PASSWORD'

Monitoring and Auditing CSR Activity

CSR logs all activity to Cloud Audit Logs. You can view them in the Logs Explorer with the filter protoPayload.serviceName="source.googleapis.com". This includes reads, writes, and IAM changes. In production, set up log-based metrics and alerts. For example, alert on protoPayload.methodName="google.devtools.source.v1.SourceRepoWriterService.Write" to detect unusual push activity. Also, export logs to BigQuery for long-term analysis. One common use case: track who pushed to a protected branch. Use the protoPayload.authenticationInfo.principalEmail field. For compliance, enable 'Data Access audit logs' for all CSR services. Be aware that audit logs can be expensive—filter to only log 'Admin Read' and 'Data Access' for sensitive repos.

# Query audit logs for pushes to main branch
gcloud logging read 'protoPayload.serviceName="source.googleapis.com" AND protoPayload.methodName="google.devtools.source.v1.SourceRepoWriterService.Write" AND protoPayload.resource.labels.repo_id="my-app"' \
  --project=my-project \
  --format=json | jq '.[].protoPayload.authenticationInfo.principalEmail'

Pub/Sub Notifications for Repository Events

CSR can publish notifications to Pub/Sub when repository events occur: repo creation, deletion, or pushes. This enables event-driven automation. For example, you can trigger a Slack notification when someone pushes to the main branch, or kick off a security scan when a new repo is created. To set it up, create a Pub/Sub topic and associate it with a project or specific repo using gcloud source repos update --add-topic. The topic receives messages in JSON or Protocol Buffers format. Subscribers can be Cloud Functions, Cloud Run, or external systems. In production, use Pub/Sub notifications to enforce compliance: log all pushes to an audit table, trigger a Cloud Function that checks for secrets in the push, or notify the security team when a repo is deleted. One caveat: Pub/Sub messages are at-least-once, so your subscriber must be idempotent to handle duplicates.

# Create a Pub/Sub topic for repo events
gcloud pubsub topics create csr-events

# Associate the topic with a project (all repos in the project)
gcloud source project-topic update \
    --topic=csr-events \
    --service-account=csr-notifier@my-project.iam.gserviceaccount.com \
    --message-format=json \
    --project=my-project

# Or associate with a specific repo
gcloud source repos update my-app \
    --add-topic=csr-events \
    --service-account=csr-notifier@my-project.iam.gserviceaccount.com \
    --message-format=protobuf \
    --project=my-project

# Create a subscription to log events
gcloud pubsub subscriptions create csr-audit-log \
    --topic=csr-events \
    --push-endpoint=https://logging.googleapis.com/v2/entries:write

# Verify
gcloud pubsub topics list-subscriptions csr-events

Security Key Detection: Blocking Secrets at the Push Level

CSR includes a built-in security key detection feature that scans git push transactions for sensitive information like service account keys, API tokens, and passwords. When enabled, it blocks the push and returns an error to the developer, preventing secrets from ever entering the repository history. This is a critical safety net—even with training, developers accidentally commit secrets. To enable it, go to 'Source Repositories > Settings > Security > Detect security keys' and toggle it on. The feature uses pattern matching to identify common credential formats (e.g., Google service account JSON keys, AWS access keys, GitHub tokens). It does not inspect existing history—only new pushes. For existing repos, you must use a separate tool like git filter-branch or BFG Repo-Cleaner to scrub history. In production, combine security key detection with pre-commit hooks and Secret Manager to create a defense-in-depth approach to secret management.

# Enable security key detection via gcloud (beta)
gcloud alpha source repos update my-app \
    --enable-security-key-detection \
    --project=my-project

# Verify
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    https://sourcerepo.googleapis.com/v1/projects/my-project/repos/my-app

# Developer experience: push with a secret
# git push origin main
# remote: ERROR: Push rejected. Security key detected.
# remote: The push contains a private key. Remove it and try again.

Code Review Tools and Branch Protection in CSR

CSR includes basic code review capabilities: you can comment on commits and changes, add approvals, and track review discussions. However, it lacks the rich branch protection UI of GitHub or GitLab. To enforce branch protections, you must combine CSR with Cloud Build triggers. For example, set up a trigger that runs on pull requests targeting main and requires tests to pass. Use Cloud Build's 'Approval' feature to add a manual review gate before deployment. You can also use a Cloud Function that listens to Pub/Sub push events and enforces policy (e.g., require at least one approval). For teams needing advanced code review, consider using Cloud Build with a third-party review tool or keeping GitHub for pull requests while using CSR mirrors for build triggers. The key trade-off: CSR's native code review is simpler but less powerful than dedicated platforms. If your team needs inline code discussions, required reviewers, or merge queues, augment CSR with an external tool.

# Set up branch protection via Cloud Build
# Create a trigger that runs on PRs to main
gcloud builds triggers create cloud-source-repositories \
    --name="pr-check-main" \
    --repo="my-app" \
    --branch-pattern="^main$" \
    --pull-request-pattern="^main$" \
    --build-config="cloudbuild.yaml" \
    --require-approval \
    --project=my-project

# The cloudbuild.yaml should include:
# - lint step (fails if code style issues)
# - test step (fails if tests fail)
# - build step (ensures the app compiles)
# The deployment step requires manual approval