Amazon ECR: Container Registry and Image Lifecycle

You pushed an image to ECR, deployed it, and three weeks later your production cluster is pulling an outdated version because your CI pipeline cached the wrong tag. That's not a bug—it's a lifecycle policy failure. Amazon ECR is more than a storage bucket for Docker images; it's the gatekeeper of your deployment integrity. Most teams treat it as a black box, only to discover too late that untagged images accumulate costs and stale images rot in the registry. ECR's lifecycle policies and image scanning aren't optional—they're the difference between a clean, auditable pipeline and a chaotic mess. If you're running containers on AWS, you need to understand how ECR works under the hood, how to enforce retention, and how to avoid the common pitfalls that lead to production outages. This isn't a tutorial on pushing your first image; it's a guide to running ECR at scale without shooting yourself in the foot.

Why ECR Exists: The Registry Problem in Production

Container registries are the backbone of any containerized deployment. Without a reliable registry, your CI/CD pipeline is just a fancy way to push broken images to production. Amazon ECR solves three core problems: secure storage, tight IAM integration, and lifecycle management. Unlike Docker Hub, ECR doesn't rate-limit pulls, and it integrates natively with ECS, EKS, and Lambda. In production, you'll hit issues like image pull throttling, stale images consuming storage costs, and permission misconfigurations that expose your registry. ECR's lifecycle policies and cross-account access controls are the tools you need to avoid these pitfalls. If you're running containers on AWS, ECR is the default choice—not because it's the best, but because it's the least painful when you're already in the ecosystem.

aws ecr create-repository \
    --repository-name my-app \
    --image-scanning-configuration scanOnPush=true \
    --region us-east-1

Authentication: Getting Past the Login Wall

Before you can push or pull images, you must authenticate with ECR. The authentication token is valid for 12 hours and is obtained via aws ecr get-login-password. In CI/CD, you need to refresh this token on every pipeline run. A common mistake is hardcoding the token or using long-lived credentials. Instead, use IAM roles for EC2, ECS, or EKS. For cross-account access, attach a policy to the target account's ECR repository allowing the source account's IAM role to push/pull. Never use root credentials. Also, note that the token is scoped to the registry, not per repository. If you're using Docker's credential helper, install amazon-ecr-credential-helper and configure Docker to use it. This avoids manual login steps and works seamlessly with Kubernetes.

aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

Pushing and Pulling Images: The Basics Done Right

Pushing an image to ECR involves tagging it with the full repository URI and then pushing. The URI format is <account-id>.dkr.ecr.<region>.amazonaws.com/<repo-name>:<tag>. Always use semantic tags like v1.2.3 and avoid latest in production—it's ambiguous and can lead to rollback nightmares. For multi-architecture images, use docker buildx to create manifests. Pulling is straightforward, but consider using imagePullSecrets in Kubernetes or task definitions in ECS. In production, you'll want to pin image digests (sha256) instead of tags for immutable deployments. This prevents drift when tags are overwritten. ECR supports immutable tags—enable it on the repository to prevent accidental overwrites.

docker tag my-app:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.0.0
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.0.0

Image Scanning: Catching Vulnerabilities Before Deployment

ECR offers two scanning modes: basic (manual) and enhanced (with Amazon Inspector). Basic scanning checks against Common Vulnerabilities and Exposures (CVEs) on push. Enhanced scanning provides continuous monitoring and integration with Inspector findings. In production, enable scanOnPush on every repository. Then, enforce a policy that blocks deployment if critical vulnerabilities exist. You can do this in your CI/CD pipeline by checking the scan results via AWS CLI or SDK. A common failure mode is ignoring scan results due to false positives. Triage by setting severity thresholds and using base images from trusted sources like Amazon EKS Distro or Alpine. Remember: scanning is not a one-time event—new CVEs are discovered daily. Enhanced scanning re-scans images periodically.

aws ecr describe-image-scan-findings \
    --repository-name my-app \
    --image-id imageTag=v1.0.0 \
    --region us-east-1

Lifecycle Policies: Automating Image Cleanup

Without lifecycle policies, your ECR repository will accumulate thousands of stale images, driving up storage costs and making it hard to find the right image. Lifecycle policies define rules to expire images based on age or count. For example, keep the last 5 tagged images and delete untagged images older than 14 days. You can also apply rules per tag prefix (e.g., keep all prod-* images forever). In production, always have a rule to delete untagged images—they are often left behind by failed builds. Test your policy with the dry-run option before applying. A common mistake is setting too aggressive a policy that deletes images still referenced by running tasks. Always ensure your running tasks use image digests, not tags, to avoid dangling references.

{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Expire untagged images older than 14 days",
            "selection": {
                "tagStatus": "untagged",
                "countType": "sinceImagePushed",
                "countUnit": "days",
                "countNumber": 14
            },
            "action": {
                "type": "expire"
            }
        },
        {
            "rulePriority": 2,
            "description": "Keep only 5 tagged images",
            "selection": {
                "tagStatus": "tagged",
                "tagPrefixList": ["v"],
                "countType": "imageCountMoreThan",
                "countNumber": 5
            },
            "action": {
                "type": "expire"
            }
        }
    ]
}

Cross-Account Access: Sharing Images Across Teams

In multi-account setups, you often need to share container images between accounts (e.g., dev, staging, prod). ECR supports cross-account access via resource-based policies. You attach a policy to the repository in the source account that grants pull (or push) permissions to a role in the target account. The target account then authenticates to its own registry and pulls via the source account's URI. A common pitfall is forgetting to set the permissions on both sides: the source repository policy and the target account's IAM role trust policy. Also, note that cross-account pulls require the source account to have a lifecycle policy that doesn't delete images needed by the target. For production, use a dedicated 'base-images' account that hosts hardened base images, and grant read-only access to all other accounts.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CrossAccountPull",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::TARGET_ACCOUNT_ID:root"
            },
            "Action": [
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer"
            ]
        }
    ]
}

Tag Mutability: Immutable Tags for Deployment Safety

By default, ECR allows tags to be overwritten (mutable). This is dangerous in production because a tag like v1.0.0 could point to different images over time, making rollbacks unreliable. Enable immutable tags at repository creation or update an existing repository. With immutable tags, you cannot push an image with a tag that already exists. This forces you to use unique tags per build (e.g., commit SHA or build number). In CI/CD, generate a unique tag for each build and use it consistently. If you need to update an image, push a new tag. This practice aligns with immutable infrastructure principles. A common objection is that it increases tag proliferation—but lifecycle policies handle cleanup.

aws ecr put-image-tag-mutability \
    --repository-name my-app \
    --image-tag-mutability IMMUTABLE \
    --region us-east-1

Repository Policies: Fine-Grained Access Control

Beyond basic IAM policies, ECR repository policies allow you to control access at the repository level. This is useful when you have multiple teams sharing a registry. You can grant specific actions (e.g., ecr:BatchCheckLayerAvailability, ecr:PutImage) to specific principals. For example, allow the CI/CD role to push images, while allowing the ECS task role to pull. A common mistake is using overly permissive policies like ecr:* for all principals. Instead, define separate policies for push and pull. Also, consider using conditions to restrict access based on source IP or VPC endpoint. In production, combine repository policies with IAM roles and use AWS Organizations SCPs for guardrails.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPush",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/CICDRole"
            },
            "Action": [
                "ecr:InitiateLayerUpload",
                "ecr:UploadLayerPart",
                "ecr:CompleteLayerUpload",
                "ecr:PutImage"
            ]
        },
        {
            "Sid": "AllowPull",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/ECSTaskRole"
            },
            "Action": [
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer"
            ]
        }
    ]
}

Integration with ECS and EKS: Pulling Images in Production

ECR integrates natively with ECS and EKS. For ECS, you specify the image URI in the task definition, and the ECS agent handles authentication using the instance's IAM role. For EKS, you need to create a secret of type docker-registry or use the ECR credential helper with kubelet. The recommended approach for EKS is to use the ECR credential helper via the kubelet image credential provider (available in Kubernetes 1.20+). This avoids storing long-lived secrets. In production, ensure your node IAM role has the necessary ECR permissions (ecr:BatchGetImage, ecr:GetDownloadUrlForLayer). A common failure is when nodes in private subnets cannot reach ECR endpoints—use VPC endpoints or NAT gateways. Also, consider using ECR pull-through cache for frequently pulled images to reduce costs and improve reliability.

{
    "family": "my-app",
    "containerDefinitions": [
        {
            "name": "my-app",
            "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.0.0",
            "memory": 512,
            "cpu": 256,
            "essential": true
        }
    ]
}

Cost Optimization: Avoiding Surprise Bills

ECR charges for storage ($0.10 per GB/month) and data transfer (free within the same region, but cross-region or internet transfers incur costs). To optimize, use lifecycle policies to delete unused images. Also, consider using ECR pull-through cache for images from public registries (e.g., Docker Hub) to avoid pull rate limits and reduce transfer costs. Another tip: use smaller base images (Alpine, distroless) to reduce storage and transfer. Monitor your ECR usage with Cost Explorer and set billing alerts. A common surprise is the cost of storing many large images—especially if you have multiple repositories per microservice. Consolidate repositories or use image tags wisely. Finally, consider using ECR's 'scan on push' only for production repositories to save on scanning costs.

aws ecr describe-repositories --region us-east-1 --query 'repositories[*].[repositoryName, imageTagMutability, createdAt]' --output table

Disaster Recovery: Backing Up Your Registry

ECR is regional. If a region goes down, you lose access to your images. For disaster recovery, replicate images to another region using cross-region replication (available via ECR replication rules). You can replicate all repositories or filter by prefix. Replication is asynchronous and happens automatically for new pushes. For existing images, you need to manually copy them using a script. In production, set up replication to a secondary region and test failover regularly. Another approach is to use a tool like skopeo to copy images to an S3 bucket or another registry. Remember that replication incurs cross-region data transfer costs. Also, ensure your CI/CD pipeline can push to both regions simultaneously or have a failover mechanism.

{
    "rules": [
        {
            "destination": {
                "region": "eu-west-1",
                "registryId": "TARGET_ACCOUNT_ID"
            },
            "repositoryFilters": [
                {
                    "filter": "my-app",
                    "filterType": "PREFIX_MATCH"
                }
            ]
        }
    ]
}

Security Best Practices: Hardening Your Registry

ECR security involves multiple layers: IAM, repository policies, encryption, scanning, and network controls. Always enable encryption at rest (default AES-256 or KMS). Use VPC endpoints to keep traffic within AWS network. Restrict public access—never set a repository policy to "Principal": "*" unless absolutely necessary. Enable image scanning and integrate with AWS Security Hub for centralized findings. Use IAM conditions to enforce MFA for push operations. Regularly audit repository policies using IAM Access Analyzer. A common vulnerability is leaving repositories with mutable tags and no scanning, allowing a compromised CI pipeline to push a malicious image. Implement a four-eye principle: require two approvals for pushes to production repositories.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyPublicAccess",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "ecr:*",
            "Condition": {
                "StringNotEquals": {
                    "aws:PrincipalAccount": "123456789012"
                }
            }
        }
    ]
}