Amazon EFS: Elastic File System for Linux Workloads

You've provisioned a fleet of EC2 instances, each with its own EBS volume. Now you need them to share files. You could build a custom NFS server, manage its capacity, and pray it doesn't become a bottleneck. Or you could use Amazon EFS and let AWS handle the scaling. EFS is a fully managed NFS file system that grows and shrinks automatically. No provisioning, no capacity planning, no patching. But it's not magic—there are trade-offs. EFS is great for shared storage across multiple instances, but it's not a drop-in replacement for local or block storage. Latency is higher than EBS, and throughput depends on your burst credits. If you're running a stateless app, you probably don't need EFS. But if you have a legacy app that requires shared file access, or you're building a content management system where multiple servers need to read and write the same files, EFS is your friend. Just don't expect it to perform like local SSD.

What Is Amazon EFS and Why Use It?

Amazon Elastic File System (EFS) is a fully managed, scalable, and elastic NFS file system for use with AWS cloud services and on-premises resources. It is designed for Linux workloads that require a shared file system accessible from multiple EC2 instances, containers, or on-premises servers via AWS Direct Connect or VPN. EFS automatically scales storage capacity up and down as you add or remove files, with no provisioning or capacity planning required. It supports two performance modes: General Purpose (for latency-sensitive applications) and Max I/O (for high-throughput, parallel workloads). EFS also offers two throughput modes: Bursting (baseline throughput that bursts) and Provisioned (fixed throughput for predictable performance). For cost optimization, EFS provides Standard and Infrequent Access storage classes, with lifecycle policies to automatically move files between them. Use EFS when you need a POSIX-compliant, concurrent file system for web serving, content management, media workflows, or container storage. Avoid EFS for Windows workloads (use FSx instead) or for workloads requiring low-latency random access (use EBS).

#!/bin/bash
# Create EFS file system and mount target in VPC
aws efs create-file-system \
    --creation-token my-efs-$(date +%s) \
    --performance-mode generalPurpose \
    --throughput-mode bursting \
    --tags Key=Name,Value=MyEFS

# Get FileSystemId from output, then create mount target
FS_ID=$(aws efs describe-file-systems --query "FileSystems[?Name=='MyEFS'].FileSystemId" --output text)
aws efs create-mount-target \
    --file-system-id $FS_ID \
    --subnet-id subnet-12345678 \
    --security-groups sg-12345678

EFS Performance Modes: General Purpose vs. Max I/O

EFS offers two performance modes that cannot be changed after creation. General Purpose mode is the default and suitable for most workloads, providing low latency (single-digit milliseconds) for file operations. It is ideal for web serving, content management, and home directories. Max I/O mode is designed for high-throughput, parallel workloads like big data analytics, media processing, and genomics. It can scale to higher levels of aggregate throughput and IOPS, but with higher latency (tens of milliseconds). Choose General Purpose if your application is sensitive to latency or has many small file operations. Choose Max I/O if you need massive parallel throughput and can tolerate higher latency. For throughput, EFS offers Bursting mode (baseline throughput based on storage size, with burst credits) and Provisioned mode (fixed throughput regardless of storage). Bursting is cost-effective for variable workloads; Provisioned is for predictable, high-throughput needs. Monitor burst credit balance with CloudWatch metrics to avoid throttling.

#!/bin/bash
# Check current performance mode and throughput mode
aws efs describe-file-systems --file-system-id fs-12345678 --query "FileSystems[0].{PerformanceMode: PerformanceMode, ThroughputMode: ThroughputMode}"

# Monitor burst credits
aws cloudwatch get-metric-statistics \
    --namespace AWS/EFS \
    --metric-name BurstCreditBalance \
    --dimensions Name=FileSystemId,Value=fs-12345678 \
    --start-time $(date -u -d '-1 hour' +%Y-%m-%dT%H:%M:%SZ) \
    --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
    --period 300 \
    --statistics Average

Mounting EFS on EC2: NFS Client Setup

To mount an EFS file system on an EC2 Linux instance, you need the NFS client (nfs-utils or nfs-common) installed. The recommended method is using the EFS mount helper (amazon-efs-utils), which simplifies mounting with TLS encryption and IAM authorization. Install the helper via yum or apt, then mount using the file system ID. For production, always mount with the 'tls' option to encrypt data in transit. Use the 'iam' option if you want to authorize mount via IAM roles. The mount command syntax: 'mount -t efs -o tls fs-12345678:/ /mnt/efs'. For automatic mounting on reboot, add an entry to /etc/fstab using the EFS mount helper syntax. Ensure security group rules allow inbound NFS traffic (port 2049) from the EC2 instance's security group. For high availability, create mount targets in multiple Availability Zones and mount from the same region. Avoid mounting from a different region due to latency and cost.

#!/bin/bash
# Install EFS mount helper on Amazon Linux 2
sudo yum install -y amazon-efs-utils

# Create mount point
sudo mkdir -p /mnt/efs

# Mount EFS with TLS encryption
sudo mount -t efs -o tls fs-12345678:/ /mnt/efs

# Verify mount
df -h /mnt/efs

# Add to /etc/fstab for auto-mount
echo "fs-12345678:/ /mnt/efs efs tls,_netdev 0 0" | sudo tee -a /etc/fstab

EFS Security: IAM Authorization and Access Points

EFS integrates with AWS IAM to control mount permissions and file system operations. You can create IAM policies that allow or deny specific actions like 'elasticfilesystem:ClientMount' and 'elasticfilesystem:ClientWrite'. For fine-grained access, use EFS access points, which enforce a specific POSIX user and directory path per application. Access points can also restrict root access and set ownership. Combine access points with IAM policies to enforce least privilege. For example, an IAM policy can allow mounting only via a specific access point. Always enable encryption at rest (default) and in transit (using TLS mount option). For compliance, use AWS KMS customer managed keys instead of AWS managed keys. Monitor file system activity with AWS CloudTrail and Amazon GuardDuty (for threat detection). Avoid using root squash unless necessary; access points provide better control.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "elasticfilesystem:ClientMount",
                "elasticfilesystem:ClientWrite"
            ],
            "Resource": "arn:aws:elasticfilesystem:us-east-1:123456789012:access-point/fsap-12345678",
            "Condition": {
                "StringEquals": {
                    "elasticfilesystem:AccessPointId": "fsap-12345678"
                }
            }
        }
    ]
}

Lifecycle Management: Moving Cold Data to EFS IA

EFS Lifecycle Management automatically transitions files to the Infrequent Access (IA) storage class after a specified number of days (30, 60, or 90) of not being accessed. IA storage costs less per GB but has a per-GB read/write fee. This is ideal for backups, logs, and archives that are rarely accessed. You can set a lifecycle policy on the file system to move files to IA after N days. Files accessed after transition are automatically moved back to Standard (if you enable 'restore' mode) or remain in IA with read costs. For cost savings, combine lifecycle policies with EFS replication for cross-region disaster recovery. Monitor storage distribution with CloudWatch metrics (StorageBytes for Standard and IA). Avoid lifecycle policies for workloads with frequent random access; the per-GB fees can outweigh savings.

#!/bin/bash
# Set lifecycle policy to move files to IA after 30 days
aws efs put-lifecycle-configuration \
    --file-system-id fs-12345678 \
    --lifecycle-policies "[{\"TransitionToIA\": \"AFTER_30_DAYS\"}]"

# Verify policy
aws efs describe-lifecycle-configuration --file-system-id fs-12345678

Backup and Disaster Recovery with EFS

EFS supports two backup strategies: AWS Backup (managed) and EFS-to-EFS replication (cross-region or cross-account). AWS Backup provides centralized backup policies, retention management, and point-in-time recovery. It creates snapshots of the file system, which can be restored to a new EFS file system. EFS Replication continuously replicates files to a destination file system in another AWS Region or account, with a Recovery Point Objective (RPO) of minutes. Replication is one-way and can be enabled at file system creation or later. For disaster recovery, use replication to a secondary region and fail over by updating DNS or mount targets. For backup compliance, use AWS Backup with lifecycle rules to expire old backups. Always test restore procedures regularly. Avoid relying solely on replication; backups provide protection against accidental deletions or corruption.

#!/bin/bash
# Enable cross-region replication (source in us-east-1, destination in us-west-2)
aws efs create-replication-configuration \
    --source-file-system-id fs-12345678 \
    --destinations "[{\"Region\": \"us-west-2\"}]"

# Check replication status
aws efs describe-replication-configurations --file-system-id fs-12345678

Monitoring EFS with CloudWatch and Metrics

EFS publishes metrics to CloudWatch every minute, including PercentIOLimit, BurstCreditBalance, StorageBytes, and Throughput. Monitor PercentIOLimit to see if you're approaching the General Purpose IOPS limit (35,000 per file system). If consistently high, consider switching to Max I/O or distributing load across multiple file systems. BurstCreditBalance indicates how many burst credits remain; if it drops to zero, throughput is throttled to baseline. Set CloudWatch alarms for BurstCreditBalance < 100 MB to proactively scale throughput mode. StorageBytes tracks total storage used, broken down by Standard and IA. Use CloudWatch Logs for audit trails via CloudTrail. For operational dashboards, use CloudWatch Dashboard or Grafana. Avoid relying solely on basic monitoring; enable detailed monitoring for faster alerting.

#!/bin/bash
# Create CloudWatch alarm for low burst credit balance
aws cloudwatch put-metric-alarm \
    --alarm-name efs-low-burst-credits \
    --alarm-description "Alert when burst credit balance < 100 MB" \
    --metric-name BurstCreditBalance \
    --namespace AWS/EFS \
    --statistic Average \
    --period 300 \
    --evaluation-periods 2 \
    --threshold 100 \
    --comparison-operator LessThanThreshold \
    --dimensions Name=FileSystemId,Value=fs-12345678 \
    --alarm-actions arn:aws:sns:us-east-1:123456789012:my-topic

Cost Optimization: EFS Standard vs. IA and Throughput Modes

EFS costs consist of storage fees (per GB-month) and throughput fees (only for Provisioned mode). Standard storage costs more per GB than IA, but IA has per-GB access charges. Use lifecycle policies to move cold data to IA. For throughput, Bursting mode is free (included in storage cost) but limited by burst credits. Provisioned mode charges per MBps provisioned, regardless of usage. Choose Bursting for variable workloads with low average throughput. Choose Provisioned for steady high throughput to avoid credit exhaustion. For cost savings, consider using EFS One Zone (in a single AZ) for non-critical data, which is 47% cheaper than Standard. However, One Zone is not resilient to AZ failures. Use Reserved Throughput (1-year or 3-year) for predictable throughput to save up to 30%. Monitor costs with AWS Cost Explorer and tag file systems for chargeback.

#!/bin/bash
# Estimate monthly cost for 1 TB Standard storage with Bursting
# Storage: 1000 GB * $0.30/GB = $300
# Throughput: included
# Total: ~$300/month

# For Provisioned Throughput (100 MBps): add $6/MBps * 100 = $600
# Total: ~$900/month

echo "Standard 1 TB Bursting: ~$300/month"
echo "Standard 1 TB Provisioned 100 MBps: ~$900/month"

Common EFS Pitfalls and How to Avoid Them

1. Performance mode mismatch: Choosing Max I/O for latency-sensitive apps causes high latency. Always use General Purpose for web servers. 2. Security group misconfiguration: Forgetting to allow inbound NFS (port 2049) from the client security group results in mount failures. 3. Burst credit exhaustion: Running out of burst credits throttles throughput. Monitor and switch to Provisioned if needed. 4. Mount target in wrong subnet: Mount targets must be in the same VPC and subnet as the client, or use VPC peering/Transit Gateway. 5. Not using TLS: Unencrypted data in transit can be intercepted. Always use 'tls' mount option. 6. Ignoring lifecycle policies: Paying Standard rates for cold data wastes money. Set lifecycle policies to move to IA. 7. Single-AZ deployment: Using EFS Standard in one AZ defeats its high-availability purpose. Ensure mount targets in multiple AZs. 8. Not testing failover: Replication is useless if you never test failover. Conduct drills. 9. Overlooking IAM permissions: Without proper IAM policies, any instance in the VPC can mount the file system. Use access points and IAM conditions. 10. Using EFS for databases: EFS latency is too high for transactional databases. Use EBS or RDS instead.

#!/bin/bash
# Troubleshoot mount failure
# Check if NFS client is installed
rpm -q nfs-utils || echo "Install nfs-utils"

# Check security group rules (from client)
aws ec2 describe-security-groups --group-ids sg-12345678 --query "SecurityGroups[0].IpPermissions[?FromPort==2049]"

# Test connectivity
nc -zv 10.0.1.123 2049

# Check mount target state
aws efs describe-mount-targets --file-system-id fs-12345678 --query "MountTargets[?LifeCycleState=='available']"

EFS Best Practices for Production Workloads

1. Enable encryption at rest and in transit: Use AWS KMS for at-rest encryption and 'tls' mount option for in-transit. 2. Use access points with IAM: Enforce per-application directories and POSIX users. 3. Set lifecycle policies: Move cold data to IA after 30 days. 4. Monitor burst credits and IOPS: Set CloudWatch alarms for BurstCreditBalance and PercentIOLimit. 5. Use Provisioned Throughput for steady workloads: Avoid burst credit exhaustion. 6. Deploy mount targets in multiple AZs: Ensure high availability. 7. Implement backup and replication: Use AWS Backup for snapshots and EFS Replication for cross-region DR. 8. Tag file systems: For cost allocation and automation. 9. Use EFS One Zone for dev/test: Save costs where resilience isn't critical. 10. Test failover and restore procedures: Regularly validate DR plans. 11. Limit mount targets per VPC: AWS default is 100 per VPC; request increase if needed. 12. Use EFS File Sync for initial data migration: For large datasets, use AWS DataSync or EFS File Sync to seed data before switching workloads.

#!/bin/bash
# Production-ready EFS setup with encryption, access point, and lifecycle policy

# Create file system with encryption
FS_ID=$(aws efs create-file-system \
    --creation-token prod-efs-$(date +%s) \
    --performance-mode generalPurpose \
    --throughput-mode bursting \
    --encrypted \
    --kms-key-id alias/aws/efs \
    --tags Key=Name,Value=ProductionEFS \
    --query 'FileSystemId' --output text)

# Create mount targets in two AZs
for subnet in subnet-1 subnet-2; do
    aws efs create-mount-target --file-system-id $FS_ID --subnet-id $subnet --security-groups sg-efs
done

# Create access point for app1
aws efs create-access-point \
    --file-system-id $FS_ID \
    --posix-user Uid=1000,Gid=1000 \
    --root-directory Path=/app1,CreationInfo='{OwnerUid=1000,OwnerGid=1000,Permissions=0755}'

# Set lifecycle policy
aws efs put-lifecycle-configuration \
    --file-system-id $FS_ID \
    --lifecycle-policies "[{\"TransitionToIA\": \"AFTER_30_DAYS\"}]"