Production-ready Python automation scripts for AWS infrastructure management using Boto3. Includes EC2 snapshot management with retention policies and S3 lifecycle optimization with cost reporting.
- Automatically creates EBS snapshots for tagged EC2 instances
- Applies descriptive tags (Name, CreatedBy, RetentionDays, SnapshotDate)
- Deletes snapshots older than configurable retention period (default: 7 days)
- Runs as AWS Lambda function or local cron job
- Handles errors gracefully with comprehensive logging
- Identifies objects older than N days across S3 prefixes
- Transitions cold data to Glacier for cost savings
- Generates cost analysis reports with storage class distribution
- Applies automated lifecycle policies (STANDARD → IA → Glacier → Expiration)
- Batch deletes with confirmation safety
- Language: Python 3.9+
- SDK: Boto3 (AWS SDK)
- Services: EC2, EBS, S3, CloudWatch Logs
- Deployment: AWS Lambda, cron, or local CLI
┌─────────────────────────────────────────────────────────────────────┐
│ python-aws-automation │
│ │
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ EC2 Snapshot Manager │ │ S3 Lifecycle Manager │ │
│ │ │ │ │ │
│ │ • Tag-based discovery │ │ • Age-based filtering │ │
│ │ • EBS snapshot creation│ │ • Storage class trans. │ │
│ │ • Auto-tagging │ │ • Cost reporting │ │
│ │ • Retention cleanup │ │ • Lifecycle policies │ │
│ │ • Lambda + CLI support │ │ • Batch delete │ │
│ └────────────┬────────────┘ └────────────┬────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ AWS EC2 │ │ AWS S3 │ │
│ │ EBS Snapshots│ │ + Lifecycle │ │
│ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
pip install -r requirements.txt
aws configure # Ensure AWS credentials are setcd python-aws-automation
# Run locally — backs up all instances tagged Backup=true
python ec2_snapshot.py --region us-east-1 --retention 7
# Run for specific tag
python ec2_snapshot.py --tag-key Environment --tag-value production --retention 14AWS Lambda Deployment:
# Set handler to: ec2_snapshot.lambda_handler
# Trigger: CloudWatch Events (cron) or EventBridge
# IAM Role needs: ec2:DescribeInstances, ec2:CreateSnapshot, ec2:DeleteSnapshot, ec2:DescribeSnapshots# Generate cost report
python s3_cleanup.py --bucket my-data-bucket --action report
# List old objects (90 days)
python s3_cleanup.py --bucket my-data-bucket --prefix logs/ --days 90 --action report
# Transition old objects to Glacier
python s3_cleanup.py --bucket my-data-bucket --prefix archives/ --days 180 --action transition
# Apply automated lifecycle policy
python s3_cleanup.py --bucket my-data-bucket --prefix uploads/ --action lifecycle
# Delete old objects (requires confirmation)
python s3_cleanup.py --bucket my-data-bucket --prefix temp/ --days 7 --action delete| Pattern | Implementation |
|---|---|
| Idempotency | Snapshots tagged with timestamp; no duplicates on same run |
| Safety | Delete operations require confirmation; snapshots only delete after retention period |
| Observability | Structured logging with timestamps and severity levels |
| Configurability | All parameters via CLI args or Lambda event payload |
| Error Handling | Try/except blocks with graceful degradation |
{
"Statement": [
{"Effect": "Allow", "Action": ["ec2:DescribeInstances", "ec2:DescribeVolumes", "ec2:DescribeSnapshots"], "Resource": "*"},
{"Effect": "Allow", "Action": ["ec2:CreateSnapshot", "ec2:DeleteSnapshot"], "Resource": "*"},
{"Effect": "Allow", "Action": ["ec2:CreateTags"], "Resource": "*"}
]
}{
"Statement": [
{"Effect": "Allow", "Action": ["s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject"], "Resource": ["arn:aws:s3:::bucket-name", "arn:aws:s3:::bucket-name/*"]},
{"Effect": "Allow", "Action": ["s3:PutLifecycleConfiguration"], "Resource": "arn:aws:s3:::bucket-name"}
]
}| File | Purpose |
|---|---|
ec2_snapshot.py |
EBS snapshot automation with retention |
s3_cleanup.py |
S3 object lifecycle and cost optimization |
requirements.txt |
Python dependencies |
README.md |
Documentation and usage guide |