Automated AWS infrastructure utilization reporting for EC2, RDS, and ECS services with customizable thresholds, multi-channel notifications, and enterprise-grade deployment options.
- Features
- Architecture
- Quick Start
- Configuration
- Deployment Options
- Report Sample
- Contributing
- License
| Feature | Description |
|---|---|
| Multi-Service Support | Monitor EC2, RDS, and ECS resources in a single report |
| Disk Metrics (Linux & Windows) | Comprehensive disk usage with GB and percentage values |
| Customizable Thresholds | Configure warning (50%) and critical (70%) levels |
| Flexible Scheduling | Daily, weekly, or monthly report generation |
| Multiple Notification Channels | Email (SES/SMTP) and SNS support |
| Enterprise Deployment | Lambda, ECS/Fargate, or Docker container options |
| Infrastructure as Code | Terraform and CloudFormation templates included |
| CI/CD Ready | GitHub Actions workflows for automated deployment |
┌─────────────────────────────────────────────────────────────────────────┐
│ AWS Utilization Report │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ CloudWatch │ │ CloudWatch │ │ CloudWatch │ │
│ │ EC2 Metrics│ │ RDS Metrics │ │ ECS Metrics │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └────────────────────┼────────────────────┘ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Lambda / ECS / │ │
│ │ Docker │ │
│ └────────┬─────────┘ │
│ │ │
│ ┌──────────────┼──────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ S3 │ │ SES/SMTP │ │ SNS │ │
│ │ Report │ │ Email │ │ Topic │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
- AWS Account with appropriate permissions
- Python 3.9+
- AWS CLI configured
- (Optional) Terraform or AWS SAM CLI for IaC deployment
# Clone the repository
git clone https://github.com/p05-cloud/aws-utilization-report.git
cd aws-utilization-report
# Install dependencies
pip install -r requirements.txt
# Copy and configure settings
cp config/config.example.yaml config/config.yaml
# Edit config/config.yaml with your settings
# Run locally
python scripts/run_local.py# Using AWS SAM
cd infrastructure/cloudformation
sam build
sam deploy --guided
# Or using Terraform
cd infrastructure/terraform
terraform init
terraform plan
terraform applyAll settings are managed via config/config.yaml:
# Report Settings
report:
name: "AWS Utilization Report"
frequency: "weekly" # daily, weekly, monthly
services:
ec2: true
rds: true
ecs: true
# Threshold Settings (percentage)
thresholds:
warning: 50 # Orange highlight
critical: 70 # Red highlight
# AWS Settings
aws:
region: "us-east-1"
s3_bucket: "my-reports-bucket"
s3_prefix: "utilization-reports/"
# Notification Settings
notifications:
email:
enabled: true
smtp_host: "email-smtp.us-east-1.amazonaws.com"
smtp_port: 587
sender: "reports@example.com"
recipients:
- "team@example.com"
# For SES, store credentials in AWS Secrets Manager
secrets_manager_key: "utilization-report-smtp"
sns:
enabled: false
topic_arn: "arn:aws:sns:us-east-1:123456789012:utilization-alerts"For Lambda/Container deployments, you can override config with environment variables:
| Variable | Description | Default |
|---|---|---|
CONFIG_PATH |
Path to config file | config/config.yaml |
AWS_REGION |
AWS region | us-east-1 |
S3_BUCKET |
S3 bucket for reports | Required |
REPORT_FREQUENCY |
daily/weekly/monthly | weekly |
THRESHOLD_WARNING |
Warning threshold % | 50 |
THRESHOLD_CRITICAL |
Critical threshold % | 70 |
ENABLE_EMAIL |
Enable email notifications | true |
ENABLE_SNS |
Enable SNS notifications | false |
SNS_TOPIC_ARN |
SNS topic ARN | - |
Using Terraform:
cd infrastructure/terraform
terraform init
terraform apply -var="s3_bucket=my-bucket" -var="email_recipients=[\"team@example.com\"]"Using CloudFormation:
cd infrastructure/cloudformation
aws cloudformation deploy \
--template-file template.yaml \
--stack-name utilization-report \
--parameter-overrides S3Bucket=my-bucket \
--capabilities CAPABILITY_IAM# Build the image
docker build -t aws-utilization-report:latest ./docker
# Run locally
docker run -v ~/.aws:/root/.aws \
-e S3_BUCKET=my-bucket \
-e AWS_REGION=us-east-1 \
aws-utilization-report:latest
# Push to ECR
aws ecr get-login-password | docker login --username AWS --password-stdin <account>.dkr.ecr.<region>.amazonaws.com
docker tag aws-utilization-report:latest <account>.dkr.ecr.<region>.amazonaws.com/aws-utilization-report:latest
docker push <account>.dkr.ecr.<region>.amazonaws.com/aws-utilization-report:latestSee docs/ecs-deployment.md for detailed ECS/Fargate deployment instructions.
The generated Excel report includes three sheets:
| Application Name | Instance ID | Private IP | State | CPU Avg (%) | Memory Avg (%) | Volume ID | Disk Size (GB) | Disk Used (GB) | Disk Used (%) | Disk Free (GB) | Disk Free (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Web Server | i-0abc123 | 10.0.1.10 | running | 45.2 | 62.1 | vol-xyz | 100 | 65.5 | 65.5 | 34.5 | 34.5 |
| DB Identifier | Endpoint | Allocated Storage (GB) | CPU Avg (%) | Free Memory (GB) | Storage Used (GB) | Storage Free (GB) |
|---|---|---|---|---|---|---|
| prod-db | prod-db.xxx.rds.amazonaws.com | 500 | 32.1 | 8.5 | 320.0 | 180.0 |
| Service Name | CPU Max (%) | Memory Utilization (%) | Min Tasks | Max Tasks | Running Tasks |
|---|---|---|---|---|---|
| api-service | 78.5 | 65.2 | 2 | 10 | 4 |
Color Coding:
- 🟠 Orange: Warning threshold (default: ≥50%)
- 🔴 Red: Critical threshold (default: ≥70%)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeVolumes",
"rds:DescribeDBInstances",
"ecs:ListClusters",
"ecs:ListServices",
"ecs:DescribeServices",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics",
"application-autoscaling:DescribeScalableTargets",
"s3:PutObject",
"s3:GetObject",
"secretsmanager:GetSecretValue",
"ses:SendRawEmail",
"sns:Publish"
],
"Resource": "*"
}
]
}# Install dev dependencies
pip install -r requirements-dev.txt
# Run tests
pytest tests/ -v
# Run with coverage
pytest tests/ --cov=src --cov-report=htmlaws-utilization-report/
├── README.md # This file
├── LICENSE # MIT License
├── CONTRIBUTING.md # Contribution guidelines
├── CHANGELOG.md # Version history
├── requirements.txt # Python dependencies
├── requirements-dev.txt # Dev dependencies
├── .gitignore # Git ignore rules
│
├── src/ # Source code
│ ├── lambda_function.py # Main Lambda handler
│ ├── metrics/ # Metric collection modules
│ ├── notifications/ # Notification handlers
│ └── utils/ # Utility functions
│
├── config/ # Configuration
│ └── config.example.yaml # Example configuration
│
├── scripts/ # Utility scripts
│ └── run_local.py # Local execution script
│
├── docker/ # Docker files
│ ├── Dockerfile # Container definition
│ └── docker-compose.yaml # Local Docker setup
│
├── infrastructure/ # IaC templates
│ ├── terraform/ # Terraform modules
│ └── cloudformation/ # SAM/CloudFormation
│
├── tests/ # Test suite
│ └── test_metrics.py # Unit tests
│
├── docs/ # Documentation
│ ├── architecture.md # Architecture details
│ ├── setup-guide.md # Detailed setup guide
│ └── images/ # Documentation images
│
└── .github/ # GitHub specific
└── workflows/ # CI/CD pipelines
├── ci.yml # Continuous Integration
└── deploy.yml # Deployment workflow
Contributions are welcome! Please read CONTRIBUTING.md for details on our code of conduct and the process for submitting pull requests.
This project is licensed under the MIT License - see the LICENSE file for details.
p05-cloud
- GitHub: @p05-cloud
- Email: pushpakpatil.cloud@gmail.com
Give a ⭐️ if this project helped you!
See CHANGELOG.md for version history.