A tool for collecting AWS Config resources and storing them in Neo4j graph database.
- Multiple collection methods:
- S3: Collect from AWS Config snapshots stored in S3
- Local: Collect from local JSON files
- API: Collect directly from AWS Config API
- Scheduled collection support
- Docker containerized deployment
- Support for AWS Config Aggregator
- Relationship mapping between resources
- Python 3.9+
- Neo4j 5.x
- AWS credentials configured
- Docker and Docker Compose (for containerized deployment)
- Create virtual environment and install dependencies:
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt- Configure AWS credentials:
aws configure- Create
.envfile with required variables:
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_password
BUCKET_NAME=your-config-bucket- Start services using docker-compose:
docker-compose up -d- Initialize Neo4j indexes:
python index.py bolt://localhost:7687 --username neo4j --password your_password- Collect resources:
- From S3:
python main.py --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password your_password s3 your-bucket --prefix config/- From local files:
python main.py --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password your_password local /path/to/config/files- From AWS Config API:
python main.py --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password your_password api --aggregator-name your-aggregator- Schedule daily collection:
python main.py --command schedule --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password your_password --schedule-time "02:00" s3 your-bucketYou can provide AWS credentials in several ways:
- Using AWS profile:
python main.py --profile your-profile-name ...- Using access keys directly:
python main.py \
--aws-access-key-id YOUR_ACCESS_KEY_ID \
--aws-secret-access-key YOUR_SECRET_ACCESS_KEY \
[--aws-session-token YOUR_SESSION_TOKEN] \
...-
Using environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)
-
Using EC2 instance profile or container role (no additional configuration needed)
Creating Aggregators for AWS Config
Just set all config recorder delivery configuration to a single bucket.
And set the bucket permissions, refer to Permissions for the Amazon S3 Bucket for the AWS Config Delivery Channel .
You can add all accounts to the AWS:SourceAccount or use aws:SourceOrgID .
Create an AWS Config configuration recorder using Quick Setup
However, Quick Setup does not support in the follow regions:
Europe (Milan)
Asia Pacific (Hong Kong)
Middle East (Bahrain)
China (Beijing)
China (Ningxia)
AWS GovCloud (US-East)
AWS GovCloud (US-West)
my-awsconfig-stackset walkthrough
main.py: Main entry point and CLI interfacecollect.py: Resource collection implementationsmerge.py: Neo4j data merging logicindex.py: Neo4j index initializationargs.py: Command line argument parsingdocker-compose.yml: Docker services configurationDockerfile: Container build configuration
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.