In our previous post, we covered Best practices and tips to reduce AWS S3 costs. In this discussion, we delve into yet another critical facet of cloud cost optimization: the identification and management of orphaned EBS snapshots, volumes, and AMIs within your AWS environment.
Automating the discovery of AWS orphaned EBS (Elastic Block Store) volumes, snapshots, and AMIs (Amazon Machine Images) is crucial for any organization looking to optimize its AWS infrastructure, reduce costs, and enhance security. Orphaned resources are no longer associated with running instances or are no longer in use, yet they continue to incur unnecessary charges and pose potential security risks.
The recommended way is to automate finding these orphaned resources and generate a list. Once you have the list, you need to work with the respective owners of these resources to figure out if they can be deleted. Finding the right owners is itself a big challenge!
By implementing an automated solution to identify and manage orphaned EBS volumes, snapshots, and AMIs, you can:
- Cost Optimization: Discovering and removing unused or forgotten resources helps you save on AWS expenses. Orphaned resources often go unnoticed, leading to unnecessary charges over time.
- Security Enhancement: Unused resources can become potential attack vectors if left unattended. Identifying and addressing these orphans ensures a more secure AWS environment.
- Operational Efficiency: Automation streamlines the process of locating orphaned resources. Manual tracking and cleanup can be time-consuming and error-prone, while automation ensures accuracy and saves valuable time.
- Resource Accountability: Maintain a clean and well-organized AWS environment by ensuring every resource has a purpose and is properly managed throughout its lifecycle.
Here is a python code that uses BOTO3 SDK with Panda to generate an excel sheet with a list of Orphaned Snapshots, EBS volumes, and AMIs.
import boto3
import pandas as pd
# Set up Boto3 clients for EC2 and CloudWatch
ec2 = boto3.client('ec2')
cloudwatch = boto3.client('cloudwatch')
# Retrieve list of AWS regions
regions = [region['RegionName'] for region in ec2.describe_regions()['Regions']]
# Create empty lists to store the orphaned EBS snapshots, volumes, and unused AMIs
orphaned_snapshots = []
orphaned_volumes = []
unused_amis = []
# Iterate over regions and find orphaned EBS snapshots, volumes, and unused AMIs
for region in regions:
# Set up Boto3 client for the region
ec2_region = boto3.client('ec2', region_name=region)
# Find all snapshots that are not associated with a running instance
snapshots = ec2_region.describe_snapshots(OwnerIds=['self'])['Snapshots']
for snapshot in snapshots:
if 'in-use' not in snapshot['State']:
orphaned_snapshots.append([region, snapshot['SnapshotId'], snapshot['StartTime']])
# Find all volumes that are not attached to a running instance
volumes = ec2_region.describe_volumes()['Volumes']
for volume in volumes:
if volume['State'] == 'available':
orphaned_volumes.append([region, volume['VolumeId'], volume['CreateTime']])
# Find all AMIs that have not been launched in the last 90 days
amis = ec2_region.describe_images(Owners=['self'])['Images']
for ami in amis:
metrics = cloudwatch.get_metric_statistics(
Namespace='AWS/EC2',
MetricName='StatusCheckFailed_Instance',
Dimensions=[{'Name': 'ImageId', 'Value': ami['ImageId']}],
StartTime=pd.Timestamp.utcnow() - pd.Timedelta(days=90),
EndTime=pd.Timestamp.utcnow(),
Period=86400,
Statistics=['Sum']
)['Datapoints']
if len(metrics) == 0 or metrics[0]['Sum'] == 0:
unused_amis.append([region, ami['ImageId'], ami['CreationDate']])
# Convert the lists into Pandas DataFrames
orphaned_snapshots_df = pd.DataFrame(orphaned_snapshots, columns=['Region', 'SnapshotId', 'StartTime'])
orphaned_volumes_df = pd.DataFrame(orphaned_volumes, columns=['Region', 'VolumeId', 'CreateTime'])
unused_amis_df = pd.DataFrame(unused_amis, columns=['Region', 'ImageId', 'CreationDate'])
# Write the DataFrames to an Excel file
writer = pd.ExcelWriter('aws_orphans.xlsx', engine='xlsxwriter')
orphaned_snapshots_df.to_excel(writer, sheet_name='Orphaned Snapshots', index=False)
orphaned_volumes_df.to_excel(writer, sheet_name='Orphaned Volumes', index=False)
unused_amis_df.to_excel(writer, sheet_name='Unused AMIs', index=False)
writer.close()
# Print a message to indicate that the Excel file has been written
print("Orphaned EBS snapshots, volumes, and unused AMIs written to aws_orphans.xlsx")
We recommend making a Lambda function and adding this code to the function. Trigger the lambda function via CloudWatch alarm every 2 weeks or once a month.
This can be run within a container, EC2 instance, or from a remote laptop as well, Lambda adds a layer of ease with Amazon Cloudwatch in this solution.
Enjoy!
