Post-Outage Account Compromises: Unpacking AWS Security Risks and Best Practices

Recent reports highlight a troubling trend: multiple AWS account compromises occurring shortly after a significant AWS outage. This coincidence has led to intense discussion, exploring whether these events are truly unrelated or if there's a more direct link, either through opportunistic exploitation by attackers or, less likely, a deeper systemic issue.

The "Coincidence" Perspective and Opportunistic Attacks

Many participants initially lean towards coincidence, positing that AWS's sheer scale (e.g., us-east-1 alone with 150+ datacenters and millions of accounts) makes isolated incidents probable. However, the timing is undeniably suspicious. A prevailing theory suggests that attackers capitalized on the chaos and heightened state of alert during the outage.

Phishing Campaigns: An outage creates a perfect storm for phishing. Users, anxious about service recovery, are more susceptible to emails purporting to be from AWS, asking them to log in to check status or "recover services." Malicious wrappers around login pages can then easily capture credentials. This tactic preys on human psychology, as even cautious individuals can be vulnerable under pressure.

Exploiting Pre-existing Access: A more sophisticated approach involves attackers who already possess compromised credentials (e.g., leaked API keys, passwords for dormant IAM users). They strategically wait for high-volume, potentially confusing events like a major outage to launch their operations. During such times, abnormal activity might blend in with legitimate troubleshooting, delaying detection. Several accounts detailed similar experiences where old, dormant accounts or leaked keys were used to create resources, request SES quota increases, or establish persistence via new IAM roles/users, sometimes weeks or months after the initial compromise.

Investigating and Mitigating Compromises

When a compromise is suspected, immediate and thorough investigation is crucial. CloudTrail logs are the primary forensic tool.

CloudTrail Forensics Checklist:

• Identify EC2 Spawns: Look for eventSource:ec2.amazonaws.com eventName:RunInstances to find out what created the instances.
• Console Logins: Check eventSource:signin.amazonaws.com eventName:ConsoleLogin for any unauthorized console access, identifying the userIdentity.type.
• STS Activity: Investigate eventSource:sts.amazonaws.com for GetSessionToken (authentication) and AssumeRole (privilege escalation) events.
• IAM Persistence: Look for eventSource:iam.amazonaws.com events like CreateUser, DeleteUser, CreateRole, DeleteRole, AttachRolePolicy, DetachRolePolicy, or CreateAccessKey to identify how attackers established or modified access.
• Resource Modifications: Check for ReplaceIamInstanceProfileAssociation to see if existing EC2s were backdoored.

Enhancing Account Security

Robust security practices are paramount to prevent such incidents:

• Phishing-Resistant MFA: Implement FIDO keys (e.g., YubiKeys) for all console access. While FIDO2 for direct API calls is limited, AWS Identity Center can be configured to require FIDO2 for web login, which then grants temporary credentials for API interaction, addressing a previous limitation.
• Temporary Credentials: Utilize AWS Security Token Service (STS) to generate short-lived credentials for programmatic access. This minimizes the impact if a key is compromised, as its lifespan is limited.
• Strict Access Control: Adhere to the principle of least privilege. Regularly review and remove dormant or unnecessary IAM accounts, roles, and access keys. If an account is old and unused, delete it.
• Secure Login Habits: Always log into cloud provider consoles using bookmarked links or by typing the URL manually. Never click on links in emails, even if they appear legitimate, especially during outages. If an email prompts action, open a new browser tab and navigate to the official portal directly.
• Alerting and Monitoring: Set up alerts for unusual activity, such as requests for quota increases (like SES email limits), creation of new resources in unusual regions, or modifications to critical IAM policies. This can provide early warnings of a compromise.

Other Potential Issues

One intriguing, though less substantiated, thread discussed brief instances of users being logged into other users' accounts during console refreshes during the outage. While some linked this to customer-side CloudFront misconfigurations (e.g., cache key issues causing session collapse hits), such reports highlight the broader potential for unexpected glitches in complex systems during stress. However, this is largely considered a separate class of problem from the credential compromises discussed by the original poster.

Ultimately, while a direct link between an AWS outage and an AWS-side breach remains unproven and unlikely given AWS's security posture, the outage undeniably created an environment ripe for opportunistic attacks. Organizations must reinforce their security hygiene and vigilance, especially during times of perceived chaos.