Real security incidents and lessons learned

Authentication and authorization bugs are among the most common vulnerabilities in production. Here are real incidents from real companies, what went wrong, and what they should have done.

Advanced

10 min read

Incident 1: Equifax data breach (2017)

What happened: Attackers exploited a known vulnerability (CVE-2017-5638) in Apache Struts to access Equifax's database, exfiltrating 147 million people's personal information including SSNs, birthdates, and credit card numbers.

What went wrong:

Equifax knew about the vulnerability but didn't patch it for weeks.
No authentication on critical admin endpoints.
Insufficient monitoring: attackers accessed the system for months undetected.

Lessons:

Patch quickly: vulnerabilities in authentication/authorization code are critical.
Use strong authentication on all admin endpoints: not just some.
Monitor for unusual access patterns: failed logins, large data exports, unusual IPs.
Never store PII in plaintext: encrypt sensitive data at rest.

Incident 2: Facebook's plaintext password storage (2019)

What happened: Facebook had been storing between 200 million and 600 million account passwords in plaintext internally for years, discovered during a routine security review in January 2019 and disclosed publicly in March 2019 after security journalist Brian Krebs reported it. The passwords were searchable by more than 20,000 Facebook employees; access logs showed around 2,000 engineers had made roughly 9 million internal queries that touched plaintext passwords.

What went wrong:

Passwords were logged in plaintext by some internal systems, instead of only ever existing as a hash.
Far too many employees had access to systems holding that plaintext data.
The practice went unnoticed internally for years.

Lessons:

Never store passwords in plaintext, anywhere — not the primary database, and not incidentally in logs or debug output either (the exact discipline the password-hashing lesson covers).
Sanitize logs: redact credentials and tokens, replace with [REDACTED].
Minimize access: most employees never need to touch raw credential data at all.
Audit regularly: this was only caught by a routine internal security review, not an external report.

Incident 3: Reddit's AiTM phishing attack (2023)

What happened: On February 5, 2023, Reddit employees were targeted by a sophisticated phishing campaign. Attackers stood up a fake site impersonating Reddit's internal intranet portal and lured an employee into entering their credentials and MFA code there. Because the fake site was an adversary-in-the-middle (AiTM) proxy sitting between the employee and Reddit's real login system, it relayed the stolen credentials and MFA token to the real system in real time, before the token expired — hijacking the resulting authenticated session. One employee was compromised; the attacker accessed some internal docs, code, and internal dashboards before the employee self-reported and Reddit's security team cut off access.

What went wrong:

Standard MFA (TOTP codes) doesn't stop AiTM phishing — the code is valid for a short window, and a fast automated relay can use it before it expires, from a device the real user never controls.
The phishing site closely impersonated a real internal portal, exploiting employee trust in familiar-looking login pages.

Lessons:

TOTP-based MFA has a real blind spot: phishing-resistant MFA (hardware security keys using WebAuthn/FIDO2, covered in the MFA lesson) can't be relayed this way, because the cryptographic proof is bound to the real site's origin.
Security training matters: the employee noticing something was wrong and self-reporting quickly is what limited the damage here.
Fast incident response: Reddit's team cut off access as soon as they were told — the gap between compromise and containment is what determines how bad an incident gets.
Session monitoring: watching for a session's activity suddenly looking different (new device, new location, unusual actions) can catch a hijacked session even after MFA was bypassed.

Incident 4: LastPass's home-computer breach (2022)

What happened: An attacker compromised a LastPass DevOps engineer's personal, home computer by exploiting a vulnerability in an unrelated third-party media-player package, gaining remote code execution and planting a keylogger. The keylogger captured the engineer's master password as it was typed — after the engineer had already authenticated with MFA — giving the attacker access to that engineer's corporate password vault, which held further decryption keys. The attacker used that access to exfiltrate a backup database and customer vault data (some fields, like site URLs, unencrypted; others, like actual passwords, still encrypted).

What went wrong:

A single employee's compromised personal device was enough to reach production credential infrastructure — corporate access wasn't isolated from what happened on that employee's own hardware.
MFA protected the login step, but a keylogger capturing the master password after login defeated it entirely for anything relying on that captured secret afterward.
Sensitive infrastructure access was reachable through a chain that started on an unmanaged personal device.

Lessons:

MFA protects login, not every secret typed afterward — a compromised endpoint can still capture what you type, regardless of how you authenticated.
Separate personal and corporate access: sensitive systems shouldn't be reachable from unmanaged personal devices at all, ideally enforced technically, not just by policy.
Encrypt everything, including "harmless" metadata: the unencrypted fields (like URLs) in the stolen backups still gave attackers useful information for follow-on attacks.
A supply-chain vulnerability in unrelated software was the actual entry point — the real risk often isn't in your own code at all.

Incident 5: Uber's MFA fatigue breach (2022)

What happened: In September 2022, an attacker (later attributed to a Lapsus$-affiliated hacker) obtained an Uber contractor's password, likely purchased from the dark web after an unrelated earlier breach. The password alone wasn't enough — Uber's MFA sent a push notification on every login attempt — so the attacker sent about 40 push notifications in 30 minutes, then contacted the contractor directly (posing as Uber IT support) and said approving the next one would make the notifications stop. The contractor, worn down and believing they were talking to real IT staff, approved it — handing the attacker full VPN access to Uber's internal network.

What went wrong:

Push-based MFA has no built-in limit on how many times it can be retried, so it can simply be worn down.
Social engineering targeted the human directly, not any technical control.
A single approved push led straight to broad internal network access, with no additional verification step.

Lessons:

Push-notification MFA is phishable through fatigue — number-matching MFA (where the user has to type a code shown on the login screen into the app, not just tap "approve") closes this specific gap.
Rate-limit MFA challenges: an account generating dozens of push notifications in minutes is itself a signal worth alerting on, not something to keep sending indefinitely.
Train staff on this exact scenario: "IT will never ask you to approve a login you didn't initiate" needs to be explicit, not assumed common sense.
Don't let one approved login be the whole perimeter: broad VPN/internal-network access shouldn't follow from a single MFA approval with nothing else in between.

Incident 6: Cloud provider API key rotation (2021)

What happened: A developer committed AWS credentials to GitHub. An attacker found the key, created EC2 instances, and mined cryptocurrency on the developer's account (costing thousands).

What went wrong:

Credentials were committed to version control.
No automatic rotation (credentials lived indefinitely).
No monitoring: the unusual instance creation wasn't detected for days.

Lessons:

Never commit credentials to version control: use environment variables or secrets managers.
Rotate credentials regularly: every 90 days at most.
Use service accounts with minimal permissions: the developer's AWS key had full access.
Monitor for unusual activity: alert on new instances, unusual regions, high costs.

Incident 7: Okta's support-system breach (2023)

What happened: From late September to October 17, 2023, an attacker used a stolen credential to access Okta's customer support case-management system. Part of Okta's support workflow involved customers uploading HAR files (browser session recordings) to help troubleshoot issues — and some of those HAR files contained live session tokens. The attacker used stolen session tokens to hijack the sessions of 5 Okta customers. Okta initially described the impact narrowly; more than three weeks later, it disclosed that the same breach had also let the attacker download the names and email addresses of all customer support system users, and access contact information for all Okta-certified users. Security researcher Brian Krebs broke the story publicly on October 20, 2023.

What went wrong:

A single stolen employee credential gave access to a system holding highly sensitive customer data (session tokens), not just configuration metadata.
The support workflow itself — asking customers to upload HAR files containing live session tokens — created a genuine risk that data would sit in a support system rather than being scrubbed first.
The full scope of the breach wasn't disclosed all at once; the broader impact only came out weeks after the initial announcement.

Lessons:

A "support system" can be just as sensitive as production — anything holding real customer session tokens deserves production-grade access controls, not lighter ones because it's "just support."
Scrub sensitive data from diagnostic uploads: HAR files and similar troubleshooting artifacts should have session tokens/cookies redacted before ever reaching a support system, automatically, not by relying on the customer to do it.
A stolen credential alone shouldn't be enough: MFA and tighter access controls on systems holding customer data reduce how far one compromised credential can reach.
Full, prompt disclosure matters: understating a breach's scope early, then revising it upward weeks later, costs trust that timely full disclosure would have preserved.

Common patterns from real incidents

1. Weak boundaries between privileged and unprivileged operations

Common mistake: treating all operations equally. Admin operations (create users, reset passwords, view all data) aren't protected differently from normal ones.

Fix:

Require MFA for admin operations.
Require re-authentication before sensitive changes.
Log all privileged actions.

2. No detection of compromise

Common mistake: assuming users will notice their account is hacked. Users often don't.

Fix:

Send emails on new devices / logins from unusual locations.
Offer "View active sessions" to revoke ones the user doesn't recognize.
Alert on unusual API usage (large data exports, bulk operations).

3. Secrets in logs/code

Common mistake: committing credentials to GitHub, logging tokens, storing secrets in configs.

Fix:

Use a secrets manager (AWS Secrets Manager, HashiCorp Vault).
Pre-commit hooks to scan for credentials.
Rotate credentials frequently.
Sanitize logs automatically.

4. Weak rate limiting

Common mistake: no limit on login attempts, password resets, or recovery codes.

Fix:

Rate limit login attempts: 5 failures → 15-minute lockout.
Rate limit password resets: 3 per day per account.
Rate limit recovery codes: require re-authentication after 1-2 uses.

5. No monitoring

Common mistake: assuming security incidents are obvious. They're not.

Fix:

Monitor failed logins.
Monitor bulk exports / unusual API usage.
Monitor new admin accounts / permission changes.
Set up alerts and review them.

Real security incidents and lessons learned

Incident 1: Equifax data breach (2017)

Incident 2: Facebook's plaintext password storage (2019)

Incident 3: Reddit's AiTM phishing attack (2023)

Incident 4: LastPass's home-computer breach (2022)

Incident 5: Uber's MFA fatigue breach (2022)

Incident 6: Cloud provider API key rotation (2021)

Incident 7: Okta's support-system breach (2023)

Common patterns from real incidents

1. Weak boundaries between privileged and unprivileged operations

2. No detection of compromise

3. Secrets in logs/code

4. Weak rate limiting

5. No monitoring

Defensive checklist

Further reading