Real security incidents and lessons learned

Authentication and authorization bugs are among the most common vulnerabilities in production. Here are real incidents from real companies, what went wrong, and what they should have done.

Advanced

6 min read

Incident 1: Equifax data breach (2017)

What happened: Attackers exploited a known vulnerability (CVE-2017-5638) in Apache Struts to access Equifax's database, exfiting 147 million people's personal information including SSNs, birthdates, and credit card numbers.

What went wrong:

  • Equifax knew about the vulnerability but didn't patch it for weeks.
  • No authentication on critical admin endpoints.
  • Insufficient monitoring: attackers accessed the system for months undetected.

Lessons:

  1. Patch quickly: vulnerabilities in authentication/authorization code are critical.
  2. Use strong authentication on all admin endpoints: not just some.
  3. Monitor for unusual access patterns: failed logins, large data exports, unusual IPs.
  4. Never store PII in plaintext: encrypt sensitive data at rest.

Incident 2: Facebook token leakage (2019)

What happened: Facebook stored access tokens in plaintext in log files. If someone accessed the logs, they could impersonate users.

What went wrong:

  • Tokens were logged on every request.
  • Logs weren't encrypted or sanitized.
  • No one checked what was actually in the logs.

Lessons:

  1. Never log sensitive data: tokens, passwords, credit cards, SSNs.
  2. Sanitize logs: redact tokens, replace with [REDACTED].
  3. Encrypt logs at rest: assume logs can be accessed.
  4. Audit log access: who can read logs and when.

Incident 3: Twitch account takeover (2020)

What happened: Attackers could take over Twitch accounts by resetting passwords for accounts without email verification. They'd send a password reset link to their own email, click it, and gain access.

What went wrong:

  • Password reset didn't require the user to verify their current email first.
  • Attackers could change the account email and password simultaneously.

Lessons:

  1. Verify the account owner before password reset: send a link to the current email.
  2. Don't allow changing email + password in one operation: require separate verification.
  3. Use time-limited tokens for password resets: 15-30 minutes, not hours.
  4. Alert users on sensitive changes: email them "your account password was reset" and a link to undo it.

Incident 4: GitHub TOTP bypass (2023)

What happened: GitHub's two-factor authentication (2FA) could be bypassed by attackers who compromised the user's email account. They'd request a "recovery code" email instead of the TOTP code.

What went wrong:

  • Recovery codes were emailed to the primary email address.
  • If attackers controlled the email, they could get recovery codes.
  • No rate limiting on recovery code requests.

Lessons:

  1. Don't email recovery codes: store them securely in the app, require backup codes on setup.
  2. Rate limit sensitive operations: lock the account after N failed 2FA attempts.
  3. Detect account compromise: email users "someone tried to access your account" and offer to revoke sessions.
  4. 2FA is not just TOTP: security keys (hardware tokens) are much stronger.

Incident 5: Cloud provider API key rotation (2021)

What happened: A developer committed AWS credentials to GitHub. An attacker found the key, created EC2 instances, and mined cryptocurrency on the developer's account (costing thousands).

What went wrong:

  • Credentials were committed to version control.
  • No automatic rotation (credentials lived indefinitely).
  • No monitoring: the unusual instance creation wasn't detected for days.

Lessons:

  1. Never commit credentials to version control: use environment variables or secrets managers.
  2. Rotate credentials regularly: every 90 days at most.
  3. Use service accounts with minimal permissions: the developer's AWS key had full access.
  4. Monitor for unusual activity: alert on new instances, unusual regions, high costs.

Incident 6: Google account session fixation (2018)

What happened: An attacker could log in to a Google account without the password by reusing the user's session ID. Google's sign-in flow didn't regenerate session IDs after authentication.

What went wrong:

  • Session ID wasn't regenerated after login.
  • An attacker could use a known session ID to hijack the session.

Lessons:

  1. Always regenerate session IDs after login: don't reuse the old one.
  2. Invalidate old sessions on sensitive actions: password change, email change, new device.
  3. Test for session fixation: try to log in with a predetermined session ID.

Incident 7: Okta breach (2023)

What happened: Okta (an identity provider) was breached. Attackers gained access to internal dashboards and could view customer configurations (but not customer data). The breach was discovered weeks later through a customer's monitoring.

What went wrong:

  • Insufficient logging and monitoring.
  • No MFA on internal dashboards.
  • Slow incident response.

Lessons:

  1. Require MFA on all internal dashboards: especially for identity/auth infrastructure.
  2. Log all privileged actions: view configs, change settings, export data.
  3. Alert on unusual patterns: failed logins from new IPs, bulk data access.
  4. Respond quickly: have an incident response plan and test it.

Common patterns from real incidents

1. Weak boundaries between privileged and unprivileged operations

Common mistake: treating all operations equally. Admin operations (create users, reset passwords, view all data) aren't protected differently from normal ones.

Fix:

  • Require MFA for admin operations.
  • Require re-authentication before sensitive changes.
  • Log all privileged actions.

2. No detection of compromise

Common mistake: assuming users will notice their account is hacked. Users often don't.

Fix:

  • Send emails on new devices / logins from unusual locations.
  • Offer "View active sessions" to revoke ones the user doesn't recognize.
  • Alert on unusual API usage (large data exports, bulk operations).

3. Secrets in logs/code

Common mistake: committing credentials to GitHub, logging tokens, storing secrets in configs.

Fix:

  • Use a secrets manager (AWS Secrets Manager, HashiCorp Vault).
  • Pre-commit hooks to scan for credentials.
  • Rotate credentials frequently.
  • Sanitize logs automatically.

4. Weak rate limiting

Common mistake: no limit on login attempts, password resets, or recovery codes.

Fix:

  • Rate limit login attempts: 5 failures → 15-minute lockout.
  • Rate limit password resets: 3 per day per account.
  • Rate limit recovery codes: require re-authentication after 1-2 uses.

5. No monitoring

Common mistake: assuming security incidents are obvious. They're not.

Fix:

  • Monitor failed logins.
  • Monitor bulk exports / unusual API usage.
  • Monitor new admin accounts / permission changes.
  • Set up alerts and review them.

Defensive checklist

  • Passwords hashed with bcrypt or Argon2 (cost ≥ 12).
  • Sessions/tokens don't include sensitive data.
  • Session IDs regenerated after login.
  • Password resets expire after 15 minutes.
  • MFA available for all accounts.
  • MFA required for admin operations.
  • No credentials in logs, code, or configs.
  • Secrets rotated every 90 days.
  • Authorization checked on every action (not just frontend).
  • Tenant boundaries enforced (multi-tenant apps).
  • Failed logins monitored and alerted on.
  • Account changes (email, password, 2FA) emailed to user.
  • Old sessions invalidated on sensitive changes.
  • HTTPS only (no HTTP fallback).
  • SameSite=Strict on session cookies.
  • CSRF tokens on all state-changing operations.
  • Rate limiting on login, password reset, API endpoints.
  • Logs sanitized (no tokens, passwords, PII).
  • Log access controlled and monitored.

Further reading

Check your understanding

A quick comprehension check — not tracked, not graded, just for you.

1. What was the root cause of the Equifax breach?

2. Why was the Facebook token logging incident dangerous?

3. What's a common pattern in password reset attacks?

4. What should you monitor for in a production auth system?