Skip to content
Incident Room

Results day, database full

SmartSchool LK, a school results portal. The results site slows to a crawl as thousands of students log in.

  • AWS
  • PostgreSQL
  • Docker
  • Nginx

About 14 minutes

PAGE: results site timing out, p95 above 20 s

Results have just been released. The site is timing out and the app reports 'too many connections'. The principal is calling.

Time since page
0 min
Customers hurt
0
Revenue lost
Trust, not cash
Status
Outage

Your system

  • Load balancerHealthy
  • Web serversDegraded
  • Results APIDown
  • Connection poolDown
  • PostgreSQLDegraded

Evidence and timeline

0 steps

You are on call and the page just went off. Do not touch anything yet. Open a dashboard or a log from the Investigate list on the right, and find out what is really happening. Every action costs minutes, and every minute hurts customers.

Look before you leap. Good engineers read the evidence first, then act.

Dashboards

Logs

Deploys

Terminal

Costs are simulated minutes.