The binary was never installed
The service never started because apt update failed at boot. You can curl the mirror just fine.
You learn: how a half-working network lies to you.
Prod is down.
Your terminal.
Fix it.
Hands-on troubleshooting labs for DevOps and SRE engineers. We hand you a broken production system; you SSH in from your own terminal and bring it back.
Get early access
The first labs are free.
Firing
02:14 UTC
checkout is returning 503
~ $ fixmyprod start disk-full --attach
creating session… variant 2… ready in 41s
root@web-01:~# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p1 20G 20G 0 100% /
root@web-01:~# du -xsh / 2>/dev/null
4.1G /
root@web-01:~# exit
~ $ fixmyprod check
solve probe.http FAIL got 503, want 200 stable 60s
~ $
01
Your terminal, not a browser tab
Plain ssh from the shell you already use — your keys, your config, your muscle memory.
02
Failures from real incidents
More than one cause, a first lead that is wrong, noise that looks like the problem. Like the pages you actually get.
03
Load and scale
Make nginx hold the traffic we send at it. Untangle scheduling on a 200-node cluster. Not just puzzles on one host.
A taste of the labs
Needs 10k rps, holds 2k
The box sits at 12% CPU and refuses connections. Every fix raises the ceiling — until you hit the next one.
You learn: where web servers really run out.
Half the pods are Pending
200 nodes, plenty of room, and the scheduler still says no. kubectl only — there is no host to SSH into.
You learn: to read the scheduler's reasons instead of guessing them.
Get early access
The first labs are free. A subscription comes later.