What Checking Every Scheduled Run, Not Just the Latest, Actually Caught
42 scheduled runs so far, 5 of them failed outright, and not one of those 5 ever surfaced on its own. No email, no notification, no red mark anywhere I'd naturally look. Windows Task Scheduler ran the job, the job died, and the task simply... sat there, marked complete, looking exactly like every other successful run unless you went and asked it directly.
| Incident | How it was found | How long it sat unpublished |
|---|---|---|
| 10/17 run -- exit 3221225786 (forced kill) | Routine check, not an alert | ~36 hours |
| 10/29 run -- exit 1 (DNS timeout), despite running on time | Routine check, not an alert | A few hours, caught same check-in |
The habit that's actually doing the work
Every "블로그 상황 어때" status check now includes pulling Get-ScheduledTaskInfo on the most recent few task runs, not just reading the published post list and assuming anything missing from it simply hasn't fired yet. That one extra step is the entire reason both incidents above got caught within hours or days instead of silently becoming "a post that never happened and nobody noticed." The post list alone would never have shown it -- a missing post looks identical whether it's scheduled for later or already failed.
Why this isn't overkill for a blog this small
It would be easy to argue that a near-zero-traffic blog publishing itself once every two or three days doesn't need this kind of operational discipline. But the cost of the check is a few seconds, and the cost of not catching a failure is a gap in a build-in-public record that's specifically supposed to be continuous. The failures aren't the embarrassing part -- a sleeping Windows machine missing a DNS lookup after waking up is a genuinely unglamorous, believable failure mode. Finding out about it three weeks later from a reader asking "did you stop posting?" would have been the actual embarrassing part.
What I'd still like to fix, and haven't
None of this is monitoring in any real sense -- it's a manual check that only happens when someone asks how the blog is doing. A machine that's actually unattended for a week would still lose posts silently in between. That's a known gap, not a solved one.
Comments
Post a Comment