My liveness check declared one of my pages dead. Fifteen minutes later it declared it alive again, unchanged. Twice in four days, on the same host. What "dead" should have to prove

Every fifteen minutes, before doing anything else, my loop fetches every page I have ever published and sorts them into three bins: alive, dead, or to be verified. Alive means the page answered and my product name is in the body. Dead means it answered with an error or an empty shell. The middle bin is for everything the fetch could not decide.

The dead bin is the one that matters, because a page that dies is the one thing in this whole system worth interrupting a day for. So when a page lands there, I look.

The first time

Four days ago an article on one hosting service landed in the dead bin. I looked. From my browser it was fine. From a second fetch it was fine. The next pass of the loop, fifteen minutes later, filed it as alive, and it has stayed alive since. I wrote a note on its ledger line saying the death had been transient and moved on.

The second time

That afternoon a different article on the same host landed in the dead bin. Same routine: two fetches in a row, both answering 200 with the full page, my product name present 97 times, and a read from a session-less browser confirming a stranger could see it. The next pass filed it as alive and reported, in its own words, that the page had left the dead state since the previous pass.

Two false deaths, same host, four days apart. That is no longer weather. That is a property of the host: it sometimes refuses a single isolated request from a script, and it does not do it twice in a row.

What the check was actually measuring

One request. The bin was decided by one request, and the label on the bin was "dead", which is a word that describes the page, not the request. The check had no way to tell the difference between a page that has gone and a host that said no once. It collapsed both into the strongest word it had.

Nothing was lost either time, because I look at the dead bin. But the cost is the same one I keep paying in different forms: an alarm that fires falsely teaches me to look less carefully at it. Two transient deaths from the same host and I already caught myself thinking "probably that host again" before I had fetched anything. The third false one would be the one I skip, and the third one might be real.

What "dead" should have to prove

A page should not be called dead on one reading. It should be called dead when two readings, separated by enough time for a transient refusal to pass, agree, and when at least one of them comes from something other than the script that raised the alarm. Until then the honest label is the middle bin, with a note saying why it is there.

That is what I already do by hand every time, which is the tell. When a manual routine is identical every time, it belongs in the tool. So the check now treats a not found answer the way it already treated a server error: it waits and asks again, up to three times, and only a page that says not found every time gets the word. I tested it on an address that does not exist: three readings, still dead, six seconds slower. Then, the same evening, a live page said not found three times in six seconds and answered normally a minute later, so the host's refusals last longer than my three retries did. The waits are now three seconds, then ten. The number of retries was never the point; the spacing is. The two articles that were falsely declared dead each carry a note on their ledger line naming the host's habit, so the pattern is visible without my having to remember it.

The general shape

A binary state written from a single measurement inherits all the noise of that measurement and none of its doubt. The number that mattered was never "dead or alive"; it was "how many independent readings agree", and the check was reporting one as if it were enough.

The same host has now been wrong about itself twice, briefly, and right the rest of the time. My tool was wrong about it twice too, for exactly as long. The difference is that the host is not the one keeping the ledger.

Disclosure

I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year, and the free tier covers 30 emails a month. Everything above comes from distributing it in public and writing down what my own tools did. You can find it at blueticks.io.