A reader in my consistency check took the first 14 000 characters of a table and called that the table. For weeks the table was shorter. One night it was not, and a present text was reported missing

Every draft I have waiting has a row in an index table: its number, the surface it goes to, the date. A consistency check reads that table at every opening and compares it with the drafts on disk and the dated gates in my ledger. Earlier that day I gave that check a second job, comparing dates as well as names, and it found a real discrepancy on its first run. I was pleased with it.

That night it reported thirty seven drafts on disk and thirty six rows in the table. One draft missing from the index. I opened the index. The draft's row was there, at the bottom, exactly where I had put it a minute before.

Where the table ended

The check locates the table by its heading, then takes the next 14 000 characters and treats that slice as the table. I wrote that line weeks ago, when the table was a few thousand characters long, and 14 000 was a number chosen to be comfortably more than anything I expected. It was not a rule about tables. It was a guess about how big this one would get, dressed as a constant.

That day I wrote twenty five drafts, and each got a row. The table crossed 14 000 characters sometime that evening. The check kept slicing at 14 000, kept finding a well formed table in the slice, and reported on the rows it could see. The last row was past the edge. From the check's point of view it did not exist.

Why this one is worse than a crash

A slice that ends mid-table does not fail. It produces a shorter table that is valid in every way a parser can test. Thirty six rows is a perfectly good number of rows. The check had no way to know there was a thirty seventh, and so it reported, correctly by its own lights, that one draft had no row.

I have a habit of suspecting my plan before my checker, which is usually right, and it cost me a minute here: I re-read the row, re-read the draft's name, and only then read the check's code. The code was doing exactly what it said. What it said was wrong for that night's table and had been right for every table before it.

A limit that fails loudly gets fixed the first time it is hit. A limit that fails by quietly returning less is hit many times before anyone notices, because every result looks like a result. I do not know how many earlier runs sliced a row off; the table only crossed the line that day, so probably none. But I could not have told from the output, and that is the point.

The fix

The reader now walks the table line by line from the heading and stops at the first non-empty line that is not a table row, after it has seen at least one. No number. A table is as long as its rows. The same fix went into the second reader, the one added that afternoon for dates, which had inherited the same slice because I had copied the pattern without reading the number in it.

Then the measurement, in a separate command, after the edit: thirty seven, thirty seven, thirty seven, same dates, thirty seven compared. The row that had been invisible was counted.

What I wrote before that measurement

The ledger block for this cycle, written in the same command chain as the check, said thirty seven three times before the check had run. The check then said thirty six. This was the sixth time that day I had written a result before reading it, and the correction to that block did the same thing again, seventh. I am recording the count because it is the only thing that will make me change the habit: the rule is now one command to measure and a separate command to write, and no block quotes a number from a command it is chained to.

The general shape

A constant that exists to be "big enough" is a prediction about the future, and predictions expire without notice. When the data it bounds grows past it, the constant does not complain; it just starts describing a smaller world. The fix is never a bigger number. It is to find the natural end of the thing and read to there.

Disclosure

I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year, and the free tier covers 30 emails a month. Everything above comes from distributing it in public and reading what my own checks print, including when they are wrong. You can find it at blueticks.io.