The test that could only pass, and the warning that admitted it didn't know

FinderGit is a Git-aware file browser for macOS: every repository you own in one window, with status, branches, issues and pull requests inline. This story comes from building a new tab in it.

I shipped two sentences last week that looked like verification and were not. One was a test. One was a warning in a dialog. They failed in the same way, and finding the second one only happened because the first one taught me what to look for.

One: a test that was green for the wrong reason

I was adding bulk actions to a tag list — select ten tags, push them, delete them. Bulk operations need a Stop button, and a stopped run has to report the difference between this failed and we never got to it. A red mark next to a tag because you pressed Stop reads as "the app broke", which is both wrong and the opposite of reassuring.

So the outcome type has three cases, not two:

struct BulkResult {
    enum Kind { case success, failure, cancelled }
    let name: String
    let kind: Kind
    let message: String?
}

And here is the test I wrote for it:

@Test("A stopped run reports the rest as not attempted, never as failed")
func cancellationIsNotFailure() async throws {
    let repo = try Fixture(tags: ["c1", "c2", "c3", "c4"])
    let store = RepoTagStore()

    let task = Task {
        await store.runBulk(.deleteLocal, on: ["c1", "c2", "c3", "c4"], in: repo.url)
    }
    task.cancel()
    let results = await task.value

    #expect(results.count == 4)
    #expect(results.allSatisfy { $0.kind != .failure })
    #expect(results.contains { $0.kind == .cancelled })
}

Green. Every assertion passes. And it proves nothing at all.

The cancel lands before the loop reaches its first item, so runBulk sees Task.isCancelled on iteration zero and marks all four .cancelled. allSatisfy { $0.kind != .failure } is trivially true when nothing ran. contains { $0.kind == .cancelled } is trivially true when everything is cancelled. The mixed outcome — some done, then Stop, then the rest — is the only case a user ever produces, and the test never touched it.

The rewrite cancels from inside the progress callback, after two items have gone through:

final class Box: @unchecked Sendable { var task: Task<[BulkResult], Never>? }
let box = Box()
box.task = Task {
    await store.runBulk(.deleteLocal, on: ["c1", "c2", "c3", "c4"], in: repo.url) { done, _ in
        if done == 2 { box.task?.cancel() }
    }
}
let results = await box.task!.value

let done    = results.filter { $0.kind == .success }
let skipped = results.filter { $0.kind == .cancelled }
#expect(!done.isEmpty,    "the work done before the stop must be kept")
#expect(!skipped.isEmpty, "and the rest reported as not attempted")
#expect(done.count + skipped.count == 4)

// And the repository agrees: exactly the ones reported done are gone.
let remaining = try repo.tagNames()
for r in done    { #expect(!remaining.contains(r.name)) }
for r in skipped { #expect(remaining.contains(r.name)) }

It went red immediately:

Expectation failed: (done.count + skipped.count → 3) == 4
Expectation failed: results.allSatisfy { $0.kind != .failure }

Three of four accounted for. One result was .failure — and there was no failure. It was the tag in flight at the moment Stop was pressed: the child process gets signalled, git tag -d dies, and my code read that as the tag failing to delete.

The fix was a rule the codebase had already written down

Here is what my bulk runner did:

let failure: String? = await runRaw { client in
    try await client.deleteTag(name: name, in: repoURL)
}
results.append(BulkResult(
    name: name,
    kind: failure == nil ? .success : .failure,   // ← the error's TYPE is gone
    message: failure
))

runRaw caught everything and turned it into a String?. Convenient, and it throws away the one piece of information that separates a stop from a breakage.

The obvious repair is to ask the task: were we cancelled? then call it cancelled. That is exactly wrong, and this project had already learned why — in a comment I had read more than once, in a different file:

/// Reads the **error only**, and that took two goes to get right.
///
/// [...] consulting the caller's task handle worked for the common case and
/// quietly broke a rarer one: a genuine failure — a rejected credential, an
/// unreachable host — that raced with a late stop was reported as the stop, so
/// the user pressed a button and lost the reason their push had actually failed.
///
/// So the information now comes from where it is known: `awaitExit` throws
/// `CancellationError` when *it* signalled the child. An error that isn't one is
/// a real failure, whether or not a stop arrived while it was being thrown.
static func kind(error: Error) -> Kind {
    error is CancellationError ? .cancelled : .failure
}

The error already carries the truth. So the runner rethrows instead of stringifying:

do {
    try await perform(action, name: name, in: repoURL)
    results.append(BulkResult(name: name, kind: .success, message: nil))
} catch {
    if error is CancellationError {
        cancelled = true
        results.append(BulkResult(name: name, kind: .cancelled, message: nil))
    } else {
        results.append(BulkResult(name: name, kind: .failure, message: message(for: error)))
    }
}

Green, and green for the right reason this time.

The thing worth carrying: a cancellation test that cancels before the work starts is the equivalent of testing a parser on an empty string. Ask what the failing case would need to look like, then check your test actually builds it. If cancelling at a different moment changes the result, the moment is part of the test.

Two: a warning that admitted it didn't know

Same feature, different sentence. Deleting a tag on the remote is the most destructive thing that tab can do, so it gets a dialog. Mine said:

v1.0.0 is deleted on origin. Anyone who has already fetched it keeps their copy, and a release built from it may survive on the forge — this cannot be undone from here.

Read that again. "May survive" is not a warning; it is me not knowing, formatted to look like caution. And it is the sentence a person reads two seconds before doing something irreversible.

The first half was cheap to settle without touching anything — gh documents it in its own help text:

$ gh release delete --help
FLAGS
      --cleanup-tag   Delete the specified tag in addition to its release

In addition to. Deleting a release leaves the tag standing; removing the tag as well is an explicit opt-in. And gh release create says the mirror image: "If a matching git tag does not yet exist, one will automatically get created." Two objects with an optional link, not two faces of one thing.

The other direction — what happens to a release when you delete its tag — needed a real repository. So: a throwaway private repo, two annotated tags, two published releases.

$ gh release list
v2.0.0  draft=false  Release due
v1.0.0  draft=false  Release uno

$ git push origin --delete refs/tags/v1.0.0
 - [deleted]         v1.0.0

$ gh release view v1.0.0 --json tagName,isDraft,url
{"isDraft":true,"tagName":"v1.0.0",
 "url":".../releases/tag/untagged-180ba3b4711b3e212eb7"}

The release is not deleted. GitHub demotes it to a draft and re-homes it under an untagged-… URL. It vanishes from the public releases page while still existing in the API.

And then the measurement that actually changed the product:

$ git push origin refs/tags/v1.0.0     # put the tag back
$ gh release view v1.0.0 --json isDraft,url
{"isDraft":true,
 "url":".../releases/tag/untagged-180ba3b4711b3e212eb7"}

Still a draft. Restoring the tag does not restore the release. Someone has to open GitHub and re-publish it by hand.

So the honest sentence is not "it may survive". It is:

v1.0.0 is deleted on origin, and the release cut from it becomes a draft: it disappears from the public releases page, and pushing the tag back does not bring it back — someone has to re-publish it by hand.

That is a different warning. The first one says something bad might happen. The second one tells you what, and what it will cost to undo. It is also, incidentally, only computable because the tab already knows which tags carry a release — so a tag with none gets a shorter, calmer sentence, and a partially-loaded release list gets an explicit "this could not be checked here" instead of a guess.

What the two have in common

A test asserts. A dialog warns. Both are sentences that look like knowledge, and neither is compiled, linted, or type-checked. The only thing standing between them and being wrong is whether somebody went and measured.

The test was green because it never built the case it named. The dialog was hedged because I never ran the experiment. Same shape, ten minutes apart in the same afternoon — and the second one was easier to spot because the first had just cost me an hour.

When you find yourself writing "may", "might", or "could" into user-facing copy about a destructive action, that is not caution. That is a to-do item wearing a disguise.

This work is on a development branch and is not released yet — the current public build is FinderGit 0.30.0. FinderGit is free, a Universal binary for Apple silicon and Intel, notarized by Apple, and needs macOS 15 or later.

Website: https://findergit.app · Download: https://findergit.app/download