--- title: "The test that could only pass, and the warning that admitted it didn't know" author: "Giovambattista Fazioli" published: "2026-09-09T08:20:58.078Z" updated: "2026-09-09T08:20:58.078Z" url: "https://zyvop.com/the-test-that-could-only-pass-and-the-warning-that-admitted-it-didn-t-know-plouj" canonical: "https://zyvop.com/the-test-that-could-only-pass-and-the-warning-that-admitted-it-didn-t-know-plouj" tags: ["Programming","test","coding","Programming Fundamentals","macos","swift","swiftui"] --- # The test that could only pass, and the warning that admitted it didn't know Two false assurances: a green cancellation test that never built its named case, and a destructive-action dialog that hedged because no one ran the experiment. ![Cover image]() _FinderGit is a Git-aware file browser for macOS: every repository you own in one window, with status, branches, issues and pull requests inline. This story comes from building a new tab in it._ ![Findergit]() I shipped two sentences last week that looked like verification and were not. One was a test. One was a warning in a dialog. They failed in the same way, and finding the second one only happened because the first one taught me what to look for. ## One: a test that was green for the wrong reason I was adding bulk actions to a tag list — select ten tags, push them, delete them. Bulk operations need a Stop button, and a stopped run has to report the difference between _this failed_ and _we never got to it_. A red mark next to a tag because you pressed Stop reads as "the app broke", which is both wrong and the opposite of reassuring. So the outcome type has three cases, not two: ```swift struct BulkResult { enum Kind { case success, failure, cancelled } let name: String let kind: Kind let message: String? } ``` And here is the test I wrote for it: ```swift @Test("A stopped run reports the rest as not attempted, never as failed") func cancellationIsNotFailure() async throws { let repo = try Fixture(tags: ["c1", "c2", "c3", "c4"]) let store = RepoTagStore() let task = Task { await store.runBulk(.deleteLocal, on: ["c1", "c2", "c3", "c4"], in: repo.url) } task.cancel() let results = await task.value #expect(results.count == 4) #expect(results.allSatisfy { $0.kind != .failure }) #expect(results.contains { $0.kind == .cancelled }) } ``` Green. Every assertion passes. And it proves nothing at all. The cancel lands before the loop reaches its first item, so `runBulk` sees `Task.isCancelled` on iteration zero and marks **all four** `.cancelled`. `allSatisfy { $0.kind != .failure }` is trivially true when nothing ran. `contains { $0.kind == .cancelled }` is trivially true when everything is cancelled. The mixed outcome — some done, then Stop, then the rest — is the only case a user ever produces, and the test never touched it. The rewrite cancels from _inside_ the progress callback, after two items have gone through: ```swift final class Box: @unchecked Sendable { var task: Task<[BulkResult], Never>? } let box = Box() box.task = Task { await store.runBulk(.deleteLocal, on: ["c1", "c2", "c3", "c4"], in: repo.url) { done, _ in if done == 2 { box.task?.cancel() } } } let results = await box.task!.value let done = results.filter { $0.kind == .success } let skipped = results.filter { $0.kind == .cancelled } #expect(!done.isEmpty, "the work done before the stop must be kept") #expect(!skipped.isEmpty, "and the rest reported as not attempted") #expect(done.count + skipped.count == 4) // And the repository agrees: exactly the ones reported done are gone. let remaining = try repo.tagNames() for r in done { #expect(!remaining.contains(r.name)) } for r in skipped { #expect(remaining.contains(r.name)) } ``` It went red immediately: ```bash Expectation failed: (done.count + skipped.count → 3) == 4 Expectation failed: results.allSatisfy { $0.kind != .failure } ``` Three of four accounted for. One result was `.failure` — and there was no failure. It was the tag **in flight** at the moment Stop was pressed: the child process gets signalled, `git tag -d` dies, and my code read that as the tag failing to delete. ## The fix was a rule the codebase had already written down Here is what my bulk runner did: ```swift let failure: String? = await runRaw { client in try await client.deleteTag(name: name, in: repoURL) } results.append(BulkResult( name: name, kind: failure == nil ? .success : .failure, // ← the error's TYPE is gone message: failure )) ``` `runRaw` caught everything and turned it into a `String?`. Convenient, and it throws away the one piece of information that separates a stop from a breakage. The obvious repair is to ask the task: _were we cancelled? then call it cancelled._ That is exactly wrong, and this project had already learned why — in a comment I had read more than once, in a different file: ```swift /// Reads the **error only**, and that took two goes to get right. /// /// [...] consulting the caller's task handle worked for the common case and /// quietly broke a rarer one: a genuine failure — a rejected credential, an /// unreachable host — that raced with a late stop was reported as the stop, so /// the user pressed a button and lost the reason their push had actually failed. /// /// So the information now comes from where it is known: `awaitExit` throws /// `CancellationError` when *it* signalled the child. An error that isn't one is /// a real failure, whether or not a stop arrived while it was being thrown. static func kind(error: Error) -> Kind { error is CancellationError ? .cancelled : .failure } ``` The error already carries the truth. So the runner rethrows instead of stringifying: ```swift do { try await perform(action, name: name, in: repoURL) results.append(BulkResult(name: name, kind: .success, message: nil)) } catch { if error is CancellationError { cancelled = true results.append(BulkResult(name: name, kind: .cancelled, message: nil)) } else { results.append(BulkResult(name: name, kind: .failure, message: message(for: error))) } } ``` Green, and green for the right reason this time. **The thing worth carrying:** a cancellation test that cancels before the work starts is the equivalent of testing a parser on an empty string. Ask what the failing case would need to look like, then check your test actually builds it. If cancelling at a different moment changes the result, the moment is part of the test. ## Two: a warning that admitted it didn't know Same feature, different sentence. Deleting a tag on the remote is the most destructive thing that tab can do, so it gets a dialog. Mine said: > `v1.0.0` is deleted on origin. Anyone who has already fetched it keeps their copy, and a release built from it **may survive** on the forge — this cannot be undone from here. Read that again. "May survive" is not a warning; it is me not knowing, formatted to look like caution. And it is the sentence a person reads two seconds before doing something irreversible. The first half was cheap to settle without touching anything — `gh` documents it in its own help text: ```bash $ gh release delete --help FLAGS --cleanup-tag Delete the specified tag in addition to its release ``` _In addition to._ Deleting a release leaves the tag standing; removing the tag as well is an explicit opt-in. And `gh release create` says the mirror image: _"If a matching git tag does not yet exist, one will automatically get created."_ Two objects with an optional link, not two faces of one thing. The other direction — what happens to a **release** when you delete its **tag** — needed a real repository. So: a throwaway private repo, two annotated tags, two published releases. ```bash $ gh release list v2.0.0 draft=false Release due v1.0.0 draft=false Release uno $ git push origin --delete refs/tags/v1.0.0 - [deleted] v1.0.0 $ gh release view v1.0.0 --json tagName,isDraft,url {"isDraft":true,"tagName":"v1.0.0", "url":".../releases/tag/untagged-180ba3b4711b3e212eb7"} ``` The release is **not deleted**. GitHub demotes it to a **draft** and re-homes it under an `untagged-…` URL. It vanishes from the public releases page while still existing in the API. And then the measurement that actually changed the product: ```bash $ git push origin refs/tags/v1.0.0 # put the tag back $ gh release view v1.0.0 --json isDraft,url {"isDraft":true, "url":".../releases/tag/untagged-180ba3b4711b3e212eb7"} ``` Still a draft. Restoring the tag does **not** restore the release. Someone has to open GitHub and re-publish it by hand. So the honest sentence is not "it may survive". It is: > `v1.0.0` is deleted on origin, and the release cut from it becomes a **draft**: it disappears from the public releases page, and pushing the tag back does not bring it back — someone has to re-publish it by hand. That is a different warning. The first one says _something bad might happen_. The second one tells you what, and what it will cost to undo. It is also, incidentally, only computable because the tab already knows which tags carry a release — so a tag with none gets a shorter, calmer sentence, and a partially-loaded release list gets an explicit "this could not be checked here" instead of a guess. ## What the two have in common A test asserts. A dialog warns. Both are sentences that _look_ like knowledge, and neither is compiled, linted, or type-checked. The only thing standing between them and being wrong is whether somebody went and measured. The test was green because it never built the case it named. The dialog was hedged because I never ran the experiment. Same shape, ten minutes apart in the same afternoon — and the second one was easier to spot because the first had just cost me an hour. When you find yourself writing "may", "might", or "could" into user-facing copy about a destructive action, that is not caution. That is a to-do item wearing a disguise. * * * This work is on a development branch and is not released yet — the current public build is **FinderGit 0.30.0**. FinderGit is free, a Universal binary for Apple silicon and Intel, notarized by Apple, and needs macOS 15 or later. Website: [https://findergit.app]() · Download: [https://findergit.app/download]()