--- title: "Mini Project 1: A Markdown Note Searcher for the Command Line" author: "Pradeep Kumar" published: "2026-11-14T07:41:01.068Z" updated: "2026-11-14T07:41:01.068Z" url: "https://zyvop.com/mini-project-1-a-markdown-note-searcher-for-the-command-line-iqloq" canonical: "https://zyvop.com/mini-project-1-a-markdown-note-searcher-for-the-command-line-iqloq" tags: ["Python","argparse","AI Engineer","Projects","cli"] --- # Mini Project 1: A Markdown Note Searcher for the Command Line Build a real command-line tool: search a folder of markdown notes, rank the results, and make it behave like a proper Unix program with exit codes, JSON output, and tests. ![Cover image]() Phase 1 gave you a lot of tools. This project puts them to work. You'll build `notes`, a command-line tool that searches a folder of markdown notes, ranks the results, and plays nicely with pipes and scripts. It's the first of four projects that close out the programming track. Everything below was built and run for real, with one exception: `pytest` couldn't be installed in my test environment, so I ran the test functions with a small stand-in, as I did in Post 13. ## What we're building Point it at a folder of notes, and ask a question: ```bash $ notes search generators --dir sample-notes 1. Python generators [python, generators] (score 19) python/generators.md:5: # Python generators python/generators.md:12: Generators keep memory small when a file is huge. 2. Decorators [python] (score 1) python/decorators.md:7: It is similar to how generators wrap a loop. ``` Each result shows the note's title, its tags, a score, and the matching lines with their line numbers, so you can jump straight to them in an editor. The tool has three commands: | Command | What it does | | --- | --- | | `notes search WORDS...` | Finds notes containing **all** the words, best matches first | | `notes list` | Lists every note, with its title and tags | | `notes stats` | Counts notes, words, and tags | And a few options: `--dir` (which folder), `--tag` (filter by tag), `--limit` (how many results), `--json` (machine-readable output), and `-v` (show which files are read). It follows the conventions of good command-line tools, which makes it usable in scripts: | Exit code | Meaning | | --- | --- | | 0 | Success, and (for `search`) something was found | | 1 | Nothing matched, or an unknown tag, like `grep` | | 2 | Bad usage: a missing folder, a bad option, no command | Results go to **stdout** and messages go to **stderr**, so piping the output into another program never mixes in warnings. ### The notes it reads Notes are plain `.md` files, optionally starting with a small block of metadata between two `---` lines: ```markdown --- title: Python generators tags: [python, generators] --- # Python generators A generator yields values lazily, one at a time. Streaming replies from a model use the same idea. ``` Without that block, the title comes from the first `# Heading`, and failing that, from the file name. The sample folder I used has four notes (`python/generators.md`, `python/decorators.md`, `ai/rag.md`, and `ideas.md`) plus a `.hidden/secret.md` that the tool should ignore. Make your own, or point the tool at real notes. ## The plan Three small modules, each with one job: ```python notesearch/ ├── pyproject.toml ├── README.md ├── .gitignore ├── src/notesearch/ │ ├── __init__.py │ ├── notes.py # read files, parse front matter (no searching, no printing) │ ├── search.py # score notes against a query (no files, no printing) │ └── cli.py # arguments, output, exit codes (the only part that prints) └── tests/ └── test_notesearch.py ``` Keeping the parts apart pays off when we test: parsing and scoring need no files or terminal at all. The scoring is deliberately simple and easy to explain. A note matches only if **every** search word appears somewhere in it. Then each word earns points: | Where the word appears | Points | | --- | --- | | In the title | 10 | | In a tag | 5 | | On a heading line (starts with `#`) | 3 per line | | On any other line | 1 per line | Ties are broken by file path, so results are always in a stable order. Along the way you'll use a good chunk of the track: generators for lazy file reading, dataclasses for the data, `heapq` for picking the best results, `difflib` for a "did you mean?" on tags, `logging`, `pathlib`, packaging with a console script, tests, and Git. ## Step 0: set up the project ```bash mkdir notesearch && cd notesearch git init -b main python -m venv .venv source .venv/bin/activate # on Windows, use the activate line from Post 12 mkdir -p src/notesearch tests sample-notes ``` Create `pyproject.toml`. The important part is `[project.scripts]`, which turns a function into a command: ```toml [project] name = "notesearch" version = "0.1.0" description = "Search and browse a folder of markdown notes from the command line" readme = "README.md" requires-python = ">=3.11" dependencies = [] [project.optional-dependencies] dev = ["pytest>=8.0"] [project.scripts] notes = "notesearch.cli:main" [build-system] requires = ["setuptools>=68"] build-backend = "setuptools.build_meta" [tool.pytest.ini_options] testpaths = ["tests"] ``` `notes = "notesearch.cli:main"` means: create a command called `notes` that runs the `main` function in `notesearch/cli.py`. Add a `.gitignore` containing `.venv/`, `__pycache__/`, `*.egg-info/`, and `.pytest_cache/`. Add a short `README.md`, and a one-line `src/notesearch/__init__.py`: ```python __version__ = "0.1.0" ``` Commit this skeleton before writing any logic, with a message like `Set up project skeleton with pyproject and README`. ## Step 1: reading notes Create `src/notesearch/notes.py`: ```python import logging from collections.abc import Iterator from dataclasses import dataclass from pathlib import Path logger = logging.getLogger("notesearch") @dataclass(frozen=True) class Note: path: Path title: str tags: tuple[str, ...] body: tuple[str, ...] # the lines after the front matter body_start: int # line number in the file of body[0], counting from 1 @property def word_count(self) -> int: return sum(len(line.split()) for line in self.body) def parse_front_matter(lines: list[str]) -> tuple[dict[str, str], int]: """Read a leading block of 'key: value' lines between two '---' lines. Returns the values and how many lines the block used (0 if there isn't one). """ if not lines or lines[0].strip() != "---": return {}, 0 for end, line in enumerate(lines[1:], start=1): if line.strip() == "---": values = {} for entry in lines[1:end]: key, separator, value = entry.partition(":") if separator: values[key.strip().lower()] = value.strip() return values, end + 1 return {}, 0 # an opening '---' with no closing one: treat it as plain text def parse_tags(raw: str) -> tuple[str, ...]: """Accept 'a, b' or '[a, b]', and return lowercase tags without duplicates.""" cleaned = raw.strip().strip("[]") tags = [tag.strip().strip("'\"").lower() for tag in cleaned.split(",")] return tuple(dict.fromkeys(tag for tag in tags if tag)) def first_heading(lines: list[str]) -> str | None: for line in lines: if line.startswith("# "): return line[2:].strip() return None def parse_note(path: Path, text: str) -> Note: lines = text.splitlines() meta, used = parse_front_matter(lines) body = lines[used:] title = meta.get("title") or first_heading(body) or path.stem return Note( path=path, title=title, tags=parse_tags(meta.get("tags", "")), body=tuple(body), body_start=used + 1, ) def iter_notes(root: Path) -> Iterator[Note]: """Yield every markdown note under root, one at a time, skipping hidden folders.""" for path in sorted(root.rglob("*.md")): relative_parts = path.relative_to(root).parts if any(part.startswith(".") for part in relative_parts): continue try: text = path.read_text(encoding="utf-8", errors="replace") except OSError as error: logger.warning("skipping %s: %s", path, error) continue logger.debug("read %s", path) yield parse_note(path, text) ``` A few choices deserve a comment. **The front matter parser is tiny on purpose.** Real front matter is YAML, and a library like PyYAML parses all of it. But we only need `title` and `tags`, and 15 lines of our own code means zero dependencies. The cost is that it understands only simple `key: value` lines, which is fine for notes, and worth a sentence in the README so nobody is surprised. **Malformed input doesn't crash it.** An opening `---` with no closing one is treated as ordinary text, and the three tag styles (`[a, b]`, `a, b`, and quoted) all work. Notes written by humans are messy, so the parser forgives. I checked the edge cases directly: ```python parse_front_matter(["---", "title: A", "tags: [x]", "---", "body"]) # ({'title': 'A', 'tags': '[x]'}, 4) parse_front_matter(["---", "title: A", "body"]) # ({}, 0) parse_tags("[a, B, a]") # ('a', 'b') ``` `body_start` **makes line numbers honest.** Search results point at lines in the _file_, but we've stripped the front matter off the body. So each note remembers where its body began, and `body_start + offset` gives the real line number. `iter_notes` **is a generator** (Post 7). It yields one note at a time instead of building a list, so `notes stats` can run over a huge folder without keeping it all in memory. It skips hidden folders like `.git`, and if one file can't be read, it logs a warning and carries on instead of dying. ## Step 2: searching and scoring Create `src/notesearch/search.py`: ```python import heapq from collections.abc import Iterable from dataclasses import dataclass from notesearch.notes import Note TITLE_POINTS = 10 TAG_POINTS = 5 HEADING_POINTS = 3 BODY_POINTS = 1 MAX_HITS_PER_NOTE = 3 @dataclass(frozen=True) class Hit: line_number: int text: str @dataclass(frozen=True) class Result: note: Note score: int hits: tuple[Hit, ...] def split_query(query: str) -> list[str]: return query.lower().split() def score_note(note: Note, terms: list[str]) -> Result | None: """Score a note, or return None unless it contains every search term.""" title = note.title.lower() tags = " ".join(note.tags) body = [line.lower() for line in note.body] everything = " ".join([title, tags, *body]) if not terms or any(term not in everything for term in terms): return None score = 0 for term in terms: if term in title: score += TITLE_POINTS if term in tags: score += TAG_POINTS for line in body: if term in line: score += HEADING_POINTS if line.startswith("#") else BODY_POINTS hits = [] for offset, (original, lowered) in enumerate(zip(note.body, body)): if any(term in lowered for term in terms): hits.append(Hit(note.body_start + offset, original.strip())) if len(hits) == MAX_HITS_PER_NOTE: break return Result(note, score, tuple(hits)) def search(notes: Iterable[Note], query: str, tag: str | None = None, limit: int = 10) -> list[Result]: """The best-scoring notes first, ties broken by file path.""" terms = split_query(query) wanted = tag.lower() if tag else None results = [] for note in notes: if wanted and wanted not in note.tags: continue result = score_note(note, terms) if result: results.append(result) return heapq.nsmallest(limit, results, key=lambda r: (-r.score, str(r.note.path))) ``` The point values are named constants, so tuning the ranking later means changing a number and not hunting through the logic. `score_note` is a **pure function**: it takes a note and some words, and returns a result or `None`. No files, no printing, so it's trivial to test. `heapq.nsmallest` picks the top results without sorting everything (Post 16). Since it finds the _smallest_ items, the key negates the score, so the biggest score comes first, and ties fall back to the path string. Check the scores by hand against the sample output. For `generators`, the "Python generators" note earns 10 (title) + 5 (tag) + 3 (the heading line mentions it) + 1 (one body line) = **19**. The "Decorators" note mentions it once in its body, so it earns **1**. ## Step 3: a crash course in argparse The command-line part uses `argparse`, which comes with Python and builds argument parsing, error messages, and `--help` for you. It's new in this series, so here are the pieces we use: - **Positional arguments** (`query`) are required and identified by position. **Options** (`--dir`, `-t`) start with dashes and usually have defaults. - `type=Path` or `type=int` converts the text for you, and rejects bad values with a clear message. - `action="store_true"` makes a flag, like `--json`, that's `False` unless given. - `nargs="+"` means "one or more values", so `notes search streaming model` gives `query == ["streaming", "model"]`. - **Subcommands**, like `git commit` and `git push`, come from `add_subparsers`. - A **parent parser** holds options shared by several subcommands. Without it, `--dir` would have to come _before_ the subcommand, and `notes search x --dir y` would be an error. - `parse_args(argv)` reads `sys.argv` by default, but accepts a list, which is exactly what tests need. - On bad usage, `argparse` prints the usage line to stderr and exits with code **2**. ## Step 4: the command line itself Create `src/notesearch/cli.py`: ```python import argparse import difflib import json import logging import os import sys from collections import Counter from pathlib import Path from notesearch import __version__ from notesearch.notes import Note, iter_notes from notesearch.search import Result, search def build_parser() -> argparse.ArgumentParser: common = argparse.ArgumentParser(add_help=False) common.add_argument("-d", "--dir", type=Path, default=Path("."), help="folder of notes (default: the current folder)") common.add_argument("-v", "--verbose", action="store_true", help="show which files are read") parser = argparse.ArgumentParser(prog="notes", description="Search and browse a folder of markdown notes.") parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}") commands = parser.add_subparsers(dest="command", required=True, metavar="command") find = commands.add_parser("search", parents=[common], help="find notes that contain all the given words") find.add_argument("query", nargs="+", help="words to look for (all of them must match)") find.add_argument("-t", "--tag", help="only look at notes with this tag") find.add_argument("-n", "--limit", type=int, default=10, help="most notes to show (default: 10)") find.add_argument("--json", action="store_true", help="print JSON instead of text") listing = commands.add_parser("list", parents=[common], help="list notes") listing.add_argument("-t", "--tag", help="only notes with this tag") listing.add_argument("--json", action="store_true", help="print JSON instead of text") commands.add_parser("stats", parents=[common], help="count notes, words, and tags") return parser def relative(note: Note, root: Path) -> str: return str(note.path.relative_to(root)) def result_to_dict(result: Result, root: Path) -> dict: return { "path": relative(result.note, root), "title": result.note.title, "tags": list(result.note.tags), "score": result.score, "hits": [{"line": hit.line_number, "text": hit.text} for hit in result.hits], } def print_results(results: list[Result], root: Path, as_json: bool) -> None: if as_json: print(json.dumps([result_to_dict(r, root) for r in results], indent=2)) return for rank, result in enumerate(results, start=1): tags = f" [{', '.join(result.note.tags)}]" if result.note.tags else "" print(f"{rank}. {result.note.title}{tags} (score {result.score})") for hit in result.hits: print(f" {relative(result.note, root)}:{hit.line_number}: {hit.text}") def report_unknown_tag(tag: str, notes: list[Note]) -> int: known = sorted({t for note in notes for t in note.tags}) message = f"notes: no notes are tagged '{tag}'." suggestions = difflib.get_close_matches(tag.lower(), known) if suggestions: message += f" Did you mean: {', '.join(suggestions)}?" print(message, file=sys.stderr) return 1 def run_search(args: argparse.Namespace, root: Path) -> int: if args.limit < 1: print("notes: error: --limit must be at least 1", file=sys.stderr) return 2 notes = list(iter_notes(root)) if args.tag and not any(args.tag.lower() in note.tags for note in notes): return report_unknown_tag(args.tag, notes) results = search(notes, " ".join(args.query), tag=args.tag, limit=args.limit) print_results(results, root, args.json) if not results and not args.json: print("No notes matched.", file=sys.stderr) return 0 if results else 1 def run_list(args: argparse.Namespace, root: Path) -> int: notes = list(iter_notes(root)) if args.tag and not any(args.tag.lower() in note.tags for note in notes): return report_unknown_tag(args.tag, notes) if args.tag: notes = [note for note in notes if args.tag.lower() in note.tags] if args.json: rows = [{"path": relative(n, root), "title": n.title, "tags": list(n.tags)} for n in notes] print(json.dumps(rows, indent=2)) else: for note in notes: tags = f" [{', '.join(note.tags)}]" if note.tags else "" print(f"{relative(note, root)} {note.title}{tags}") return 0 def run_stats(args: argparse.Namespace, root: Path) -> int: note_count = 0 word_count = 0 tag_counts: Counter[str] = Counter() for note in iter_notes(root): # one note at a time: nothing is kept in memory note_count += 1 word_count += note.word_count tag_counts.update(note.tags) print(f"notes: {note_count}") print(f"words: {word_count}") if tag_counts: print("tags:") for tag, count in tag_counts.most_common(): print(f" {tag}: {count}") return 0 def main(argv: list[str] | None = None) -> int: args = build_parser().parse_args(argv) logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING, format="%(levelname)s: %(message)s") root: Path = args.dir if not root.is_dir(): print(f"notes: error: {root} is not a folder", file=sys.stderr) return 2 handlers = {"search": run_search, "list": run_list, "stats": run_stats} try: return handlers[args.command](args, root) except BrokenPipeError: # The reader (for example `head`) stopped early. That's normal for a pipe, so exit quietly. os.dup2(os.open(os.devnull, os.O_WRONLY), sys.stdout.fileno()) return 0 if __name__ == "__main__": sys.exit(main()) ``` Several decisions here are about being a good citizen on the command line: - `main` **returns an int, and the entry point exits with it.** The generated `notes` command effectively runs `sys.exit(main())`. Returning codes instead of calling `sys.exit` all over the place keeps `main` easy to test. - **Messages go to stderr, results to stdout.** "No notes matched." is printed to stderr in text mode, so `notes search x | something` receives an empty stream, not a sentence. In JSON mode the tool prints `[]` and still exits with 1. - **The unknown-tag message uses** `difflib`, the "did you mean?" idea from Post 17. A typo like `--tag pyton` gets a helpful reply instead of a silent empty result. - `search` **and** `list` **keep the notes in a list**, because they need two passes (check the tag, then search). `stats` streams them one at a time. - **The** `BrokenPipeError` **handler came from a bug I hit myself.** I tested `notes list --json | head -8`, and `head` closed the pipe after eight lines, which made Python's next `print` raise `BrokenPipeError`, with a long traceback. That's not an error in your program. It's the reader saying "that's enough." The fix redirects stdout to `/dev/null` so Python's exit cleanup doesn't complain, and exits quietly. After the fix, `notes list --json | head -2` printed two lines, nothing on stderr, and both programs exited with code 0. Here's the help text `argparse` generated for us, for free (the exact layout can vary by Python version): ```sql $ notes --help usage: notes [-h] [--version] command ... Search and browse a folder of markdown notes. positional arguments: command search find notes that contain all the given words list list notes stats count notes, words, and tags options: -h, --help show this help message and exit --version show program's version number and exit ``` ## Step 5: tests The tests use a fixture that builds a small notes folder in a temporary directory (Post 13), so no real files are touched. Create `tests/test_notesearch.py`: ```python import json from pathlib import Path import pytest from notesearch.cli import main from notesearch.notes import iter_notes, parse_front_matter, parse_note, parse_tags from notesearch.search import score_note, search, split_query def write(path: Path, text: str) -> None: path.parent.mkdir(parents=True, exist_ok=True) path.write_text(text, encoding="utf-8") @pytest.fixture def notes_dir(tmp_path): write(tmp_path / "python" / "generators.md", "---\ntitle: Python generators\ntags: [python, generators]\n---\n" "# Python generators\n\nA generator yields values lazily.\nStreaming replies use the same idea.\n") write(tmp_path / "python" / "decorators.md", "---\ntitle: Decorators\ntags: python\n---\n" "A decorator wraps a function.\nIt is similar to how generators wrap a loop.\n") write(tmp_path / "ai" / "rag.md", "---\ntitle: RAG basics\ntags: [ai, search]\n---\n" "Retrieval augmented generation looks up documents first.\n") write(tmp_path / "ideas.md", "# Project ideas\n\n- A streaming chat UI\n") write(tmp_path / ".hidden" / "secret.md", "generators streaming secret\n") return tmp_path # ---- parsing ---- def test_front_matter_is_parsed(): values, used = parse_front_matter(["---", "title: A", "tags: [x]", "---", "body"]) assert values == {"title": "A", "tags": "[x]"} assert used == 4 @pytest.mark.parametrize("lines", [ ["no front matter"], ["---", "title: A", "body"], # the closing marker is missing [], ]) def test_text_without_a_complete_front_matter_block_is_plain_text(lines): assert parse_front_matter(lines) == ({}, 0) @pytest.mark.parametrize("raw, expected", [ ("[a, B, a]", ("a", "b")), ("python, generators", ("python", "generators")), ("['x', \"y\"]", ("x", "y")), ("", ()), ]) def test_parse_tags(raw, expected): assert parse_tags(raw) == expected def test_title_falls_back_to_the_first_heading_then_the_file_name(): assert parse_note(Path("a/x.md"), "---\ntitle: Given\n---\n# Heading").title == "Given" assert parse_note(Path("a/x.md"), "intro\n# The Heading\nmore").title == "The Heading" assert parse_note(Path("a/untitled.md"), "plain text only").title == "untitled" def test_line_numbers_count_the_front_matter(): note = parse_note(Path("x.md"), "---\ntitle: T\ntags: a\n---\nfirst\nsecond needle\n") assert note.body_start == 5 result = score_note(note, ["needle"]) assert result is not None assert [hit.line_number for hit in result.hits] == [6] def test_hidden_folders_are_skipped(notes_dir): names = [note.path.name for note in iter_notes(notes_dir)] assert "secret.md" not in names assert sorted(names) == ["decorators.md", "generators.md", "ideas.md", "rag.md"] # ---- searching ---- def test_every_term_must_match(): note = parse_note(Path("x.md"), "alpha beta") assert score_note(note, ["alpha", "beta"]) is not None assert score_note(note, ["alpha", "gamma"]) is None assert score_note(note, []) is None def test_a_title_match_outranks_a_body_match(notes_dir): results = search(iter_notes(notes_dir), "generators") assert [r.note.title for r in results] == ["Python generators", "Decorators"] assert results[0].score > results[1].score def test_search_can_filter_by_tag(notes_dir): titles = [r.note.title for r in search(iter_notes(notes_dir), "generators", tag="Python")] assert titles == ["Python generators", "Decorators"] assert search(iter_notes(notes_dir), "generators", tag="ai") == [] def test_limit_and_tie_order(notes_dir): # 'streaming' appears once in ideas.md and once in generators.md: equal scores, ordered by path both = search(iter_notes(notes_dir), "streaming") assert [r.score for r in both] == [1, 1] assert [r.note.path.name for r in both] == ["ideas.md", "generators.md"] assert len(search(iter_notes(notes_dir), "streaming", limit=1)) == 1 def test_split_query_lowercases_and_splits(): assert split_query(" Generators STREAMING ") == ["generators", "streaming"] # ---- the command line ---- def test_search_prints_the_best_note_first(notes_dir, capsys): code = main(["search", "generators", "--dir", str(notes_dir)]) out = capsys.readouterr().out assert code == 0 assert out.splitlines()[0].startswith("1. Python generators") assert "python/generators.md:" in out def test_json_output_is_valid_json(notes_dir, capsys): code = main(["search", "retrieval", "-d", str(notes_dir), "--json"]) data = json.loads(capsys.readouterr().out) assert code == 0 assert data[0]["title"] == "RAG basics" assert data[0]["tags"] == ["ai", "search"] assert data[0]["hits"][0]["line"] == 5 def test_no_results_gives_exit_code_1_and_keeps_stdout_clean(notes_dir, capsys): code = main(["search", "quantum", "-d", str(notes_dir)]) captured = capsys.readouterr() assert code == 1 assert captured.out == "" assert "No notes matched" in captured.err def test_a_missing_folder_gives_exit_code_2(tmp_path, capsys): code = main(["search", "x", "-d", str(tmp_path / "nope")]) assert code == 2 assert "is not a folder" in capsys.readouterr().err def test_an_unknown_tag_suggests_the_closest_one(notes_dir, capsys): code = main(["search", "generators", "--tag", "pyton", "-d", str(notes_dir)]) assert code == 1 assert "Did you mean: python?" in capsys.readouterr().err def test_stats_counts_notes_and_tags(notes_dir, capsys): assert main(["stats", "-d", str(notes_dir)]) == 0 out = capsys.readouterr().out assert "notes: 4" in out assert " python: 2" in out def test_a_command_is_required(): with pytest.raises(SystemExit) as excinfo: main([]) assert excinfo.value.code == 2 ``` Count what these cover: the parser and its edge cases (including text with no closing marker), the line-number arithmetic, hidden folders, the AND rule, ranking, tag filters, tie ordering, and every exit code, with both text and JSON output. The CLI tests call `main([...])` directly with a list, and read what it printed with the `capsys` fixture, so there's no subprocess and no real terminal. The suite has 18 test functions, which expand to 23 test cases because of `parametrize`, and all 23 passed. Then came the check that matters, the one from Post 13: can the tests fail? I broke the code three ways, one at a time: | Deliberate bug | Tests that failed | | --- | --- | | Line numbers off by one | 2 (the line-number test and the JSON test) | | Match **any** word instead of **all** | 1 (`test_every_term_must_match`) | | Exit with 0 even when nothing is found | 1 (the no-results test) | After restoring the code, all 23 passed again. (My first attempt at this check gave a confusing result, because Python had cached the old compiled version of a file whose size hadn't changed. If you ever see a "fixed" bug still failing, clear the `__pycache__` folders.) ## Step 6: install it and use it Install the project in editable mode, so edits take effect immediately, along with the dev tools: ```bash pip install -e ".[dev]" pytest which notes ``` (In my offline test environment, I installed without the `dev` extras and used the stand-in runner for the tests. Your `pytest` run should end with 23 passed.) Now try it. Searching with two words requires both: ```sql $ notes search streaming model -d sample-notes 1. RAG basics [ai, search] (score 2) ai/rag.md:6: Chunking and embeddings decide what the model sees. ai/rag.md:7: Streaming the answer makes the app feel fast. 2. Python generators [python, generators] (score 2) python/generators.md:8: Streaming replies from a model use the same idea. ``` Both notes score 2, so the tie is broken alphabetically by path. Filter by tag, and ask for JSON: ```typescript $ notes search retrieval -d sample-notes --json [ { "path": "ai/rag.md", "title": "RAG basics", "tags": ["ai", "search"], "score": 1, "hits": [ {"line": 5, "text": "Retrieval augmented generation looks up documents first."} ] } ] ``` (The tool prints the arrays on separate lines with `indent=2`; I've compacted them here to save space.) Browse and count: ```yaml $ notes list -d sample-notes ai/rag.md RAG basics [ai, search] ideas.md Project ideas python/decorators.md Decorators [python] python/generators.md Python generators [python, generators] $ notes stats -d sample-notes notes: 4 words: 95 tags: python: 2 ai: 1 search: 1 generators: 1 ``` Note that the hidden `.hidden/secret.md` appears in none of these. And the failure cases behave the way the table promised: | Command | What happens | Exit code | | --- | --- | --- | | `notes search quantum -d sample-notes` | prints `No notes matched.` to stderr | 1 | | `notes search generators --tag pyton -d sample-notes` | `no notes are tagged 'pyton'. Did you mean: python?` | 1 | | `notes search x -d does-not-exist` | `error: does-not-exist is not a folder` | 2 | | `notes search x -n 0 -d sample-notes` | `error: --limit must be at least 1` | 2 | | `notes search x -n many -d sample-notes` | `argument -n/--limit: invalid int value: 'many'` | 2 | | `notes` (no command) | usage message, `the following arguments are required: command` | 2 | Because the tool follows the conventions, it composes with everything else you know from the shell. Use the exit code in a script: ```bash if notes search quantum -d sample-notes 2>/dev/null; then echo "found" else echo "nothing found" fi # nothing found ``` Or feed the JSON to another program: ```bash notes search generators -d sample-notes --json | python3 -c "import json,sys; print([r['title'] for r in json.load(sys.stdin)])" # ['Python generators', 'Decorators'] ``` And `-v` shows what's happening under the hood, through the `logging` module, on stderr: ```bash $ notes stats -d sample-notes -v DEBUG: read sample-notes/ai/rag.md DEBUG: read sample-notes/ideas.md DEBUG: read sample-notes/python/decorators.md DEBUG: read sample-notes/python/generators.md notes: 4 words: 95 ... ``` ## Step 7: a clean history Don't commit everything in one lump at the end. Commit as you build, with messages that say what changed (Post 22). The history I made while building this was: ```sql c1f1bb0 Add tests for parsing, search, and the CLI 2fa7cda Add command line interface with search, list, and stats 8775e14 Add scoring and search with tag filter 302e7fc Parse markdown notes with simple front matter 3e14d32 Set up project skeleton with pyproject and README ``` `git status` was clean afterward, and `.gitignore` kept `.venv/` out. Create a repository on GitHub, push it, and add the project to your `ai-engineer-journey` repo as a pull request, as in Post 22's mini project. ## Where to take it next 1. **Add a real search index.** SQLite has a built-in full-text search engine called FTS5, and it's already in Python's `sqlite3` (Post 21). It ranks results with a proper algorithm and handles prefixes. Here's a working sketch that indexes the sample notes: ```python import sqlite3 from pathlib import Path from notesearch.notes import iter_notes root = Path("sample-notes") conn = sqlite3.connect(":memory:") conn.execute("CREATE VIRTUAL TABLE notes USING fts5(path, title, body)") for note in iter_notes(root): conn.execute("INSERT INTO notes VALUES (?, ?, ?)", (str(note.path.relative_to(root)), note.title, "\n".join(note.body))) rows = conn.execute( "SELECT path, title, snippet(notes, 2, '[', ']', '...', 6) " "FROM notes WHERE notes MATCH ? ORDER BY bm25(notes)", ("generators",), ).fetchall() ``` For `generators` it returned the generators note first, then the decorators note, with the matches wrapped in brackets. Searching `gener*` also found "generation" in the RAG note. Save the index to a file, and you can skip re-reading every note on each run. 2. **Color the matches** when output goes to a terminal (`sys.stdout.isatty()`), and keep plain text when it's piped. 3. **Add** `notes new "Title" --tags a,b`, which creates a note with front matter already filled in. 4. **Read a default folder from an environment variable** such as `NOTES_DIR`, so you don't need `--dir` every time. 5. **Support** `"exact phrases"` and a `--regex` option. 6. **Add a type checker and a linter** (Post 24), and run them with `pre-commit` (Post 22). 7. **Try it on your own notes**, and tune the point values until the ranking feels right. This is the real test. 8. **Add semantic search.** Searching by _meaning_, so "memory-efficient iteration" finds the generators note even though it doesn't contain those words, uses embeddings. You'll build that in Post 52, and this project is a fine place to plug it in. ## A definition of done Before you call it finished, check each of these: - Every command works with `--help`, and the README shows how to install and use the tool. - Errors go to stderr, results go to stdout, and exit codes are 0, 1, and 2 as promised. - The tests pass, and at least one of them fails when you break the code on purpose. - The code is formatted consistently, with no leftover debug prints. - Your history reads like a story, and the project is on GitHub. ## What to take away - A command-line tool is a function with a good front door. Keep parsing, scoring, and printing apart, and the logic becomes easy to test. - `argparse` gives you subcommands, type conversion, errors, and `--help` for almost nothing. Return an exit code from `main`. - Follow the conventions: results to stdout, messages to stderr, 0 for success, 1 for "nothing found", 2 for bad usage, and a `--json` option for other programs. - Be forgiving with messy input, and keep going when one file is unreadable. - Handle `BrokenPipeError`, so `| head` doesn't print a traceback. - Generators let a tool handle a folder of any size, one file at a time. - A test that can't fail is worth nothing. Break the code on purpose to find out. * * * **Previous:** Post 26, _Intro to C++/Rust_ **Next in the series:** Post 28, _Mini Project 2_, where we build a REST API with tests, Docker, and continuous integration.