--- title: "Intro to C++ and Rust: Just Enough to Read the Fast Code Under AI" author: "Pradeep Kumar" published: "2026-11-11T06:11:54.749Z" updated: "2026-11-11T06:11:54.749Z" url: "https://zyvop.com/intro-to-c-and-rust-just-enough-to-read-the-fast-code-under-ai-4af5t" canonical: "https://zyvop.com/intro-to-c-and-rust-just-enough-to-read-the-fast-code-under-ai-4af5t" tags: ["AI Engineer","c","Native Code","performance","Rust"] --- # Intro to C++ and Rust: Just Enough to Read the Fast Code Under AI Learn to read the C++ and Rust that powers NumPy, PyTorch, and tokenizers, see why native code isn’t automatically fast, and call C++ from Python yourself. ![Cover image]() Python is the language of AI, but it isn't the language doing the heavy lifting. Underneath NumPy, PyTorch, and many tools you'll use sits C, C++, Rust, and CUDA. You don't need to master them, but you should be able to read them. This post is optional, and it's about reading, not writing. By the end you'll recognize the main features of C++ and Rust code, understand why native code is fast (and when it isn't), and have called a C++ function from Python yourself. A note on what I ran. The C++ code was compiled with g++ 13 on Ubuntu 24.04, and the outputs and compiler messages are real. I had no Rust toolchain in my test environment, so the Rust snippets are standard idioms that I did _not_ compile. Treat them as reading material, and compile them yourself before trusting them. ## Why the fast parts aren't written in Python Back in Post 15, we saw that Python's flexibility costs speed, and that libraries get around it by doing the heavy work in compiled code while Python just gives the orders. Some well-known examples: | Project | What it is | Core written in | | --- | --- | --- | | NumPy | Arrays and math | C | | PyTorch | Deep learning | C++ (plus CUDA for GPUs) | | Hugging Face `tokenizers` | Fast tokenization | Rust | | llama.cpp | Running language models locally | C and C++ | | Polars | Data frames | Rust | | Pydantic (version 2) | Data validation | Rust core | | Ruff and uv | A linter, and a package manager | Rust | So when you hit a confusing bug or a performance mystery inside one of these, the answer often lives in code that isn't Python. Reading it won't be your daily job, but being _able_ to will set you apart. You'll rarely need to _write_ it. The ladder from Post 15 still applies: a better algorithm first, then a library that already does the job, then NumPy vectorization, and only then native code. As you'll see below, even that last step can disappoint. ## What's different about C++ and Rust Three differences explain almost everything that feels strange: 1. **They're compiled.** You don't run the source directly. A compiler translates it into machine code first, and it refuses to build if it finds certain mistakes. That's why a type error stops you _before_ the program runs. 2. **Types are fixed, and written out.** Every variable has a declared type (`int`, `float`, `std::string`), and it can't change. 3. **You think about memory.** In Python, memory is looked after for you (Post 15). In C++ and Rust you decide who owns data, and who may look at it or change it. | | Python | C++ | Rust | | --- | --- | --- | --- | | Typing | Dynamic | Static | Static | | Memory | Automatic | You manage it (modern C++ helps a lot) | The compiler enforces rules | | Mistakes like out-of-range access | Raise an error | Can silently misbehave | Caught, or panic safely | | Build tool | none / `pip`, `uv` | `g++`, `make`, CMake | `cargo` | | Learning curve | Gentle | Wide and old, with a lot of history | Steep at first, strict compiler | ## Reading C++ Here's the cycle. You write `file.cpp`, compile it, and run the result: ```bash g++ -std=c++20 -Wall -Wextra -O2 basics.cpp -o basics ./basics ``` `-Wall -Wextra` turns on helpful warnings, and `-O2` turns on optimization, which matters a great deal (more on that below). Here's a program that packs in most of the syntax you'll meet when reading AI code. Skim it first, and then I'll explain the cues: ```cpp #include #include #include #include #include struct Message { std::string role; std::string content; int tokens = 0; }; int estimate_tokens(const std::string& text) { return static_cast(text.size() + 3) / 4; // roughly 4 characters per token } void add_one(int& x) { // a reference: changes the caller's variable x += 1; } template T sum(const std::vector& values) { T total = 0; for (const T& v : values) { total += v; } return total; } int main() { int count = 3; double price = 0.5; std::string name = "Asha"; std::cout << "Hello, " << name << "! count=" << count << " price=" << price << "\n"; std::vector numbers = {1, 2, 3, 4}; numbers.push_back(5); std::cout << "size=" << numbers.size() << " first=" << numbers[0] << " sum=" << sum(numbers) << "\n"; std::cout << "float sum=" << sum(std::vector{0.5f, 1.5f}) << "\n"; add_one(count); int* p = &count; // a pointer: the address of count *p = *p + 10; // follow the pointer std::cout << "count after add_one and pointer: " << count << "\n"; std::vector words = {"the", "cat", "the", "dog"}; std::unordered_map counts; for (const auto& word : words) { counts[word] += 1; } std::cout << "the=" << counts["the"] << " cat=" << counts["cat"] << "\n"; Message m{"user", "How do generators work?"}; m.tokens = estimate_tokens(m.content); std::cout << m.role << ": " << m.content << " (" << m.tokens << " tokens)\n"; struct Doc { std::string id; float score; }; std::vector docs = {{"a", 0.31f}, {"b", 0.92f}, {"c", 0.55f}}; std::sort(docs.begin(), docs.end(), [](const Doc& x, const Doc& y) { return x.score > y.score; }); for (const auto& d : docs) { std::cout << d.id << ":" << d.score << " "; } std::cout << "\n"; try { std::vector small = {1, 2, 3}; std::cout << small.at(10) << "\n"; } catch (const std::out_of_range& e) { std::cout << "caught: " << e.what() << "\n"; } return 0; } ``` It prints: ```bash Hello, Asha! count=3 price=0.5 size=5 first=1 sum=15 float sum=2 count after add_one and pointer: 14 the=2 cat=1 user: How do generators work? (6 tokens) b:0.92 c:0.55 a:0.31 caught: vector::_M_range_check: __n (which is 10) >= this->size() (which is 3) ``` ### The reading cues You don't need to write any of that from memory. You need to recognize what you're looking at: | You see | It means | | --- | --- | | `#include ` | Like `import`: pull in a library header | | `std::` | "From the standard library," like a module prefix. `std::string`, `std::vector` | | `int count = 3;` | The **type comes first**, then the name, and every statement ends with `;` | | `{ ... }` | Blocks use braces, not indentation | | `std::vector` | A list of ints. The `<...>` is a **template** parameter, like Python's `list[int]` | | `template ` | A function written once that works for any type `T`. Generics (Post 25) | | `const` | "This won't be changed." Everywhere, and a good sign | | `int&` / `const std::string&` | A **reference**: another name for the same variable, with no copy | | `int* p = &count;` | A **pointer**. `&count` is "the address of count," and `*p` is "the thing it points to" | | `p->field` | The same as `(*p).field`: reach through a pointer | | `auto` | "Let the compiler work out the type" | | `[](...) { ... }` | A **lambda**, an inline function like Python's `lambda`, but with a full body | | `for (const auto& x : items)` | Python's `for x in items`, without copying each item | | `std::cout << ...` | Print. Think of it as `print` | The reference rule is the one worth understanding properly, because it's all about the copying trap from Post 5. By default, C++ passes arguments **by value**, meaning it copies them. Passing a million-float vector by value copies a million floats. So you'll see `const std::vector&` constantly: "a reference to the vector, which I promise not to change," with no copy and no cost. And a non-`const` reference like `int& x` is how a function changes its caller's variable, as `add_one` did above. ### Memory and the danger of silence You rarely see raw memory management in modern C++. A `std::vector` owns its memory and frees it automatically when it goes out of scope. But C++ still lets you do dangerous things quietly. If you write `small[10]` on a 3-item vector, there's no error. It reads whatever happens to be in memory at that spot: garbage, or a crash, or something that looks right and isn't. That's called **undefined behavior**, and it's the source of countless bugs and security problems. The checked version, `.at(10)`, raises an exception, which is what the last lines of the program above demonstrate. I'm not going to run the dangerous version for you, because the whole point is that its behavior can't be predicted. Remember it when you read C++: where Python would raise an `IndexError`, C++ may not tell you anything went wrong. ### What compiler errors look like The compiler is strict, and its messages can be long, but the first error usually has the answer. Two real ones: ```python broken.cpp:3:17: error: invalid conversion from 'const char*' to 'int' [-fpermissive] 3 | int count = "three"; ``` You tried to put text into an `int`. And a missing semicolon: ```cpp broken2.cpp:3:22: error: expected ';' before 'return' 3 | std::cout << "hi" ``` Read the first error, fix it, and recompile. Later errors are often just fallout from the first one. ## Reading Rust Rust has the same speed goals as C++, with a very different way of staying safe. I'll keep this short, and remember that I haven't compiled these snippets. Here's the dot product and cosine similarity from Post 15 in Rust: ```rust fn dot(a: &[f32], b: &[f32]) -> f32 { a.iter().zip(b).map(|(x, y)| x * y).sum() } fn norm(v: &[f32]) -> f32 { dot(v, v).sqrt() } fn cosine(a: &[f32], b: &[f32]) -> f32 { dot(a, b) / (norm(a) * norm(b)) } fn main() { let a = vec![1.0, 2.0, 3.0]; let b = vec![4.0, 5.0, 6.0]; println!("dot = {}", dot(&a, &b)); println!("cosine = {:.4}", cosine(&a, &b)); } ``` That should print `dot = 32` and `cosine = 0.9746` (I checked those numbers separately in Python). Here's how to read it: | You see | It means | | --- | --- | | `fn dot(a: &[f32], b: &[f32]) -> f32` | A function. Names come first, then `: type`, and `-> f32` is the return type, like Python hints | | `&[f32]` | A **borrowed slice** of floats: a read-only view, with no copy | | `a.iter().zip(b).map(\|(x, y)\| x * y).sum()` | An **iterator chain**. Close to Python's `sum(x * y for x, y in zip(a, b))` | | `\|x\| x * 2` | A closure, Rust's lambda | | no `return`, no `;` on the last line | The last expression is the return value | | `let a = ...` / `let mut a = ...` | Variables are **immutable** unless you say `mut` | | `vec![1.0, 2.0]` | Build a `Vec`, Rust's growable list | | `println!(...)` | A macro (the `!` gives it away) | | `use std::collections::HashMap;` | Import. Rust's dictionary is `HashMap` | A few more features come up all the time: ```rust struct Message { role: String, content: String, tokens: Option, // a value that might be missing } impl Message { fn token_estimate(&self) -> u32 { self.tokens.unwrap_or((self.content.len() as u32 + 3) / 4) } } fn parse_port(text: &str) -> Result { let port: u16 = text.trim().parse()?; // ? means: on error, return it right away Ok(port) } ``` - `struct` plus `impl` is Rust's version of a class (Post 9): data in one block, methods in another. - `Option` replaces `None`. A value is either `Some(x)` or `None`, and the compiler _forces_ you to handle both. There's no equivalent of the `NoneType has no attribute` error from Post 11. - `Result` replaces exceptions. A function that can fail returns either `Ok(value)` or `Err(error)`, and `?` passes errors upward. - `match` is like a more powerful `if/elif` that must cover every case, much like the exhaustive `switch` from Post 25. ### Ownership: the idea behind Rust Rust's big idea is **ownership**. Every value has exactly one owner, and when the owner goes away, the value is freed. You can lend a value out with `&` (read-only, many at once) or `&mut` (writable, only one at a time). The compiler checks all of this _before_ the program runs, which is how Rust can be fast with no garbage collector and still avoid many memory bugs. Here's the famous error: ```rust let a = vec![1, 2, 3]; let b = a; // ownership MOVES to b println!("{:?}", a); // error: borrow of moved value: `a` ``` After `let b = a`, `a` is no longer valid. If you see an error like that, it isn't a bug in your logic. Rust is telling you that you used something after giving it away. The usual fixes are to borrow (`&a`) or to copy deliberately (`a.clone()`). It's the strictest part of the language, and the reason Rust feels steep at first. Rust projects use **Cargo**: `cargo new myproject` creates one, `cargo run` builds and runs, and `cargo build --release` produces the optimized version. (Like C++'s `-O2`, **release mode matters**: Rust's default debug builds can be many times slower.) ## The performance reality check Now the part that surprised me, and that I think matters most. Native code is fast, but "I wrote it in C++" does not mean "it's faster than what I had." I ran four experiments, all in C++ on my small test machine. Treat the numbers as examples, since yours will differ. ### 1\. Compiler flags matter a lot The same dot product of two 50-million-float arrays, compiled three ways: ```java -O0 149 ms -O2 51 ms -O3 -march=native 57 ms ``` Without optimization (`-O0`, the default if you forget), it's about three times slower than with `-O2`. Always benchmark with optimization on. ### 2\. The comparison with NumPy For the same dot product, a plain Python loop would take about **5 seconds** (I timed a million items at 0.1 seconds and scaled up). My C++ took about 51 ms, roughly a hundred times faster. And NumPy's `np.dot` took **24 ms**, which is _twice as fast as my C++_. That tells you something from Post 19: this job is **memory-bound**. There's very little math per number, so the limit is how fast data can be read from memory, and NumPy's version is tuned to do exactly that well. ### 3\. Fast and wrong When I first printed the C++ result, something looked off. Each dot product should be 50,000,000 × 3 = 150,000,000. The simple loop with a `float` accumulator gave: ```yaml expected: 150000000 float accumulator: 67108864 double accumulator: 150000000 NumPy float32 np.dot: 150000000.0 ``` The fast loop was **silently wrong**, and not by a little. The cause is Post 19's lesson on precision. A `float` has about seven digits of precision, so once the running total gets large, adding a small number to it changes nothing, and the sum stalls. Check: `float32` can't distinguish 16,777,217 from 16,777,216. Here's the fix, a `double` accumulator, in a function that otherwise reads the same: ```cpp float dot_float(const float* a, const float* b, int n) { float sum = 0.0f; for (int i = 0; i < n; ++i) sum += a[i] * b[i]; return sum; // wrong for long inputs } double dot_double(const float* a, const float* b, int n) { double sum = 0.0; for (int i = 0; i < n; ++i) sum += static_cast(a[i]) * b[i]; return sum; // correct } ``` NumPy got it right because it adds numbers in a smarter order. The lesson is the one from Post 15: **always check that the fast version gives the right answer.** Nothing crashed, nothing warned, and the timing looked great. ### 4\. Memory order matters, again Here's the memory-locality idea from Post 19, in C++. Multiplying two matrices takes three nested loops, and you can nest them in different orders. Both versions do exactly the same arithmetic: ```cpp // i-j-k order: B is read by jumping n elements at a time void matmul_ijk(const float* A, const float* B, float* C, int n) { for (int i = 0; i < n; ++i) for (int j = 0; j < n; ++j) { float sum = 0.0f; for (int k = 0; k < n; ++k) sum += A[i * n + k] * B[k * n + j]; C[i * n + j] = sum; } } // i-k-j order: B is read in order void matmul_ikj(const float* A, const float* B, float* C, int n) { for (int i = 0; i < n; ++i) for (int k = 0; k < n; ++k) { float a = A[i * n + k]; for (int j = 0; j < n; ++j) C[i * n + j] += a * B[k * n + j]; } } ``` For 768 × 768 matrices, the first took about **627 ms** and the second about **222 ms**, with exactly the same answer (the biggest difference was 0). Same work, three times faster, only because the second one walks through memory in order. And NumPy's `A @ B` took **6 ms**, about 100 times faster than the first version and 35 times faster than the second. Libraries like it use techniques such as splitting the matrices into blocks that fit in cache, and special instructions that process several numbers at once. This is why "just rewrite it in C++" so often disappoints, and why the heavy math in AI always goes through tuned libraries (and GPUs). The takeaway from all four: native code _can_ be much faster than Python, but speed comes from understanding memory, precision, and the hardware, not from the language. When a library already does the job, use it. ## Calling native code from Python So when _is_ it worth writing? When no library does exactly what you need, and the profile (Post 15) says that's where your time goes. Python can call compiled code in several ways, and I'll show the simplest, which needs nothing installed. The basic recipe: write the function with a plain **C interface** (`extern "C"`), build it into a **shared library** (a `.so` file on Linux, `.dylib` on macOS, `.dll` on Windows), and load it with Python's built-in `ctypes` module. Other tools do the same job with more comfort: `cffi`; `pybind11` and `nanobind` for C++; Cython; and for Rust, PyO3 with a build tool called maturin. (I haven't used those here, so check their documentation.) Two properties of `ctypes` are worth knowing. It does **no checking whatsoever**: it trusts that you passed the right types, as we'll see go wrong in a moment. And it releases Python's GIL (Post 14) while the C function runs, so several threads can run native code at the same time. ## Reading unfamiliar native code When you open a C++ or Rust file in an AI project, here's a field guide: - **Start from the edges.** Find the function that Python calls (the bindings) or `main`, and work inward. Read the **signatures** (names, parameters, return types) before any function bodies. They tell the story. - **Understand the data layout.** A tensor in native code is usually a pointer to numbers plus a **shape**, a **dtype**, and **strides** (how far to jump to reach the next row). That's the memory picture from Post 19. - **Find the hot loop.** Somewhere there's the loop that runs millions of times. Everything else is setup. - **Ignore the noise.** Heavy template syntax, macros, and long type names look scary. Skim them, and read the logic. - **Recognize GPU code.** A function marked `__global__` and launched with `<<>>` is a CUDA kernel. We'll look at those in Post 70. - **Use the tools you already have.** Build with debug symbols, add prints, read the tests, and use `git log -p` and `git blame` (Post 22) to see _why_ a line exists. On safety: C++ can have data races when threads share data, and out-of-range accesses that fail silently. Safe Rust rules out data races and out-of-range memory errors at compile time. An `unsafe` block in Rust means "the compiler can't check this, so I promise it's fine." In both languages, that's where to look first when something strange happens. ## Should you learn to write it? | If you... | Then | | --- | --- | | Build apps and pipelines on top of models | Reading ability is plenty. Stay in Python and TypeScript | | Hit a hot spot that profiling proves | Try a library, then NumPy, then Numba or Cython, before writing C++ or Rust | | Work on inference engines, custom GPU kernels, or tokenizers | Learn C++ (and CUDA), or Rust, properly. It's the job | | Want to write safe, fast tools and libraries | Rust is a great choice, and the ecosystem is growing | | Work on a project that's already C++ | Learn C++ | If you do pick one: C++ has an enormous existing codebase in ML and GPU work, so it's the one you'll most often _need_ to read. Rust has a gentler path to writing correct code once you get past the compiler's strictness, and it's a popular choice for newer tools. Both are long learning journeys. Neither is required to become an excellent AI engineer. ## Mini project: a C++ search function called from Python Let's do the full loop. We'll take the cosine search from Post 15, write the scoring in C++, build it as a shared library, call it from Python, check it against NumPy, and measure. Create `fastsearch.cpp`: ```cpp #include #include extern "C" { // scores[q * n_docs + d] = cosine similarity between query q and document d. // All arrays are float32, stored row by row. void cosine_scores(const float* docs, int n_docs, const float* queries, int n_queries, int dim, float* scores) { std::vector doc_norms(n_docs); for (int d = 0; d < n_docs; ++d) { double sum = 0.0; for (int i = 0; i < dim; ++i) { double x = docs[d * dim + i]; sum += x * x; } doc_norms[d] = std::sqrt(sum); } for (int q = 0; q < n_queries; ++q) { const float* query = queries + q * dim; double query_sum = 0.0; for (int i = 0; i < dim; ++i) { query_sum += static_cast(query[i]) * query[i]; } double query_norm = std::sqrt(query_sum); for (int d = 0; d < n_docs; ++d) { const float* doc = docs + d * dim; double dot = 0.0; for (int i = 0; i < dim; ++i) { dot += static_cast(doc[i]) * query[i]; } scores[q * n_docs + d] = static_cast(dot / (doc_norms[d] * query_norm)); } } } } // extern "C" ``` Notice what we learned going in: `double` accumulators, so the sums stay accurate, and all arrays passed as plain pointers with their sizes as separate arguments. That's the "pointer plus shape" layout from the field guide. The `extern "C"` block gives the function a plain, unmangled name that Python can find. Build it into a shared library (`-shared -fPIC` makes a loadable library): ```bash g++ -std=c++20 -Wall -Wextra -O2 -shared -fPIC fastsearch.cpp -o libfastsearch.so ``` Now the Python side. Create `use_fastsearch.py`: ```python import ctypes import time import numpy as np lib = ctypes.CDLL("./libfastsearch.so") lib.cosine_scores.argtypes = [ ctypes.c_void_p, ctypes.c_int, # docs, n_docs ctypes.c_void_p, ctypes.c_int, # queries, n_queries ctypes.c_int, # dim ctypes.c_void_p, # scores (output) ] lib.cosine_scores.restype = None def cosine_scores_cpp(docs, queries): docs = np.ascontiguousarray(docs, dtype=np.float32) queries = np.ascontiguousarray(queries, dtype=np.float32) scores = np.empty((len(queries), len(docs)), dtype=np.float32) lib.cosine_scores( docs.ctypes.data, len(docs), queries.ctypes.data, len(queries), docs.shape[1], scores.ctypes.data, ) return scores def cosine_scores_numpy(docs, queries): docs = docs / np.linalg.norm(docs, axis=1, keepdims=True) queries = queries / np.linalg.norm(queries, axis=1, keepdims=True) return queries @ docs.T def best_time(func, *args, repeats=10): times = [] for _ in range(repeats): start = time.perf_counter() func(*args) times.append(time.perf_counter() - start) return min(times) rng = np.random.default_rng(0) for n_docs in (2_000, 100_000): docs = rng.standard_normal((n_docs, 128)).astype(np.float32) queries = rng.standard_normal((20, 128)).astype(np.float32) from_cpp = cosine_scores_cpp(docs, queries) from_numpy = cosine_scores_numpy(docs, queries) same_top3 = (np.argsort(-from_cpp, axis=1)[:, :3] == np.argsort(-from_numpy, axis=1)[:, :3]).all() t_cpp = best_time(cosine_scores_cpp, docs, queries, repeats=5) t_np = best_time(cosine_scores_numpy, docs, queries, repeats=5) print(f"{n_docs:>7,} docs: C++ {t_cpp * 1000:8.2f} ms | NumPy {t_np * 1000:8.2f} ms | " f"max difference {np.abs(from_cpp - from_numpy).max():.1e} | same top 3: {same_top3}") # the silent-mistake demo: hand ctypes the wrong dtype docs64 = rng.standard_normal((2_000, 128)) # float64 by default! queries64 = rng.standard_normal((20, 128)) wrong = np.empty((20, 2_000), dtype=np.float32) lib.cosine_scores(docs64.ctypes.data, 2_000, queries64.ctypes.data, 20, 128, wrong.ctypes.data) right = cosine_scores_numpy(docs64.astype(np.float32), queries64.astype(np.float32)) print("float64 passed straight in: max difference from the right answer =", f"{np.abs(wrong - right).max():.2f}") ``` Run `python use_fastsearch.py`. On my machine it printed: ```yaml 2,000 docs: C++ 3.81 ms | NumPy 0.52 ms | max difference 1.8e-07 | same top 3: True 100,000 docs: C++ 197.86 ms | NumPy 36.53 ms | max difference 2.4e-07 | same top 3: True float64 passed straight in: max difference from the right answer = nan ``` Three things to take from it. **It's correct.** The C++ scores match NumPy's to about one part in ten million, which is just `float32` rounding, and the top three results are identical for every query. **It's much faster than pure Python, and slower than NumPy.** In Post 15, pure Python took about 0.65 seconds for the 2,000-document case. The C++ version took 3.8 milliseconds, around 170 times faster. But NumPy took 0.5 milliseconds, about seven times faster than my C++. NumPy turns the whole job into one matrix multiplication handled by a heavily tuned library, while my version is straightforward loops. Compiler flags help (with `-O3 -march=native -ffast-math` the 100,000-document case dropped to about 75 ms, and with `-O0` it was about 765 ms), but `-ffast-math` lets the compiler rearrange floating-point math, so results can change slightly, and it still didn't catch NumPy. `ctypes` **checked nothing.** In the last test, I passed `float64` arrays straight in, without converting them to `float32`. There was no error and no crash. The function just read 4-byte floats out of memory that actually held 8-byte doubles, and produced garbage, printed as `nan`. That's why the wrapper function calls `np.ascontiguousarray(..., dtype=np.float32)` first: it guarantees the exact type and memory layout the C++ code expects. With raw pointers, the types are a promise _you_ keep. ### Make it yours 1. Write a test (Post 13) that compares `cosine_scores_cpp` with NumPy on random inputs, and that deliberately fails if you remove the dtype conversion. 2. Do the same work with the loop order changed, so the inner loop walks through memory in order. Does it help here? 3. Add top-k selection inside C++ with `std::partial_sort`, so Python only receives the top three document numbers. 4. Try `-fopenmp` to spread the work across cores (`#pragma omp parallel for` on the outer loop). It needs OpenMP support, so check your compiler. 5. Rewrite the `dot` functions in Rust, build them as a `cdylib`, and call them from Python with `ctypes`. 6. Look up `cblas_sgemm`. Calling a real BLAS library from your C++ is how you'd get NumPy-like speed. ## Practice About 20 minutes of real effort on each before asking for help. 1. Compile and run the C++ example, then change one thing at a time: pass `sum` a vector of `double`s, break a type on purpose, and read the compiler's error. 2. Read a small open-source C++ or Rust file (the source of a tokenizer or a small utility works well). Write down each function signature in plain English before reading any body. 3. Benchmark a function at `-O0`, `-O2`, and `-O3`. Explain the differences you see. 4. Reproduce the `float` versus `double` accumulator bug with your own numbers. At about what total does the `float` sum stall? 5. Change the matrix multiplication to a third loop order (j-k-i) and measure it. Predict first, and explain the result using the memory picture from Post 19. 6. Write a `C++` function that takes a `const std::vector&` and returns the largest value, and a version that takes the vector by value. What would you measure to show the difference? 7. Translate the Rust `Message` and `parse_port` snippets into Python, using `dataclass` and exceptions. What does the Rust version force you to handle? 8. Use `git log -p` and `git blame` on a file in an open-source native library, and find the commit that explains a surprising line. Then ask an AI tutor to review your answers, and don't let it rewrite them. ## What to take away - The fast parts of AI are mostly C, C++, Rust, and CUDA, and Python gives the orders. You'll read this code far more often than you write it. - C++ and Rust are compiled, statically typed, and make you think about memory. Learn the reading cues: `std::`, ``, `&`, `*`, `const`, `auto`, lambdas, and in Rust `fn`, `let mut`, `Option`, `Result`, `match`, and `?`. - Pass big things by reference. In C++, out-of-range access can fail silently, and Rust's ownership rules catch many of those mistakes before the program runs. - Always benchmark with optimization on, and always check that the fast version gives the right answer. A `float` accumulator silently produced the wrong total. - Your first C++ is probably not faster than NumPy. Speed comes from memory order, precision, and tuned libraries, not from the language. - `ctypes` plus `extern "C"` is the simplest bridge from Python, and it trusts you completely about types and memory layout. - Read native code from its edges: the bindings, then the signatures, the data layout, and the hot loop. * * * **Previous:** Post 25, _TypeScript for AI Engineers_ **Next in the series:** Post 27, _Mini Project 1_, where we build the first Phase 1 project: a command-line tool, such as a markdown note searcher.