Most Rust learning resources teach concepts one at a time. I learn faster by building something and getting stuck.

Why a JSON parser

A JSON tokenizer sounds like a toy problem. It's not. JSON parsing sits at the core of nearly every web service, and the libraries that do it well are heavily optimized. serde is one of the most downloaded crates in the Rust ecosystem, and Pydantic v2 rewrote its validation core in Rust for performance.

Building one from scratch forces you into real decisions immediately: how do you iterate through characters, when do you consume vs peek, how do you handle state transitions between token types. These aren't hypothetical questions, they're design choices with consequences you discover as soon as you start testing.

The bug that taught pattern matching

Here's the first version of the number parser:

'-' | '0'..='9' => {
    let mut number_str = String::new();
    if ch == '-' {
        number_str.push(ch);
        chars.next();
    }
    while let Some(&c) = chars.peek() {
        if c.is_ascii_digit() || c == '.' {
            number_str.push(c);
            chars.next();
        } else {
            break;
        }
    }
    if let Ok(num) = number_str.parse::<f64>() {
        tokens.push(Token::Number(num));
    }
}

This handles -42 and 0.5 correctly. All tests green. Then I tested 1.2.3:

Input: "1.2.3"

The loop consumes every digit and every dot:
  '1' → digit, push
  '.' → dot, push
  '2' → digit, push
  '.' → dot, push   ← no guard against a second dot
  '3' → digit, push

number_str = "1.2.3"
parse::<f64>() → Err(...)  ← invalid float, token silently dropped!

Notice the if let Ok(num) — when parse fails, the token is silently dropped. No error, no warning. We'll come back to that problem in a future article.

The condition c.is_ascii_digit() || c == '.' treats every dot the same. It has no memory of whether it already saw one. The fix adds a has_dot flag:

let mut has_dot = false;
while let Some(&c) = chars.peek() {
    if c.is_ascii_digit() {
        number_str.push(c);
        chars.next();
    } else if c == '.' && !has_dot {
        has_dot = true;
        number_str.push(c);
        chars.next();
    } else {
        break;
    }
}

Now 1.2.3 stops consuming at the second dot, producing 1.2 — and the .3 gets handled by the catch-all arm. A small state variable, but you only discover you need it when a test case forces the issue.

The toolchain teaches too

Exercises teach match syntax. A project teaches you to run cargo clippy and discover your match arms have redundant patterns. Exercises teach String vs &str. A project teaches you to read compiler errors that say "expected &str, found String" and understand what to change.

The actual learning loop looks like this:

cargo test                    # red
# read the error, think about why
# edit the code
cargo test                    # green (or red again)
cargo clippy -- -D warnings   # catch what you missed
cargo fmt                     # consistent style

That cycle of writing, breaking, reading the error, fixing, and linting builds a feedback loop that's tighter than reading a chapter and doing the exercises at the end. You remember the fix because you needed it.

Beyond exercises

Rustlings and the Rust Book are great resources for teaching concepts in isolation: one exercise for pattern matching, one for iterators, one for enums. Building a tokenizer forces you to combine all three in one function, and the interesting bugs live at the intersections.

String parsing, for example, needs to handle opening and closing quotes, escape sequences like \", and the decision to break or keep consuming. That's peek(), next(), and conditional logic all interacting inside one loop. Each concept is easy to grasp in isolation. The challenge is making them work together without losing track of where you are in the input.

Modules, project structure, test organization, Cargo.toml dependencies: none of these show up in isolated exercises, but they're the first things you deal with in a real project.

The tokenizer works — but it silently skips anything it doesn't recognize. That's fine until you build a parser on top of it and need to know why something failed. That's a problem worth solving — I'll cover it in my next Rust article.