Build a word counter
This page builds one small program from an empty folder: wordcount, a command-line tool that reads a text file, counts how often each word appears, and prints the most common ones. It uses most of the Ownership and Data parts of the course: ownership, borrowing, structs, enums, Option, Result, String and &str, Vec and HashMap.
Where one of those ideas decides how the code is written, a short note says which way it went and why, and links the lesson. The page does not re-teach the lessons.
RememberA &, a String, a clone and a move are each a decision. This page says which one was made, and why, wherever it shapes the design.
To follow along, run cargo new wordcount and copy each file into it as it appears (every code panel has a Copy button). Every file is shown in full, so there is nothing to download. Two small text files, below, go next to Cargo.toml. Every output is a real run in the course’s pinned image (Linux, Rust 1.99.1).
What we are building
Section titled “What we are building”wordcount [FILE] [--top N]- It reads
FILE, or standard input when there is no file (or the file is-). - A word is a run of letters and apostrophes. Case is ignored:
Theandtheare one word. - It prints the number of words, the number of distinct words, then the
Nmost common ones (10 by default). Words with the same count come out in code point order, which for English is alphabetical order. - On failure it prints a message on standard error and exits with a non-zero exit code, the number a program hands back to the shell. 0 means success.
We use only the standard library. Real tools use the clap crate for arguments (the clap page), but a generated parser would hide the Strings and Options this page is about. Save this as sample.txt:
The quick brown fox jumps over the lazy dog. The dog, being lazy, did not mind."Don't worry," said the fox. "I'm only passing through."The dog yawned; the fox left. Passing through is what foxes do.Was the dog worried? Not at all. Dogs don't worry about foxes.Several things here have a close cousin in the language you know. This table is the map; the notes do not repeat it.
| In this program | Java | Go | Python | C++ |
|---|---|---|---|---|
*counts.entry(k).or_insert(0) += 1 |
map.merge(k, 1, Integer::sum) |
m[k]++ (a missing key reads as 0) |
Counter, defaultdict(int) |
++m[k] ([] inserts a default) |
sort_by(...) with then_with(...) |
Comparator.comparing(...).thenComparing(...) |
sort.Slice with an if for ties |
sorted(key=(-n, w)) |
std::sort with a lambda |
? after a call that can fail |
an exception propagating, as with throws |
if err != nil { return err } |
an exception propagating | an exception, or an error code |
&str |
a String object, shared by reference |
a string: pointer and length | str |
std::string_view: pointer and length |
.clone() |
not needed: assigning a String copies a reference |
assigning a string copies its small header | not needed | the copy constructor, run silently on pass by value |
using counts after top_n(counts, n) took it |
no such error: both names share the map | no such error | no such error | std::move, and the moved-from object is still readable |
Step 1: read a file
Section titled “Step 1: read a file”$ cargo new wordcount
Creating binary (application) `wordcount` package
note: see more `Cargo.toml` keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
Replace src/main.rs with this. Cargo.toml stays as it is, with no dependencies.
use std::env;use std::fs;use std::io;
fn main() -> Result<(), io::Error> { let path = env::args().nth(1).expect("usage: wordcount FILE"); let text = fs::read_to_string(&path)?; println!( "{path}: {} bytes, {} lines", text.len(), text.lines().count() ); Ok(())}sample.txt: 264 bytes, 4 lines$ cargo run -- missing.txt
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }$ cargo runthread 'main' (58) panicked at src/main.rs:6:35: usage: wordcount FILE note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
env::args() gives the command line, one String per argument. The first is traditionally the program’s own name, so .nth(1) skips it and returns an Option<String>, None if no file was given. .expect(...) stops the program with its message on None (Option). fs::read_to_string returns a Result<String, io::Error>, and ? means “if this failed, return the error from main now” (Errors). That works because main itself returns a Result, much as a Java method declares throws IOException, except that here it is in the return type.
The two failing runs show a program with no real error handling. A missing file prints the error’s Debug form, which is meant for programmers, and never names the file. Forgetting the argument is a panic. Steps 6 to 8 fix both.
Step 2: split the text into words
Section titled “Step 2: split the text into words”A new file, src/words.rs, plus mod words; in main.rs so Rust knows the file exists (modules). pub makes the function visible outside its file; private is the default, so pub is Rust’s capital letter.
/// Splits text into words: runs of letters and apostrophes, with apostrophes/// trimmed off the ends. The words are slices of `text`, so nothing is copied.pub fn words(text: &str) -> Vec<&str> { let mut found = Vec::new(); for piece in text.split(|c: char| !(c.is_alphabetic() || c == '\'')) { let word = piece.trim_matches('\''); if !word.is_empty() { found.push(word); } } found}mod words;
use std::env;use std::fs;use std::io;
fn main() -> Result<(), io::Error> { let path = env::args().nth(1).expect("usage: wordcount FILE"); let text = fs::read_to_string(&path)?; let found = words::words(&text); let first = &found[..found.len().min(8)]; println!("{} words; the first eight: {first:?}", found.len()); Ok(())}49 words; the first eight: ["The", "quick", "brown", "fox", "jumps", "over", "the", "lazy"]
text.split(...) cuts the text wherever the closure says “this char is a separator”. A closure is a small unnamed function written inline (Closures). Ours says: anything that is not a letter or an apostrophe separates words. trim_matches('\'') removes apostrophes from the ends, so 'quoted' becomes quoted while don't keeps its own.
Unlike string_view, these slices cannot dangle. The compiler gives every reference a lifetime: how long the thing it points at stays alive. With one reference parameter, the references in the result get that parameter’s lifetime (Returning references). So this does not compile:
mod words;
fn main() { let found; { let text = String::from("the cat sat"); found = words::words(&text); } println!("{found:?}");}error[E0597]: `text` does not live long enough --> src/main.rs:7:30 | 4 | let found; | ----- variable `found` declared here 5 | { 6 | let text = String::from("the cat sat"); | ---- binding `text` declared here 7 | found = words::words(&text); | ^^^^^ borrowed value does not live long enough 8 | } | - `text` dropped here while still borrowed 9 | println!("{found:?}"); | ----- borrow later used here | = note: `found` is a collection that stores borrowed references, but `text` does not live long enough to be stored in it = help: buffer reuse with borrowed references requires unsafe code or restructuring
The equivalent C++ with a vector<string_view> compiles, and reads freed memory.
Optional: what counts as a letter, and where this breaks
char::is_alphabetic decides what is a word, so look at what it says. This side program (not part of wordcount) runs it on some characters, and words on some strings:
mod words;
fn main() { let chars = [ ('a', "LATIN SMALL LETTER A"), ('é', "LATIN SMALL LETTER E WITH ACUTE"), ('ß', "LATIN SMALL LETTER SHARP S"), ('你', "a Han character"), ('Ⅳ', "ROMAN NUMERAL FOUR"), ('٣', "ARABIC-INDIC DIGIT THREE"), ('5', "DIGIT FIVE"), ('-', "HYPHEN-MINUS"), ('\u{2019}', "RIGHT SINGLE QUOTATION MARK"), ('\u{301}', "COMBINING ACUTE ACCENT"), ('😀', "GRINNING FACE"), ]; for (c, name) in chars { println!( "U+{:04X} is_alphabetic: {:<5} {name}", c as u32, c.is_alphabetic() ); }
println!(); let texts = [ "state-of-the-art", "3rd place", "don't", "don\u{2019}t", "caf\u{e9}", "cafe\u{301}", "你好世界", "हिन्दी", "ก็ได้", ]; for text in texts { println!("{text:?} -> {:?}", words::words(text)); }
println!(); println!("lowercase of Straße: {}", "Straße".to_lowercase()); println!("lowercase of STRASSE: {}", "STRASSE".to_lowercase()); println!("lowercase of ISTANBUL: {}", "ISTANBUL".to_lowercase()); println!( "chars in the lowercase of İ (U+0130): {}", "İ".to_lowercase().chars().count() ); println!("lowercase of ΟΔΥΣΣΕΥΣ: {}", "ΟΔΥΣΣΕΥΣ".to_lowercase());}U+0061 is_alphabetic: true LATIN SMALL LETTER A
U+00E9 is_alphabetic: true LATIN SMALL LETTER E WITH ACUTE
U+00DF is_alphabetic: true LATIN SMALL LETTER SHARP S
U+4F60 is_alphabetic: true a Han character
U+2163 is_alphabetic: true ROMAN NUMERAL FOUR
U+0663 is_alphabetic: false ARABIC-INDIC DIGIT THREE
U+0035 is_alphabetic: false DIGIT FIVE
U+002D is_alphabetic: false HYPHEN-MINUS
U+2019 is_alphabetic: false RIGHT SINGLE QUOTATION MARK
U+0301 is_alphabetic: false COMBINING ACUTE ACCENT
U+1F600 is_alphabetic: false GRINNING FACE
"state-of-the-art" -> ["state", "of", "the", "art"]
"3rd place" -> ["rd", "place"]
"don't" -> ["don't"]
"don’t" -> ["don", "t"]
"café" -> ["café"]
"cafe\u{301}" -> ["cafe"]
"你好世界" -> ["你好世界"]
"हिन\u{94d}दी" -> ["हिन", "दी"]
"ก\u{e47}ได\u{e49}" -> ["ก", "ได"]
lowercase of Straße: straße
lowercase of STRASSE: strasse
lowercase of ISTANBUL: istanbul
chars in the lowercase of İ (U+0130): 2
lowercase of ΟΔΥΣΣΕΥΣ: οδυσσευς- Precomposed letters in Latin, Cyrillic and Greek count, and so do Han characters:
é,ß,你. So does the Roman numeralⅣ, which may surprise you: Rust tests Unicode’s Alphabetic property, and that includes it. - Digits do not count, so
3rdbecomesrd. Hyphens split words:state-of-the-artis four. - A typographic apostrophe (U+2019, which many word processors produce) is not the ASCII
', sodon’tbecomesdonandt. - A combining accent is a separate, non-alphabetic character.
cafétyped aseplus U+0301 loses its accent. Scripts whose words contain combining marks break the same way: the Hindi and Thai words above are split inside. - Chinese is written without spaces, so
你好世界is one “word” here. Splitting it properly needs a dictionary or a library. to_lowercasefollows Unicode but not any language’s rules.StraßebecomesstraßeandSTRASSEbecomesstrasse, so they count as two words.ISTANBULbecomesistanbul, which is wrong in Turkish, andİ(U+0130) lowercases to two characters. Greek is fine: the final sigma inΟΔΥΣΣΕΥΣcame out right.
In short: the program is reasonable for English and for text in precomposed Latin, Cyrillic and Greek. For anything else it needs more than the standard library, and should say so.
Step 3: count the words
Section titled “Step 3: count the words”We want a map from word to count. words gives &str, so the first thing most people try is a map keyed by those: HashMap<&str, usize>. Each word is also lowercased so that The and the match, and to_lowercase() returns a new String. counts.entry(key) looks the key up once and gives a handle to its slot. .or_insert(0) fills the slot with 0 if the word is new and returns a &mut usize, and *... += 1 adds one through it (Maps). Here is the first try:
use std::collections::HashMap;
use crate::words::words;
pub fn count_words(text: &str) -> HashMap<&str, usize> { let mut counts = HashMap::new(); for word in words(text) { let lower = word.to_lowercase(); *counts.entry(lower.as_str()).or_insert(0) += 1; } counts}error[E0515]: cannot return value referencing local variable `lower` --> src/count.rs:11:5 | 9 | *counts.entry(lower.as_str()).or_insert(0) += 1; | ----- `lower` is borrowed here 10 | } 11 | counts | ^^^^^^ returns a value referencing data owned by the current function
lower is created in each pass through the loop and dropped at the end of it, and the map would keep pointing at the freed text. The compiler reports it where the map leaves the function (E0515). This is the C++ unordered_map<string_view, ...> over a temporary string, caught at compile time. A &str key works only when the text it points into outlives the map.
The same loop in a function that does not return the map
Count the distinct words and return only a number, and the same mistake is reported at the borrow itself (E0597, “does not live long enough”):
use std::collections::HashMap;
use crate::words::words;
pub fn distinct_words(text: &str) -> usize { let mut counts = HashMap::new(); for word in words(text) { let lower = word.to_lowercase(); *counts.entry(lower.as_str()).or_insert(0) += 1; } counts.len()}error[E0597]: `lower` does not live long enough --> src/count.rs:9:23 | 8 | let lower = word.to_lowercase(); | ----- binding `lower` declared here 9 | *counts.entry(lower.as_str()).or_insert(0) += 1; | ^^^^^ borrowed value does not live long enough 10 | } | - `lower` dropped here while still borrowed 11 | counts.len() | ------ borrow later used here
The fix is to let the map own its keys: HashMap<String, usize>. Because the function returns the map, the caller owns it and it no longer depends on text. (HashMap::new() needs no type written: it comes from the return type.)
use std::collections::HashMap;
use crate::words::words;
/// Counts how often each word appears, ignoring case.pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { *counts.entry(word.to_lowercase()).or_insert(0) += 1; } counts}mod count;mod words;
use std::env;use std::fs;use std::io;
fn main() -> Result<(), io::Error> { let path = env::args().nth(1).expect("usage: wordcount FILE"); let text = fs::read_to_string(&path)?; let counts = count::count_words(&text); let total: usize = counts.values().sum(); println!("{total} words, {} distinct", counts.len()); println!( "\"the\" appears {} times", counts.get("the").copied().unwrap_or(0) ); Ok(())}49 words, 31 distinct "the" appears 7 times
In main, counts.get("the") accepts a &str although the keys are Strings: a map may be asked with any borrowed form of its key, which C++ calls heterogeneous lookup. It returns Option<&usize>, a reference to the number inside the map; .copied() copies the number out and unwrap_or(0) supplies 0 for a missing word.
Two more errors
Section titled “Two more errors”Using a String after it moved into the map. A debug print after the insert is all it takes:
use std::collections::HashMap;
use crate::words::words;
pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { let lower = word.to_lowercase(); *counts.entry(lower).or_insert(0) += 1; println!("counted {lower}"); } counts}error[E0382]: borrow of moved value: `lower` --> src/count.rs:10:28 | 8 | let lower = word.to_lowercase(); | ----- move occurs because `lower` has type `String`, which does not implement the `Copy` trait 9 | *counts.entry(lower).or_insert(0) += 1; | ----- value moved here 10 | println!("counted {lower}"); | ^^^^^ value borrowed here after move | help: consider cloning the value if the performance cost is acceptable | 9 | *counts.entry(lower.clone()).or_insert(0) += 1; | ++++++++
This is the least natural error here if you come from Java or Python, where a variable and a map share one string and both stay usable. In Rust the map took the String, so there is one owner. (C++ lets you read a moved-from string; Rust refuses. Ownership.) The help: offers .clone(), which would copy every word just for a print. Better: delete the print, or run it before the move.
Holding a reference into the map while changing it. Say you want to print the old count as you go:
use std::collections::HashMap;
use crate::words::words;
pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { let lower = word.to_lowercase(); let before = counts.get(&lower); counts.insert(lower, before.copied().unwrap_or(0) + 1); println!("{word}: was {before:?}"); } counts}error[E0502]: cannot borrow `counts` as mutable because it is also borrowed as immutable --> src/count.rs:10:9 | 9 | let before = counts.get(&lower); | ------ immutable borrow occurs here 10 | counts.insert(lower, before.copied().unwrap_or(0) + 1); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here 11 | println!("{word}: was {before:?}"); | ------ immutable borrow later used here
counts.get(&lower) returns a reference into the map, kept in before. The println! after the insert still uses it, so the map is borrowed while insert needs to change it: many readers or one writer, never both (Mutable borrows). The error is about holding the reference, not about calling get and then insert. Copy the number out first and it compiles:
use std::collections::HashMap;
use crate::words::words;
pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { let lower = word.to_lowercase(); let before = counts.get(&lower).copied().unwrap_or(0); counts.insert(lower, before + 1); } counts}49 words, 31 distinct "the" appears 7 times
It looks the key up twice, where entry looks once. C++’s unordered_map happens to keep references to its elements valid across inserts; Rust’s HashMap may move its entries when it grows, so a reference into it must not be held across a change.
Optional: what the String keys cost, measured
Every word allocates a String, and a word seen a thousand times drops 999 of them. Does that matter? This side program counts allocations by wrapping the system allocator in a counter. (GlobalAlloc is an unsafe trait because a broken allocator can break memory safety. This one forwards alloc, dealloc and realloc to the real allocator and counts the calls to alloc and realloc. You can skip reading it.) It builds the same map four ways from the sample repeated 1,000 times:
mod words;
use std::alloc::{GlobalAlloc, Layout, System};use std::collections::HashMap;use std::fs;use std::sync::atomic::{AtomicUsize, Ordering};
/// Passes every request on to the system allocator, and counts two kinds:/// new blocks (`alloc`) and blocks resized in place or moved (`realloc`).struct Counting;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);static REALLOCS: AtomicUsize = AtomicUsize::new(0);
unsafe impl GlobalAlloc for Counting { unsafe fn alloc(&self, layout: Layout) -> *mut u8 { ALLOCS.fetch_add(1, Ordering::Relaxed); unsafe { System.alloc(layout) } }
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { unsafe { System.dealloc(ptr, layout) } }
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 { REALLOCS.fetch_add(1, Ordering::Relaxed); unsafe { System.realloc(ptr, layout, new_size) } }}
#[global_allocator]static GLOBAL: Counting = Counting;
/// Runs `build`, and returns what it built with the number of `alloc` and/// `realloc` calls it made.fn measure<T>(build: impl FnOnce() -> T) -> (T, usize, usize) { let allocs = ALLOCS.load(Ordering::Relaxed); let reallocs = REALLOCS.load(Ordering::Relaxed); let built = build(); ( built, ALLOCS.load(Ordering::Relaxed) - allocs, REALLOCS.load(Ordering::Relaxed) - reallocs, )}
fn sorted<K: ToString>(counts: &HashMap<K, usize>) -> Vec<(String, usize)> { let mut all: Vec<(String, usize)> = counts.iter().map(|(k, n)| (k.to_string(), *n)).collect(); all.sort(); all}
fn main() { // 1,000 copies of the sample, built before anything is counted. let text = fs::read_to_string("sample.txt").unwrap().repeat(1000); let lower = text.to_lowercase(); let _warm_up: HashMap<String, usize> = HashMap::new();
// A: what count_words does. A new lowercase String for every word. let (a, a_allocs, a_reallocs) = measure(|| { let mut counts: HashMap<String, usize> = HashMap::new(); for word in words::words(&text) { *counts.entry(word.to_lowercase()).or_insert(0) += 1; } counts });
// B: lowercase the whole text once, and keep &str keys that point into it. let lower_once = measure(|| text.to_lowercase()); let (b, b_allocs, b_reallocs) = measure(|| { let mut counts: HashMap<&str, usize> = HashMap::new(); for word in words::words(&lower_once.0) { *counts.entry(word).or_insert(0) += 1; } counts }); let (b_allocs, b_reallocs) = (b_allocs + lower_once.1, b_reallocs + lower_once.2);
// C: owned keys, cloned for every word, even when the key is already in the map. let (c, c_allocs, c_reallocs) = measure(|| { let mut counts: HashMap<String, usize> = HashMap::new(); for word in words::words(&lower) { *counts.entry(word.to_string()).or_insert(0) += 1; } counts });
// D: owned keys, cloned only the first time a word is seen. let (d, d_allocs, d_reallocs) = measure(|| { let mut counts: HashMap<String, usize> = HashMap::new(); for word in words::words(&lower) { match counts.get_mut(word) { Some(n) => *n += 1, None => { counts.insert(word.to_string(), 1); } } } counts });
println!( "words: {}, distinct: {}", words::words(&text).len(), a.len() ); println!( "same counts in all four: {}", sorted(&a) == sorted(&b) && sorted(&a) == sorted(&c) && sorted(&a) == sorted(&d) ); println!(); println!(" alloc calls realloc calls"); let rows = [ ("A entry(word.to_lowercase())", a_allocs, a_reallocs), ("B one lowercase copy, &str keys", b_allocs, b_reallocs), ("C entry(word.to_string())", c_allocs, c_reallocs), ("D get_mut, insert(word.to_string())", d_allocs, d_reallocs), ]; for (name, allocs, reallocs) in rows { println!("{name:<38} {allocs:>11} {reallocs:>13}"); }}words: 49000, distinct: 31
same counts in all four: true
alloc calls realloc calls
A entry(word.to_lowercase()) 49006 14
B one lowercase copy, &str keys 7 14
C entry(word.to_string()) 49006 14
D get_mut, insert(word.to_string()) 37 14| Variant | What it does |
|---|---|
| A | what count_words does: a new lowercase String per word |
| B | lowercase the whole text once, keep &str keys pointing into that copy |
| C | owned keys, cloned for every word |
| D | owned keys, cloned only when the word is new (get_mut, then insert) |
The output is from a debug build. A and C allocate about once per word, B a handful of times in all, D about once per distinct word. The realloc column is the Vec of words and the map growing, the same for every variant. The program checks that all four built identical maps. We counted allocations and timed nothing: A allocates more, which does not mean it is slower.
D keeps owned keys and is a small change. B removes almost everything, but the lowercase copy must outlive the map, which is the first error again: the function could no longer return the map on its own. We keep A, the simplest correct version.
Step 4: sort, and the tie problem
Section titled “Step 4: sort, and the tie problem”A HashMap has no order, so we copy the pairs into a Vec and sort that. Add top_n to count.rs:
use std::collections::HashMap;
use crate::words::words;
/// Counts how often each word appears, ignoring case.pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { *counts.entry(word.to_lowercase()).or_insert(0) += 1; } counts}
/// The `n` most common words, most common first. Words with the same count/// come out in code point order (for English, plain alphabetical order).pub fn top_n(counts: HashMap<String, usize>, n: usize) -> Vec<(String, usize)> { let mut entries: Vec<(String, usize)> = counts.into_iter().collect(); entries.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0))); entries.truncate(n); entries}mod count;mod words;
use std::env;use std::fs;use std::io;
const TOP: usize = 10;
fn main() -> Result<(), io::Error> { let path = env::args().nth(1).expect("usage: wordcount FILE"); let text = fs::read_to_string(&path)?; let counts = count::count_words(&text); let total: usize = counts.values().sum(); let distinct = counts.len(); let top = count::top_n(counts, TOP); println!("{total} words, {distinct} distinct"); for (word, count) in &top { println!("{count} {word}"); } Ok(())}49 words, 31 distinct 7 the 4 dog 3 fox 2 don't 2 foxes 2 lazy 2 not 2 passing 2 through 2 worry
counts.into_iter().collect() turns the map into a Vec<(String, usize)>, a list of (word, count) pairs (Iterators, Tuples). sort_by takes a closure that compares two entries, each a reference to a pair. b.1.cmp(&a.1) compares the counts (cmp borrows both sides) with b first, so bigger counts sort first. If they are equal, then_with compares the words. A String compares by code point, so zoo comes before école, and Latin before Cyrillic before Han. truncate(n) drops everything after the first n.
Why the second comparison matters
Section titled “Why the second comparison matters”Here is the same program comparing only the counts. sort_by is stable: equal items keep the order they arrived in, and they arrive from a HashMap whose seed is random (Maps). Two runs on the same file:
$ ./target/debug/wordcount sample.txt 49 words, 31 distinct 7 the 4 dog 3 fox 2 lazy 2 through 2 passing 2 worry 2 not 2 don't 2 foxes $ ./target/debug/wordcount sample.txt 49 words, 31 distinct 7 the 4 dog 3 fox 2 lazy 2 through 2 foxes 2 don't 2 passing 2 not 2 worry
The top three are fixed, but the seven words tied at 2 (places four to ten) come out in a different order each time, and with --top 5 it would be a different set. A Python dict remembers insertion order, so its ties would repeat; Go’s maps are random like Rust’s and sort.Slice is not stable, so Go needs the second key too. With the tie-break the result depends only on the input, which also makes it testable in step 9.
Step 5: print aligned columns
Section titled “Step 5: print aligned columns”/// Prints the totals, then one line per row: rank, word, count.pub fn print(total: usize, distinct: usize, rows: &[(String, usize)]) { println!("{total} words, {distinct} distinct"); let width = rows .iter() .map(|(word, _)| word.chars().count()) .max() .unwrap_or(0); for (i, (word, count)) in rows.iter().enumerate() { println!("{:>3} {word:<width$} {count:>5}", i + 1); }}mod count;mod report;mod words;
use std::env;use std::fs;use std::io;
const TOP: usize = 10;
fn main() -> Result<(), io::Error> { let path = env::args().nth(1).expect("usage: wordcount FILE"); let text = fs::read_to_string(&path)?; let counts = count::count_words(&text); let total: usize = counts.values().sum(); let distinct = counts.len(); let top = count::top_n(counts, TOP); report::print(total, distinct, &top); Ok(())}The format {:>3} {word:<width$} {count:>5} right-aligns the rank in 3 columns, left-aligns the word in width columns, and right-aligns the count in 5. width counts characters (chars().count()), not bytes (len()), or café would count as five (Strings).
To see more than English, save this as mixed.txt and run cargo run -- mixed.txt:
Café café CAFÉ naïve Naïve.Straße STRASSEПривет привет мир你好世界 你好世界49 words, 31 distinct 1 the 7 2 dog 4 3 fox 3 4 don't 2 5 foxes 2 6 lazy 2 7 not 2 8 passing 2 9 through 2 10 worry 2$ cargo run -- mixed.txt
12 words, 7 distinct 1 café 3 2 naïve 2 3 привет 2 4 你好世界 2 5 strasse 1 6 straße 1 7 мир 1
Café, café and CAFÉ become one word through lowercasing, and straße and strasse stay separate (see the optional box in step 2). Padding counts characters, but many terminals draw Chinese characters two columns wide, so the 你好世界 row looks out of line.
Step 6: errors as a type
Section titled “Step 6: errors as a type”Step 1 showed a bare io::Error: no file name, and not written for users. We want errors in our program’s own terms. There are two kinds of failure, and they carry different data, which is what an enum is for (Enums): a message for a wrong command line, and the input’s name plus the io::Error for an unreadable input. Create src/error.rs:
use std::fmt;use std::io;
pub enum CliError { /// The command line is wrong. Exit code 2. Usage(String), /// The input could not be read. Exit code 1. Read { what: String, cause: io::Error },}
impl CliError { pub fn exit_code(&self) -> u8 { match self { CliError::Usage(_) => 2, CliError::Read { .. } => 1, } }}
impl fmt::Display for CliError { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { match self { CliError::Usage(message) => write!(f, "{message}"), CliError::Read { what, cause } => write!(f, "cannot read {what}: {cause}"), } }}exit_codeis a method (Methods) with amatchthat must cover both variants. It returnsu8because an exit code is a number from 0 to 255, which is exactly what au8holds. We use 0 for success, 1 for unreadable input and 2 for a wrong command line. That is a convention (many tools use 2 for bad usage), not a Rust rule.impl fmt::Displaysays how{err}prints.Displayis Rust’stoStringfor people;Debug, used by{:?}, is the form for programmers that step 1 printed. A trait is a set of methods a type can promise to have; the Traits lesson comes later.CliErrordoes not implementstd::error::Error, the trait that lets errors be boxed and chained. Nothing in this program needs that. A library would; Errors in real apps shows how.
Step 7: reading the command line
Section titled “Step 7: reading the command line”src/args.rs turns the arguments into a Config, a struct holding what the user asked for (Structs), or into a CliError:
use crate::error::CliError;
pub const DEFAULT_TOP: usize = 10;
pub struct Config { /// `None` means "read standard input". pub path: Option<String>, pub top: usize,}
fn usage(message: impl Into<String>) -> CliError { CliError::Usage(message.into())}
pub fn parse_args(mut args: impl Iterator<Item = String>) -> Result<Config, CliError> { let mut path = None; let mut top = DEFAULT_TOP; while let Some(arg) = args.next() { match arg.as_str() { "--top" => { let value = args.next().ok_or_else(|| usage("--top needs a number"))?; top = match value.parse::<usize>() { Ok(n) if n >= 1 => n, Ok(_) => return Err(usage("--top must be at least 1")), Err(e) => return Err(usage(format!("invalid value {value:?} for --top: {e}"))), }; } flag if flag.starts_with('-') && flag != "-" => { return Err(usage(format!("unknown option {flag}"))); } _ if path.is_some() => return Err(usage("only one input file is allowed")), "-" => {} _ => path = Some(arg), } } Ok(Config { path, top })}parse_argstakesimpl Iterator<Item = String>: “some type that hands out ownedStrings”. The compiler builds the function for the exact type, as with a template, so there is no virtual call.env::args().skip(1)is one. (Step 1 usednth(1)to take one argument; here we need all of them after the program name.)while let Some(arg) = args.next()loops until the arguments run out. After--topit takes the next argument, andvalue.parse::<usize>()returns aResult:Ok(n)for a number,Err(e)for text likeabc. We also refuse 0.impl Into<String>inusageaccepts either a&stror aString.- Arm order matters:
_ if path.is_some()comes before the arm that stores a path, so a second file is an error.--top=5is not supported; it would be one more arm.
Step 8: reading the input, and exit codes
Section titled “Step 8: reading the input, and exit codes”src/input.rs reads the file, or standard input when path is None:
use std::fs;use std::io::{self, Read};
use crate::error::CliError;
pub fn read_input(path: Option<&str>) -> Result<String, CliError> { match path { Some(path) => fs::read_to_string(path).map_err(|cause| CliError::Read { what: path.to_string(), cause, }), None => { let mut text = String::new(); io::stdin() .read_to_string(&mut text) .map_err(|cause| CliError::Read { what: "standard input".to_string(), cause, })?; Ok(text) } }}read_to_string loads the whole input into memory, which is fine for text you can hold in RAM. It comes from the Read trait, hence use std::io::Read. &mut text lends the String out to be changed (Mutable borrows).
Now main. First a decision: how should main fail?
| Option | What happens | Verdict |
|---|---|---|
fn main() -> Result<(), E> |
On Err, Rust prints Error: and the error’s Debug form, and exits with 1. Java’s analogue is letting an exception escape main. |
Debug text for programmers, and always code 1, so we could not use 2 for bad usage. |
std::process::exit(code) once, at the end of main, after run() returns |
The Go habit, os.Exit(run()). |
Works, and is close to what we do. It ends the process at once without running destructors (the std docs say so), as std::exit does in C++, and System.exit skips Java’s finally blocks. A buffered writer or a lock file added later would not be cleaned up. |
std::process::exit(code) where the error happens |
Ends the program on the spot. | Scatters the decisions about messages and codes through the program. |
fn main() -> ExitCode calling run() -> Result<(), CliError> |
run uses ?. One match in main prints the message and picks the code. |
This one. main returns normally, so destructors run, and one place decides what the user sees. |
wordcount/ ├── src/ │ ├── args.rs Config and parse_args │ ├── count.rs counting and ranking (and tests) │ ├── error.rs CliError │ ├── input.rs read_input │ ├── main.rs run() and main() │ ├── report.rs the aligned output │ └── words.rs splitting text ├── Cargo.lock ├── Cargo.toml ├── mixed.txt accents, Cyrillic, Chinese └── sample.txt the file we count
mod args;mod count;mod error;mod input;mod report;mod words;
use std::env;use std::process::ExitCode;
use error::CliError;
fn run() -> Result<(), CliError> { let config = args::parse_args(env::args().skip(1))?; let text = input::read_input(config.path.as_deref())?; let counts = count::count_words(&text); let total: usize = counts.values().sum(); let distinct = counts.len(); let top = count::top_n(counts, config.top); report::print(total, distinct, &top); Ok(())}
fn main() -> ExitCode { match run() { Ok(()) => ExitCode::SUCCESS, Err(err) => { eprintln!("wordcount: {err}"); if matches!(err, CliError::Usage(_)) { eprintln!("usage: wordcount [FILE] [--top N]"); } ExitCode::from(err.exit_code()) } }}matches!(err, CliError::Usage(_)) asks “is this the Usage variant?” without taking the error apart, so err is still there for err.exit_code(). From here we run ./target/debug/wordcount directly, which makes the exit code easy to show; these terminals merge standard error and standard output. First the success cases: a file, then a pipe.
$ cargo build $ ./target/debug/wordcount sample.txt --top 5 49 words, 31 distinct 1 the 7 2 dog 4 3 fox 3 4 don't 2 5 foxes 2 $ printf 'one two two three three three\n' | ./target/debug/wordcount --top 2 6 words, 3 distinct 1 three 3 2 two 2
With no file and nothing piped, the program waits for you to type; Ctrl-D ends the input. Now the errors the program checks for, each followed by the exit code:
$ ./target/debug/wordcount missing.txt; echo "exit code: $?" wordcount: cannot read missing.txt: No such file or directory (os error 2) exit code: 1 $ ./target/debug/wordcount src; echo "exit code: $?" wordcount: cannot read src: Is a directory (os error 21) exit code: 1 $ printf 'caf\xe9 au lait\n' > latin1.txt; ./target/debug/wordcount latin1.txt; echo "exit code: $?" wordcount: cannot read latin1.txt: stream did not contain valid UTF-8 exit code: 1 $ ./target/debug/wordcount sample.txt --top abc; echo "exit code: $?" wordcount: invalid value "abc" for --top: invalid digit found in string usage: wordcount [FILE] [--top N] exit code: 2 $ ./target/debug/wordcount sample.txt --top 0; echo "exit code: $?" wordcount: --top must be at least 1 usage: wordcount [FILE] [--top N] exit code: 2 $ ./target/debug/wordcount sample.txt --top; echo "exit code: $?" wordcount: --top needs a number usage: wordcount [FILE] [--top N] exit code: 2 $ ./target/debug/wordcount --verbose; echo "exit code: $?" wordcount: unknown option --verbose usage: wordcount [FILE] [--top N] exit code: 2 $ ./target/debug/wordcount sample.txt mixed.txt; echo "exit code: $?" wordcount: only one input file is allowed usage: wordcount [FILE] [--top N] exit code: 2 $ : > empty.txt; ./target/debug/wordcount empty.txt; echo "exit code: $?" 0 words, 0 distinct exit code: 0
The file errors name the file, and the operating system’s reason comes from the io::Error. src is a directory, and latin1.txt holds a byte that is not valid UTF-8, so read_to_string refuses the whole file instead of guessing.
Two things still panic, with exit code 101, because they are not checked. env::args() panics on an argument that is not valid Unicode (args_os() does not, and a tool that must accept any file name should use it). And println! panics when standard output fails, shown here with /dev/full, a device that rejects every write:
$ ./target/debug/wordcount "$(printf '\xff')"; echo "exit code: $?" thread 'main' (962) panicked at /rustc/58cf8f9c35b355199727627d07ae34a4e2846d67/library/std/src/env.rs:878:51: called `Result::unwrap()` on an `Err` value: "\xFF" note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace exit code: 101 $ ./target/debug/wordcount sample.txt > /dev/full; echo "exit code: $?" thread 'main' (964) panicked at /rustc/58cf8f9c35b355199727627d07ae34a4e2846d67/library/std/src/io/stdio.rs:1168:9: failed printing to stdout: No space left on device (os error 28) note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace exit code: 101
Step 9: test the pure part
Section titled “Step 9: test the pure part”count_words and top_n are worth testing alone: text in, numbers out, no files, no terminal. A function like that, with no side effects, is called pure, and it is easy to test. The finished count.rs, with the tests at the bottom:
use std::collections::HashMap;
use crate::words::words;
/// Counts how often each word appears, ignoring case.pub fn count_words(text: &str) -> HashMap<String, usize> { let mut counts = HashMap::new(); for word in words(text) { *counts.entry(word.to_lowercase()).or_insert(0) += 1; } counts}
/// The `n` most common words, most common first. Words with the same count/// come out in code point order (for English, plain alphabetical order).pub fn top_n(counts: HashMap<String, usize>, n: usize) -> Vec<(String, usize)> { let mut entries: Vec<(String, usize)> = counts.into_iter().collect(); entries.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0))); entries.truncate(n); entries}
#[cfg(test)]mod tests { use super::*;
#[test] fn ignores_case_and_punctuation() { let counts = count_words("The cat. the CAT, the dog!"); assert_eq!(counts["the"], 3); assert_eq!(counts["cat"], 2); assert_eq!(counts["dog"], 1); assert_eq!(counts.len(), 3); }
#[test] fn empty_and_punctuation_only_text_has_no_words() { assert!(count_words("").is_empty()); assert!(count_words(" ... !!! \n").is_empty()); }
#[test] fn apostrophes_stay_inside_words() { let counts = count_words("Don't 'quote' don't"); assert_eq!(counts["don't"], 2); assert_eq!(counts["quote"], 1); }
#[test] fn accented_letters_are_letters() { let counts = count_words("Café café CAFÉ"); assert_eq!(counts["café"], 3); }
#[test] fn ties_come_out_in_code_point_order() { let counts = count_words("b a c a b"); let top = top_n(counts, 3); let expected = vec![ ("a".to_string(), 2), ("b".to_string(), 2), ("c".to_string(), 1), ]; assert_eq!(top, expected); }
#[test] fn accented_words_sort_after_plain_ones() { let top = top_n(count_words("école zoo"), 2); assert_eq!(top[0].0, "zoo"); assert_eq!(top[1].0, "école"); }
#[test] fn top_n_keeps_only_n() { let counts = count_words("a b c d"); assert_eq!(top_n(counts, 2).len(), 2); }}running 7 tests test count::tests::accented_letters_are_letters ... ok test count::tests::accented_words_sort_after_plain_ones ... ok test count::tests::apostrophes_stay_inside_words ... ok test count::tests::empty_and_punctuation_only_text_has_no_words ... ok test count::tests::ignores_case_and_punctuation ... ok test count::tests::ties_come_out_in_code_point_order ... ok test count::tests::top_n_keeps_only_n ... ok test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
#[cfg(test)] means “compile this module only for cargo test”. use super::*; brings the parent’s names in. A #[test] function passes unless it panics, and assert_eq! panics when its two sides differ. There is no test framework to install; the standard library has one. counts["the"] panics if the key is missing, which is what you want in a test. Tests run on several threads, so the output order is not the file order. They cover case, punctuation, empty text, an apostrophe inside a word, accented letters, ties, and an accented word sorting after a plain one. More in Testing.
The same program in Python and Go
Section titled “The same program in Python and Go”We wrote the tool in Python and Go too and ran all of them in the same image. A Python programmer would not hand-roll an argument parser, so there are two Python files: a short one with argparse and Counter, and a long one with the same options, messages and exit codes as the Rust, for a fair comparison. We ran Python and Go because we had to pick two garbage-collected languages; Java, Kotlin and C# sit with them on the points below (managed memory, exceptions). C++ is the language whose decisions match Rust’s; the table near the top is its comparison, and we did not run a C++ version.
import argparseimport reimport sysfrom collections import Counter
parser = argparse.ArgumentParser(prog="wordcount")parser.add_argument("file", nargs="?", default="-")parser.add_argument("--top", type=int, default=10)args = parser.parse_args()
text = sys.stdin.read() if args.file == "-" else open(args.file, encoding="utf-8").read()counts = Counter(w.lower() for w in re.findall(r"[^\W\d_']+(?:'+[^\W\d_']+)*", text))rows = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))[: args.top]width = max((len(w) for w, _ in rows), default=0)print(f"{sum(counts.values())} words, {len(counts)} distinct")for rank, (word, n) in enumerate(rows, 1): print(f"{rank:>3} {word:<{width}} {n:>5}")import sys
DEFAULT_TOP = 10USAGE = "usage: wordcount [FILE] [--top N]"
class UsageError(Exception): pass
def words(text): cleaned = "".join(c if c.isalpha() or c == "'" else " " for c in text) stripped = (w.strip("'") for w in cleaned.split()) return [w for w in stripped if w]
def count_words(text): counts = {} for word in words(text): word = word.lower() counts[word] = counts.get(word, 0) + 1 return counts
def top_n(counts, n): ranked = sorted(counts.items(), key=lambda item: (-item[1], item[0])) return ranked[:n]
def parse_args(args): path, top = None, DEFAULT_TOP i = 0 while i < len(args): arg = args[i] i += 1 if arg == "--top": if i == len(args): raise UsageError("--top needs a number") value = args[i] i += 1 try: top = int(value) except ValueError: raise UsageError(f"invalid value {value!r} for --top") if top < 1: raise UsageError("--top must be at least 1") elif arg.startswith("-") and arg != "-": raise UsageError(f"unknown option {arg}") elif path is not None: raise UsageError("only one input file is allowed") elif arg != "-": path = arg return path, top
def read_input(path): if path is None: return sys.stdin.read() with open(path, encoding="utf-8") as f: return f.read()
def main(): try: path, top = parse_args(sys.argv[1:]) except UsageError as e: print(f"wordcount: {e}\n{USAGE}", file=sys.stderr) return 2 try: text = read_input(path) except (OSError, UnicodeDecodeError) as e: print(f"wordcount: cannot read {path or 'standard input'}: {e}", file=sys.stderr) return 1
counts = count_words(text) rows = top_n(counts, top) print(f"{sum(counts.values())} words, {len(counts)} distinct") width = max((len(word) for word, _ in rows), default=0) for rank, (word, count) in enumerate(rows, 1): print(f"{rank:>3} {word:<{width}} {count:>5}") return 0
sys.exit(main())package main
import ( "errors" "fmt" "io" "os" "sort" "strconv" "strings" "unicode" "unicode/utf8")
const defaultTop = 10const usage = "usage: wordcount [FILE] [--top N]"
func words(text string) []string { pieces := strings.FieldsFunc(text, func(r rune) bool { return !unicode.IsLetter(r) && r != '\'' }) var found []string for _, piece := range pieces { if word := strings.Trim(piece, "'"); word != "" { found = append(found, word) } } return found}
type entry struct { word string count int}
func countWords(text string) map[string]int { counts := map[string]int{} for _, word := range words(text) { counts[strings.ToLower(word)]++ } return counts}
func topN(counts map[string]int, n int) []entry { entries := make([]entry, 0, len(counts)) for word, count := range counts { entries = append(entries, entry{word, count}) } sort.Slice(entries, func(i, j int) bool { if entries[i].count != entries[j].count { return entries[i].count > entries[j].count } return entries[i].word < entries[j].word }) if len(entries) > n { entries = entries[:n] } return entries}
func parseArgs(args []string) (path string, top int, err error) { top = defaultTop for i := 0; i < len(args); i++ { arg := args[i] switch { case arg == "--top": i++ if i == len(args) { return "", 0, errors.New("--top needs a number") } n, convErr := strconv.Atoi(args[i]) if convErr != nil { return "", 0, fmt.Errorf("invalid value %q for --top", args[i]) } if n < 1 { return "", 0, errors.New("--top must be at least 1") } top = n case strings.HasPrefix(arg, "-") && arg != "-": return "", 0, errors.New("unknown option " + arg) case path != "": return "", 0, errors.New("only one input file is allowed") case arg != "-": path = arg } } return path, top, nil}
func readInput(path string) (string, error) { var data []byte var err error if path == "" { data, err = io.ReadAll(os.Stdin) } else { data, err = os.ReadFile(path) } return string(data), err}
func run() int { path, top, err := parseArgs(os.Args[1:]) if err != nil { fmt.Fprintf(os.Stderr, "wordcount: %v\n%s\n", err, usage) return 2 } text, err := readInput(path) if err != nil { what := path if what == "" { what = "standard input" } fmt.Fprintf(os.Stderr, "wordcount: cannot read %s: %v\n", what, err) return 1 }
counts := countWords(text) rows := topN(counts, top) total := 0 for _, n := range counts { total += n } fmt.Printf("%d words, %d distinct\n", total, len(counts)) width := 0 for _, row := range rows { width = max(width, utf8.RuneCountInString(row.word)) } for i, row := range rows { fmt.Printf("%3d %-*s %5d\n", i+1, width, row.word, row.count) } return 0}
func main() { os.Exit(run()) }All four programs print the same bytes on the sample and on the mixed-script file:
$ python3 --version Python 3.13.5 $ go version go version go1.27.3 linux/arm64 $ python3 wordcount.py sample.txt > py.txt; go run . sample.txt > go.txt; ./target/debug/wordcount sample.txt > rs.txt; python3 wordcount_short.py sample.txt > short.txt; cmp py.txt go.txt && cmp go.txt rs.txt && cmp rs.txt short.txt && echo "all four outputs are byte for byte the same" all four outputs are byte for byte the same $ python3 wordcount.py mixed.txt > py.txt; go run . mixed.txt > go.txt; ./target/debug/wordcount mixed.txt > rs.txt; python3 wordcount_short.py mixed.txt > short.txt; cmp py.txt go.txt && cmp go.txt rs.txt && cmp rs.txt short.txt && echo "all four outputs are byte for byte the same" all four outputs are byte for byte the same
Lines that are not blank and not comments, with the Rust tests left out:
$ echo "Python, by hand: $(grep -Evc '^[[:space:]]*(#.*)?$' wordcount.py) lines"
echo "Python, argparse and Counter: $(grep -Evc '^[[:space:]]*(#.*)?$' wordcount_short.py) lines"
echo "Go: $(grep -Evc '^[[:space:]]*(//.*)?$' main.go) lines"
echo "Rust: $(for f in src/*.rs; do sed '/#\[cfg(test)\]/,$d' $f; done | grep -Evc '^[[:space:]]*(//.*)?$') lines"
for f in src/*.rs; do echo " $f: $(sed '/#\[cfg(test)\]/,$d' $f | grep -Evc '^[[:space:]]*(//.*)?$')"; done
Python, by hand: 66 lines
Python, argparse and Counter: 15 lines
Go: 123 lines
Rust: 142 lines
src/args.rs: 32
src/count.rs: 15
src/error.rs: 22
src/input.rs: 21
src/main.rs: 31
src/report.rs: 11
src/words.rs: 10
The short Python is the honest price list: Rust costs many more lines. What those lines buy is that every decision is written down. The short Python has no error handling of its own. argparse gives usage errors for free (as clap would in Rust), and a missing file is a traceback:
$ python3 wordcount_short.py sample.txt --top abc; echo "exit code: $?" usage: wordcount [-h] [--top TOP] [file] wordcount: error: argument --top: invalid int value: 'abc' exit code: 2 $ python3 wordcount_short.py missing.txt; echo "exit code: $?" Traceback (most recent call last): File "/home/you/wordcount/wordcount_short.py", line 11, in <module> text = sys.stdin.read() if args.file == "-" else open(args.file, encoding="utf-8").read() ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'missing.txt' exit code: 1
The messages for a missing file differ in wording across the three full programs (the exit codes match because we wrote them to). The exit status 1 line in the Go output comes from go run:
$ python3 wordcount.py missing.txt; echo "exit code: $?" wordcount: cannot read missing.txt: [Errno 2] No such file or directory: 'missing.txt' exit code: 1 $ go run . missing.txt; echo "exit code: $?" wordcount: cannot read missing.txt: open missing.txt: no such file or directory exit status 1 exit code: 1 $ ./target/debug/wordcount missing.txt; echo "exit code: $?" wordcount: cannot read missing.txt: No such file or directory (os error 2) exit code: 1
On a file that is not valid UTF-8, Rust refuses to read it, and so does Python (opened with encoding="utf-8"), by raising an exception that our program turns into a message. Go reads the bytes without checking: the bad byte is not a letter, so it ends the word caf, and the run succeeds.
$ python3 wordcount.py latin1.txt; echo "exit code: $?" wordcount: cannot read latin1.txt: 'utf-8' codec can't decode byte 0xe9 in position 3: invalid continuation byte exit code: 1 $ go run . latin1.txt; echo "exit code: $?" 3 words, 3 distinct 1 au 1 2 caf 1 3 lait 1 exit code: 0 $ ./target/debug/wordcount latin1.txt; echo "exit code: $?" wordcount: cannot read latin1.txt: stream did not contain valid UTF-8 exit code: 1
Each language also has its own idea of “a letter”. Rust’s is_alphabetic said true for Ⅳ in step 2. Python’s isalpha and Go’s unicode.IsLetter test letter categories instead of the Alphabetic property, and both say:
$ python3 -c "print('Ⅳ'.isalpha())" False $ go run ./isletter unicode.IsLetter('Ⅳ'): false
What Rust made us decide that the others did not:
- Who owns each word. Where memory is managed for you, a map key is just a value you can keep. In Rust we chose
&strforwordsandStringfor the keys, and the signatures say so. - Borrow or consume.
top_ntakes the map by value;printborrows the rows. - What an error is. We defined an enum and its exit codes. The Python and Go versions use exceptions and
errorvalues from the language, and did not force us to list the ways the program can fail. - When a copy happens. Every
to_string,to_lowercaseandcloneis visible. We did not measure Python or Go.
The step 3 errors are the other side of the trade. The others have no equivalent because the collector keeps keys alive. In Rust you may choose variant B, one shared buffer behind all the keys, and the compiler checks it.
What you used
Section titled “What you used”| Idea | Where it appeared | Lesson |
|---|---|---|
Owned String from a function |
read_to_string, to_lowercase |
Ownership, Strings |
&str and slices |
words, counts.get("the") |
Strings, Borrowing |
| Move into a function or a map | entry(word.to_lowercase()), top_n(counts, n), path = Some(arg) |
Ownership |
& borrow |
&path, &text, &top |
Borrowing |
&mut borrow |
read_to_string(&mut text), or_insert(0) |
Mutable borrows |
| Returning a borrow | words(&str) -> Vec<&str> |
Returning references |
| Struct, methods | Config, CliError::exit_code |
Structs, Methods |
Enum and match |
CliError, parse_args |
Enums |
Option, Result and ? |
nth(1), get, read_input, run |
Option, Errors |
Vec, HashMap and entry |
the words, the rows, count_words |
Vectors, Maps |
| Closures and iterators | split, sort_by, map_err, into_iter().collect() |
Closures, Iterators |
| Modules and tests | mod, pub, #[test] |
Modules, Testing |
Where to next
Section titled “Where to next”- The same tool with
clap,--helpand usage errors for free: Command-line tools with clap. - Errors that carry context without writing your own enum: Errors in real apps.
Paths in Cargo’s output are shown as /home/you. Allocation counts are from a debug build in the course’s Docker image.
Learn Rust is an independent project, not affiliated with the Rust Project or the Rust Foundation. Code and text are under the MIT licence.Source on GitHub.