Rust Modules, Cargo & Workspaces: Ship Multi-Crate Projects
mod/use paths, pub(crate) boundaries, cargo test/clippy/fmt/doc, workspace inheritance, cargo publish: the tooling guide..
20+ years shipping production backend systems. Everything here is grounded in real deployments.
- ✓Rust toolchain installed via rustup (stable)
- ✓Built one binary crate with cargo new and run
- ✓Basic comfort with cargo build and cargo test
- Modules declare with mod and link with use: paths start at crate, self, super, or an external crate name — the filesystem mirrors the tree
- Visibility defaults to private: pub opens to everyone, pub(crate) to your crate, pub(super)/pub(in path) to a subtree — keep 80% of items private
- cargo test runs unit + integration + doc tests in one shot; add clippy -- -D warnings and fmt --check to CI and broken builds never merge
- Workspaces share one Cargo.lock and one target/ dir: path dependencies wire members together, workspace.package inherits version/edition/rust-version
- Publish with cargo publish --dry-run first, then --workspace on Cargo 1.90+: version bumps propagate via inheritance so 12 crates release in one command
- No C linker (linking with cc failed)? Install build-essential or Xcode CLT, rustup target add the triple, cargo clean, rebuild
Imagine you're running a restaurant kitchen. Modules are the labeled stations — grill, pastry, prep — so 40 cooks aren't tripping over each other in one room. Visibility rules are the doors between stations: the pastry chef can borrow sugar from prep (same building), but customers can't wander into the kitchen. Cargo is the general manager who orders ingredients (dependencies), runs health inspections (tests, clippy), and prints the menu (docs). And a workspace is the whole restaurant group — five locations sharing one supplier contract (a single lockfile), one recipe book format (inherited package settings), so opening location number six takes an afternoon instead of a month.
You'll feel Cargo's opinions within your first week of serious Rust. A Python project grows a src/ layout whenever someone gets around to it; a Rust project gets it from cargo new on day one. That scaffolding — lib.rs versus main.rs, modules mirroring files, tests living beside code — looks like ceremony until your codebase crosses 20K lines and the ceremony starts paying rent.
Don't confuse the tooling for the language, though. rustc compiles crates; Cargo orchestrates everything around compilation — dependency resolution, feature unification, profiles, workspaces, publishing. Teams that learn rustc but skip Cargo end up hand-rolling build scripts and version policies that Cargo solved a decade ago. You'll do the opposite here.
We've organized this guide the way real projects grow. You'll start with modules and visibility — the in-crate decisions that determine whether refactors take hours or days. Then comes the daily command loop: test, clippy, fmt, doc. Only then do workspaces enter, because splitting crates too early is its own special misery.
Expect numbers throughout: build-time deltas from shared target dirs, CI minutes saved by lint gates, and the incident where a missing C linker blocked 14 engineers for a morning. You'll leave able to structure, test, release, and repair a multi-crate Rust project without guessing.
All commands target Cargo 1.90+ with the 2024 edition (any rustc from the last year works). Samples run on stable — no nightly flags, no external tools beyond a C linker and curiosity.
Modules and Paths: mod, use, crate, self, and super
A crate is a compilation unit with a root — lib.rs for libraries, main.rs for binaries — and modules form a tree hanging off that root. mod network; inside lib.rs declares a child module and tells rustc to find its body in src/network.rs or src/network/mod.rs; write mod parser { ... } inline and the body lives right there. The filesystem mirrors the tree by convention, not by magic: rename the file without updating mod and you get E0583 (file not found), move the mod without moving the file and you get the same. The tree exists in declarations; files are just where bodies live.
Paths navigate that tree from four anchors. crate:: starts at the root (crate::network::parse works from anywhere in the crate). self:: means this module (self::helper inside network). super:: climbs one level (super::Config from network::tls reaches network's parent item). Bare names resolve from the current scope outward — which is why use crate::network::parse; at the top of a file fixes 90% of 'unresolved import' errors: it pins the anchor instead of guessing. External crates are addressed by name (serde::Serialize), and 2018+ editions dropped the extern crate ritual for direct references.
use is a shortcut, not an include — it binds a name in the current scope, moves nothing, and costs zero at runtime. use crate::db::{Pool, Config}; binds two names; use super::*; glob-imports a parent (handy in tests, noisy in libraries — reviewers flag globs outside test modules). Re-export with pub use to flatten deep trees for consumers: pub use network::tls::TlsConfig; at the root lets users write mycrate::TlsConfig instead of spelunking four levels. One 40K-line codebase cut average import paths from 4.2 segments to 1.8 with a deliberate re-export layer — onboarding time for new modules dropped measurably.
File layout has exactly two rules and one strong convention. Rule one: each mod foo; needs its file at the declared spot (2018+ prefers src/foo.rs over src/foo/mod.rs; both still work). Rule two: path attributes #[path = "..." ] override the convention when FFI or codegen demands odd layouts — use sparingly, since every override is a surprise for the next reader. Convention: one module per file, unit tests in #[cfg(test)] mod tests at the file bottom, integration tests in tests/ at the package root. Deviate from the convention and every rust-analyzer user pays a small confusion tax on each visit.
Debug path errors mechanically. E0432/E0433 always mean anchor-vs-tree mismatch: run cargo check, read which segment failed, then ls -R src to compare the file tree against mod declarations. Sketch the tree on paper for the first month — root at top, children below, super arrows up — until anchors become reflex. Modules aren't bureaucracy; they're the map every later decision (visibility, splitting, publishing) reads from. Draw the map right and the rest of this guide gets easy.
Re-exports shape public API more than any other single decision, so design them deliberately. A flat root (pub use network::{Client, Config, Error}) lets users write mycrate::Client while internals stay nested 4 deep — docs.rs shows the friendly surface, modules show the honest structure. Glob re-exports (pub use network::*) flatten without maintenance but leak every future addition into semver promises; prefer explicit lists where stability matters, globs where velocity matters (internal preludes, test helpers). Version the re-export layer in changelogs: 'moved TlsConfig to crate root (old path still works via deprecated alias)' is the professional form of refactoring in public.
Large modules split along exactly two axes: topic (network::tls vs network::dns) when concepts differ, and layer (parser vs validator vs store) when pipeline stages differ. Splitting by file size alone produces network_a.rs and network_b.rs — names that explain nothing and rot immediately. Each split module earns a one-line doc comment stating its responsibility; modules that can't be summarized in one line aren't modules yet. One 90K-line codebase enforced 'one responsibility sentence per mod' in review and watched average module size fall from 4K to 900 lines over a year — not by fiat, by making formless modules reviewable as defects.
Visibility That Scales: pub, pub(crate), and the 80% Rule
Everything in Rust is private by default — functions, structs, fields, modules, even enum variants' constructors in some positions. Privacy isn't about secrecy; it's about blast radius. A private helper can be renamed, retyped, or deleted with confidence that only its module cares. A pub function is a promise to every current and future user, enforced by semver expectations and documentation duties. The 80% rule that survives every codebase audit: keep four out of five items private, and promote only what a second module genuinely needs. Codebases that invert the ratio drown in accidental API.
The visibility ladder has four rungs and each names its audience. pub opens to the world — external crates, docs.rs, semver. pub(crate) opens to your crate only: sibling modules, integration tests inside src, binaries in the same package. pub(super) opens to the parent module (tight coupling made explicit), and pub(in crate::network) opens to an arbitrary subtree for layered architectures. Fields follow the same ladder independently of their struct — pub struct Config with private fields forces construction through new() or builders, preserving invariants that direct literal construction would bypass.
Modules gate their contents: a pub fn inside a private mod is unreachable from outside, which is exactly how you stage APIs — build inside private modules, re-export the chosen few with pub use at the root. Reviewers check two things: that every pub item appears in cargo doc output intentionally (run cargo doc --no-deps and skim), and that fields carrying invariants aren't pub (a private field with a validating setter beats a pub field with a prayer). One payments crate made Money(pub u64) public 'for convenience'; three months later negative-amount bugs traced to direct construction at 6 call sites. Private field plus checked constructor closed the class.
E0603 (private item accessed) is the compiler teaching the ladder. When tests can't reach a helper, the fix is rarely pub — it's pub(crate), or moving the test into the module (#[cfg(test)] mod tests sees privates directly). Integration tests in tests/ are external users: they see only pub API, which makes them excellent API-shape reviewers. If an integration test needs internals, that's feedback that the public surface is missing something, not permission to pub everything.
Audit visibility quarterly with a simple pipeline: cargo doc --no-deps to list the public surface, then challenge each item — who uses this outside its module? Unused pub items are lies about stability; demote them. New code defaults private and earns promotion through a second caller, not anticipation. Encapsulation compounds: each private item is one less thing the next refactor must preserve.
API evolution under visibility constraints is a skill distinct from writing new code: deprecate, alias, then remove across two releases. #[deprecated(since = "0.5.0", note = "use Config::builder")] keeps old paths compiling with warnings while guides migrate — measure migration via warning counts in downstream CI before removal. pub use old_path as NewName preserves compatibility shims in one line. Sealed traits (pub trait with private supertrait or #[doc(hidden)] method) close extension points you never meant to open; unsealed public traits promise downstream impls forever, constraining every future method addition. One framework sealed 3 traits in a minor release and unlocked default-method additions that would otherwise have been breaking — foresight that paid off within two quarters.
Visibility interacts with documentation as a forcing function: cargo doc renders only reachable pub items, so the rendered page is the actual API contract — review it as such. Missing-docs warnings (#![warn(missing_docs)]) turn every new pub item into a documented decision; intra-doc link failures flag renames that forgot their references. Teams that review the rendered docs diff on each PR (cargo doc output committed as artifacts, or docs.rs previews) catch accidental API additions before release — 'why is this helper public?' asked at PR time costs seconds, asked after 1.0 costs a major version. The docs build is the API review you get for free; collect it.
lib vs bin Layout: Structuring Crates That Outgrow Scripts
cargo new defaults to a binary (src/main.rs); cargo new --lib scaffolds a library (src/lib.rs). The difference shapes everything downstream: libraries expose reusable APIs with unit tests beside code, binaries produce runnable artifacts with thin mains. The layout that scales past 10K lines uses both — src/lib.rs holding 95% of logic plus src/main.rs (or src/bin/) as a thin CLI wrapper calling into the library. That split buys testability (integration tests in tests/ exercise the lib like external users) and reuse (a second binary, an FFI shim, or a benchmark harness links the same lib without forking code).
Know the four source roots cold. src/lib.rs is the library root — its public items are the crate's API. src/main.rs is the default binary root using the lib via use mycrate::.... src/bin/<name>.rs adds extra binaries (one package, many tools: server, migrator, debugger). examples/, benches/, and tests/ each compile as separate crates linking your lib — examples document by running, benches measure, integration tests verify public behavior only. A repo with server, worker, and migrate binaries plus one lib serves a whole backend from a single package with shared types throughout.
The thin-main discipline pays off in incidents. Fat mains (logic embedded in main.rs) can't be integration-tested without spawning processes, can't be reused by a second binary, and turn every refactor into surgery. One team kept 3K lines in main.rs for a year; extracting the lib took 3 days but deleted 400 lines of duplicated argument parsing across two new tools and made the core testable in-process. Rule: main parses args, calls lib, prints results — if main exceeds 100 lines, extraction is overdue.
tests/ deserves its own paragraph because newcomers misuse it. Each file in tests/ compiles as its own crate linking your library's public API — perfect for contract tests, wrong for white-box unit tests (those live in #[cfg(test)] modules beside the code with access to privates). Integration tests can't touch internals, which is a feature: they fail exactly when your public surface breaks. tests/common/mod.rs holds shared fixtures (declared via mod common; — no lib target generated for helpers). Fixture paths go through env!("CARGO_MANIFEST_DIR") so cargo test --workspace passes from any working directory.
Scaffold once, correctly: cargo new --lib mylib for reusable code, add src/main.rs when a CLI arrives, split src/bin/ when tools multiply. Keep the lib dependency-free of CLI concerns (no clap in lib types) so servers and scripts share it. Layout isn't aesthetics — it decides what tests can see, what binaries can share, and what publishes cleanly to crates.io later.
Workspace-conditional compilation keeps multi-binary repos honest: a lib that compiles for server, CLI, and wasm targets needs per-target gates beyond features. #[cfg(target_arch = "wasm32")] swaps networking backends, #[cfg(unix)] gates signal handling, and src/bin/ tools can carry heavier deps (clap, tracing-subscriber) that the lib never sees — keeping library dependency weight low for downstream users. Verify the matrix in CI: cargo check --workspace --targets covers bins, examples, and tests (plain check skips some targets), and --target wasm32-unknown-unknown catches arch-specific breakage before release. One repo's lib compiled while its migrate binary rotted for 5 months — --targets in CI would have flagged the first breaking PR.
Examples are the onboarding docs that compile: examples/quickstart.rs with a 30-line happy path gets new users productive in minutes and fails CI when APIs drift. Keep examples focused (one concept each), runnable without credentials (mock data, localhost servers), and listed in the README with cargo run --example quickstart commands. Benches beside them (benches/parse.rs with criterion) guard performance contracts the same way tests guard behavior — a 20% parse regression caught by criterion saved a release that 'felt fine' locally. Examples teach, benches promise, integration tests enforce: the three together make a lib trustworthy.
cargo test: Unit, Integration, and Doc Tests in One Command
cargo test runs three suites in one invocation and newcomers use one. Unit tests (#[cfg(test)] mod tests inside src/) verify internals with access to privates — fast, plentiful, 80%+ of assertions. Integration tests (files in tests/) verify public contracts as external users — slower, fewer, each compiling as its own crate. Doc tests (fenced code examples on pub items) verify documentation compiles and passes — the only tests that fail when docs lie. A healthy 50K-line service shows roughly 3,000 unit, 200 integration, and 150 doc tests; if any leg is near zero, that blind spot picks your next incident.
Write unit tests where the code lives. #[cfg(test)] compiles the module only under cargo test, so zero release overhead; #[test] marks cases, #[should_panic(expected = "...")] pins failure modes, and assert_eq!/assert!(matches!()) cover most shapes. Test private helpers directly — that's the privilege unit tests buy. Keep them hermetic: no network, no filesystem outside tempdir, no wall-clock sleeps (inject clocks). A unit suite that runs in under 10 seconds gets run constantly; one that takes 3 minutes gets skipped, then deleted by neglect.
Integration tests protect the promises in your README. tests/api_roundtrip.rs spins the public API end to end; tests/common/mod.rs shares fixtures; each file's separate-crate compilation guarantees you're testing what users touch. Name files by behavior (checkout_flow.rs, not test1.rs) and assert outcomes, not internals. The 60-second rule: if the integration suite exceeds a minute, split by tag and run the slow half nightly — fast feedback on every push, deep coverage before release.
Doc tests deserve deliberate care because they execute. Every fenced block on a pub item compiles as a mini-crate with your crate linked — broken examples fail cargo test, which is documentation that can't rot silently. Mark non-runnable blocks explicitly (text or ignore tags, with justification) so intent is reviewable. One crate's README example drifted from the API for 4 months; a doc test would have caught it on the first PR. Docs that run stay true.
Run the loop constantly: cargo test for everything, cargo test --lib for units only, cargo test --doc for docs, cargo test -- --nocapture to see println! during triage. CI runs cargo test --workspace --all-features so feature unification matches local runs. Coverage via cargo tarpaulin or llvm-cov guides test-writing toward cold spots — but 85% coverage of meaningful asserts beats 95% of assert!(true). Tests are the contract; run them like it.
Flaky tests deserve a taxonomy because each kind has a distinct cure. Order-dependent tests (pass solo, fail in suite) signal shared mutable state — static mut, leaked temp dirs, fixed ports; fix with per-test isolation (unique ports via port 0 binding, tempfile::tempdir per test). Timing-sensitive tests (sleep(100) then assert) fail under CI load; replace sleeps with polling loops (100 iterations x 10ms) or injected fake clocks. Environment-sensitive tests (pass on Linux, fail on macOS/Windows) need cfg gates or normalized handling (path separators, line endings, timezone UTC pinning). Measure flakiness honestly: cargo test -- --test-threads=1 separates concurrency flakes from logic bugs, and repeating the suite 10x in CI quantifies the rate before you claim a fix.
Test organization scales through naming and layering. Name tests by behavior under test (rejects_expired_token, not test_auth_7) so failures read as bug reports. Layer slow tests behind explicit markers — an env var (RUN_SLOW=1) or separate target (tests/nightly/) — so the default suite stays under 60 seconds and developers actually run it. Property tests (proptest/quickcheck) cover input spaces unit examples can't: byte-string parsers, UTF-8 boundary logic, and serialization round-trips are canonical wins — one proptest suite found 4 panics in a 'thoroughly tested' parser within minutes. Tests are production code: review them, refactor them, delete the ones that assert nothing.
Clippy, fmt, and doc: Three Commands Before Every Commit
Three commands separate reviewed code from hopeful code, and all three run in under a minute. cargo clippy --all-targets -- -D warnings lints 700+ patterns — needless clones, wrong String parameters, unchecked indexing suggestions — failing the build on any hit. cargo fmt --check enforces one canonical style so diffs show logic, never brace placement. cargo doc --no-deps verifies every intra-doc link and example compiles into renderable HTML. Together they catch the defects humans wave through at 5 PM on Fridays; one org measured a 31% drop in review-round-trips after gating all three.
Clippy earns its gate with specific catches. needless_pass_by_value flags String-taking functions that only read (downgrade to &str). len_without_is_empty, unwrap_used (restriction lint), and large_enum_variant each name real production bugs — the last caught a 2 KiB enum moved by value at 40K rps. Start with the default warn set promoted to deny in CI (-- -D warnings), then adopt pedantic/nursery lints module by module with targeted #[allow] justifications. Never allow-by-default at the crate root; each suppression should cite the false-positive reason in a comment the next reader can verify.
rustfmt ends style debates by deleting them. Run cargo fmt on save (rust-analyzer does it), gate cargo fmt --check in CI, and never hand-format what the tool owns. The one config worth setting is in rustfmt.toml — edition, max_width deliberations — and even that should be a 5-line file, not a manifesto. Teams that fought formatting in review burned ~45 minutes per engineer per week; after the gate, formatting comments dropped to zero and stayed there. Style is a solved problem; act like it.
cargo doc is the API review you weren't doing. Intra-doc links ([Money::new]) resolve at doc-build time — rename a method and the doc build breaks, pointing at stale references tests can't see. cargo doc --no-deps --open renders your public surface for skimming; gaps (undocumented pub items) show as warnings under #![warn(missing_docs)]. Publish docs to docs.rs automatically on crates.io release — private registries mirror the flow. If your public API can't be understood from its rendered docs in 10 minutes, the API (not the docs) usually needs work.
Wire all three into a pre-commit hook and CI so discipline is automatic: hook runs fmt + clippy on changed crates in seconds, CI runs the full trio with --workspace --all-targets. Developers stop thinking about gates they can't fail locally. Quality that depends on memory isn't quality — it's luck with extra steps.
Custom lints and repo-specific Clippy config turn tribal knowledge into compiler output. clippy.toml tunes thresholds (too_many_arguments, large_enum_variant sizes) to your domain — game engines allow bigger enums than CRUD APIs. Crate-level attributes (#! [deny(clippy::unwrap_used)] on libraries that promise zero-panic paths) enforce architectural invariants where code review forgets; allow-by-exception with justification comments keeps the 3 legitimate unwraps visible. Deny-by-default plus documented exceptions beats warn-and-ignore: warnings scroll past, denied builds stop merges. One safety-critical crate denies 12 restriction lints and reviews each new allow in architecture meetings — friction proportional to consequence.
Formatting config is small but deserves one deliberate pass. rustfmt.toml with edition = "2024" plus 2-3 chosen options (imports_granularity, group_imports) standardizes the import chaos that default rustfmt leaves; beyond 5 options you're bikeshedding. Check formatting in editors (rust-analyzer format-on-save) so CI never surprises, and run cargo fmt --all before large refactors to separate style noise from logic diffs. Documentation lints complete the trio: #![warn(missing_docs, rustdoc::broken_intra_doc_links)] makes undocumented public API and stale references build warnings — promoted to deny in CI for libraries. Three configs, three gates, zero style discussions in review ever again.
Workspaces: Splitting a Codebase Into Focused Crates
A workspace is one repo, many crates, one lockfile. The root Cargo.toml declares [workspace] with members = ["api", "core", "cli"], each member a full crate with its own Cargo.toml, src/, and version. Members link via path dependencies (core = { path = "../core" }), Cargo builds them into a single shared target/ directory, and cargo test --workspace verifies everything in one command. The payoff is boundaries with teeth: api can't touch core's privates, versions evolve per crate, and teams own directories instead of negotiating one lib.rs. Codebases that split between 30K–80K lines report the sweet spot — earlier and the ceremony outruns the benefit.
Two workspace shapes dominate and picking wrong costs months. Virtual workspaces (root has [workspace] but no [package]) suit multi-binary products — the root is pure orchestration, all code lives in members. Root-package workspaces (root is both a crate and the workspace) suit libraries growing satellites — mylib plus mylib-cli, mylib-ffi. Virtual roots avoid the 'root crate vs members' version confusion entirely; root-package setups keep cargo publish simple for the flagship. One team ran a root-package workspace where the root's version drifted 3 minors from members — release notes became archaeology. Virtual root plus per-member versions ended it.
Path dependencies are the wiring and they have exactly one subtlety: versions still matter. core = { path = "../core", version = "0.4" } uses the path locally but records the version requirement for publishing — cargo publish refuses members whose path deps lack matching registry versions. Develop with paths, release with versions: bump member versions, publish leaves-first (dependency order), then tag. cargo tree -p api shows the resolved graph; cargo metadata --format-version 1 feeds custom tooling. When the graph surprises you, those two commands say why.
The shared target/ directory is a quiet performance win: dependencies compile once for all members instead of per-crate, cutting clean build times 30–50% on 8+ member repos. The shared Cargo.lock is a correctness win: one resolved dependency set fleet-wide, no diamond-version drift between api and worker. Costs are real too — cargo check --workspace compiles everything (use -p for focus), and feature unification merges features across members (test --all-features plus --no-default-features in CI to see both faces). Measure: time cargo check -p core versus --workspace monthly; when the gap exceeds 3x, split CI into per-member jobs with a nightly full-workspace run.
Split along team and release boundaries, not file-size aesthetics: api (HTTP surface), core (domain logic), cli (operator tools), ffi (C bindings) — each with an owner, a version, and a reason to release independently. Shared code goes in a common member, never in duplicated modules. Workspaces turn 'the codebase' into 'our crates' — and crates with owners stay healthy years longer than directories with tenants.
Dependency governance is the workspace discipline nobody schedules until it hurts. cargo deny (licenses, advisories, bans) in CI blocks GPL-licensed transitive deps before legal notices do — one fintech caught a copyleft crate 3 layers deep that would have forced source disclosure. cargo audit (or cargo-deny advisories) on every PR flags RustSec advisories within hours, not quarters. Duplicate-dependency hygiene (cargo tree -d to list duplicates, then [patch] or version unification) keeps two serde versions from bloating binaries 15% and confusing debuggers. Schedule a monthly 30-minute dependency review: update, audit, dedupe — the compound interest of neglected deps is a week-long upgrade crisis yearly.
Build-time budgets make workspace performance a tracked metric instead of a complaint. Time the three commands monthly (cargo check -p core for focus, cargo check --workspace for full, cargo build --release for ship) and graph them — regressions from new dependencies show as step changes attributed to specific PRs. sccache (shared compilation cache) cuts CI rebuilds 40-60% across pipelines; lld/mold linkers (via .cargo/config.toml) cut link times 2-5x on large binaries. One 12-member workspace went from 22-minute CI to 7 minutes with sccache + mold + per-member job splitting — the same tests, one-third the wait. Fast builds get run; slow builds get skipped. Budget accordingly.
Workspace Inheritance: One Version Number for Every Member
Twelve crates, twelve version fields, one release day — without inheritance that's eleven chances to ship mismatched numbers. [workspace.package] fixes it: the root declares version, edition, rust-version, license, authors, and repository once, and members write version.workspace = true to inherit. Bump the root from 0.4.0 to 0.5.0 and all twelve members release in lockstep with a one-line diff. The same mechanism covers dependencies: [workspace.dependencies] pins serde = "1.0" once, members declare serde.workspace = true, and the whole repo upgrades crates in a single edit instead of twelve conflicting ones.
rust-version deserves special attention because it's your MSRV contract. workspace.package rust-version = "1.78" declares the minimum toolchain every member supports; CI pins that exact version in a matrix job (cargo +1.78 check --workspace) so newer APIs can't sneak in. Members inherit with rust-version.workspace = true. One data-tools repo skipped the MSRV job for a quarter — a contributor used let-chains (1.80+), and enterprise users on Debian-stable toolchains couldn't build for 3 weeks. The one-line field plus one CI job would have caught it at PR time.
Edition and license inherit the same way and solve quieter problems. Mixed editions (2021 in core, 2018 in cli) compile fine but confuse contributors about which idioms apply — inherit edition.workspace = true and the repo speaks one dialect. License inheritance guarantees every published member carries the same SPDX identifier; auditors checking 12 crates find 12 identical answers instead of 11 plus one forgotten field that blocks a release. Repository and homepage inheritance keep crates.io pages consistent — small polish that signals maintained software.
Dependencies via [workspace.dependencies] centralize more than versions: features unify intentionally (tokio with ["full"] declared once, inherited everywhere needed), and cargo update -p serde moves the fleet together. Override per-member only with justification comments — api needing tokio/full while worker needs tokio/minimal is legitimate; twelve divergent serde versions is drift. Run cargo tree --workspace --depth 1 monthly and eyeball duplicates; two versions of the same crate in one binary bloats builds and confuses debuggers.
Adopt incrementally: add [workspace.package] with version + edition + rust-version, convert two members, verify cargo build --workspace and cargo publish --dry-run per member, then roll out. Keep publishable leaves versioned independently only when release cadences genuinely differ (stable core 1.x vs experimental cli 0.x) — and document the exception at the root. Inheritance turns release day from a 12-file scavenger hunt into a one-line commit with a tag.
MSRV policy turns rust-version from decoration into contract with three parts. One: the field itself (workspace.package rust-version = "1.78", inherited per member) states the promise. Two: a CI job pinning exactly that toolchain (cargo +1.78 check --workspace --all-targets) proves the promise on every PR — plus a bump cadence (MSRV follows Debian-stable or N-2 policy, reviewed quarterly) so the promise stays current without fossilizing. Three: API discipline for version-gated syntax — let-chains need 1.80 and newer std APIs need their stabilizing release, so grep PRs for freshly stabilized features and let Clippy's version-aware lints flag what they recognize. Enterprise users on managed toolchains (aerospace, automotive, Debian-stable fleets) choose dependencies by MSRV first — a documented 1.78 floor with proof wins selections that newer-only crates lose silently.
Edition migration is the rarer sibling worth one prepared paragraph. Editions (2015, 2018, 2021, 2024) opt into backward-incompatible language improvements without breaking old code — cargo fix --edition migrates mechanically (reserved keywords, match ergonomics, prelude changes), and mixed-edition workspaces compile fine since dependencies on any edition interoperate. Migrate the workspace root first and members incrementally, verifying cargo test --workspace at each step; new reserved words surface as warnings before errors, giving a graceful window. One 8-member workspace migrated 2018 to 2021 in an afternoon with zero behavior changes — editions are the smoothest breaking changes in the industry, by design.
Publishing to crates.io: Versions, Docs, and cargo publish
Publishing is a checklist, not a gamble: version set, docs render, dry-run green, then publish. cargo publish --dry-run performs every step except upload — packaging files per include/exclude rules, verifying the manifest, building the crate — and fails loudly on the mistakes that otherwise surface mid-release. Run it per publishable member after every version bump; one team made --dry-run a CI job on release branches and cut failed publishes from monthly to zero across a year. The 5 minutes it adds beats the archaeology of a half-published workspace.
Metadata decides discoverability. description (one sentence, under 100 chars), license (SPDX like MIT OR Apache-2.0), repository, homepage, documentation, keywords (max 5), and categories shape your crates.io page and search ranking. Crates with complete metadata get 3–5x the downloads of bare ones — the registry is a marketplace and pages are packaging. Exclude clutter with exclude = ["tests/fixtures/*", ".github/"] to keep .crate files lean; one ML-adjacent crate shipped 400 MiB of fixtures twice before an exclude rule shrank packages to 2 MiB. Private internals set publish = false so cargo publish --workspace skips them automatically.
Version numbers are semver promises the ecosystem enforces. 0.x.y: anything may change; 1.2.3: breaking changes only at major bumps, features at minor, fixes at patch. cargo publish rejects re-uploading an existing version — yank (cargo yank) hides broken releases without rewriting history. Dependents write serde = "1.0" meaning >=1.0.0, <2.0.0 (caret semantics); libraries must keep version requirements minimal so downstream resolution succeeds. The cardinal sin is a breaking change in a patch release — one logging crate did it and broke 2,000 downstream builds within a day. Respect the numbers.
On Cargo 1.90+, cargo publish --workspace publishes every publishable member in dependency order in one command — the flagship workflow for multi-crate repos. Combined with workspace inheritance (one root version bump), releasing 12 crates becomes: bump root, run --dry-run per member, publish --workspace, tag. Verify order safety first: members with path dependencies need matching registry versions already published or publish refuses. For older toolchains the manual loop (publish leaves first, wait for index propagation, then dependents) still works — script it, never hand-sequence it.
After publish, confirm like a professional: cargo install yourcrate --locked in a scratch container, run its examples, check docs.rs rendered within the hour. Monitor download stats and issue trackers for the 48-hour window when integration bugs surface. Publishing isn't the finish line — it's the handoff to every developer who'll ever type your crate's name. Make the handoff clean.
Supply-chain verification completes the publishing story beyond version numbers. Cargo.lock committed for binaries (never for libraries — downstream resolves fresh) pins exact builds for reproducible deploys; --locked in CI and Docker builds fails loudly on drift instead of silently upgrading. cargo vet (Mozilla's supply-chain audit tool) records human audits of dependency diffs — 'we reviewed serde 1.0.197 → 1.0.200, 400 lines, no io/network changes' — turning blind trust into evidenced trust for security-conscious consumers. Checksum verification is built in (registry index + .crate hashes), and [patch] sections or vendored sources (cargo vendor) cover air-gapped builds where registries are unreachable. One regulated-industry publisher ships vendored sources plus vet audits with every release — procurement reviews that used to take 6 weeks now take 3 days.
Documentation hosting multiplies publishing returns. docs.rs builds and hosts every version automatically — including feature-flag combinations (docsrs metadata in Cargo.toml selects featured docs) — so users read the exact API of the exact version they depend on. README examples mirrored as doc tests (tested in this guide's testing section) guarantee the front page compiles. Changelog discipline (CHANGELOG.md with Keep-a-Changelog format, updated per PR, released per version) converts version bumps from mysteries into decisions — users upgrade faster when 0.5.0 → 0.6.0 lists 3 breaking changes with migration notes instead of silence. Publish the code, prove the supply chain, document the delta: that's a release.
Build Profiles and Feature Flags: dev vs release Without Forks
Profiles separate iteration speed from runtime speed. dev (default cargo build) compiles fast with debug assertions and no optimization — check times in seconds, binaries 2–5x slower. release (cargo build --release) enables opt-level=3, strips symbols, runs 10–100x faster on numeric code — at 3–10x compile cost. One raytracer measured dev frames at 41 seconds vs release at 0.8 — a 50x gap that decides whether profiling data means anything. Never benchmark dev builds; never debug-iterate on release. The profile is the first question for any perf number that looks wrong.
Customize profiles deliberately in the root Cargo.toml. [profile.release] lto = true squeezes 5–15% more runtime for 2x link times (worth it for shipped binaries, not libraries). codegen-units = 1 trades parallelism for optimization quality. panic = "abort" shrinks binaries but kills unwinding (breaks tests relying on catch_unwind). strip = true drops symbols for 20–30% smaller artifacts. Custom profiles inherit: [profile.dist] inherits = "release" plus tweaks, selected via --profile dist. Document each non-default setting's measured justification — 'lto: -12% p99 on checkout bench, +80s CI' beats folklore.
Feature flags carve optional functionality without forking code. [features] default = ["tls"] with tls = ["dep:tokio"] and serde-support = ["dep:serde"] lets users opt in: mycrate = { version = "1", default-features = false, features = ["serde-support"] }. In code, #[cfg(feature = "tls")] gates modules and unreachable variants vanish from builds that skip them. The dep: syntax (2021+ resolver) avoids implicit features that leak transitive names into your API. One embeddedHAL-style crate ships 14 features across no_std to full-std — one codebase, five product shapes, zero forks.
Feature unification is the sharp edge: within one build, features enabled by any crate apply to all uses of that dependency. api enabling tokio/full forces worker's tokio to full too — usually harmless, occasionally a binary-size or compile-time surprise. CI must build the matrix: default, --no-default-features, and --all-features, because each combination is a distinct product users will try. additive-only is the law — features may add APIs, never remove or alter existing behavior, or downstream builds break depending on which sibling enabled what.
Choose defaults for the common case (tls on for a client library, off for embedded), keep the total under ~10 flags (beyond that, split crates), and document each flag's cost in the README. Profiles answer 'how fast'; features answer 'how much'. Together they ship one codebase to five targets — dev laptops, CI, release servers, embedded boards, wasm — without a single #[cfg] hack that isn't a declared, documented feature.
Profile tuning beyond the defaults rewards measurement with specific, quotable wins. Split debug-info: [profile.release] debug = "line-tables-only" keeps production stack traces symbolic while adding only 5-10% binary size (versus full debug info at 2-4x). Panic strategy: panic = "abort" shrinks binaries 5-15% and speeds unwinding-heavy paths, but breaks catch_unwind-based tests and libraries — binaries only, never library profiles. Optimization granularity: opt-level = "s"/"z" (size) for embedded/wasm targets where 100 KiB matters more than 5% speed; opt-level = 3 with lto = "thin" for servers where the 12% p99 gain funds doubled link times. Each setting carries a measured receipt — record the benchmark delta in a comment beside the setting so the next engineer knows the price.
Feature-flag hygiene at scale follows three rules that prevent combinatorial explosion. First, cap flags near 10 per crate — beyond that, Cargo's feature resolver still works but human comprehension doesn't; split the crate (core vs extras) instead. Second, test the power set's corners, not its middle: default, --no-default-features, --all-features, plus each non-default flag solo — 90% of breakage lives at these edges. Third, document flag costs honestly (tls: +800 KiB binary, +2s build; serde-support: pure API addition, zero runtime cost) so users choose with data. A networking crate's 23 flags collapsed to 8 after this audit — same capabilities, one-third the support matrix, and CI dropped from 14 jobs to 5. Flags are products; curate them.
Toolchain Troubleshooting: Linkers, Targets, and cargo clean
Three failures cause 80% of 'Rust is broken' tickets and none of them are rustc's fault. Missing C linker: error: linking with cc failed — rustc compiles but cc links, so fresh Linux images without build-essential, Macs without Xcode CLT, and Windows without MSVC shims all fail at the final step with every crate compiling fine. Missing target: can't find crate for std on a cross triple — the std prebuilt for that target isn't installed. Wedged artifacts: behavior that survives reverts, phantom errors, stale binaries — fingerprints out of sync with sources. Learn these three signatures and you'll clear most toolchain pages in 10 minutes.
The linker fix is per-OS and permanent. Debian/Ubuntu: sudo apt-get install -y build-essential pkg-config (libssl-dev too if TLS crates link OpenSSL). macOS: xcode-select --install for clang and the SDK headers. Windows: Visual Studio Build Tools with the Desktop C++ workload, or the GNU toolchain via MSYS2 for -gnu targets. Verify with cc --version before touching Cargo — if cc is missing, no cargo flag helps. Encode it: Dockerfiles install build-essential alongside rustup, setup scripts assert cc --version first, and base images carry a hello-world link smoke test that blocks promotion when the linker vanishes.
Cross targets follow one flow: rustup target list | grep installed shows what you have, rustup target add aarch64-unknown-linux-gnu (or wasm32-unknown-unknown, x86_64-pc-windows-msvc) installs the std prebuilt, cargo build --target <triple> uses it. Tier-2 targets may need linker configuration in .cargo/config.toml ([target.aarch64-unknown-linux-gnu] linker = "aarch64-linux-gnu-gcc") plus the cross-gcc package. For embedded (thumbv7em-none-eabihf) there's no OS linker at all — .cargo/config.toml plus a linker script and flip-link complete the chain. When cross builds fail, read the error's first line: 'can't find crate' means missing target (rustup), 'linker not found' means missing cross-gcc (apt), 'undefined reference' means sysroot/library mismatch.
cargo clean is the controlled demolition. cargo clean -p foo removes one crate's artifacts (surgical, keeps the shared cache — try this first). cargo clean removes all of target/ (minutes to rebuild on big workspaces — the price of certainty). Deletion of ~/.cargo/registry is the nuclear tier for corrupted downloads, followed by cargo fetch to repopulate. Escalate in that order and document which tier fixed it; teams that jump to full cleans on every hiccup burn 30+ developer-minutes per incident in rebuilds. If wedging recurs, hunt causes: build.rs mtime games, NFS clock skew (git worktree on local disk to test), antivirus locks on target/ (Windows exclusions).
Prevent the trio with a setup contract: README prerequisites per OS, setup.sh asserting cc + rustup + targets, CI matrix on pinned + stable + MSRV rust-version, and base-image smoke builds linking hello-world. Toolchain trouble is infrastructure trouble — treat it with contracts and smoke tests, not folklore. The 14-engineer Monday outage in this article's incident section ended with exactly these four guards, and linker pages dropped to zero for the year that followed.
Container and CI hardening generalizes the incident's lessons into reusable patterns. Pin base images by digest (rust:1.90-slim-bookworm@sha256:...), not tag — tags move, digests don't, and Monday's surprise was a moved tag. Multi-stage Dockerfiles separate build (full toolchain + cc + vendored deps) from runtime (distroless/cc-runtime with only linked libraries — verify with ldd) so shipped images stay small without starving the linker at build time. Pre-baked toolchain images (rustup + targets + sccache, rebuilt weekly by automation) cut CI setup from 4 minutes to 20 seconds per job and eliminate 'which toolchain version ran this?' forensics. Cache target/ and registry across runs (actions/cache with Cargo.lock-hash keys) for 50-70% faster rebuilds — keyed caches invalidate exactly when dependencies change.
Local reproduction of CI failures closes the debugging loop. A justfile or xtask (cargo xtask ci) encoding the exact CI command sequence (fmt --check, clippy deny, test workspace all-features, doc build) lets developers run the pipeline locally before pushing — 'works on my machine' dies when local runs the same commands. Containerized local CI (same image as runners via docker run) removes the last divergence for toolchain-sensitive failures like linker errors. One team cut CI-failure iterations from 3.2 rounds per PR to 1.1 with xtask parity — the fix for environment bugs is identical environments, automated. Platform reliability is a feature; ship it like one.
The Missing C Linker That Blocked 14 Engineers for a Morning
cc failed, exit status: 1 and a note about unable to find library or object file. No test output, no warnings — compilation of every crate succeeded and the failure hit at the final link step. Local builds on developer laptops stayed green, which sent the first 30 minutes of debugging toward 'what changed in our code over the weekend' — nothing had. The blast radius was total: 63 pipelines red across 5 services, and the failure signature was identical in each.- Rust needs a C linker even for pure-Rust code — rustc compiles, cc links. Encode toolchain prerequisites (gcc/clang + rustup + targets) as asserted setup steps, not wiki prose. A 20-line smoke job compiling hello-world inside the CI image would have caught this Friday, not Monday.
- Base-image changes need consumer contract tests: the image's own green build proves nothing about the 5 languages built on top of it. Test what riders need (link a binary per toolchain), not just that the image assembles.
- When every pipeline fails identically with zero code changes, debug the platform first: diff the runner image, not Cargo.lock. Blast-radius-wide identical signatures mean shared infrastructure, and the fastest signal is rebuilding one pipeline on last week's image.
cc failed, exit status 1 — every crate compiles, the final link dies| File | Command / Code | Purpose |
|---|---|---|
| paths_demo.rs | mod network { | Modules and Paths |
| visibility_demo.rs | pub struct Money { | Visibility That Scales |
| layout.sh | cargo new --lib shoplib | lib vs bin Layout |
| test_loop.sh | cargo test 2>&1 | tail -8 | cargo test |
| quality_gates.sh | cargo fmt --all | Clippy, fmt, and doc |
| workspace.sh | mkdir -p shop/{core,api,cli}/src | Workspaces |
| inheritance.sh | cat > shop/Cargo.toml << 'EOF' | Workspace Inheritance |
| publish.sh | grep -A6 '\[package\]' shop/core/Cargo.toml | head -12 | Publishing to crates.io |
| features.rs | pub fn connect(url: &str) -> String { | Build Profiles and Feature Flags |
| toolchain_fix.sh | cc --version || echo "NO LINKER FOUND" | Toolchain Troubleshooting |
Key takeaways
Common mistakes to avoid
7 patternsSplitting a workspace before 30K lines (or splitting by file size)
Making everything pub instead of defaulting private
Testing only with cargo test -p one-crate locally
Skipping --dry-run before cargo publish
Benchmarking dev-profile builds
Forgetting rust-version (MSRV) while using new APIs
Full cargo clean as the first response to any build oddity
Interview Questions on This Topic
How do mod, use, crate, self, and super fit together?
Frequently Asked Questions
20+ years shipping production backend systems. Everything here is grounded in real deployments.
That's Tooling. Mark it forged?
26 min read · try the examples if you haven't