Change HashMaps and HashSets in Cargo to use Fxhasher by Kobzol · Pull Request #17169 · rust-lang/cargo

Kobzol · 2026-07-02T12:19:15Z

What does this PR try to resolve?

While profiling the build performance of bors, one of the things that kind of screamed me in the performance profiles were many calls to SipHasher, the default hashing algorithm from the Rust standard library.

I tried to see what would be the performance effect of:

Replacing the hottest instances of hashing (in HashMap and HashSet) that I found in the profiles by rustc_hash::{HashMap, HashSet} (first commit)
Replacing pretty much all hashmaps and hashsets in Cargo with the FxHasher maps/sets from rustc_hash (second commit)

Note that the benchmark results below are based on #17167.

First commit

Benchmark 1: /projects/personal/rust/cargo/target/release/cargo test --no-run
Time (mean ± σ):     315.2 ms ±   2.2 ms    [User: 206.2 ms, System: 111.9 ms]
Range (min … max):   312.2 ms … 318.3 ms    10 runs

Benchmark 2: ./cargo-baseline test --no-run
Time (mean ± σ):     319.2 ms ±   1.2 ms    [User: 210.6 ms, System: 112.3 ms]
Range (min … max):   317.4 ms … 320.7 ms    10 runs

Summary
/projects/personal/rust/cargo/target/release/cargo test --no-run ran
1.01 ± 0.01 times faster than ./cargo-baseline test --no-run

# perf stat (baseline)
1 301 866 736      instructions                                                          

# perf stat (new)
1 244 381 900      instructions

Approximately a 1% win on wall-time, and 4.5% on instruction counts.

Second commit

Benchmark 1: ./cargo-perf test --no-run
Time (mean ± σ):     305.5 ms ±   2.0 ms    [User: 199.1 ms, System: 110.2 ms]
Range (min … max):   303.8 ms … 310.6 ms    10 runs

Benchmark 2: ./cargo-baseline test --no-run
Time (mean ± σ):     318.7 ms ±   1.9 ms    [User: 205.6 ms, System: 116.9 ms]
Range (min … max):   317.1 ms … 321.7 ms    10 runs

Summary
./cargo-perf test --no-run ran
1.04 ± 0.01 times faster than ./cargo-baseline test --no-run

# perf stat (baseline)
1 302 028 601      instructions 

# perf stat (new)
1 192 872 611      instructions

Approximately a 4% win on wall-time and 9% on instruction counts.

Similar micro-optimizations are a bit tricky, and I'd understand if you wouldn't want to land this. On one hand, the first commit has relatively few changes, but the perf. wins are much smaller than with the second commit. But the second commit has a lot of changes, and some of them are not performance-sensitive (but the wins are kind of spread throughout Cargo, so it's very difficult to estimate which ones are worth it or not).

For my personal projects where I care about performance and don't care about DoS hashing attacks, I usually create an alias for a HashMap and HashSet that uses e.g. FxHasher, and ideally also hashbrown with the inline-more feature. rustc kind of does the same, and tries to use the Fx hash maps/sets everywhere to have better performance overall.

I think that Cargo could do the same, but I don't know if you'd like to have different conventions about it than what I did in the second commit.

rustbot · 2026-07-02T12:19:23Z

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: @ehuss, @epage, @weihanglo
@ehuss, @epage, @weihanglo expanded to ehuss, epage, weihanglo
Random selection from ehuss, epage, weihanglo

epage · 2026-07-02T14:46:01Z

I believe this would resolve #15649. On that, there was some discussion on what hasher to use.

Haven't checked the implementation yet but I'm fully on board with aliases. Also, did you check our IndexMaps?

I also wonder about moving away from hashmaps of unit -> structs in compilation to tracking an index and using struct-of-arrays to just remove any of this overheadd

epage · 2026-07-02T15:04:15Z


    gctx.shell()
-        .print_json(&HashMap::from([("success", "true")]))?;
+        .print_json(&HashMap::from_iter([("success", "true")]))?;


Aside: would love to have easy access to a const array-map.

View changes since the review

epage · 2026-07-02T15:05:37Z

 use cargo_util::paths;
 use cargo_util_schemas::manifest::RustVersion;
-use std::collections::hash_map::{Entry, HashMap};
+use std::collections::hash_map::Entry;


Still using a HashMap?

View changes since the review

I assume it's because Entry is compatible. Feels weird from an auditing perspective to import related type from two different places.

@weihanglo, any thoughts?

rustc_hash still re-exports the stdlib's HashMap, just with a different hasher, so it is indeed compatible. It doesn't export any Entry though, so I think that this is the only way to do it (or we can just re-export the Entry from util::data_structures 🙃).

epage · 2026-07-02T15:12:11Z

    // Move the old token location to the new one.
    if let Some(token) = toml.remove("token") {
-        let map = HashMap::from([("token".to_string(), token)]);
+        let map = std::collections::HashMap::from([("token".to_string(), token)]);


I assume this is needed for interop?

View changes since the review

Yes, the TOML implementation is not present for the FxHasher version.

Kobzol · 2026-07-02T15:44:19Z

Also, did you check our IndexMaps?

Not that many of those. I converted them to FxHash, but there doesn't seem to be a lot of changes on top of the second commit. Maybe a 0.5% instruction count reduction, but no significant wall-time change.

That being said, I think that creating an alias for IndexMap/Set is still useful, as it would allow us to more easily change the implementation or hash in the future.

Kobzol · 2026-07-02T18:47:29Z

I missed a bunch of hashmaps and hashsets previously, now it should be all of them.

I wanted to use Clippy's disallowed-types config to forbid using the stdlib's types in the future, but I don't know how to do that only for the main cargo crate; we want to allow their usage in the other workspace crates, I suppose?

rustbot · 2026-07-05T19:45:17Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kobzol · 2026-07-05T19:45:26Z

Added the types to Clippy's disallowed_types set.

weihanglo · 2026-07-05T21:04:42Z

https://github.com/rust-lang/cargo/actions/runs/28752700767/job/85254494553#step:5:713

Mind bumping patch versions for them?

…ructures We want to use Cargo's optimized HashMap/HashSet and IndexMap/IndexSet versions with a fast hashing function instead. Allow using these types in various auxiliary crates, because we cannot currently tell Clippy to only apply the lint in the Cargo crate.

weihanglo

Thanks. I just fixed the outdated lockfile issue. This looks good to me to merge now.

As for Entry, we could deal with it in a follow-up. It would be less confusing indeed though not strictly necessary.

View changes since this review

Kobzol · 2026-07-06T17:16:14Z

For future reference, I tried to depend directly on hashbrown, and also switch to foldhash, on top of this change. Neither seemed to have any additional effect on perf.

Update cargo submodule 4 commits in 2f0e7011e0e9cb30cb772bf2ec1d69dce4dff4f3..59800466c5c41c444d264b1010b4d57e85a7117f 2026-07-05 12:10:34 +0000 to 2026-07-07 15:52:22 +0000 - Revert "feat: Stablize build-dir layout v2" (rust-lang/cargo#17187) - Use a set when checking visited workspace members (rust-lang/cargo#17180) - Stabilize `build-dir` layout v2 (rust-lang/cargo#16807) - Change HashMaps and HashSets in Cargo to use Fxhasher (rust-lang/cargo#17169)

rustbot assigned epage Jul 2, 2026

epage reviewed Jul 2, 2026

View reviewed changes

Kobzol force-pushed the hashmap branch from f00d779 to 00e789a Compare July 5, 2026 19:45

rustbot added A-cli-help Area: built-in command-line help A-credential-provider Area: credential provider for storing and retreiving credentials A-infrastructure Area: infrastructure around the cargo repo, ci, releases, etc. A-testing-cargo-itself Area: cargo's tests labels Jul 5, 2026

weihanglo force-pushed the hashmap branch from d96d0a6 to e8a2efa Compare July 6, 2026 03:17

weihanglo approved these changes Jul 6, 2026

View reviewed changes

Bump cargo-credential/cargo-test-support/rustfix

32cdb8a

weihanglo force-pushed the hashmap branch from e8a2efa to 32cdb8a Compare July 6, 2026 03:24

weihanglo enabled auto-merge July 6, 2026 03:24

weihanglo added this pull request to the merge queue Jul 6, 2026

weihanglo mentioned this pull request Jul 6, 2026

Use a set when checking visited workspace members #17180

Merged

Merged via the queue into rust-lang:master with commit f3cae1a Jul 6, 2026
31 checks passed

Kobzol deleted the hashmap branch July 6, 2026 06:28

weihanglo mentioned this pull request Jul 6, 2026

Update cargo submodule rust-lang/rust#158861

Merged

weihanglo mentioned this pull request Jul 6, 2026

Use another hashing algorithm? 250K calls to Siphash #15649

Closed

rustbot added this to the 1.99.0 milestone Jul 8, 2026

Uh oh!

Uh oh!

Conversation

Kobzol commented Jul 2, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR try to resolve?

First commit

Second commit

Uh oh!

rustbot commented Jul 2, 2026

Uh oh!

epage commented Jul 2, 2026

Uh oh!

epage Jul 2, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

epage Jul 2, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

epage Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Kobzol Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

epage Jul 2, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kobzol Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Kobzol commented Jul 2, 2026

Uh oh!

Kobzol commented Jul 2, 2026

Uh oh!

rustbot commented Jul 5, 2026

Uh oh!

Kobzol commented Jul 5, 2026

Uh oh!

weihanglo commented Jul 5, 2026

Uh oh!

weihanglo left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Kobzol commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kobzol commented Jul 2, 2026 •

edited by rustbot

Loading

epage Jul 2, 2026 •

edited by rustbot

Loading

epage Jul 2, 2026 •

edited by rustbot

Loading

epage Jul 2, 2026 •

edited by rustbot

Loading

weihanglo left a comment •

edited by rustbot

Loading