Skip to content

Add --base <run_id> to compare local run with another run#273

Open
GuillaumeLagrange wants to merge 3 commits intomainfrom
cod-1796-make-comparison-between-two-local-runs-easier
Open

Add --base <run_id> to compare local run with another run#273
GuillaumeLagrange wants to merge 3 commits intomainfrom
cod-1796-make-comparison-between-two-local-runs-easier

Conversation

@GuillaumeLagrange
Copy link
Contributor

@GuillaumeLagrange GuillaumeLagrange commented Mar 20, 2026

$ codspeed run -m walltime                                                                                       1 ↵

 Running the benchmarks

     Walltime - Running `./rust/target/release/graph-benchmark` with exec-harness
    ╭─ 511 lines above ───────────────────────────────────────────────────────────────────────────────────────────────╮
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

 Uploading results

  Performance data uploaded
  Linked repository: GuillaumeLagrange/local-runs
    Uploading results 3.4s

 Benchmark results

  Rust Fractal Exec: best 6.47 ms (396 iterations, rel. stddev: 3.35%, total 2.80 s)

  View full report: http://localhost:3000/GuillaumeLagrange/local-runs/runs/69bd6dc2724308afd1b36c4d
  To compare future runs against this one, use: --base 69bd6dc2724308afd1b36c4d
$ codspeed run -m walltime --base 69bd60894af4516d0aac60c7

 Running the benchmarks

     Walltime - Running `./rust/target/release/graph-benchmark` with exec-harness
    ╭─ 514 lines above ───────────────────────────────────────────────────────────────────────────────────────────────╮
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    │ Fractal computation result: 157504                                                                              │
    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

 Uploading results

  Performance data uploaded
  Linked repository: GuillaumeLagrange/local-runs
    Uploading results 1.1s

 Benchmark results

   Impact: +778.3%
  ╭────────────────────────────────────────────────────────────────╮
  │                            Walltime                           │
  ├───────────────────┬──────────┬─────────┬─────────┬─────────────┤
  │ Benchmark         │     Base │    Head │  Change │      Status │
  ├───────────────────┼──────────┼─────────┼─────────┼─────────────┤
  │ Rust Fractal Exec │ 56.94 ms │ 6.48 ms │ +778.3% │ Improvement │
  ╰───────────────────┴──────────┴─────────┴─────────┴─────────────╯

  View comparison report: http://localhost:3000/GuillaumeLagrange/local-runs/runs/compare/69bd60894af4516d0aac60c7..69bd6e18724308afd1b36c4e

@GuillaumeLagrange GuillaumeLagrange changed the title Cod 1796 make comparison between two local runs easier Add --base <run_id> to compare local run with another run Mar 20, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 20, 2026

Merging this PR will not alter performance

✅ 4 untouched benchmarks


Comparing cod-1796-make-comparison-between-two-local-runs-easier (2d85f8f) with main (6e55caf)

Open in CodSpeed

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch from f190462 to 8673a9c Compare March 20, 2026 14:34
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch 2 times, most recently from 7e5e798 to 753dc1f Compare March 20, 2026 14:47
Add --base cli argument and simplify polling code
1. If a base run id is provided, we now use the `compareRunsPaginated`
   resolver and display results from there
2. If a base run id is not provided or if the base run id is not correct
   (missing, executor mismatch...), we display the single mode results

This overall simplifies the polling code and gets rid of the artificial
"for_run" and "for_exec" distinctions.
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch from 0fffde9 to 2d85f8f Compare March 20, 2026 15:57
@GuillaumeLagrange GuillaumeLagrange marked this pull request as ready for review March 20, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant