Enable full LTO on Python 3.12 and 3.13 by zanieb · Pull Request #529 · astral-sh/python-build-standalone

zanieb · 2025-02-13T15:08:58Z

Closes #528

charliermarsh · 2025-02-13T15:15:51Z


 if [ -n "${CPYTHON_LTO}" ]; then
-    CONFIGURE_FLAGS="${CONFIGURE_FLAGS} --with-lto"
+    # On Python 3.12 and 3.13, `--with-lto` enables ThinLTO by default, while on other versions it


What about 3.14? Did it change in 3.14?

3.14a5 and below still defaults to ThinLTO. 3.14a6 (upcoming) will default back to fullLTO.

It's safe to just pass --with-lto=full to all versions of 3.14. That will always work.

Yes the linked documentation says they're reverting the ThinLTO default in 3.14.

Seems best to just set it on all versions.

zanieb · 2025-02-13T16:07:53Z

I'll try to do a brief analysis of the performance and build time trade-offs here.

zanieb · 2025-02-13T16:26:35Z

Total job time (main / branch)

Linux 34m / 37m
macOS 1h 13m / 1h 3m

Target job time (main / branch)

aarch64-apple-darwin / 3.12 / pgo+lto 15m / 16m
aarch64-apple-darwin / 3.13 / pgo+lto 14m / 16m
aarch64-apple-darwin / 3.14 / pgo+lto 15m / 16m
x86_64-apple-darwin / 3.12 / pgo+lto 15m / 17m
x86_64-apple-darwin / 3.13 / pgo+lto 16m / 19m
x86_64-apple-darwin / 3.14 / pgo+lto 16m / 18m
x86_64-unknown-linux-gnu / 3.13 / pgo+lto 28m / 28m

zanieb · 2025-02-13T16:32:13Z

Surprisingly, this is consistently a bit slower on the pystones benchmark used for #524 on aarch64-apple-darwin

❯ hyperfine "$(uv python find 3.12.9) ./pystones.py" "./python/install/bin/python ./pystones.py"
Benchmark 1: /Users/zb/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/bin/python3.12 ./pystones.py
  Time (mean ± σ):      75.1 ms ±   1.0 ms    [User: 72.3 ms, System: 2.1 ms]
  Range (min … max):    73.2 ms …  76.6 ms    39 runs
 
Benchmark 2: ./python/install/bin/python ./pystones.py
  Time (mean ± σ):      79.2 ms ±   1.0 ms    [User: 76.0 ms, System: 2.5 ms]
  Range (min … max):    77.6 ms …  81.7 ms    36 runs
 
Summary
  /Users/zb/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/bin/python3.12 ./pystones.py ran
    1.05 ± 0.02 times faster than ./python/install/bin/python ./pystones.py

❯ hyperfine "$(uv python find 3.13.2) ./pystones.py" "./python/install/bin/python3.13 ./pystones.py"
Benchmark 1: /Users/zb/workspace/python-build-standalone/.venv/bin/python3 ./pystones.py
  Time (mean ± σ):      75.7 ms ±   2.3 ms    [User: 72.4 ms, System: 2.4 ms]
  Range (min … max):    72.6 ms …  84.6 ms    39 runs
 
Benchmark 2: ./python/install/bin/python3.13 ./pystones.py
  Time (mean ± σ):      80.6 ms ±   2.4 ms    [User: 76.5 ms, System: 3.0 ms]
  Range (min … max):    77.1 ms …  88.4 ms    34 runs
 
Summary
  /Users/zb/workspace/python-build-standalone/.venv/bin/python3 ./pystones.py ran
    1.06 ± 0.05 times faster than ./python/install/bin/python3.13 ./pystones.py

zanieb · 2025-02-13T16:37:24Z

This is less pronounced (but still present?) if I compare to an artifact from main instead of a uv-installed binary

❯ hyperfine "./main/install/bin/python3.13 ./pystones.py" "./branch/install/bin/python3.13 ./pystones.py" --min-runs 100
Benchmark 1: ./main/install/bin/python3.13 ./pystones.py
  Time (mean ± σ):      77.4 ms ±   1.1 ms    [User: 74.5 ms, System: 2.3 ms]
  Range (min … max):    75.1 ms …  80.4 ms    100 runs
 
Benchmark 2: ./branch/install/bin/python3.13 ./pystones.py
  Time (mean ± σ):      79.2 ms ±   1.1 ms    [User: 76.3 ms, System: 2.3 ms]
  Range (min … max):    77.4 ms …  85.0 ms    100 runs
 
Summary
  ./main/install/bin/python3.13 ./pystones.py ran
    1.02 ± 0.02 times faster than ./branch/install/bin/python3.13 ./pystones.py

zanieb · 2025-02-13T16:40:59Z

I'll find another benchmark to try.

The summary at python/cpython#122580 (comment) is relatively compelling.

zanieb · 2025-02-13T16:47:40Z

With the script from python/cpython#122580 (comment) — which was the original reproduction for a regression, I also don't see an improvement

❯ hyperfine "./main/install/bin/python3.13 ./repro.py" "./branch/install/bin/python3.13 ./repro.py"
Benchmark 1: ./main/install/bin/python3.13 ./repro.py
  Time (mean ± σ):      5.926 s ±  0.048 s    [User: 5.900 s, System: 0.021 s]
  Range (min … max):    5.833 s …  5.975 s    10 runs
 
Benchmark 2: ./branch/install/bin/python3.13 ./repro.py
  Time (mean ± σ):      5.988 s ±  0.035 s    [User: 5.962 s, System: 0.018 s]
  Range (min … max):    5.927 s …  6.050 s    10 runs
 
Summary
  ./main/install/bin/python3.13 ./repro.py ran
    1.01 ± 0.01 times faster than ./branch/install/bin/python3.13 ./repro.py

Fidget-Spinner · 2025-02-13T16:47:42Z

@zanieb It might be that this was a bug in Apple Clang but fixed in normal Clang. The version of clang Ned used was clang-1500 but knowing Apple, that isn't actually clang 15 but some older internal version. The Clang version here is pretty new in comparison.

So I'm sorry for the noise if that turns out to be the case.

Fidget-Spinner · 2025-02-13T16:48:17Z

I think my guess is probably right. I will close the issue as the build time tradeoff is not worth it. So sorry for the noise!

zanieb · 2025-02-13T16:56:18Z

Does this mean I want to explicitly request thin on 3.14 now? :)

Fidget-Spinner · 2025-02-13T17:03:28Z

Does this mean I want to explicitly request thin on 3.14 now? :)

I think I will revert the change upstream. It's probably somewhat disruptive now anyways to flip-flop. Thanks.

zanieb · 2025-02-25T03:54:09Z

I'm going to explore this again in the context of #535 and #539 — I think thin LTO is one of the few configuration differences between our build and the conda-forge one.

I'll test some of the binaries here with the pyperformance suite.

Enable full LTO on Python 3.12 and 3.13

4d43c13

zanieb force-pushed the zb/lto branch from 61fcdd7 to 4d43c13 Compare February 13, 2025 15:09

charliermarsh reviewed Feb 13, 2025

View reviewed changes

Fidget-Spinner mentioned this pull request Feb 13, 2025

Revert to default fullLTO on Clang python/cpython#130048

Closed

zanieb closed this Feb 13, 2025

zanieb mentioned this pull request Feb 25, 2025

Enable computed gotos #539

Closed

Conversation

zanieb commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charliermarsh Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

Fidget-Spinner Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

zanieb Feb 13, 2025

Choose a reason for hiding this comment

Uh oh!

zanieb commented Feb 13, 2025

Uh oh!

zanieb commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zanieb commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zanieb commented Feb 13, 2025

Uh oh!

zanieb commented Feb 13, 2025

Uh oh!

zanieb commented Feb 13, 2025

Uh oh!

Fidget-Spinner commented Feb 13, 2025

Uh oh!

Fidget-Spinner commented Feb 13, 2025

Uh oh!

zanieb commented Feb 13, 2025

Uh oh!

Fidget-Spinner commented Feb 13, 2025

Uh oh!

zanieb commented Feb 25, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zanieb commented Feb 13, 2025 •

edited

Loading

zanieb commented Feb 13, 2025 •

edited

Loading

zanieb commented Feb 13, 2025 •

edited

Loading