TASK: determine feasibility of llvm18/clang18 removal wrt python3 JIT

[yselkowitz]

What does the ELN SIG need to do?

The new JIT in Python 3.13 has clang as a build (but not runtime) dependency. Unfortunately, the code is locked to a particular version; 3.13 uses llvm/clang 18, main (the future 3.14) uses 19. Therefore, a clang18 compat package is needed to build the JIT in Fedora 41 (which normally uses llvm/clang 19). On Fedora this isn't a big deal, but on ELN this is pulling in llvm18/clang18 into the buildroot (all other consumers have already been ported to the current 19).

For RHEL though this seems untenable. The default python3.Y package must remain buildable for the entire lifetime of RHEL, but llvm is continuously updated to newer versions. RHEL 10 has Python 3.12 as default, so there is no precedence yet, but this will come up should any additional versions be provided in parallel (albeit with shorter scope).

Therefore, we need clarification on these points:

• Should the JIT be enabled in RHEL/ELN builds of python 3.13+?
• If so, will a llvm compat version be kept in the buildroot for as long as necessary to continue building the JIT?
• Or, will support for newer llvm versions be backported? Can that even be done safely, particularly over such a long span?

/cc @hroncok

[yselkowitz]

FTR, the changes to port the JIT from 18 to 19 are extensive: python/cpython#125499

[hroncok]

Thanks for asking the question, this is definitely something that needs to be solved.

---

The default python3.Y package must remain buildable for the entire lifetime of RHEL, but llvm is continuously updated to newer versions.

That's definitely something we need to consider when we decide whether or not c11s will have Python JIT enabled. Or when we'll add Python 3.14 to c9s/c10s.

Should the JIT be enabled in RHEL/ELN builds of python 3.13+?

The CPython JIT is experimental, so it definitively isn't "enterprise material". At this point, I would not enable it in RHEL.

On the other hand, ELN is a place to experiment for future RHEL. Chances are that the JIT will become more stable in time for RHEL 11 (which will very likely contain Python 3.15/3.26).

So I'd like to keep this enabled in ELN for now (if it's not a big deal) and revisit this in 2 years. I would certainly not want to pull some old LLVM version to RHEL-proper, but but having it in ELN "for now" is probably OK.

If so, will a llvm compat version be kept in the buildroot for as long as necessary to continue building the JIT?

If so, then yes.

Will support for newer llvm versions be backported?

Time will tell. No idea yet.

[hroncok]

Upstreams considers supporting ranges of LLVM versions: https://discuss.python.org/t/llvm-19-for-jit-on-3-13-maybe-19-for-3-14/73425/2

[yselkowitz]

FTR, LLVM 20 is planned for F42, while Python will stay on 3.13. That means, barring any other changes, LLVM 18 will need to remain in for the duration of F42, as well as in ELN until 3.14. Hopefully 3.14 will be compatible with 20 and 21 in due course, otherwise Python (with its yearly cadence) will never keep up with LLVM (with its semi-yearly cadence).

As much as I wish to drop llvm/clang 18 from ELN, I guess there is nothing to do until 3.14 lands, at which point we can revisit this.

[hroncok]

See also this PEP draft: https://peps.python.org/pep-0774/

tl;dr we might be able to conditionalize the dependency and keep JIT even in EL(N)

[yselkowitz]

But FWIW the proposal only mentions Tier 1 platforms, which IIUC does not include ppc64le/s390x, unless you would just pregenerate those and keep them in dist-git?

[hroncok]

That's still to be determined but keeping them in distgit is an option. We currently only build the JIT on aarch64 and x86_64 anyway.