bpo-45637: Store the frame pointer in the cframe

[markshannon]

This provides better machine level debugger support. Since the cframe is always in memory, it can be reliably found by debuggers, unlike the frame local variable which might be (ideally should be) in a register.

In theory performance might improve a bit as the cframe is on the C stack, the fastest memory to access in any system. However benchmarking results are just noise; at least it isn't obviously slower.

https://bugs.python.org/issue45637

[pablogsal]

This PR would break all current out-of-process debuggers and profilers. All these tools are reliying on iterating over all the active frames by accessing the current frame from the thread states that are obtained from the _PyRuntime symbol. Removing it from there will be a masive breakage for all these tools.

Edit: See new comment

[bedevere-bot]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

[pablogsal]

Hmmm, I just saw the current frame is still accessible through the cframe pointer on the thread state. I think that is certainly better, although it complicates things still because more memory needs to be fetched (one extra level) and this memory is stack memory, which may complicate doing this from core files. I think we should leave the pointer in the thread state still as it has a negligible cost and it makes things much more simple and with less work to adapt.

In short, I still prefer this approach: #29257

[markshannon]

Either we store a pointer to the current frame in the cframe or in tstate; not both.
Storing it in both is error prone, as they could all too easily get out of sync.

Regarding efficiency, storing it the cframe is (in theory) more efficient as the hottest accesses to the frame are in the interpreter.

[pablogsal]

Regarding efficiency, storing it the cframe is (in theory) more efficient as the hottest accesses to the frame are in the interpreter.

I agree, but we are talking about not making life more difficult for other tools not about efficiency.

I think this would be workable, but I am a bit concerned about the extra hoop through the cframe. I will experiment a bit to see if this is going to be a real problem or not. For now, let's land this PR and we can revert it if it results a big problem,.

[pablogsal]

Wait, hold on, I am cheching and this approach sadly doesn't work with threads as cframe is still optimized out in threads :(

[pablogsal]

See: #29257 (comment)

[pablogsal]

I think we need to just mark the test as skippable if Python is compiled with optimizations, which is the case in the ASAN build.

[markshannon]

What does 'optimized out' mean here? cframe is in memory. It cannot be replaced by scalars.

[pablogsal]

What does 'optimized out' mean here? cframe is in memory. It cannot be replaced by scalars.

It means that gdb cannot read the value and shows it as "optimized out". This is a common problem if the DWARF data is not good enough to reconstruct whatever dance the optimizer is doing.