Does the concept of a compiler fence make any sense?

[RalfJung]

Atomic fences are specified in terms of things that happen when certain atomic reads occur:

• on thread A: first a fence, then some (atomic) store
• on thread B: first an (atomic) load, then a fence

In that particular situation, if the load reads-from the store, then the fences kick in and have an effect. That is the only effect they have, I think.

So, if your program contains no atomic accesses, but some atomic fences, those fences do nothing.
We also think that an atomic fence has at least all the effects of a compiler fence, i.e., a compiler fence is strictly weaker than an atomic fence. But that means a compiler fence has no effect on programs without atomic accesses -- which is just wrong, that's not how they are supposed to behave.

So what is the operational spec of a compiler fence? I have no idea.
See the discussion here for some more details. Let's go on discussing here.

[Diggsey]

I think atomic fences have additional guarantees that cannot be specified in terms of the abstract machine. Compiler fences provide a subset of those additional guarantees.

These guarantees would probably have to be specified in the same way that FFI or inline assembly is specified - as a relation between the abstract machine state and the underlying lower level state of the machine.

If you look at the uses for a compiler fence: https://stackoverflow.com/a/18454971

Both uses boil down to a single thread of execution being interrupted (either by an actual interrupt, or by a context switch) to run some other code, and the programmer wants to synchronize access to memory between the interrupted code and the interrupting code. This cannot be specified at an abstract machine level, because the abstract machine doesn't have a concept of a thread being interrupted in this way.

Going back to "atomic fences have additional guarantees that cannot be specified in terms of the abstract machine":

I think this is evidenced by the fact that we would expect atomic access from Rust code to be able to synchronize with atomic accesses from another language (or indeed assembly) in the same program. Therefore there must be "more to" atomic accesses than just their semantics within the abstract machine.

[RalfJung]

So you are going in the direction of what was discussed around here.

These guarantees would probably have to be specified in the same way that FFI or inline assembly is specified - as a relation between the abstract machine state and the underlying lower level state of the machine.

That would block a lot more optimizations that you would want when writing concurrent software with atomics. In particular we want there to be a meaningful difference between atomic fences with different orderings, which I do not think is the case under your proposal. So I don't think atomic fences should have any effect like that.

So maybe the statement that an atomic fence is strictly stronger than a compiler fence is wrong?

I think this is evidenced by the fact that we would expect atomic access from Rust code to be able to synchronize with atomic accesses from another language (or indeed assembly) in the same program.

I don't agree. Atomics are a feature inside the language and its memory model, so one should only expect them to synchronize inside that memory model. So whatever the other code does must be cast in terms of that memory model.

C++ atomics are not some abstract way of talking about what the hardware does, they are their completely own thing that is then implemented in terms of what the hardware does. There are sometimes even multiple different mutually incompatible schemes to implement the same model on a given hardware!

@DemiMarie (in the old thread)

Are there semantics for interrupts at all? That is a prerequisite for having semantics for compiler fences.

I guess part of the question here is whether interrupts exist as a language feature (that we'd have to add to the AM) and compiler fences interact with that, or whether we consider them to be outside the AM and compiler-fence to be a way to somehow coordinate with out-of-AM activities.

Atomic operations are not an out-of-AM mechanism. This is one of the many key differences between atomic accesses and volatile accesses. (E.g., the compiler is allowed to reorder and remove atomic accesses when that preserve the semantics of the abstract memory model.) So it would be very strange to me if atomic fences were ouf-of-AM mechanisms, and indeed that would put a lot of extra cost on concurrent algorithms that are perfectly happy staying entirely inside the AM.

[DemiMarie]

I think this is evidenced by the fact that we would expect atomic access from Rust code to be able to synchronize with atomic accesses from another language (or indeed assembly) in the same program.

I don't agree. Atomics are a feature inside the language and its memory model, so one should only expect them to synchronize inside that memory model. So whatever the other code does must be cast in terms of that memory model.

Cross-FFI atomics need to work. The question is how to specify atomics in such a way that they do.

[Diggsey]

In particular we want there to be a meaningful difference between atomic fences with different orderings, which I do not think is the case under your proposal. So I don't think atomic fences should have any effect like that.

That seems like a bit of a leap? I assume you are referring to:

There are sometimes even multiple different mutually incompatible schemes to implement the same model on a given hardware!

I would expect atomics to be like part of the ABI - ie. the compiler is not expected to make any kind of atomic accesses from FFI code work correctly, but as long as the FFI code follows the same rules (for that ordering) as the Rust compiler does, then they would be expected to synchronize. I don't see why this would make orderings redundant.

[Diggsey]

I don't agree. Atomics are a feature inside the language and its memory model, so one should only expect them to synchronize inside that memory model. So whatever the other code does must be cast in terms of that memory model.

It's my understanding that atomics are the expected way to do IPC via shared memory. I don't see how to reconcile your statement with that. (This is certainly a use-case supported by C++'s atomics)

[RalfJung]

It's my understanding that atomics are the expected way to do IPC via shared memory. I don't see how to reconcile your statement with that. (This is certainly a use-case supported by C++'s atomics)

For shared memory with other code that follows the same memory model. Basically, with other instances of (roughly) the same AM.

[Lokathor]

I'm unclear: are you saying that Rust atomics can sync with C++ atomics, or not?

[RalfJung]

That seems like a bit of a leap? I assume you are referring to:

No that's not what I am referring to. I'm saying atomic::fence has this ordering parameter and we agree it is relevant for the semantics, right? Because under your spec I don't see how it is.

I'm unclear: are you saying that Rust atomics can sync with C++ atomics, or not?

We are using the C++ memory model so that works fine.

But syncing e.g. with assembly code only makes sense if whatever that assembly code does can be expressed in terms of this memory model. You can't expect an atomic fence to have any effect other than what it says in the memory model.

[RalfJung]

I guess part of the question here is whether interrupts exist as a language feature (that we'd have to add to the AM) and compiler fences interact with that, or whether we consider them to be outside the AM and compiler-fence to be a way to somehow coordinate with out-of-AM activities.

To be clear, I think saying that compiler_fence is an out-of-AM syncing mechanism is not a bad idea. However, that would make compiler_fence incomparable in strength with atomic fences (and having atomic::compiler_fence just makes no sense).

That is actually consistent with what @comex wrote

But if we do want to allow reordering, I'd say that compiler_fence should only have a specified effect in conjunction with volatile accesses, similar to how atomic fences only have an effect in conjunction with atomic accesses.

Volatile accesses and atomic accesses are also incomparable in strength.

However, I just realized compiler_fence has an Ordering, and I have no idea what that is supposed to mean then...

[DemiMarie]

I'm unclear: are you saying that Rust atomics can sync with C++ atomics, or not?

We are using the C++ memory model so that works fine.

I think we should seriously consider looking at the Linux kernel memory model as well. This has the advantage that it is known to work for low-level programming at scale, whereas I am not aware of the C++ memory model being used for such code. Rust will need to integrate with the Linux kernel memory model to be used as part of the Linux kernel anyway.

[Diggsey]

I'm saying atomic::fence has this ordering parameter and we agree it is relevant for the semantics, right? Because under your spec I don't see how it is.

Oh I see - because the memory ordering is part of the C++/Rust memory model.

I guess another option would be to add the concept of "interruption" to the abstract machine, and then define compiler fences in terms of that. I think it's possible to define everything about "interruption" except for what triggers it - that's the only part that's "outside" what we can define.

[RalfJung]

I think we should seriously consider looking at the Linux kernel memory model as well. This has the advantage that it is known to work for low-level programming at scale, whereas I am not aware of the C++ memory model being used for such code. Rust will need to integrate with the Linux kernel memory model to be used as part of the Linux kernel anyway.

I have opinions about that, but if you want to propose/discuss this, please make it a new issue. This one is about fences. :)

[m-ou-se]

But that means a compiler fence has no effect on programs without atomic accesses -- which is just wrong, that's not how they are supposed to behave.

Why is that wrong?

Single-threaded / signal / compiler fences are relevant for signal handlers, interrupts, and things like Linux' SYS_membarrier.

[Lokathor]

What's wrong is the "that means a compiler fence has no effect..." part, because they're supposed to have some sort of effect.

We all seem to agree they should do something.