Is there any archs where a memory barrier is implemented even with a cache flush? I read that memory barrier affects only CPU reordering but I read statements related to the memory barriers: ensures all the cpu will see the value..., but for me it means a cache flush/invalidation.
On pretty much all modern architectures, caches (like the L1 and L2 caches) are ensured coherent by hardware. There is no need to flush any cache to make memory visible to other CPUs.
One could imagine hypothetically a system that was not cache coherent in hardware, but it wouldn't look anything like the current systems that run operating systems like Windows and Linux.
Memory barriers are needed on these architectures to do three things:
The CPU may pre-fetch a read that's invalidated by a write on another core. This must be prevented. (Though on x86, this is prevented in hardware. The pre-fetch is locked to the L1 cache line, so if another CPU invalidates the cache line, the pre-fetch is invalidated as well.)
The CPU may "post" writes and not put them in its L1 cache yet. These writes must be completed at least to L1 cache.
The CPU may re-order reads and writes on one side of the memory barrier with reads and writes on the other side. Depending on the type of memory barrier, some of these re-orderings must be prohibited. (For example, read x; read y; doesn't ensure the reads happen in that order. But read x; memory_barrier(); read y; typically does.)
The exact impact of a memory barrier depends on the specific architecture
CPUs employ performance optimizations that can result in out-of-order
execution. The reordering of memory operations (loads and stores)
normally goes unnoticed within a single thread of execution, but
causes unpredictable behaviour in concurrent programs and device
drivers unless carefully controlled. The exact nature of an ordering
constraint is hardware dependent, and defined by the architecture's
memory ordering model. Some architectures provide multiple barriers
for enforcing different ordering constraints.
http://en.wikipedia.org/wiki/Memory_barrier
Current Intel architectures ensure automatic cache consistency across all CPU's, without explicit use of memory barrier or a cache flush instructions.
In symmetric multiprocessor (SMP) systems, each processor has a local
cache. The memory system must guarantee cache coherence. False sharing
occurs when threads on different processors modify variables that
reside on the same cache line. This invalidates the cache line and
forces an update, which hurts performance.
http://software.intel.com/en-us/articles/avoiding-and-identifying-false-sharing-among-threads/
Related
Consider the following example taken from Wikipedia, slightly adapted, where the steps of the program correspond to individual processor instructions:
x = 0;
f = 0;
Thread #1:
while (f == 0);
print x;
Thread #2:
x = 42;
f = 1;
I'm aware that the print statement might print different values (42 or 0) when the threads are running on two different physical cores/processors due to the out-of-order execution.
However I don't understand why this is not a problem on a single core machine, with those two threads running on the same core (through preemption). According to Wikipedia:
When a program runs on a single-CPU machine, the hardware performs the necessary bookkeeping to ensure that the program executes as if all memory operations were performed in the order specified by the programmer (program order), so memory barriers are not necessary.
As far as I know single-core CPUs too reorder memory accesses (if their memory model is weak), so what makes sure the program order is preserved?
The CPU would not be aware that these are two threads. Threads are a software construct (1).
So the CPU sees these instructions, in this order:
store x = 42
store f = 1
test f == 0
jump if true ; not taken
load x
If the CPU were to re-order the store of x to the end, after the load, it would change the results. While the CPU is allowed out of order execution, it only does this when it doesn't change the result. If it was allowed to do that, virtually every sequence of instructions would possibly fail. It would be impossible to produce a working program.
In this case, a single CPU is not allowed to re-order a store past a load of the same address. At least, as far the CPU can see it is not re-ordered. As far the as the L1, L2, L3 cache and main memory (and other CPUs!) are concerned, maybe the store has not been committed yet.
(1) Something like HyperThreads, two threads per core, common in modern CPUs, wouldn't count as "single-CPU" w.r.t. your question.
The CPU doesn't know or care about "context switches" or software threads. All it sees is some store and load instructions. (e.g. in the OS's context-switch code where it saves the old register state and loads the new register state)
The cardinal rule of out-of-order execution is that it must not break a single instruction stream. Code must run as if every instruction executed in program order, and all its side-effects finished before the next instruction starts. This includes software context-switching between threads on a single core. e.g. a single-core machine or green-threads within on process.
(Usually we state this rule as not breaking single-threaded code, with the understanding of what exactly that means; weirdness can only happen when an SMP system loads from memory locations stored by other cores).
As far as I know single-core CPUs too reorder memory accesses (if their memory model is weak)
But remember, other threads aren't observing memory directly with a logic analyzer, they're just running load instructions on that same CPU core that's doing and tracking the reordering.
If you're writing a device driver, yes you might have to actually use a memory barrier after a store to make sure it's actually visible to off-chip hardware before doing a load from another MMIO location.
Or when interacting with DMA, making sure data is actually in memory, not in CPU-private write-back cache, can be a problem. Also, MMIO is usually done in uncacheable memory regions that imply strong memory ordering. (x86 has cache-coherent DMA so you don't have to actually flush back to DRAM, only make sure its globally visible with an instruction like x86 mfence that waits for the store buffer to drain. But some non-x86 OSes that had cache-control instructions designed in from the start do requires OSes to be aware of it. i.e. to make sure cache is invalidated before reading in new contents from disk, and to make sure it's at least written back to somewhere DMA can read from before asking a device to read from a page.)
And BTW, even x86's "strong" memory model is only acq/rel, not seq_cst (except for RMW operations which are full barriers). (Or more specifically, a store buffer with store forwarding on top of sequential consistency). Stores can be delayed until after later loads. (StoreLoad reordering). See https://preshing.com/20120930/weak-vs-strong-memory-models/
so what makes sure the program order is preserved?
Hardware dependency tracking; loads snoop the store buffer to look for loads from locations that have recently been stored to. This makes sure loads take data from the last program-order write to any given memory location1.
Without this, code like
x = 1;
int tmp = x;
might load a stale value for x. That would be insane and unusable (and kill performance) if you had to put memory barriers after every store for your own reloads to reliably see the stored values.
We need all instructions running on a single core to give the illusion of running in program order, according to the ISA rules. Only DMA or other CPU cores can observe reordering.
Footnote 1: If the address for older stores isn't available yet, a CPU may even speculate that it will be to a different address and load from cache instead of waiting for the store-data part of the store instruction to execute. If it guessed wrong, it will have to roll back to a known good state, just like with branch misprediction.
This is called "memory disambiguation". See also Store-to-Load Forwarding and Memory Disambiguation in x86 Processors for a technical look at it, including cases of narrow reload from part of a wider store, including unaligned and maybe spanning a cache-line boundary...
Commonly, cacheline is 64B but atomicity of non-volatile memory is 8B.
For example:
x[1]=100;
x[2]=100;
clflush(x);
x is cacheline aligned, and is initially set to 0.
System crashs in clflush();
Is it possible x[1]=0, x[2]=100 after reboot？
Under the following assumptions:
I assume that the code you've shown represents a sequence of x86 assembly instructions rather than actual C code that is yet to be compiled.
I also assume that the code is being executed on a Cascade Lake processor and not on a later generation of Intel processors (I think CPL or ICX with Barlow Pass support eADR, meaning that explicit flushing is not required for persistence because the caches are in the persistence domain). This answer also applies to existing AMD+NVDIMM platforms.
The global observablility order of stores may differ from the persist order on Intel x86 processors. This is referred to as relaxed persistency. The only case in which the order is guaranteed to be the same is for a sequence of stores of type WB to the same cache line (but a store reaching GO doesn't necessarily meant it's become durable). This is because CLFLUSH is atomic and WB stores cannot be reordered in global observability. See: On x86-64, is the “movnti” or "movntdq" instruction atomic when system crash?.
If the two stores cross a cache line boundary or if the effective memory type of the stores is WC:
The x86-TSO memory model doesn't allow reordering stores, so it's impossible for another agent to observe x[2] == 100 and x[1] != 100 during normal operation (i.e., in the volatile state without a crash). However, if the system crashed and rebooted, it's possible for the persistent state to be x[2] == 100 and x[1] != 100. This is possible even if the system crashed after retiring clflush because the retirement of clflush doesn't necessarily mean that the cache line flushed has reached the persistence domain.
If you want to eliminate that possibly, you can either move clflush as follows:
x[1]=100;
clflush(x);
x[2]=100;
clflush on Intel processors is ordered with respect to all writes, meaning that the line is guaranteed to reach the persistence domain before any later stores become globally observable. See: Persistent Memory Programming Primary (PDF) and the Intel SDM V2. The second store could be to the same line or any other line.
If you want x[1]=100 to become persistent before x[2]=100 becomes globally observable, add sfence after clflush on Intel CSX or mfence on AMD processors (clflush is only ordered by mfence on AMD processors). clflush by itself sufficient to control persist order.
Alternatively, use the sequenceclflushopt+sfence (or clwb+sfence) as follows:
x[1]=100;
clflushopt(x);
sfence;
x[2]=100;
In this case, if a crashed happened and if x[2] == 100 in the persistent state, then it's guaranteed that x[1] == 100. clflushopt by itself doesn't impose any persist ordering.
(Also see #Hadi's answer: x86 TSO store ordering does not guarantee persistence ordering even within one line. This answer doesn't try to address that. My best guess based on Hadi's answer is that a single atomic store to one 32-byte half of a cache line will persist atomically, but that's based on how current HW works, transferring lines in 2 32-byte halves between cores, caches, and memory controllers. If this really matters, look for docs or ask Intel.)
Remember that store data can propagate out of cache (into DRAM or NVDIMM) on its own, before explicit flushing.
The following sequence of events is possible:
x[2]=100; store the 3nd byte of the cache line first. (Compile-time reordering: this is a C not asm question and x is apparently plain uint8_t x[64], not _Atomic or volatile so x[1]=100; and x[2]=100; aren't guaranteed to happen in that order in the asm.)
An interrupt arrives; at some point the cache line containing x[] is evicted all the way out of cache, into the persistence domain. (Perhaps after a context-switch to another thread, so lots of other code runs between those two asm stores).
The system crashes before execution resumes. (Or before x[1]=100; finishes becoming durable.)
If you want to depend on x86 memory ordering rules to control durability order within a cache line, you need to make sure C respects that. volatile would work, or _Atomic with memory_order_release for at least the 2nd store. (Or better, get them done as a single store if it's within an aligned 8-byte chunk.) (x86 asm memory model = program order with a store buffer; no StoreStore reordering.)
Compile-time reordering doesn't usually happen for no reason (but it can); more often due to surrounding code making it appealing to do so. But surrounding code could cause this. (And of course x[1]=100; / x[2]=0; can happen by this mechanism without any compile-time reordering, if it's 2 separate stores.)
I think a necessary pre-condition for atomicity of durability is being done as a single atomic store. e.g. guaranteed atomic by the ISA, or with a single wider SIMD store1 because Intel CPUs in practice don't split those up (but there's no on-paper guarantee of that). Being atomic wrt. interrupts (i.e. a single instruction) but not a single store uop makes it harder to split up but still completely possible2 and thus not guaranteed safe. e.g. a 10-byte x87 fstp tbyte that involves 2 separate store-data uops can be split up by an invalidation from another core that's possible even without false sharing. (See footnote 2 again.)
Without any on-paper atomicity guarantee for 16-byte or wider SIMD stores, you'd be depending on implementation details for SIMD stores or misaligned stores to not be split up.
Even ISA-guaranteed atomicity isn't sufficient, though: a lock cmpxchg that spans a cache-line boundary is still guaranteed atomic wrt. other cores and DMA readers. (Supporting this is very very slow, don't do it.) But there's no way to guarantee those two lines become durable at the same time. But outside of that special case for atomicity, IDK, I can't rule out whole-line atomicity. It's certainly plausible that a plain-store into a single line which is atomic in asm will become durable atomically, with no chance of tearing.
Within a single cache line, I don't know.
I'd guess that an atomic store within an 8-byte-aligned block will make it to persistence atomically or not at all, but I haven't checked Intel's docs. (And in practice perhaps even a whole 64-byte line, which you could store with AVX512). The point of this answer is that you don't even have a single atomic store so there are lots of other mechanisms for breaking your test-case.
Footnote 1: Modern Intel CPUs commit SIMD stores to L1d cache as a single transaction as long as they don't span a cache line. Intel hasn't made a CPU that splits SIMD stores into 2 halves since Sandy/Ivy Bridge which had full-width 256-bit AVX execution units but only 128-bit wide paths to/from cache in the load units and AFAIK in the store-buffer-commit stuff. (The store-data execution unit also took 2 cycles to write 32-byte store data to the store buffer).
Footnote 2: For separate store uops that are part of the same instruction like in fstp tbyte [rdi], this might be possible:
first part commits from the store buffer to L1d cache
an RFO or share-request arrives and is handled before the 2nd store from the same instruction commits: This core's copy is now Invalid or Shared so the commit from the store buffer to L1d is blocked until it regains exclusive ownership. The 2nd part of the store from that one instruction is at the head of the store buffer, not in coherent cache.
The other core that was doing an RFO followed up their store with a clflush, evicting this line to persistent memory before the first core can get it back and finish committing the other data from that one instruction.
An NT store like movnti by another core would force eviction of the line as part of committing the NT store, like a normal store + clflushopt.
This scenario requires false-sharing between two threads trying to persist 2 separate things in the same line, so can be avoided if you avoid false-sharing e.g. with padding. (Or some insane true sharing, or firing off clflush without storing first, on memory that other threads might be in the middle of writing).
(Or more plausible for software, much less plausible for hardware): The line gets evicted on its own before the first writer gets it back, even though a core has a pending RFO for it. (As soon as it loses ownership, the first core would send out an RFO).
(Or fully plausible without false-sharing): Forced eviction from L2/L1d at any time due to eviction from an inclusive cache-line tracking structure. This could be triggered by demand for lines that merely happen to alias the same set in L3, not false sharing.
Skylake-server (SKX) has non-inclusive L3, as do later Intel server CPUs. Cascade Lake (CSX) was the first to support persistent memory. Even though it has a non-inclusive L3, the snoop filter is inclusive and a fill conflict that causes an eviction does cause a back invalidation in the entire NUMA node.
So an invalidate request can arrive at any time, and it's likely the core / store buffer isn't going to hold onto the line for more cycles to commit an unknown number of more stores to the same line.
(By that point, the fact that both store-buffer entries were part of one instruction is probably lost. It's possible for an access pattern to create a stream of store buffer entries that store different parts of the same cache line indefinitely, so waiting until "all stores for this line are done" could let unprivileged code create a denial-of-service for a core that wanted to read it. So I think it's unlikely that HW would have a mechanism to avoid releasing ownership of a cache line between stores that came from the same instruction.)
I have a Linux kernel module which calculates network packet statistics among several CPUs (in kernel address space). Periodically I clear the corresponding memory chunk and strongly need this action to take immediate effect for all CPUs, otherwise it will distort the subsequent statistics values. My target CPU is a Power PC, so its cache coherency is very relaxed. Thus I need to manually flush data caches of all CPUs just after zeroing the memory.
So what should I place just after my clearing procedure:
memset(ptr, 0, size);
// what's going here?
After some reflection I realised that the problem here is not really linked to data cache flushing. Actually I try to avoid a banal race condition (the first cpu clears value while the second increments it). In my case it's too expensive to protect data by mutex, so it's worth using atomic flag to notify the owning CPU to clear the values by itself.
I am a little confused as to the real issues between multi-core and multi-cpu environments when it comes to shared memory, with particular reference to mmap in C.
I have an application that utilizes mmap to share multiple segments of memory between 2 processes. Each process has access to:
A Status and Control memory segment
Raw data (up to 8 separate raw data buffers)
The Status and Control segment is used essentially as an IPC. IE, it may convey that buffer 1 is ready to receive data, or buffer 3 is ready for processing or that the Status and Control memory segment is locked whilst being updated by either parent or child etc etc.
My understanding is, and PLEASE correct me if I am wrong, is that in a multi-core CPU environment on a single boarded PC type infrastructure, mmap is safe. That is, regardless of the number of cores in the CPU, RAM is only ever accessed by a single core (or process) at any one time.
Does this assumption of single-process RAM access also apply to multi-cpu systems? That is, a single PC style board with multiple CPU's (and I guess, multiple cores within each CPU).
If not, I will need to seriously rethink my logic to allow for multi-cpu'd single-boarded machines!
Any thoughts would be greatly appreciated!
PS - by single boarded I mean a single, standalone PC style system. This excludes mainframes and the like ... just to clarify :)
RAM is only ever accessed by a single core (or process) at any one time.
Take a step back and think about your assumption means. Theoretically, yes, this statement is true, but I don't think it means what you think it means. There are no practical conclusions you can draw from this other than maybe "the memory will not catch fire if two CPUs write to the same address at the same time". Let me explain.
If one CPU/process writes to a memory location, then a different CPU/process writes to the same location, the memory writes will not happen at the same time, they will happen one at a time. You can't generally reason about which write will happen before the other, you can't reason about if a read from one CPU will happen before the write from the other CPU, one some older CPUs you can't even reason if multi-byte (multi-word, actually) values will be stored/accessed one byte at a time or multiple bytes at a time (which means that reads and writes to multibyte values can get interleaved between CPUs or processes).
The only thing multiple CPUs change here is the order of memory reads and writes. On a single CPU reading memory you can be pretty sure that your reads from memory will see earlier writes to the same memory (iff no other hardware is reading/writing the memory, then all bets are off). On multiple CPUs the order of reads and writes to different memory locations will surprise you (cpu 1 writes to address 1 and then 2, but cpu 2 might just see the new value at address 2 and the old value at address 1).
So unless you have specific documentation from your operating system and/or CPU manufacturer you can't make any assumptions (except that when two writes to the same memory location happen one will happen before the other). This is why you should use libraries like pthreads or stdatomic.h from C11 for proper locking and synchronization or really dig deep down into the most complex parts of the CPU documentation to actually understand what will happen. The locking primitives in pthreads not only provide locking, they are also guarantee that memory is properly synchronized. stdatomic.h is another way to guarantee memory synchronization, but you should carefully read the C11 standard to see what it promises and what it doesn't promise.
One potential issue is that each core has it's own cache (usually just level1, as level2 and level3 caches are usually shared). Each cpu would also have it's own cache. However most systems ensure cache coherency, so this isn't the issue (except for performance impact of constantly invalidating caches due to writes to the same memory shared in a cache line by each core or processor).
The real issue is that there is no guarantee against reordering of reads and writes due to optimizations by the compiler and/or the hardware. You need to use a Memory Barrier to flush out any pending memory operations to synchronize the state of the threads or shared memory of processes. The memory barrier will occur if you use one of the synchronization types such as an event, mutex, semaphore, ... . Not all of the shared memory reads and writes need to be atomic, but you need to use synchronization between threads and/or processes before accessing any shared memory possibly updated by another thread and/or process.
This does not sound right to me. Two processes on two different cores can both load and store data to RAM at the same time. In addition to this caching strategies can result in all kinds of strangeness-es. So please make sure all access to shared memory is properly synchronized using (interprocess) synchronization objects.
My understanding is, and PLEASE correct me if I am wrong, is that in a multi-core CPU environment on a single boarded PC type infrastructure, mmap is safe. That is, regardless of the number of cores in the CPU, RAM is only ever accessed by a single core (or process) at any one time.
Even if this holds true for some particular architecture, such an assumption is entirely wrong in general. You should have proper synchronisation between the processes that modify the shared memory segment, unless atomic intrinsics are used and the algorithm itself is lock-free.
I would advise you to put a pthread_mutex_t in the shared memory segment (shared across all processes). You will have to initialise it with the PTHREAD_PROCESS_SHARED attribute:
pthread_mutexattr_t mutex_attr;
pthread_mutexattr_init(&mutex_attr);
pthread_mutexattr_setpshared(&mutex_attr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(mutex, &mutex_attr);
I know that modern CPUs can execute out of order, However they always retire the results in-order, as described by wikipedia.
"Out of Oder processors fill these "slots" in time with other instructions that are ready, then re-order the results at the end to make it appear that the instructions were processed as normal."
Now memory fences are said to be required when using multicore platforms, because owing to Out of Order execution, wrong value of x can be printed here.
Processor #1:
while f == 0
;
print x; // x might not be 42 here
Processor #2:
x = 42;
// Memory fence required here
f = 1
Now my question is, since Out of Order Processors (Cores in case of MultiCore Processors I assume) always retire the results In-Order, then what is the necessity of Memory fences. Don't the cores of a multicore processor sees results retired from other cores only or they also see results which are in-flight?
I mean in the example I gave above, when Processor 2 will eventually retire the results, the result of x should come before f, right? I know that during out of order execution it might have modified f before x but it must have not retired it before x, right?
Now with In-Order retiring of results and cache coherence mechanism in place, why would you ever need memory fences in x86?
This tutorial explains the issues: http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-95-7.pdf
FWIW, where memory ordering issues happen on modern x86 processors, the reason is that while the x86 memory consistency model offers quite strong consistency, explicit barriers are needed to handle read-after-write consistency. This is due to something called the "store buffer".
That is, x86 is sequentially consistent (nice and easy to reason about) except that loads may be reordered wrt earlier stores. That is, if the processor executes the sequence
store x
load y
then on the processor bus this may be seen as
load y
store x
The reason for this behavior is the afore-mentioned store buffer, which is a small buffer for writes before they go out on the system bus. Load latency is, OTOH, a critical issue for performance, and hence loads are permitted to "jump the queue".
See Section 8.2 in http://download.intel.com/design/processor/manuals/253668.pdf
The memory fence ensures that all changes to variables before the fence are visible to all other cores, so that all cores have an up to date view of the data.
If you don't put a memory fence, the cores might be working with wrong data, this can be seen especially in scenario's, where multiple cores would be working on the same datasets. In this case you can ensure that when CPU 0 has done some action, that all changes done to the dataset are now visible to all other cores, whom can then work with up to date information.
Some architectures, including the ubiquitous x86/x64, provide several
memory barrier instructions including an instruction sometimes called
"full fence". A full fence ensures that all load and store operations
prior to the fence will have been committed prior to any loads and
stores issued following the fence.
If a core were to start working with outdated data on the dataset, how could it ever get the correct results? It couldn't no matter if the end result were to be presented as-if all was done in the right order.
The key is in the store buffer, which sits between the cache and the CPU, and does this:
Store buffer invisible to remote CPUs
Store buffer allows writes to memory and/or caches to be saved to
optimize interconnect accesses
That means that things will be written to this buffer, and then at some point will the buffer be written to the cache. So the cache could contain a view of data that is not the most recent, and therefore another CPU, through cache coherency, will also not have the latest data. A store buffer flush is necessary for the latest data to be visible, this, I think is essentially what the memory fence will cause to happen at hardware level.
EDIT:
For the code you used as an example, Wikipedia says this:
A memory barrier can be inserted before processor #2's assignment to f
to ensure that the new value of x is visible to other processors at or
prior to the change in the value of f.
Just to make explicit what is implicit in the previous answers, this is correct, but is distinct from memory accesses:
CPUs can execute out of order, However they always retire the results in-order
Retirement of the instruction is separate from performing the memory access, the memory access may complete at a different time to instruction retirement.
Each core will act as if it's own memory accesses occur at retirement, but other cores may see those accesses at different times.
(On x86 and ARM, I think only stores are observably subject to this, but e.g., Alpha may load an old value from memory. x86 SSE2 has instructions with weaker guarentees than normal x86 behaviour).
PS. From memory the abandoned Sparc ROCK could in fact retire out-of-order, it spent power and transistors determining when this was harmless. It got abandoned because of power consumption and transistor count... I don't believe any general purpose CPU has been bought to market with out-of-order retirement.