In the state machines, it is said that it only holds information about the current state and based on the input, a next state is transitioned to.
What about a situation where there are additional conditions, such as:
State A (input X) ---> State B
State B
(input X) AND (SomeValue>=100) ---> State C
(input X) AND (SomeValue < 100) ---> State D
Is this still a state machine?
State machines can have no memory (like finite automata), memory whose access is restricted in some way (such as pushdown automata with stack access), or memory whose access is essentially unlimited (such as a Turing machine or Random Access Machine (RAM)). I think it's fair to call all of these things state machines since they change behavior based upon their internal states.
If the automaton is not writing memory, but only reading memory, and it has no ability to "go back" and read memory it had read before, then no matter what memory it's reading, it's equivalent to having no memory and simply responding to the normal inputs it receives. For instance, a Turing Machine that cannot write and can only read its tape from left to right is equivalent to a finite automaton; a pushdown automaton that cannot push symbols onto the stack is equivalent to a finite automaton; etc.
If the automaton can write the contents of memory and has an ability to eventually read those contents back - and if the amount of memory which can be so manipulated is not fixed - then it remains a state machine but is no longer equivalent to a finite automaton. Note that I say the amount of memory must not be fixed: if the amount of memory is fixed, then any machine using it is equivalent to a finite automaton with repeated states for every possible configuration of all memory. Even the computer you're on right now is no more powerful than a finite automaton: in fact, your computer is infinitely less powerful than a general finite automaton, since there are infinitely many regular languages that cannot possibly be accepted by any physically realizable computer.
Related
In a non-OOP programming language, like C, If we only allow local variables to be mutated in every possible way (change internal fields, re-assigning, ...) but disallow mutation of function arguments, will it help us prevent shared mutable state?
Note that in this case, function main can start 10 threads (functions) and each of those 10 threads will receive an immutable reference to the same variable (defined in main). But the main function can still change the value of that shared variable. So can this cause problem in a concurrent/parallel software?
I hope the question is clear, but let me know if it's not.
P.S. Can "software transactional memory (STM)" solve the potential problems? Like what Clojure offers?
Yes and no... this depends on the platform, the CPU, the size of the shared variable and the compiler.
On an NVIDIA forum, in relation to GPU operations, a similar question was very neatly answered:
When multiple threads are writing or reading to/from a naturally aligned location in global memory, and the datatype being read or written is the same by all threads, and the datatype corresponds to one of the supported types for single-instruction thread access ...
(Many GPU single-instruction can handle 16 Byte words (128bit) when it's known in advance, but most CPUs use single-instruction 32bits or 64bit limits)
I'm leaving aside the chance that threads might read from the CPU registers instead of the actual memory (ignoring updates to the data), these are mostly solvable using the volatile keyword in C.
However, conflicts and memory corruption can still happen.
Some memory storage operations are handled internally (by the CPU) or by your compiler (the machine code) using a number of storage calls.
In these cases, mostly on multi-core machines (but not only), there's the risk that the "reader" will receive information that was partially updated and has no meaning whatsoever (i.e., half of a pointer is valid and the other isn't).
Variables larger than 32 bits or 64 bits, will usually get updated a CPU "word" (not an OS word) at a time (32bits or 64 bits).
Byte sized variables are super safe, That's why they are are often used as flags... but they should probably be handled using the atomic_* store/write operations provided by the OS or the compiler.
I have this situation where I have a state variable; int state = (2, 1, 0)
and an infinite loop:
ret = BoolCompareAndSwap(state, 1, 2)
if (ret) {
// Change something ...
state = 0;
}
Would this state setting be atomic?
Assuming to set a variable you must:
Take out from memory
Change value
Set new value
If some other thread came and compared the variable, it would be atomic since the actual value doesn't change until it it re-set in memory, Correct?
Strictly speaking, C compilers would still be standard conforming if they wrote the state bitwise. After writing the first few bits, some other thread can read any kind of garbage.
Most compilers do no such thing (with the possible exception of compilers for ancient 4bit processors or even narrower ...), because it would be a performance loss.
Also, and more practically relevant, if any other thread writes (instead of only reading) to the state, that written value can get lost if you do not protect the described code against racing conditions.
As a side note, the described state change (read, modify, write) is never atomic. The question however when that non-atomicity is vulnerable, is valid and it is what I tried to answer above.
More generically speaking, thinking through all possible combinations of concurrent access is a valid protection mechanism. It is however extremely costly in many ways (design effort, test effort, risk during maintenance...).
Only if those costs are in total smaller than the intended saving (possibly performance), it is feasible to go that way, instead of using an appropriate protection mechanism.
Yesterday I can across this obfuscated C code implementing Conway's Game of Life. As a pseudorandom generator, it writes code to this effect:
int pseudoRand = (int) &pseudoRand;
According to the author's comments on the program:
This is a big number that should be different on each run, so it works nicely as a seed.
I am fairly confident that the behavior here is either implementation-defined or undefined. However, I'm not sure why this value would vary from run to run. My understanding of how most OS's work is that, due to virtual memory, the stack is initialized to the same virtual address each time the program is run, so the address should be the same each time.
Will this code actually produce different results across different runs on most operating systems? Is it OS-dependent? If so, why would the OS map the same program to different virtual addresses on each run?
Thanks!
While the assignment of addresses to objects with automatic storage is unspecified (and the conversion of an address to an integer is implementation-defined), what you're doing in your case is simply stealing the entropy the kernel assigned to the initial stack address as part of Address space layout randomization (ASLR). It's a bad idea to use this as a source of entropy which may leak out of your program, especially in applications interacting over a network with untrusted, possibly malicious remote hosts, since you're essentially revealing the random address base the kernel gave you to an attacker who might want to know it and thereby defeating the purpose of ASLR. (Even if you just use this as a seed, as long as the attacker knows the PRNG algorithm, they can reverse it to get the seed.)
While learning C, I made some mistakes and printed elements of a character array that were uninitialized.
If I expand the size of the array to be quite large, say 1 million elements in size and then print the contents, what comes out is not always user unreadable, but seems to contain some runtime info.
Consider the following code:
#include <stdio.h>
main() {
char s[1000000];
int c, i;
printf("Enter input string:\n");
for (i = 0; ( c = getchar()) != '\n'; i++) {
s[i] = c;
}
printf("Contents of input string:\n");
for (i = 0; i < 999999; i++) {
putchar(s[i]);
}
printf("\n");
return 0;
}
Just scrolling through the output, I find things such as:
???l????????_dyldVersionNumber_dyldVersionString_dyld_all_image_infos_dyld_fatal_error_dyld_shared_cache_ranges_error_string__mh_dylinker_header_stub_binding_helper_dyld_func_lookup_offset_to_dyld_all_image_infos__dyld_start__ZN13dyldbootstrapL30randomizeExecutableLoadAddressEPK12macho_headerPPKcPm__ZN13dyldbootstrap5startEPK12macho_headeriPPKcl__ZN4dyldL17setNewProgramVarsERK11ProgramVars__ZN4dyld17getExecutablePathEv__ZN4dyld22mainExecutablePreboundEv__ZN4dyld14mainExecutableEv__ZN4dyld21findImageByMachHeaderEPK11mach_header__ZN4dyld26findImageContainingAddressEPKv
and also,
Apple Inc.1&0$U ?0?*?H??ot CA0?"0ple Certification Authority10U
?䑩 ??GP??^y?-?6?WLU????Kl??"0?>?P ?A?????f?$kУ????z
?G?[?73??M?i??r?]?_???d5#KY?????P??XPg? ?ˬ,
op??0??C??=?+I(??ε??^??=?:??? ?b??q?GSU?/A????p??LE~LkP?A??tb
?!.t?<
?A?3???0X?Z2?h???es?g^e?I?v?3e?w??-??z0?v0U?0U?0?0U+?iG?v ??k?.#??GM^0U#0?+?iG?v ??k?.#??GM^0?U
0?0? ?H??cd0??0+https://www.apple.com/appleca/0?+0????Reliance on
this certificate by any party assumes acceptance of the then
applicable standard terms and conditions of use, certificate
poli?\6?L-x?팛??w??v?w0O????=G7?#?,Ա?ؾ?s???d?yO4آ>?x?k??}9??S ?8ı??O
01?H??[d?c3w?:,V??!ںsO??6?U٧??2B???q?~?R??B$*??M?^c?K?P????????7?uu!0?0??0
I believe one time my $PATH environment variable was even printed out.
Can the contents of an uninitialized variable ever pose a security risk?
Update 1
Update 2
So it seems clear from the answers that this is indeed a security risk. This surprises me.
Is there no way for a program to declare its memory content protected to allow the OS to restrict any access to it other than the program that initialized that memory?
Most C programs use malloc to allocate memory. A common misunderstanding is that malloc zeros out the memory returned. It actually does not.
As a result, due to the fact that memory chunks are "recycled" it is quite possible to get one with information of "value".
An example of this vulnerability was the tar program on Solaris which emitted contents of /etc/passwd. The root cause was the fact that the memory allocated to tar to read a block from disk was not initialized and before getting this memory chunk the tar utility made a OS system call to read /etc/passwd. Due to the memory recycling and the fact that tar did not initialize the chunk fragments of /etc/passwd were printed to logs. This was solved by replacing malloc with calloc.
This is an actual example of security implication if you don't explicitly and properly initialize memory.
So yes, do initialize your memory properly.
Update:
Is there no way for a program to declare its memory content protected
to allow the OS to restrict any access to it other than the program
that initialized that memory?
The answer is yes (see in the end) and no.
I think that you view it the wrong way here. The more appropriate question would be for example, why doesn't malloc initialize the memory on request or clears the memory on release but instead recycles it?
The answer is that the designers of the API explicitly decided not to initialize (or clear memory) as doing this for large blocks of memory 1)would impact performance and 2)is not always necessary (for example you may not deal, in your application or several parts in your application with data that you actually care if they are exposed). So the designers decided not to do it, as it would inadvertently impact performance, and to drop the ball to the programmer to decide on this.
So carrying this also to the OS, why should it be the OS's responsibility to clear the pages? You expect from your OS to hand you memory in a timely manner but security is up to the programmer.
Having said that there are some mechanism provided that you could use to make sure that sensitive data are not stored in swap using mlock in Linux.
mlock() and mlockall() respectively lock part or all of the calling
process's virtual address space into RAM, preventing that memory
from being paged to the swap area. munlock() and munlockall()
perform the converse operation, respectively unlocking part or all
of the calling process's virtual address space, so that pages in the
specified virtual address range may once more to be swapped out if
required by the kernel memory manager. Memory locking and unlocking
are performed in units of whole pages.
Yes, at least on systems where the data may be transmitted to outside users.
There have been a whole series of attacks on webservers (and even iPods) where you get it to dump the contents of memory from other process - and so get details of the type and version of the OS, the data in other apps and even things like password tables
It's quite possible to perform some sensitive work in an area of memory, and not clear that buffer.
A future invocation can then retrieve that uncleared work via a call to malloc() or by checking the heap (via an unitiaised buffer/array declaration). It could inspect it (maliciously) or inadvertently copy it. If you're doing anything sensitive it thus makes sense to clear that memory before binning it (memset() or similar), and perhaps before using/copying it.
From the C standard:
6.7.8 Initialization
"If an object that has automatic storage duration is not initialized
explicitly, its value is indeterminate."
indeterminate value is defined as:
either an unspecified value or a trap representation.
Trap representation is defined as:
Certain object representations need not represent a value of the
object type. If the stored value of an object has such a
representation and is read by an lvalue expression that does not have
character type, the behavior is undefined. If such a representation is
produced by a side effect that modifies all or any part of the object
by an lvalue expression that does not have character type, the
behavior is undefined.41) Such a representation is called a trap
representation.
Accessing such a values leads to undefined behaviour and can pose security threats.
This paper Attacks on uninitialized variables can give some insights on they can be used to exploit the system.
If you are concerned about security, safest way is to allways initialize every variable you're going to use. It may even help you find some bugs.
There may be some good reasons for not initializing memory, but in most cases initializing every variable/memory will be a good thing.
Reading uninitialized memory leads to undefined behavior. Bear in mind that what it means to be initialized depends on the invariant of a particular type. For example, it may be required for some pointer to be non-null, some enum to be from a valid range or a certain parameter to be a power of two. Situation complicates further with compound structures. An arbitrary sequence of bytes may not represent a valid object. This is why zeroing memory is not enough. If the expected invariant is broken, some code path relying on it will behave in an undefined manner and may pose a security issue.
What are the real dangers of simultaneous read/write to a single variable?
If I use one thread to write a variable and another to read the variable in a while loop and there is no danger if the variable is read while being written and an old value is used what else is a danger here?
Can a simultaneous read/write cause a thread crash or what happens on the low level when an exact simultaneous read/write occurs?
If two threads access a variable without suitable synchronization, and at least one of those accesses is a write then you have a data race and undefined behaviour.
How undefined behaviour manifests is entirely implementation dependent. On most modern architectures, you won't get a trap or exception or anything from the hardware, and it will read something, or store something. The thing is, it won't necessarily read or write what you expected.
e.g. with two threads incrementing a variable, you can miss counts, as described in my article at devx: http://www.devx.com/cplus/Article/42725
For a single writer and a single reader, the most common outcome will be that reader sees a stale value, but you might also see a partially-updated value if the update requires more than one cycle, or the variable is split across cache lines. What happens then depends on what you do with it --- if it's a pointer and you get a partially updated value then it might not be a valid pointer, and won't point to what you intended it to anyway, and then you might get any kind of corruption or error due to dereferencing an invalid pointer value. This may include formatting your hard disk or other bad consequences if the bad pointer value just happens to point to a memory mapped I/O register....
In general you get unexpected results. Wikipedia defines two distinct racing conditions:
A critical race occurs when the order in which internal variables are changed determines the eventual state that the state machine will end up in.
A non-critical race occurs when the order in which internal variables are changed does not alter the eventual state. In other words, a non-critical race occurs when moving to a desired state means that more than one internal state variable must be changed at once, but no matter in what order these internal state variables change, the resultant state will be the same.
So the output will not always get messed up, it depends on the code. It's good practice to always deal with racing conditions for later code scaling and preventing possible errors. Nothing is more annoying then not being able to trust your own data.
Two threads reading the same value is no problem at all.
The problem begins when one thread writes a non-atomic variable and another thread reads it. Then the results of the read are undefined. Since a thread may be preempted (stopped) at any time. Only operations on atomic variables are guaranteed to be non-breakable. Atomic actions are usually writes to int type variables.
If you have two threads accessing the same data, it is best practice + usually unavoidable to use locking (mutex, semaphore).
hth
Mario
Depends on the platform. For example, on Win32, then read and write ops of aligned 32bit values are atomic- that is, you can't half-read a new value and half-read an old value, and if you write, then when someone comes to read, either they get the full new value or the old value. That's not true for all values, or all platforms, of course.
Result is undefined.
Consider this code:
global int counter = 0;
tread()
{
for(i=0;i<10;i++)
{
counter=counter+1;
}
}
Problem is that if you have N threads result can be anything between 10 and N*10.
This is because it might happen all treads read same value increase it and then write value +1 back. But you asked if you can crash program or hardware.
It depends. In most cases are wrong results useless.
For solving this locking problem you need mutex or semaphore.
Mutex is lock for code. In upper case you would lock part of code in line
counter = counter+1;
Where semaphore is lock for variable
counter
Basicaly same thing for solving same type of problem.
Check for this tools in your tread library.
http://en.wikipedia.org/wiki/Mutual_exclusion
The worst that will happen depends on the implementation. There are so many completely independent implementations of pthreads, running on different systems and hardware, that I doubt anyone knows everything about all of them.
If p isn't a pointer-to-volatile then I think that a compiler for a conforming Posix implementation is allowed to turn:
while (*p == 0) {}
exit(0);
Into a single check of *p followed by an infinite loop that doesn't bother looking at the value of *p at all. In practice, it won't, so it's a question of whether you want to program to the standard, or program to undocumented observed behavior of the implementations you're using. The latter generally works for simple cases, and then you build on the code until you do something complicated enough that it unexpectedly doesn't work.
In practice, on a multi-CPU system that doesn't have coherent memory caches, it could be a very long time before that while loop ever sees a change made from a different CPU, because without memory barriers it might never update its cached view of main memory. But Intel has coherent caches, so most likely you personally won't see any delays long enough to care about. If some poor sucker ever tries to run your code on a more exotic architecture, they may end up having to fix it.
Back to theory, the setup you're describing could cause a crash. Imagine a hypothetical architecture where:
p points to a non-atomic type, like long long on a typical 32 bit architecture.
long long on that system has trap representations, for example because it has a padding bit used as a parity check.
the write to *p is half-complete when the read occurs
the half-write has updated some of the bits of the value, but has not yet updated the parity bit.
Bang, undefined behavior, you read a trap representation. It may be that Posix forbids certain trap representations that the C standard allows, in which case long long might not be a valid example for the type of *p, but I expect you can find a type for which trap representations are permitted.
If the variable being written to and from can not be updated or read atomically then it is possible for the reader to pick up a corrupt "partially updated" value.
You can see a partial update (e.g. you may see a long long variable with half of it coming from the new value and the other half coming from the old value).
You are not guaranteed to see the new value until you use a memory barrier (pthread_mutex_unlock() contains an implicit memory barrier).