I recently found this theorem here, (at the bottom):
Any program can be transformed into a semantically equivalent program of one procedure containing one switch statement inside a while loop.
The Article went on to say :
A corollary to this theorem is that any program can be rewritten into a program consisting of a single recursive function containing only conditional statements
My questions are, are both these theorems applicable today ? Does similarly transforming a program reap any benefits ? I mean to say, is such a code optimized ? (Although recursion calls are slower, I know)
I read, from here, that switch-cases are almost always faster when optimized by the compiler. Does that make a difference. ?
PS: I'm trying to get some idea about compiler optimizations from here
And I've added the c tag as that's the only language I've seen optimized.
Its true. A Turing machine is essentially a switch statement on symbols that repeats forever, so its based pretty directly on Turing-machines-compute everything. A switch statement is just a bunch of conditionals, so you can clearly write such a program as a loop with just conditionals. Once you have that, making the loop from recursion is pretty easy although you may have to pass a lot of state variables as parameters if your language doesn't have true lexical scoping.
There's little reason to do any of this in practice. Such programs generally operate more slowly than the originals, and may take more space. So why would you possibly slow your program down, and/or make its load image bigger?
The only place this makes sense is if you intend to obfuscate the code. This kind of technique is often used as "control flow obfuscation".
This is basically what happens when a compiler translates a program into machine code. The machine code runs on a processor, which executes instructions one-by-one in a loop. The complex structure of the program has become part of the data in memory.
Recursive loops through a switch statement can be used to create a rudimentary virtual machine. If your virtual machine is Turing complete then, in theory, any program could be rewritten to work on this machine.
int opcode[] {
PUSH,
ADD
....
};
while (true) {
switch (*opcode++) {
case PUSH:
*stack++ = <var>;
break;
case ADD:
stack[-1] += stack[0];
--stack;
break;
....
}
}
Of course writing a compiler for this virtual machine would be another matter.
:-)
Related
I am writing a .y4m video generator in C that takes all the .bmp images in a directory and writes them to a .y4m video file (after appropriately converting the colours to YCbCr format).
One of the command-line options I am allowing to be specified is whether the .bmp files should be deleted during the video generation. Right now, this happens right at the end of the program, but it would be best for it to occur as images are written (to not increase disk space usage by more than 1 frame at at a time, since the .y4m files are uncompressed video, so can get pretty big).
Thus, if the delete option is specified, the deleting should take place within one of the 5 main loops I have (there is one for each colour-subsampling scheme). The loops have lots of code within, however, so I really would like to avoid duplicating them.
In summary, even though this:
if (delete) {
while (bmps_remain) {
// do lots of funky stuff
remove(path_to_single_bmp);
}
}
else {
while (bmps_remain) {
// do the exact same funky stuff as above
// ... but do not delete bmp file
}
}
... is better than this:
while (bmps_remain) {
// lots and lots of code
if (delete)
remove(path_to_single_bmp);
}
... how much of a difference does it really make, and how frowned upon is it to opt for the second option (taking performance into account as much as possible), given that the second option re-evaluates the (unchanging) condition during each iteration of the loop?
Even though it would probably get compiled into some kind of cmp instruction (probably followed by a kind of jnz) which would only take a fraction of a second to perform, this situation occurs commonly in programming, so I would be interested to hear people's opinions.
Thanks.
P.S. The first option would produce a lot of code duplication in my case (and I would prefer to not stick everything into functions, given the previous layout of the program).
The remove( path_to_single_bmp ); instruction is several orders of magnitude slower than your if( invariant ); therefore, it makes absolutely no sense to be worrying the slightest bit about the overhead of re-executing it in the loop.
More generally, an if( invariant ) is so trivial as to never be worth considering for optimization, even if it only controls trivial code.
Even more generally, things like if( invariant ) will be optimized by the C compiler in any way it sees fit, regardless of what your intentions are, so they are generally not worth considering.
And even more-more generally, one of the most important qualities of code is readability, (second only to correctness,) and more code is less readable than less code, so any approach that results in less code is preferable. In other words, any approach that requires more code is generally far worse than any approach that requires less code. Exceptions to this rule tend to be algorithmic optimizations, where you introduce an entire algorithm to achieve performance. (E.g. a hash map instead of a naive array scan.) Tweaking and hacking code all over the place to squeeze clock cycles here and there never results in anything good.
From the point of view of optimizing the code run time, is there a thumb rule for where to use "nested if" statement and when to use "switch case" statements ?
I doubt you will ever find a real-life application where the difference between a nested if and a switch case is even worth measuring. Disk access, web access, etc. take many many orders of magnitude more time.
Choose what is easiest to read and debug.
Also see What is the difference between IF-ELSE and SWITCH? (possible duplicate) as well as Advantage of switch over if-else statement. Interestingly, a proponent of switch writes
In the worst case the compiler will
generate the same code as a if-else
chain, so you don't lose anything. If
in doubt put the most common cases
first into the switch statement.
In the best case the optimizer may
find a better way to generate the
code. Common things a compiler does is
to build a binary decission tree
(saves compares and jumps in the
average case) or simply build a
jump-table (works without compares at
all).
If you have more than 2-3 comparisons
then "switch"
else "if"
try to apply some patterns before you go to switch like strategy...
I don't believe it will make any difference for a decision structure that could be implemented using either method. It's highly likely that your compiler would produce the same instructions in the executable.
I thought expressions like this would cause Haskell to evaluate forever. But the behaviors in both GHCi and the compiled program surprised me.
For example, in GHCi, these expressions blocked until I Control+C, but consumed no CPU. Looked like it was sleeping.
let loop = loop
let loop = 1 + loop
I tried compiling these programs with GHC:
main = print loop
where loop = 1 + loop
main = print loop
where loop = if True then loop else 1
What was printed was:
Main: <<loop>>
So my question is: Obviously these expressions are compiled to something different than loops or recursive calls in imperative languages. What are they compiled to? Is this a special rule to handle 0-arg functions that have themselves in the right hand side, or it's a special case of something more general that I don't know?
[EDIT]:
One more question: If this happens to be a special handling from the compiler, what is the reason behind doing this when it's impossible to check for all infinite loops? 'Familiar' languages don't care about cases like while (true); or int f() { return f();}, right?
Many thanks.
GHC implements Haskell as a graph reduction machine. Imagine your program as a graph with each value as a node, and lines from it to each value that value depends on. Except, we're lazy, so you really start with just one node -- and to evaluate that node, GHC has to "enter" it and open it up to a function with arguments. It then replaces the function call with the body of the function, and attempts to reduce it enough to get it into head normal form, etc.
The above being very handwavy and I'm sure eliding some necessary detail in the interest of brevity.
In any case, when GHC enters a value, it generally replaces it with a black hole while the node is being evaluated (or, depending on your terminology, while the closure is being reduced) This has a number of purposes. First, it plugs a potential space leak. If the node references a value which is used nowhere else, the black hole allows that value to be garbage-collected even while the node is being evaluated. Second, this prevents certain types of duplicate work, since in a multi-threaded environment, two threads may attempt to enter the same value. The black-hole will cause the second thread to block rather than evaluate the value already being evaluated. Finally, this happens to allow for a limited form of loop detection, since if a thread attempts to re-enter its own black hole, we can throw an exception.
Here's a bit of a more metaphorical explanation. If I have a series of instructions that moves a turtle (in logo) around the screen, there's no one way to tell what shape they will produce, or whether that shape terminates without running them. But if, while running them, I notice that the path of the turtle has crossed itself, I can indicate to the user "aha! the turtle has crossed its path!" So I know that the turtle has reached a spot it has been before -- if the path is a circuit through evaluating the nodes of a graph, then that tells us we're in a loop. However, the turtle can also go in, for example, an expanding spiral. And it will never terminate, but it will also never cross its prior path.
So, because of the use of black holes, for multiple reasons, we have some notion of a marked "path" that evaluation has followed. And if the path crosses itself, we can tell and throw an exception. However, there are a million ways for things to diverge that don't involve the path crossing itself. And in those cases, we can't tell, and don't throw an exception.
For super-geeky technical detail about the current implementation of black holes, see Simon Marlow's talk from the recent Haskell Implementors Workshop, "Scheduling Lazy Evaluation on Multicore" at the bottom of http://haskell.org/haskellwiki/HaskellImplementorsWorkshop/2010.
In some, limited cases, the compiler can determine such a loop exists as part of its other control flow analyses, and at that point replaces the looping term with code that throws an appropriate exception. This cannot be done in all cases, of course, but only in some of the more obvious cases, where it falls out naturally from other work the compiler is doing.
As for why Haskell finds this more often than other languages:
These cases do not occur in languages which are strict such as C. These loops specifically happen when a lazy variable's computation depends on its own value.
Languages such as C have very specific semantics in loops; ie, what order to do what in. As such, they are forced to actually execute the loop. Haskell, however defines a special value _|_ ("the bottom"), which is used to represent erroneous values. Values which are strict on themselves - ie, they depend on their own value to compute - are _|_. The result of pattern-matching on _|_ can either be an infinite loop or an exception; your compiler is choosing the latter here.
The Haskell compiler is very interested in performing strictness analysis - ie, proving that a certain expression depends on certain other expressions - in order to perform certain optimizations. This loop analysis falls out naturally as an edge case in the strictness analyzer which must be handled in one way or another.
In C there is a do while loop and pascal's (almost) equivalent is the repeat until loop, but there is a small difference between the two, while both structures will iterate at least once and check whether they need to do the loop again only in the end, in pascal you write the condition that need to met to terminate the loop (REPEAT UNTIL something) in C you write the condition that needs to be met to continue the loop (DO WHILE something). Is there a reason why there is this difference or is it just an arbitrary decision?
There's no fundamental difference at all, and no advantage to one over the other. It's just "syntactic sugar" — a change to the language's syntax that doesn't change its behavior in any real way. Some people find "repeat until" easier to conceptualize, while others find "repeat while" easier.
If, in C, you encounter a situation where "until" is what's desire, you can always just negate the condition:
do {
excitingThings();
} while ( !endOfTheWorld() );
In C the statement
while(some_condition);
might either be a "do nothing" loop or might have become detached from a "do ... while" loop.
do {
statement;
statement;
statement;
lots more statements;
}
while(some_condition);
Using a different keyword - until - avoids this possible misinterpretation.
Not such a problem these days when everybody turns on all compiler warnings and heeds them, don't they?
Still, I suspect that most veteran C programmers have wished - at some time or other - that C used "until" in this case.
I'm not sure about historical influences, but in my opinion C is more consistent, in the sense that ifs require a condition to be true for the code to run, as do whiles and do whiles.
The design of Pascal was motivated in part by the structured-programming work of the 1960s, including Edsger Dijkstra's groundbreaking work A Discipline of Programming. Dijkstra (the same man who considered goto harmful) invented methods for creating programs that were correct by construction. These methods including methods for writing loops that focus on the postcondition established when the loop terminates. In creating the repeat... until form, Wirth was inspired by Dijkstra to make the termination condition, rather than its complement, explicit in the code.
I have always admired languages like Smalltalk and Icon, which offer two syntactic forms, thus allowing the programmer to express his or her intent clearly, without necessarily having to rely on an easily missed complement operator. (In Icon the forms are while e1 do e2 and until e1 do e2; in Smalltalk they are block1 whileTrue: block2 and block1 whileFalse: block2.) From my perspective neither C nor Pascal is a fully built out, orthogonal design.
It's just an arbitrary decision. Some languages have both. The QBASIC/VB DO...LOOP statement supports all four combinations of pretest/posttest and WHILE/UNTIL.
There was no "decision" that would in any way connect the behavior of Pascal repeat/until loop with the behavior of C do/while loop, neither deliberate nor arbitrary. These are simply two completely unrelated issues.
Just some information.
Road Runner : while(not edge) { run(); }
Wily E Coyote : do { run(); } while(not edge);
I've always found UNTIL loops backwards, but that might just be because I'm from a C background. There are modern languages like Perl that provide both, but there isn't any particular advantage for one over the other
The C syntax requires no extra keywords.
In C, the two keywords do and while work for two kinds of loops. Pascal requires four keywords: while, do, repeat, and until.
I have programmed an embedded software (using C of course) and now I'm considering ways to improve the running time of the system. The most important single module in my system is one very large nested for loop module.
That module consists of two nested for loops that loops max 122500 times. That's not very much yet, but the problem is that inside that nested for loop I have a function call to a function that is in another source file. That specific function consists mostly of two another nested for loops which loops always 22500 times. So now I have to make a function call 122500 times.
I have made that function that is to be called a lot lighter and shorter (yet still works as it should) and now I started to think that would it be faster to rip off that function call and write that process directly inside those first two for loops?
The processor in that system is ARM7TDMI and its frequency is 55MHz. The system itself isn't very time critical so it doesn't have to be real time capable. However the faster it can process its duties the better.
Also would it be also faster to use while loops instead of fors? And any piece of advice about how to improve the running time is appreciated.
-zaplec
TRY IT AND SEE!!
It'll almost certainly make a difference. Function call overhead isn't usually that much of an issue, but at over 100K repetitions it starts to add up.
...But whether or not it makes any real-world difference is something only you can answer, after trying it and timing the results.
As for for vs while... it shouldn't matter unless you actually change the behavior when changing the loop. If in doubt, make your compiler spit out assembler code for both and compare... or just change it and time it.
You need to be careful in the optimizations you make because you aren't always clear on which optimizations the compiler is making for you. Pre-optimization is a common mistake people make. Is it important that your code is readable and easily maintained or slightly faster? Like others have suggested, the best approach is to benchmark the different ways and see if there is a noticeable difference.
If you don't believe your compiler does much in the way of optimization I would look at some older concepts in optimizing C (searches on SO or google should provide some good links).
The ARM processor has an instruction pipeline (cache). When the processor encounters a branch (call) instruction, it must clear the pipeline and reload, thus wasting some time. One objective when optimizing for speed is to reduce the number of reloads to the instruction pipeline. This means reducing branch instructions.
As others have stated in SO, compile your code with optimization set for speed, and profile. I prefer to look at the assembly language listing as well (either printed from the compiler or displayed interwoven in the debugger). Use this as a baseline. If you can't profile, you can use assembly instruction counting as a rough estimate.
The next step is to reduce the number of branches; or the number times a branch is taken. Unrolling loops helps to reduce the number of times a branch is taken. Inlining helps reduce the number of branches. Before applying this fine-tuning techniques, review the design and code implementation to see if branches can be reduced. For example, reduce the number of "if" statements by using Boolean arithmetic or using Karnaugh Maps. My favorite is reducing requirements and eliminating code that doesn't need to be executed.
In the code implementation, move code that doesn't change outside of the for or while loops. Some loops may be reduce to equations (example, replacing a loop of additions with a multiplication). Also, reduce the quantity of iterations, by asking "does this loop really need to be executed this many times").
Another technique is to optimize for Data Oriented Design. Also check this reference.
Just remember to set a limit for optimizing. This is where you decide any more optimization is not generating any ROI or customer satisfaction. Also, apply optimizations in stages; which will allow you to have a deliverable when your manager asks for one.
Run a profiler on your code. If you are just guessing at where you are spending your time, you are probably wrong. A profiler will show what function is taking the most time and you can focus on that. You could be doing something in the function that takes longer than the function call itself. Did you look to see if you can change floating operations to integer, or integer math to shifts? You can spend a lot of time fiddling with things that don't make much difference. Run a profiler on your code and know for sure that the things you are changing will make a difference.
For function vs. inline, unfortunately there is no easy answer. I.e. it depends. See this FAQ. For "for" vs. "while", I wouldn't think there is any significant difference in performance.
In general, a function call should have more overhead than inlining. You really should profile however, as this can be affected quite a bit by your compiler (especially the compile/optimization settings). Some compilers will automatically inline code for example.