Are simple loops as powerful as nested loops in terms of Turing completeness?
In terms of Turing completeness, yes they are.
Proof: It's possible to write a Brainf*** interpreter using a simple loop, for example here:
http://www.hevanet.com/cristofd/brainfuck/sbi.c
For loops with a fixed number of steps (LOOP, FOR and similar): Imagine the whole purpose of a loop is to count to n. Why should make it a difference if I loop i times in an outer loop and j times in an inner loop as opposed n = i * j in just a single loop?
Assume that no WHILE, GOTO or similar constructs are allowed in a program (just assignment, IF, and fixed loops). Then all these programs end after a finite number of steps.
The next step to more expressibility is to allow loops, where the number of iterations is e.g. determined by a condition, and it is not sure, whether this condition is ever satisfied (e.g. WHILE). Then is may happen, that a program won't halt. (This type of expressiveness is also known as Turing-completeness).
Corresponding to these two forms of programs are two kinds of functions, which were historically developed around the same time and which are called primitive recursive functions and μ-recursive functions.
The number of nestings doesn't play a role in this.
Related
I came across this video which is discussing how most recursive functions can be written with for loops but when I thought about it, I couldn't see the logical difference between the two. I found this topic here but it only focuses on the practical difference as do many other similar topics on the web so what is the logical difference in the way a loop and a recursion are handled?
Bottom line up front -- recursion is more versatile but in practice is generally less efficient than looping.
A loop could in principle always be implemented as a recursion if you wished to do so. In practice the limits of stack resources put serious constraints on the size of the problems you can address. I can and have built loops that iterate a billion times, something I'd never try with recursion unless I was certain the compiler could and would convert the recursion into a loop. Because of the stack limits and efficiency, people often try to find a looping equivalent for recursions.
Tail recursions can always be converted to loops. However, there are recursions that can't be converted. As an example, I work with statistical design of experiments. Sometimes a large design is constructed by "crossing" several smaller sub-designs. Crossing is where you concatenate every row of a second design to each row of the first. For two sub-designs, all this needs is simple nested looping, but for three or more designs you need to increase the level of nesting, adding one level of nesting for each additional sub-design. So while this is nested looping in principle, in practice the amount of nesting is variable. If you tried to implement it with looping you'd have to revise your program to add/subtract nested loops every time you were dealing with a different number of sub-designs to be crossed, so you can't write an immutable loop-based version. This can easily be implemented with recursion. In this case, I'm happy to trade a slight amount of efficiency, because I wrote and debugged the code 6 years ago and haven't had to revise it since, despite creating lots of crossed designs of varying complexity since then.
One way to think through this is that the choice for recursion or iteration depends on how you think about the problem being solved. Certain "ways of thinking" lead more naturally to recursive solutions, and other ways of thinking lead to more iterative solutions. For any problem, you can in principle think in a way that gives you a recursive solution or a way that gives you an iterative solution. (Sometimes the iterative solution will just end up simulating a recursion stack, but there is no actual recursion there.)
Here's an example. You have an array of integers (positive or negative), and you want to find the maximum segment sum. A segment is a piece of the array that is contiguous. So in the array [3, -4, 2, 1, -2, 4], the maximum segment sum is 5, and you get that from the segment [2, 1, -2, 4]; its sum is 5.
OK - so how might we solve this problem? One thing you might do is reason like this: "if I knew the maximum segment sum in the left half, and the maximum segment sum in the right half, then maybe I could somehow jam those together and figure out the maximum segment sum overall". This idea would require you to find the maximum segment sum on the two subhalves, and this is a smaller instance of the original problem. This is recursion, and a direct translation of this idea into code would therefore be recursive.
But the maximum segment sum problem isn't "recursive" or "iterative" -- it can be both, depending on how you think about the solution. I gave a recursive thought process above. Here is an iterative process: "well, if I add up the elements in each of the segments that start at some index i and end at some index j, I can just take the maximum of these to solve the problem". And directly trying to code this approach would give you triply nested loops (and a bad mark on an assignment because it's horribly inefficient!).
So, the same problem, depending on how the problem is conceptualized, can lead to a recursive or iterative solution. Now, I happened to choose a problem where there are many ways of solving it, and where there are reasonable recursive and iterative solutions. Some problems, however, admit only one type of solution, and that solution may be most naturally implemented using recursion or iteration. For example, if I asked you to write a function that keeps asking the user to enter a letter until they enter y or n, you might start thinking: "keep repeating the prompt and asking for input..." and before you know it you have some iterative code. Perhaps you might instead think recursively: "if the user enters y or n, I am done; otherwise ask the user for y or n"... in which case you'd generate a recursive algorithm. But the recursion here doesn't give you much: it unnecessarily uses a stack and doesn't make the program any faster. (Recursion sometimes makes it easier to prove correctness, in which case you might present something recursively even though you could alternately give a reasonable iterative solution.)
I was under the impression that recursive CTEs were set based, but in a recent SO post someone mentioned that they are loops.
Are recursive CTEs set based? Am I wrong to assume that a set based operation cannot be a loop?
If it is recursive it is still considered a loop.
Although one statement is set based, calling it over and over can be considered a loop. This is an argument about the definition or wording based on the context being used. They are set based statements but the processing is considered in simple terms a looping process.
For those interested here is a nice little write up about performance with CTE's:
http://explainextended.com/2009/11/18/sql-server-are-the-recursive-ctes-really-set-based/
They are set based. Recursive sets are still sets.
But all set operations are, if you look with a powerful enough magnifier glass, loops. Ultimately the code runs on CPUs and CPUs execute a stream of serial instructions that operate on discrete regions of memory. In other words, there is no set oriented hardware. Being 'set oriented' is a logical concept. The fact that all SQL operations are ultimately implemented using some form of a loop is an implementation detail.
I think the distinction that needs to be done is "tail recursion" versus "general recursion".
All tail recursions can be implemented as loops - without the need for a Stack.
General recursion support can also be implemented as a Loop - but with a stack.
Recursive CTEs are Tail recursions and hence essentially a Loop. The only difference is the terminating condition is handled by SQL semantics/execution engine. The output from each Loop iteration is UNIONed or whatever set op you specify.
I came across "loops must be folded to enusre termination" in a paper on formal methods (abstract interpretation to be precise). I am clear on what termination means, but I do not know what a folded loop is, nor how to perform folding on a loop.
Could someone please explain to me what a folded loop is please? And if it is not implicit in, or does not follow immediately for the definition of a folded loop, how this ensures termination?
Thanks
Folding a loop is the opposite action from the better-known loop unfolding, which itself is better known as loop unrolling. Given a loop, to unfold it means to repeat the body several times, so that the loop test is executed less often. When the number of executions of the loop in advance, the loop can be completely unfolded, leaving a simple sequence of instructions. For example, this loop
for i := 1 to 4 do
writeln(i);
can be unfolded to
writeln(1);
writeln(2);
writeln(3);
writeln(4);
See C++ loop unfolding, bounds for another example with partial unfolding.
The metaphor is that the program is folded on itself many times over; unfolding means removing some or all of these folds.
In some static analysis techniques, it is difficult to deal with loops, because finding a precondition for the entry point of the loop (i.e. finding a loop invariant) requires a fixpoint computation which is unsolvable in general. Therefore some analyses unfold loops; this requires having a reasonable bound on the number of iterations, which limits the scope of programs that can be analyzed.
In other static analysis techniques, finding a good invariant is a critical part of the analysis. It doesn't help to unfold the loop, in fact partially unfolding the loop would make the loop body larger and so would make it more difficult to determine a good invariant; and completely unfolding the loop would be impractical or impossible if the number of iterations was large or unbounded. Without seeing the paper, I find the statement a bit surprising, because the code could have been written with the unfolded form, but there can be programs that the analysis only works on in a more folded form.
I have no knowledge of abstract interpretation, so I'll take the functional programming approach to folding. :-)
In functional programming, a fold is an operation applied to a list to do something with each element, updating a value each iteration. For example, you can implement map this way (in Scheme):
(define (map1 func lst)
(fold-right (lambda (elem result)
(cons (func elem) result))
'() lst))
What that does is that it starts with an empty list (let's call that the result), and then for each element of the list, from the right-hand-side moving leftward, you call func on the element and cons its result onto the result list.
The key here, as far as termination goes, is that with a fold, the loop is guaranteed to terminate as long as the list is finite, since you're iterating to the next element of the list each time, and if the list is finite, then eventually there will be no next element.
Contrast this with a more C-style for loop, that doesn't operate on a list, but instead have the form for (init; test; update). The test is not guaranteed to ever return false, and so the loop is not guaranteed to complete.
Say I have a nested loop as:
for(m=0;m<10;m++)
for(n=0;n<10;n++)
result[n][m]=result[m-3][n-2]
+result[n+1];
Would we call any of the two loops parallelizeable? My understanding is there is dependence on both the two variables n and m so we can't parallelize any of the loops.
Please clarify. Also, what type of dependences is this?
Thanks!
You are correct that the example loop there is not parallelizeable (or at least not easily), but it's not because the inner contents rely on both m and n (as those could be passed as arguments to a new thread or whatever). It's because later calculations depend on the results of earlier calculations. For example, the value of result[10][12] depends on result[9][8] which depends on result[5][7] etc.
When using formal aspects to create some code is there a generic method of determining a loop invariant or will it be completely different depending on the problem?
It has already been pointed out that one same loop can have several invariants, and that Calculability is against you. It doesn't mean that you cannot try.
You are, in fact, looking for an inductive invariant: the word invariant may also be used for a property that is true at each iteration but for which is it not enough to know that it hold at one iteration to deduce that it holds at the next. If I is an inductive invariant, then any consequence of I is an invariant, but may not be an inductive invariant.
You are probably trying to get an inductive invariant to prove a certain property (post-condition) of the loop in some defined circumstances (pre-conditions).
There are two heuristics that work quite well:
start with what you have (pre-conditions), and weaken until you have an inductive invariant. In order to get an intuition how to weaken, apply one or several forward loop iterations and see what ceases to be true in the formula you have.
start with what you want (post-conditions) and strengthen until you have an inductive invariant. To get the intuition how to strengthen, apply one or several loop iterations backwards and see what needs to be added so that the post-condition can be deduced.
If you want the computer to help you in your practice, I can recommend the Jessie deductive verification plug-in for C programs of Frama-C. There are others, especially for Java and JML annotations, but I am less familiar with them. Trying out the invariants you think of is much faster than working out if they work on paper. I should point out that verifying that a property is an inductive invariant is also undecidable, but modern automatic provers do great on many simple examples. If you decide to go that route, get as many as you can from the list: Alt-ergo, Simplify, Z3.
With the optional (and slightly difficult to install) library Apron, Jessie can also infer some simple invariants automatically.
It's actually trivial to generate loop invariants. true is a good one for instance. It fulfills all three properties you want:
It holds before loop entry
It holds after each iteration
It holds after loop termination
But what you're after is probably the strongest loop invariant. Finding the strongest loop invariant however, is sometimes even an undecidable task. See article Inadequacy of Computable Loop Invariants.
I don't think it's easy to automate this. From wiki:
Because of the fundamental similarity of loops and recursive programs, proving partial correctness of loops with invariants is very similar to proving correctness of recursive programs via induction. In fact, the loop invariant is often the inductive property one has to prove of a recursive program that is equivalent to a given loop.
I've written about writing loop invariants in my blog, see Verifying Loops Part 2. The invariants needed to prove a loop correct typically comprise 2 parts:
A generalisation of the state that is intended when the loop terminates.
Extra bits needed to ensure that the loop body is well-formed (e.g. array indices in bounds).
(2) is straightforward. To derive (1), start with a predicate expressing the desired state after termination. Chances are it contains a 'forall' or 'exists' over some range of data. Now change the bounds of the 'forall' or 'exists' so that (a) they depend on variables modified by the loop (e.g. loop counters), and (b) so that the invariant is trivially true when the loop is first entered (usually by making the range of the 'forall' or 'exists' empty).
There are a number of heuristics for finding loop invariants. One good book about this is "Programming in the 1990s" by Ed Cohen. It's about how to find a good invariant by manipulating the postcondition by hand. Examples are: replace a constant by a variable, strengthen invariant, ...