I have written a small program to calculate the factorial of a number using Dynamic Programming Technique.
#include<stdio.h>
int fact(int n)
{
int f[n],i;
f[0] = 1;
for(i=1;i<=n;i++)
f[i] = i * f[i-1];
return f[n];
}
int main(void)
{
printf("\n Factorial of %d is %d ",5,fact(5));
return 0;
}
Is the approach of memorization correct? Because, dynamic programming involves recursion. But I have not included it here. So I am not sure of my approach.
Yes, your approach of solving the problem is a very simple case of Dynamic Programming, where you store previously solved sub-problems to help you solve the actual problem. While the example you provided would be considered Dynamic Programming, it usually isn't called Memoization
When someone says Memoization, it usually involves in a top-down approach of solving problems, where you assume you have already solved the sub-problems by structuring your program in a way that will solve sub-problems recursively.
You store, or memoize, the results of these sub-problems so that they will not be computed multiple times.
Let me illustrate Memoization through an example:
Here is a simple example of computing the nth Fibonacci of a number:
int fib(int n)
{
if (n <= 1)
return n;
return fib(n-1) + fib(n-2);
}
The above code uses recursion to solve sub-problems (fib(n-1) and fib(n-2)) so that fib(n) can be solved in the end. It assumes that fib(n-1) and fib(n-2) are already solved in the way that it is structured.
Though this code looks elegant, the running time is exponential, because you can solve fib(i), where i is a number less than n, multiple times. You can look at the diagram presented here to see the tree generated by this problem: http://www.geeksforgeeks.org/program-for-nth-fibonacci-number.
To avoid the unnecessary re-computation, Memoization is used to optimizes run-time by using memory.
Here is an optimized example of computing the nth Fibonacci number using Memoization:
/*Global array initialized to 0*/
int a[100];
int fib(int n)
{
/*base case*/
if (n <= 1)
return n;
/*if fib(n) has not been computed, compute it*/
if (a[n] == 0) {
a[n] = fib(n - 1) + fib(n - 2);
}
*/Otherwise, simply get a[n] and return it*/
return a[n];
}
As you can see, the overall structure is not that much different from the recursive solution, but it runs linear time instead of exponential time because fib(i) will only be computed only if we have not computed already.
If I were to use your approach, Dynamic Programming, for the Fibonacci problem, it would look something like this:
int fib(int n)
{
/* just like the array you declared in your solution */
int f[n+1];
int i;
/* set up the base cases, just like how you set f[0] to 1*/
f[0] = 0;
f[1] = 1;
for (i = 2; i <= n; i++)
{
/* using previously solved problem to solve further problems*/
f[i] = f[i-1] + f[i-2];
}
/*return the final result*/
return f[n];
}
There are more subtle differences, trade offs, and implications between Dynamic Programming and Memoization. Some consider Memoization a subset of Dynamic Programming. You can read more about the difference here:
Dynamic programming and memoization: bottom-up vs top-down approaches
Yes this is dynamic programming : going from base cases up to final case. Of course your example (factorial) is too simple so you have been able to simplify many things by yourself : you eliminated the recursion and never use a test in the memoization. But anyway that's it.
For the general scheme of memoization see http://en.wikipedia.org/wiki/Memoization.
For explanation about Dynamic programming see http://en.wikipedia.org/wiki/Dynamic_programming, you will be able to read the section about Fibonacci sequence and its computation using a bottom-up approach.
Related
I have been given a problem to write a recursive function which uses Pascal's Rule. I completed the function and it is working, however, I know it can be improved using memoization. I'm not too sure how to go about this as it is my first time implementing memoization. Any help would be appreciated! Here is my code:
long choose(int n, int k) {
if (k == 0 || n == k) {
return 1;
}
else {
return choose(n - 1, k) + choose(n - 1, k - 1);
}
}
And here is how I am testing it:
int main(int argc, char **argv) {
int n = atoi(argv[1]);
int k = atoi(argv[2]);
printf("%ld \n", choose(n, k));
}
If this is an exercise for a class where you're learning about recursion, then yes you can "improve" it by memoization. However, this is just a band-aid on the underlying grossly inefficient algorithm. If you find yourself using recursion, you're either attacking the problem the wrong way, or you're dealing with a really hard problem that has no efficient solution. Recursive relationships are useful as mathematical identities for proving properties by induction. They're not useful as implementation strategies. Sadly this is not taught well in most CS programs.
In the case of implementing "n choose k", Pascal's triangle is just the wrong way to implement it. Instead you use Pascal's triangle to develop the closed-form formula in terms of factorial, then implement that with a loop.
This is my idea of solving 'nth term of fibonacci series with least processing power'-
int fibo(int n, int a, int b){
return (n>0) ? fibo(n-1, b, a+b) : a;
}
main(){
printf("5th term of fibo is %d", fibo(5 - 1, 0, 1));
}
To print all the terms, till nth term,
int fibo(int n, int a, int b){
printf("%d ", a);
return (n>0)? fibo(n-1, b, a+b): a;
}
I showed this code to my university professor and as per her, this is a wrong approach to solve Fibonacci problem as this does not abstract the method. I should have the function to be called as fibo(n) and not fibo(n, 0, 1). This wasn't a satisfactory answer to me, so I thought of asking experts on SOF.
It has its own advantage over traditional methods of solving Fibonacci problems. The technique where we employ two parallel recursions to get nth term of Fibonacci (fibo(n-1) + fibo(n-2)) might be slow to give 100th term of the series whereas my technique will be lot faster even in the worst scenario.
To abstract it, I can use default parameters but it isn't the case with C. Although I can use something like -
int fibo(int n){return fiboN(n - 1, 0, 1);}
int fiboN(int n, int a, int b){return (n>0)? fiboN(n-1, b, a+b) : a;}
But will it be enough to abstract the whole idea? How should I convince others that the approach isn't wrong (although bit vague)?
(I know, this isn't sort of question that I should I ask on SOF but I just wanted to get advice from experts here.)
With the understanding that the base case in your recursion should be a rather than 0, this seems to me to be an excellent (although not optimal) solution. The recursion in that function is tail-recursion, so a good compiler will be able to avoid stack growth making the function O(1) soace and O(n) time (ignoring the rapid growth in the size of the numbers).
Your professor is correct that the caller should not have to deal with the correct initialisation. So you should provide an external wrapper which avoids the need to fill in the values.
int fibo(int n, int a, int b) {
return n > 0 ? fibo(b, a + b) : a;
}
int fib(int n) { return fibo(n, 0, 1); }
However, it could also be useful to provide and document the more general interface, in case the caller actually wants to vary the initial values.
By the way, there is a faster computation technique, based on the recurrence
fib(a + b - 1) = f(a)f(b) + f(a - 1)f(b - 1)
Replacing b with b + 1 yields:
fib(a + b) = f(a)f(b + 1) + f(a - 1)f(b)
Together, those formulas let us compute:
fib(2n - 1) = fib(n + n - 1)
= fib(n)² + fib(n - 1)²
fib(2n) = fib(n + n)
= fib(n)fib(n + 1) + fib(n - 1)fib(n)
= fib(n)² + 2fib(n)fib(n - 1)
This allows the computation to be performed in O(log n) steps, with each step producing two consecutive values.
Your result will be 0, with your approaches. You just go in recursion, until n=0 and at that point return 0. But you have also to check when n==1 and you should return 1; Also you have values a and b and you do nothing with them.
i would suggest to look at the following recursive function, maybe it will help to fix yours:
int fibo(int n){
if(n < 2){
return n;
}
else
{
return (fibo(n-1) + fibo(n-2));
}
}
It's a classical problem in studying recursion.
EDIT1: According to #Ely suggest, bellow is an optimized recursion, with memorization technique. When one value from the list is calculated, it will not be recalculated again as in first example, but it will be stored in the array and taken from that array whenever is required:
const int MAX_FIB_NUMBER = 10;
int storeCalculatedValues[MAX_FIB_NUMBER] = {0};
int fibo(int n){
if(storeCalculatedValues[n] > 0)
{
return storeCalculatedValues[n];
}
if(n < 2){
storeCalculatedValues[n] = n;
}
else
{
storeCalculatedValues[n] = (fibo(n-1) + fibo(n-2));
}
return storeCalculatedValues[n];
}
Using recursion and with a goal of least processing power, an approach to solve fibonacci() is to have each call return 2 values. Maybe one via a return value and another via a int * parameter.
The usual idea with recursion is to have a a top level function perform a one-time preparation and check of parameters followed by a local helper function written in a lean fashion.
The below follows OP's idea of a int fibo(int n) and a helper one int fiboN(int n, additional parameters)
The recursion depth is O(n) and the memory usage is also O(n).
static int fib1h(int n, int *previous) {
if (n < 2) {
*previous = n-1;
return n;
}
int t;
int sum = fib1h(n-1, &t);
*previous = sum;
return sum + t;
}
int fibo1(int n) {
assert(n >= 0); // Handle negatives in some fashion
int t;
return fib1h(n, &t);
}
#include <stdio.h>
int fibo(int n);//declaring the function.
int main()
{
int m;
printf("Enter the number of terms you wanna:\n");
scanf("%i", &m);
fibo(m);
for(int i=0;i<m;i++){
printf("%i,",fibo(i)); /*calling the function with the help of loop to get all terms */
}
return 0;
}
int fibo(int n)
{
if(n==0){
return 0;
}
if(n==1){
return 1;
}
if (n > 1)
{
int nextTerm;
nextTerm = fibo(n - 2) + fibo(n - 1); /*recursive case,function calling itself.*/
return nextTerm;
}
}
solving 'nth term of fibonacci series with least processing power'
I probably do not need to explain to you the recurrence relation of a Fibonacci number. Though your professor have given you a good hint.
Abstract away details. She is right. If you want the nth Fibonacci number it suffices to merely tell the program just that: Fibonacci(n)
Since you aim for least processing power your professor's hint is also suitable for a technique called memoization, which basically means if you calculated the nth Fibonacci number once, just reuse the result; no need to redo a calculation. In the article you find an example for the factorial number.
For this you may want to consider a data structure in which you store the nth Fibonacci number; if that memory has already a Fibonacci number just retrieve it, otherwise store the calculated Fibonacci number in it.
By the way, didactically not helpful, but interesting: There exists also a closed form expression for the nth Fibonacci number.
This wasn't a satisfactory answer to me, so I thought of asking
experts on SOF.
"Uh, you do not consider your professor an expert?" was my first thought.
As a side note, you can do the fibonacci problem pretty much without recursion, making it the fastest I know approach. The code is in java though:
public int fibFor() {
int sum = 0;
int left = 0;
int right = 1;
for (int i = 2; i <= n; i++) {
sum = left + right;
left = right;
right = sum;
}
return sum;
}
Although #rici 's answer is mostly satisfactory but I just wanted to share what I learnt solving this problem. So here's my understanding on finding fibonacci using recursion-
The traditional implementation fibo(n) { return (n < 2) n : fibo(n-1) + fibo(n-2);} is a lot inefficient in terms of time and space requirements both. This unnecessarily builds stack. It requires O(n) Stack space and O(rn) time, where r = (√5 + 1)/2.
With memoization technique as suggested in #Simion 's answer, we just create a permanent stack instead of dynamic stack created by compiler at run time. So memory requirement remains same but time complexity reduces in amortized way. But is not helpful if we require to use it only the once.
The Approach I suggested in my question requires O(1) space and O(n) time. Time requirement can also be reduced here using same memoization technique in amortized way.
From #rici 's post, fib(2n) = fib(n)² + 2fib(n)fib(n - 1), as he suggests the time complexity reduces to O(log n) and I suppose, the stack growth is still O(n).
So my conclusion is, if I did proper research, time complexity and space requirement both cannot be reduced simultaneously using recursion computation. To achieve both, the alternatives could be using iteration, Matrix exponentiation or fast doubling.
Before this gets accused of being a duplicate, I have looked everywhere on StackOverflow for this answer and have not been able to find something that can explain this to me, so please read the entirety first.
Suppose you need to write a function that takes an integer n and returns the sum of the positive integers from 1..n (I will use C).
int sum_of_integers(int n) {
int i, counter = 0;
for(i = 1; i <= n; i++)
counter += i;
return counter;
}
Obviously, this algorithm is in O(n) time, since the number of instructions it runs is proportional to the input size n.
However, consider this implementation, using the mathematical truth that 1+...+n = (n)(n+1)/2.
int sum_of_integers(int n) {
//Trying to avoid potential int overflow
//And take into account int division
if(n % 2 == 0)
return (n/2)*(n+1);
return ((n+1)/2)*n;
}
My Question: Since multiplication is technically in big-O > O(n), is the first implementation preferred? Is the second implementation considered to be in O(1) or not?
To me, because it does not matter what the size of n is for the second implementation since the same operations are being performed the same amount of times, I feel like it should be in O(1). On the other hand, the second implementation may actually be running more instructions based on the implementation of the multiplication operator.
Schoolbook multiplication takes time O(b^2) where b is the number of bits in the numbers, so using the formula n(n+1)/2 takes time O((log n)^2) which is much faster than O(n).
What is the benefit to use recursion then loop(for,while,do-while)?
Using Recursion(Here i am getting the sum of a given number, suppose number is 5 then 5 + 4 + 3 + 2 + 1:
#include<stdio.h>
int sum(int n);
void main()
{
int sums =0, n;
scanf("%d",&n);
sums = sum(n);
printf("%d",sums);
while (1)
{
}
}
int sum(int n)
{
if (n==1)
return 1;
return n + sum(n-1);
}
Without recursion(Here i am getting the sum of a given number suppose number is 5 then 5 + 4 + 3 + 2 + 1:
#include<stdio.h>
void main()
{
int sum =0, n;
scanf("%d",&n);
for(int i=n;i>=1;i--)
{
sum = sum + i;
}
printf("%d",sum);
while (1)
{
}
}
You can always make a recursive function an iterative one, and vice versa (Turing said).
In some cases, it's better to use recursion (e.g. traveling on a tree), it's more natural to "think recursively" in such cases. However, if using loops isn't more complicated and much more difficult than a recursion, I prefer them.
Recursion is more costly in memory, but sometimes it clearer and a more readable, using loops increases the performance, but recursion can sometimes be better for the programmer (and his performance).
Deciding what to use - recursion or iteration, depends on what you want to implement, and what's more important for you (readability? performance?), asking recursion or iteration is somehow like asking elegance or performance.
recursion can solve all those problems which can be solved by for loop. But it is difficult or quite impossible to solve some problems by for loop. For example 'json' parsing, 'xml' parsing etc. For this kind of problems you should use recursion. Some dynamic problems can be solved easily by recursion instead of for loop. For a hard problem you do not have to think much solving by recursion instead of for loop. You just form the Recurrence_relation.
I prefer loop over recursive functions. See if this helps : Read it
"How to write an algorithm that, given a number n, prints out all the simplified vulgar fractions that have the denominator 1..n"
(I hope I could phrase it well, feel free to rephrase.)
Example: If n is 3, the output should be like "1/2 1/3 2/3"
We were talking about this question in the end of the last class. He showed us a solution and asked us to try to understand the code. Here it is:
#include<stdio.h>
void main()
{
int p,m,n,i,j,a,b;
p=7;
m=0;
n=1;
do
{
printf("%d/%d\n",m,n);
i=j=1;
for(b=2; b<=p; b++)
{
a=m*b/n+1;
if(a*j<b*i)
{
i=a;
j=b;
}
}
m=i;
n=j;
}
while(i<j);
}
I'm new to C and just learning to code, to be honest I couldn't figure out what this code does. And it also prints "0/1", I also wonder why that is, I think it shouldn't be printing that.
Here is my elementary approach to this problem:
#include <stdio.h>
int gcd(int a, int b) // Finds the GCD of two numbers.
{
int temp;
while (a) {
temp = a;
a = b % a;
b = temp;
}
return b;
}
int main(void)
{
int i, j;
for (i = 1; i <= 7; i++) // Denominator goes from 1 to 7
for (j = 1; j < i; j++) // Numerator goes from 1 to denominator
if (gcd(i, j) == 1)
printf("%d/%d ", j, i); // If the numerator and the denominator
// are coprimes then print the fraction
return 0;
}
"n" is 7 in both of the codes. I checked the execution times with much bigger numbers and my algorithm is faster than the other one. So I don't understand what the other code is for. Also any suggestions/corrections about my code is appreciated.
Your professor's code looks like it may have been purposely complicated, maybe as a learning exercise. If that's the case, I can't say I agree with the practice.
Your approach of nested for loops is exactly how I would have approached the solution.
And it also prints "0/1", I also wonder why that is, I think it shouldn't be printing that.
Simply put, it prints "0/1" because of this line:
printf("%d/%d\n",m,n);
The values m and n are initialized to 0 and 1 right before the do loop, so on the first pass it prints exactly that.
Your code is better than the first paste in a few ways. The loop over b is a crappy way of trying to find a common prime factor for m and n. But it only runs to b=7, so the first program can print 11/121 and other non-reduced fractions!
If the loop over b were properly coded, it would take O(sqrt(n)) time. Your gcd() (using the Euclidean Algorithm well) has O(log(n)) time.
The other code is exceptionally poorly written. Single-letter variables for non-idiomatic uses? void main()? No comments? Nobody should be expected to understand that code, and especially not learners.
Your code seems pretty competent - it's far clearer and cleaner than the other code and pretty much superior in every way. The only suggestion I would make is, firstly, you should take in N as user input from the console, to make rerunning the program for different values simpler, and you should also comment the GCD function explaining it's operations.