Approach 1:
C(n,r) = n!/(n-r)!r!
Approach 2:
In the book Combinatorial Algorithms by wilf, i have found this:
C(n,r) can be written as C(n-1,r) + C(n-1,r-1).
e.g.
C(7,4) = C(6,4) + C(6,3)
= C(5,4) + C(5,3) + C(5,3) + C(5,2)
. .
. .
. .
. .
After solving
= C(4,4) + C(4,1) + 3*C(3,3) + 3*C(3,1) + 6*C(2,1) + 6*C(2,2)
As you can see, the final solution doesn't need any multiplication. In every form C(n,r), either n==r or r==1.
Here is the sample code i have implemented:
int foo(int n,int r)
{
if(n==r) return 1;
if(r==1) return n;
return foo(n-1,r) + foo(n-1,r-1);
}
See output here.
In the approach 2, there are overlapping sub-problems where we are calling recursion to solve the same sub-problems again. We can avoid it by using Dynamic Programming.
I want to know which is the better way to calculate C(n,r)?.
Both approaches will save time, but the first one is very prone to integer overflow.
Approach 1:
This approach will generate result in shortest time (in at most n/2 iterations), and the possibility of overflow can be reduced by doing the multiplications carefully:
long long C(int n, int r) {
if(r > n - r) r = n - r; // because C(n, r) == C(n, n - r)
long long ans = 1;
int i;
for(i = 1; i <= r; i++) {
ans *= n - r + i;
ans /= i;
}
return ans;
}
This code will start multiplication of the numerator from the smaller end, and as the product of any k consecutive integers is divisible by k!, there will be no divisibility problem. But the possibility of overflow is still there, another useful trick may be dividing n - r + i and i by their GCD before doing the multiplication and division (and still overflow may occur).
Approach 2:
In this approach, you'll be actually building up the Pascal's Triangle. The dynamic approach is much faster than the recursive one (the first one is O(n^2) while the other is exponential). However, you'll need to use O(n^2) memory too.
# define MAX 100 // assuming we need first 100 rows
long long triangle[MAX + 1][MAX + 1];
void makeTriangle() {
int i, j;
// initialize the first row
triangle[0][0] = 1; // C(0, 0) = 1
for(i = 1; i < MAX; i++) {
triangle[i][0] = 1; // C(i, 0) = 1
for(j = 1; j <= i; j++) {
triangle[i][j] = triangle[i - 1][j - 1] + triangle[i - 1][j];
}
}
}
long long C(int n, int r) {
return triangle[n][r];
}
Then you can look up any C(n, r) in O(1) time.
If you need a particular C(n, r) (i.e. the full triangle is not needed), then the memory consumption can be made O(n) by overwriting the same row of the triangle, top to bottom.
# define MAX 100
long long row[MAX + 1];
int C(int n, int r) {
int i, j;
// initialize by the first row
row[0] = 1; // this is the value of C(0, 0)
for(i = 1; i <= n; i++) {
for(j = i; j > 0; j--) {
// from the recurrence C(n, r) = C(n - 1, r - 1) + C(n - 1, r)
row[j] += row[j - 1];
}
}
return row[r];
}
The inner loop is started from the end to simplify the calculations. If you start it from index 0, you'll need another variable to store the value being overwritten.
I think your recursive approach should work efficiently with DP. But it will start giving problems once the constraints increase. See http://www.spoj.pl/problems/MARBLES/
Here is the function which i use in online judges and coding contests. So it works quite fast.
long combi(int n,int k)
{
long ans=1;
k=k>n-k?n-k:k;
int j=1;
for(;j<=k;j++,n--)
{
if(n%j==0)
{
ans*=n/j;
}else
if(ans%j==0)
{
ans=ans/j*n;
}else
{
ans=(ans*n)/j;
}
}
return ans;
}
It is an efficient implementation for your Approach #1
Your Recursive Approach is fine but using DP with your approach will reduce the overhead of solving subproblems again.Now since we already have two Conditions-
nCr(n,r) = nCr(n-1,r-1) + nCr(n-1,r);
nCr(n,0)=nCr(n,n)=1;
Now we can easily build a DP solution by storing our subresults in a 2-D array-
int dp[max][max];
//Initialise array elements with zero
int nCr(int n, int r)
{
if(n==r) return dp[n][r] = 1; //Base Case
if(r==0) return dp[n][r] = 1; //Base Case
if(r==1) return dp[n][r] = n;
if(dp[n][r]) return dp[n][r]; // Using Subproblem Result
return dp[n][r] = nCr(n-1,r) + nCr(n-1,r-1);
}
Now if you want to further otimise, Getting the prime factorization of the binomial coefficient is probably the most efficient way to calculate it, especially if multiplication is expensive.
The fastest method I know is Vladimir's method. One avoids division all together by decomposing nCr into prime factors. As Vladimir says you can do this pretty efficiently using Eratosthenes sieve.Also,Use Fermat's little theorem to calculate nCr mod MOD(Where MOD is a prime number).
Using dynamic programming you can easily find the nCr here is the solution
package com.practice.competitive.maths;
import java.util.Scanner;
public class NCR1 {
public static void main(String[] args) {
try (Scanner scanner = new Scanner(System.in)) {
int testCase = scanner.nextInt();
while (testCase-- > 0) {
int n = scanner.nextInt();
int r = scanner.nextInt();
int[][] combination = combination();
System.out.println(combination[n][r]%1000000007);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public static int[][] combination() {
int combination[][] = new int[1001][1001];
for (int i = 0; i < 1001; i++)
for (int j = 0; j <= i; j++) {
if (j == 0 || j == i)
combination[i][j] = 1;
else
combination[i][j] = combination[i - 1][j - 1] % 1000000007 + combination[i - 1][j] % 1000000007;
}
return combination;
}
}
unsigned long long ans = 1,a=1,b=1;
int k = r,i=0;
if (r > (n-r))
k = n-r;
for (i = n ; k >=1 ; k--,i--)
{
a *= i;
b *= k;
if (a%b == 0)
{
a = (a/b);
b=1;
}
}
ans = a/b;
Related
I found this post How to do iterative quicksort without using stack in c?
but the answer suggested does use a inline stack array! (Only constant amount of extra space is permitted)
The code in the page in reference makes a bold claim:
STACK My implementation does not use the stack to store data...
Yet the function definition has many variables with automatic storage, among them 2 arrays with 1000 entries, which will end up using a fixed but substantial amount of stack space:
// quickSort
//
// This public-domain C implementation by Darel Rex Finley.
//
// * Returns YES if sort was successful, or NO if the nested
// pivots went too deep, in which case your array will have
// been re-ordered, but probably not sorted correctly.
//
// * This function assumes it is called with valid parameters.
//
// * Example calls:
// quickSort(&myArray[0],5); // sorts elements 0, 1, 2, 3, and 4
// quickSort(&myArray[3],5); // sorts elements 3, 4, 5, 6, and 7
bool quickSort(int *arr, int elements) {
#define MAX_LEVELS 1000
int piv, beg[MAX_LEVELS], end[MAX_LEVELS], i=0, L, R ;
beg[0]=0; end[0]=elements;
while (i>=0) {
L=beg[i]; R=end[i]-1;
if (L<R) {
piv=arr[L]; if (i==MAX_LEVELS-1) return NO;
while (L<R) {
while (arr[R]>=piv && L<R) R--; if (L<R) arr[L++]=arr[R];
while (arr[L]<=piv && L<R) L++; if (L<R) arr[R--]=arr[L]; }
arr[L]=piv; beg[i+1]=L+1; end[i+1]=end[i]; end[i++]=L; }
else {
i--; }}
return YES; }
The indentation style is very confusing. Here is a reformatted version:
#define MAX_LEVELS 1000
bool quickSort(int *arr, int elements) {
int piv, beg[MAX_LEVELS], end[MAX_LEVELS], i = 0, L, R;
beg[0] = 0;
end[0] = elements;
while (i >= 0) {
L = beg[i];
R = end[i] - 1;
if (L < R) {
piv = arr[L];
if (i == MAX_LEVELS - 1)
return NO;
while (L < R) {
while (arr[R] >= piv && L < R)
R--;
if (L < R)
arr[L++] = arr[R];
while (arr[L] <= piv && L < R)
L++;
if (L < R)
arr[R--] = arr[L];
}
arr[L] = piv;
beg[i + 1] = L + 1;
end[i + 1] = end[i];
end[i++] = L;
} else {
i--;
}
}
return YES;
}
Note that 1000 is large but not sufficient for pathological cases on moderately large arrays that are already sorted. The function returns NO on such arrays with a size of 1000 only, which is unacceptable.
A much lower value would suffice with an improved version of the algorithm where the larger range is pushed into the array and the loop iterates on the smaller range. This ensures that an array of N entries can handle a set of 2N entries. It still has quadratic time complexity on sorted arrays but at least would sort arrays of all possible sizes.
Here is a modified and instrumented version:
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#define MAX_LEVELS 64
int quickSort(int *arr, size_t elements) {
size_t beg[MAX_LEVELS], end[MAX_LEVELS], L, R;
int i = 0;
beg[0] = 0;
end[0] = elements;
while (i >= 0) {
L = beg[i];
R = end[i];
if (L + 1 < R--) {
int piv = arr[L];
if (i == MAX_LEVELS - 1)
return -1;
while (L < R) {
while (arr[R] >= piv && L < R)
R--;
if (L < R)
arr[L++] = arr[R];
while (arr[L] <= piv && L < R)
L++;
if (L < R)
arr[R--] = arr[L];
}
arr[L] = piv;
if (L - beg[i] > end[i] - R) {
beg[i + 1] = L + 1;
end[i + 1] = end[i];
end[i++] = L;
} else {
beg[i + 1] = beg[i];
end[i + 1] = L;
beg[i++] = L + 1;
}
} else {
i--;
}
}
return 0;
}
int testsort(int *a, size_t size, const char *desc) {
clock_t t = clock();
size_t i;
if (quickSort(a, size)) {
printf("%s: quickSort failure\n", desc);
return 1;
}
for (i = 1; i < size; i++) {
if (a[i - 1] > a[i]) {
printf("%s: sorting error: a[%zu]=%d > a[%zu]=%d\n",
desc, i - 1, a[i - 1], i, a[i]);
return 2;
}
}
t = clock() - t;
printf("%s: %zu elements sorted in %.3fms\n",
desc, size, t * 1000.0 / CLOCKS_PER_SEC);
return 0;
}
int main(int argc, char *argv[]) {
size_t i, size = argc > 1 ? strtoull(argv[1], NULL, 0) : 1000;
int *a = malloc(sizeof(*a) * size);
if (a != NULL) {
for (i = 0; i < size; i++)
a[i] = rand();
testsort(a, size, "random");
for (i = 0; i < size; i++)
a[i] = i;
testsort(a, size, "sorted");
for (i = 0; i < size; i++)
a[i] = size - i;
testsort(a, size, "reverse sorted");
for (i = 0; i < size; i++)
a[i] = 0;
testsort(a, size, "constant");
free(a);
}
return 0;
}
Output:
random: 100000 elements sorted in 7.379ms
sorted: 100000 elements sorted in 2799.752ms
reverse sorted: 100000 elements sorted in 2768.844ms
constant: 100000 elements sorted in 2786.612ms
Here is a slighlty modified version more resistant to pathological cases:
#define MAX_LEVELS 48
int quickSort(int *arr, size_t elements) {
size_t beg[MAX_LEVELS], end[MAX_LEVELS], L, R;
int i = 0;
beg[0] = 0;
end[0] = elements;
while (i >= 0) {
L = beg[i];
R = end[i];
if (R - L > 1) {
size_t M = L + ((R - L) >> 1);
int piv = arr[M];
arr[M] = arr[L];
if (i == MAX_LEVELS - 1)
return -1;
R--;
while (L < R) {
while (arr[R] >= piv && L < R)
R--;
if (L < R)
arr[L++] = arr[R];
while (arr[L] <= piv && L < R)
L++;
if (L < R)
arr[R--] = arr[L];
}
arr[L] = piv;
M = L + 1;
while (L > beg[i] && arr[L - 1] == piv)
L--;
while (M < end[i] && arr[M] == piv)
M++;
if (L - beg[i] > end[i] - M) {
beg[i + 1] = M;
end[i + 1] = end[i];
end[i++] = L;
} else {
beg[i + 1] = beg[i];
end[i + 1] = L;
beg[i++] = M;
}
} else {
i--;
}
}
return 0;
}
Output:
random: 10000000 elements sorted in 963.973ms
sorted: 10000000 elements sorted in 167.621ms
reverse sorted: 10000000 elements sorted in 167.375ms
constant: 10000000 elements sorted in 9.335ms
As a conclusion:
yes quick sort can be implemented without recursion,
no it cannot be implemented without any local automatic storage,
yes only a constant amount of extra space is necessary, but only because we live is a small world where the maximum size of the array is bounded by available memory. A size of 64 for the local objects handles arrays larger than the size of the Internet, much larger than current 64-bit systems could address.
Apparently, it is possible to implement a non-recursive quicksort with only constant amount of extra space as stated here. This builds upon the Sedgewick's work for non-recursive formulation of quicksort. Instead of preserving the boundary values(low and high) it essentially performs a linear scan to determine these bounds.
Can quicksort be implemented in C without stack and recursion?
Quicksort requires two paths be followed forward from each non-trivial partitioning: a new partitioning of each (sub)partition. Information about the previous partitioning (the bounds of one of the resulting partitions) needs to be carried forward to each new partitioning. The question, then, is where does that information live? In particular, where does the information about one partition live while the program is working on the other?
For a serial algorithm, the answer is that the information is stored on a stack or a queue or a functional equivalent of one of those. Always, because those are our names for data structures that serve the needed purpose. In particular, recursion is a special case, not an alternative. In a recursive quicksort, the data are stored on the call stack. For an iterative implementation you can implement a stack in a formal sense, but it's possible to instead use a simple and relatively small array as a makeshift stack.
But stack and queue equivalents can go a lot farther than that. You could append data to a file, for example, for later read-back. You could write it to a pipe. You could transmit it to yourself asynchronously over a communications network.
If you're clever, you can even make the input array itself serve the need for a stack, by encoding the partition bounds using relative element order or some other element property, as described by Ďurian, for example. This involves a space vs speed tradeoff that is probably not a good deal in most cases. However, it has lower space overhead (O(1)) than do typical quicksort implementations (O(log N)), and it does not change the algorithm's O(N log N) asymptotic time complexity.
If you wanted to go crazy, you could even nest iterations in place of recursing. That would impose a hard upper bound on the size of the arrays that could be handled, but not as tight of one as you might think. With some care and a few tricks, you could handle billion-element arrays with a 25-loop nest. Such a deep nest would be ugly and crazy, but nevertheless conceivable. A human could write it by hand. And in that case, the series of nested loop scopes, with their block-scoped variables, serves as a stack equivalent.
So the answer depends on what exactly you mean by "without stack":
yes, you can use a queue instead, though it would need to have about the same capacity as there are elements to sort;
yes, you can use an array or some other kind of sequential data storage, including the input array itself, to emulate a formal stack or queue;
yes, you can encode a suitable stack equivalent directly into the structure of your program;
yes, you can probably come up with other, more esoteric versions of stacks and queues;
but no, you cannot perform a quicksort without something filling the multi-level data-storage role for which a stack or stack-equivalent is conventionally used.
Well, it can, because I implemented a quicksort in fortran IV (it was a long time ago, and before the language supported recursion - and it was for a bet). However you do need somewhere (a large array would do) to remember your state as you do individual bits of work.
It's a lot easier recursively...
Quicksort is by definition a "divide and conquer" searching algorithm, the idea is that you split the given array into smaller partitions. So you are dividing the problem into subproblems, that is easier to solve.
When using Quicksort without recursion you need a struct of some sort to store the partitions you are not using at the time.
That's why the answer of the post uses an array to make quicksort non recursive.
I was thinking of various approaches for finding the triplet sum and I came across this finding a triplet having a given sum. So I thought of giving it a try.
My algorithm:
1) Sort the numbers //O(nlogn)
2) Initialize low=0 and high=size-1
3) loop till low<high
a) if sum-arr[high]-arr[low]< 0 , then decrement high
b) if third number is not in range of (low,high) then break the loop
c) else search for third number using binary search
But I am not getting correct output. I don't know where my logic is wrong because its pretty much simple binary search. Below is what I have implemented.
Please let me know my mistake
static boolean isTriplet(int a[], int size, int sum)
{
Arrays.sort(a); //sort the numbers
int low = 0;
int high = size-1;
while(low<high)
{
int third = sum - a[high] - a[low];
if(third<0)
high--;
else if(!(sum - a[high] - a[low]>a[low] && sum - a[high] - a[low]< a[high])) //if the number is not within the range then no need to find the index
break;
else
{
int index = binarySearch(a,low+1,high-1,third);
if(index != -1)
{
System.out.println(a[low]+" "+a[index]+" "+a[high]);
return true;
}
else
low++;
}
}
return false;
}
I tried it with input {1,2,3,4,5,6} and sum=6 but it returns false and when input was {3,4,8,1,2,7,5} and sum=20 it returned true
I partially understood your idea, not completely. It seems like you are trying to solve the problem in O(n log(n)) time complexity and I am not confident that it is possible. I am not sure how you decide to do this:
else
low++;
I doubt that in some cases maybe you should do
high--
there. Also I am not sure about this piece of code:
if(third<0)
high--;
What if third > 0, but it's less than low?
I read the other question and it proposes a O(n^2 logn) solution, so I provided here such a solution (in Java).
The idea is: iterate with 2 nested for loops (i, j) through all pairs of elements and look up for the third element which would complement the triplet in the rest of the array (that lookup is done using binary search - the while loop.
public class TripletSum {
static boolean solve(int[] a, int k) {
Arrays.sort(a);
int n = a.length;
for(int i = 0; i < n; i++) {
for(int j = i + 1; j < n; j++) {
int low = j + 1, high = n - 1;
while(low <= high) {
int middle = (low + high) / 2;
int sum = a[middle] + a[i] + a[j];
if(sum == k) {
return true;
}
if(sum > k) {
high = middle - 1;
}
if(sum < k) {
low = middle + 1;
}
}
}
}
return false;
}
public static void main(String[] args) {
int[] a = {1,2,3,4,5,6};
System.out.println(solve(a, 20));
}
}
EDIT:
I did some research and couldn't find a O(N logN) solution for this problem. Turns out this problem is popular as 3SUM. You can see on the Wiki page there is a Quadratic solution which beats the O(N^2 logN).
I am trying to solve this: SPOJ problem.
And after some research I found out that it comes down to a simple calculation of the nth fib number, however n can get really large so an O(n) solution won't do any good. Googling around, I found that you can calculate the nth fib number in O(logn) and also a code sample that does exactly that:
long long fibonacci(int n) {
long long fib[2][2] = {{1,1},{1,0}}, ret[2][2] = {{1,0},{0,1}}, tmp[2][2] = {{0,0},{0,0}};
int i, j, k;
while (n) {
if (n & 1) {
memset(tmp, 0, sizeof tmp);
for (i = 0; i < 2; i++)
for (j = 0; j < 2; j++)
for (k = 0; k < 2; k++)
tmp[i][j] = (tmp[i][j] + ret[i][k] * fib[k][j]);
for (i = 0; i < 2; i++)
for (j = 0; j < 2; j++)
ret[i][j] = tmp[i][j];
}
memset(tmp, 0, sizeof tmp);
for (i = 0; i < 2; i++)
for (j = 0; j < 2; j++)
for (k = 0; k < 2; k++)
tmp[i][j] = (tmp[i][j] + fib[i][k] * fib[k][j]);
for (i = 0; i < 2; i++)
for (j = 0; j < 2; j++)
fib[i][j] = tmp[i][j];
n /= 2;
}
return (ret[0][1]);
}
I tried to modify it for the problem and am still getting WA: http://ideone.com/3TtE5m
Am I calculating the modular arithmetic wrong? Or is something else the issue?
You mean the nth Fibonacci number I hope.
In order to do it you need a matrix decomposition of Fibonacci numbers described here.
The basic idea is you take the Donald E. Knuth matrix identity form for a Fibonacci number which is:
And instead of calculating the Fibonacci numbers in the traditional way you will try and find the matrix to the power of (k) where k is the given number.
So this is solving the problem in k matrix multiplications, not really helpful since we can do it in much easier way.
But wait! We can optimise the matrix multiplication. Instead of doing the k multiplications we can square it first and then do the half of the multiplications. And we can keep on doing it. So if the given number is 2a then we can do it in a steps. By keeping squaring the matrix.
If the number is not a power of 2 we can do the binary decomposition of a number and see whether to take the given squared matrix into final product or not.
In your case after each multiplication you also need to apply modulo operator 123456 to each matrix element.
Hope my explanation helps if not see the link for a clearer and longer one.
There is actually one more caveat of the task: as you are asked to provide some Fibonacci number modulo a given number, you should also prove that taking the remainder of each matrix element doesn't change the result. In other words if we multiply matrices and take remainder that we are actually still getting the Fibonacci number remainders. But since the remainder operation is distributive in addition and multiplication it actually does produce the correct results.
The Fibonacci numbers occur as the ratio of successive convergents of the continued fraction for , and the matrix formed from successive convergents of any continued fraction has a determinant of +1 or −1.
The matrix representation gives the following closed-form expression for the Fibonacci numbers i.e.
The matrix is multiplied n time because then only we can get the (n+1)th Fibonacci number as the element at the row and the column (0, 0) in the resultant matrix.
If we apply the above method without using recursive matrix multiplication, then the Time Complexity: O(n) and Space Complexity: O(1).
But we want Time Complexity: O(log n), so we have to optimize the above method, and this can be done by recursive multiplication of matrix to get the nth power.
Implementation of the above rule can be found below.
#include <stdio.h>
void multiply(int F[2][2], int M[2][2]);
void power(int F[2][2], int n);
/*
The function that returns nth Fibonacci number.
*/
int fib(int n) {
int F[2][2] = {{1, 1}, {1, 0}};
if (n == 0)
return 0;
power(F, n - 1);
return F[0][0];
}
/*
Optimized using recursive multiplication.
*/
void power(int F[2][2], int n) {
if ( n == 0 || n == 1)
return;
int M[2][2] = {{1, 1}, {1, 0}};
power(F, n / 2);
multiply(F, F);
if (n % 2 != 0)
multiply(F, M);
}
void multiply(int F[2][2], int M[2][2]) {
int x = F[0][0] * M[0][0] + F[0][1] * M[1][0];
int y = F[0][0] * M[0][1] + F[0][1] * M[1][1];
int z = F[1][0] * M[0][0] + F[1][1] * M[1][0];
int w = F[1][0] * M[0][1] + F[1][1] * M[1][1];
F[0][0] = x;
F[0][1] = y;
F[1][0] = z;
F[1][1] = w;
}
int main() {
printf("%d\n", fib(15));
/*
15th Fibonacci number is 610.
*/
return 0;
}
There is a very simple algorithm, using only integers:
long long fib(int n) {
long long a, b, p, q;
a = q = 1;
b = p = 0;
while (n > 0) {
if (n % 2 == 0) {
long long qq = q*q;
q = 2*p*q + qq;
p = p*p + qq;
n /= 2;
} else {
long long aq = a*q;
a = b*q + aq + a*p;
b = b*p + aq;
n -= 1;
}
}
return b;
}
This is based on the identities of the Lucas sequence.
Given a snipplet of code, how will you determine the complexities in general. I find myself getting very confused with Big O questions. For example, a very simple question:
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
System.out.println("*");
}
}
The TA explained this with something like combinations. Like this is n choose 2 = (n(n-1))/2 = n^2 + 0.5, then remove the constant so it becomes n^2. I can put int test values and try but how does this combination thing come in?
What if theres an if statement? How is the complexity determined?
for (int i = 0; i < n; i++) {
if (i % 2 ==0) {
for (int j = i; j < n; j++) { ... }
} else {
for (int j = 0; j < i; j++) { ... }
}
}
Then what about recursion ...
int fib(int a, int b, int n) {
if (n == 3) {
return a + b;
} else {
return fib(b, a+b, n-1);
}
}
In general, there is no way to determine the complexity of a given function
Warning! Wall of text incoming!
1. There are very simple algorithms that no one knows whether they even halt or not.
There is no algorithm that can decide whether a given program halts or not, if given a certain input. Calculating the computational complexity is an even harder problem since not only do we need to prove that the algorithm halts but we need to prove how fast it does so.
//The Collatz conjecture states that the sequence generated by the following
// algorithm always reaches 1, for any initial positive integer. It has been
// an open problem for 70+ years now.
function col(n){
if (n == 1){
return 0;
}else if (n % 2 == 0){ //even
return 1 + col(n/2);
}else{ //odd
return 1 + col(3*n + 1);
}
}
2. Some algorithms have weird and off-beat complexities
A general "complexity determining scheme" would easily get too complicated because of these guys
//The Ackermann function. One of the first examples of a non-primitive-recursive algorithm.
function ack(m, n){
if(m == 0){
return n + 1;
}else if( n == 0 ){
return ack(m-1, 1);
}else{
return ack(m-1, ack(m, n-1));
}
}
function f(n){ return ack(n, n); }
//f(1) = 3
//f(2) = 7
//f(3) = 61
//f(4) takes longer then your wildest dreams to terminate.
3. Some functions are very simple but will confuse lots of kinds of static analysis attempts
//Mc'Carthy's 91 function. Try guessing what it does without
// running it or reading the Wikipedia page ;)
function f91(n){
if(n > 100){
return n - 10;
}else{
return f91(f91(n + 11));
}
}
That said, we still need a way to find the complexity of stuff, right? For loops are a simple and common pattern. Take your initial example:
for(i=0; i<N; i++){
for(j=0; j<i; j++){
print something
}
}
Since each print something is O(1), the time complexity of the algorithm will be determined by how many times we run that line. Well, as your TA mentioned, we do this by looking at the combinations in this case. The inner loop will run (N + (N-1) + ... + 1) times, for a total of (N+1)*N/2.
Since we disregard constants we get O(N2).
Now for the more tricky cases we can get more mathematical. Try to create a function whose value represents how long the algorithm takes to run, given the size N of the input. Often we can construct a recursive version of this function directly from the algorithm itself and so calculating the complexity becomes the problem of putting bounds on that function. We call this function a recurrence
For example:
function fib_like(n){
if(n <= 1){
return 17;
}else{
return 42 + fib_like(n-1) + fib_like(n-2);
}
}
it is easy to see that the running time, in terms of N, will be given by
T(N) = 1 if (N <= 1)
T(N) = T(N-1) + T(N-2) otherwise
Well, T(N) is just the good-old Fibonacci function. We can use induction to put some bounds on that.
For, example, Lets prove, by induction, that T(N) <= 2^n for all N (ie, T(N) is O(2^n))
base case: n = 0 or n = 1
T(0) = 1 <= 1 = 2^0
T(1) = 1 <= 2 = 2^1
inductive case (n > 1):
T(N) = T(n-1) + T(n-2)
aplying the inductive hypothesis in T(n-1) and T(n-2)...
T(N) <= 2^(n-1) + 2^(n-2)
so..
T(N) <= 2^(n-1) + 2^(n-1)
<= 2^n
(we can try doing something similar to prove the lower bound too)
In most cases, having a good guess on the final runtime of the function will allow you to easily solve recurrence problems with an induction proof. Of course, this requires you to be able to guess first - only lots of practice can help you here.
And as f final note, I would like to point out about the Master theorem, the only rule for more difficult recurrence problems I can think of now that is commonly used. Use it when you have to deal with a tricky divide and conquer algorithm.
Also, in your "if case" example, I would solve that by cheating and splitting it into two separate loops that don; t have an if inside.
for (int i = 0; i < n; i++) {
if (i % 2 ==0) {
for (int j = i; j < n; j++) { ... }
} else {
for (int j = 0; j < i; j++) { ... }
}
}
Has the same runtime as
for (int i = 0; i < n; i += 2) {
for (int j = i; j < n; j++) { ... }
}
for (int i = 1; i < n; i+=2) {
for (int j = 0; j < i; j++) { ... }
}
And each of the two parts can be easily seen to be O(N^2) for a total that is also O(N^2).
Note that I used a good trick trick to get rid of the "if" here. There is no general rule for doing so, as shown by the Collatz algorithm example
In general, deciding algorithm complexity is theoretically impossible.
However, one cool and code-centric method for doing it is to actually just think in terms of programs directly. Take your example:
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
System.out.println("*");
}
}
Now we want to analyze its complexity, so let's add a simple counter that counts the number of executions of the inner line:
int counter = 0;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
System.out.println("*");
counter++;
}
}
Because the System.out.println line doesn't really matter, let's remove it:
int counter = 0;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
counter++;
}
}
Now that we have only the counter left, we can obviously simplify the inner loop out:
int counter = 0;
for (int i = 0; i < n; i++) {
counter += n;
}
... because we know that the increment is run exactly n times. And now we see that counter is incremented by n exactly n times, so we simplify this to:
int counter = 0;
counter += n * n;
And we emerged with the (correct) O(n2) complexity :) It's there in the code :)
Let's look how this works for a recursive Fibonacci calculator:
int fib(int n) {
if (n < 2) return 1;
return fib(n - 1) + fib(n - 2);
}
Change the routine so that it returns the number of iterations spent inside it instead of the actual Fibonacci numbers:
int fib_count(int n) {
if (n < 2) return 1;
return fib_count(n - 1) + fib_count(n - 2);
}
It's still Fibonacci! :) So we know now that the recursive Fibonacci calculator is of complexity O(F(n)) where F is the Fibonacci number itself.
Ok, let's look at something more interesting, say simple (and inefficient) mergesort:
void mergesort(Array a, int from, int to) {
if (from >= to - 1) return;
int m = (from + to) / 2;
/* Recursively sort halves */
mergesort(a, from, m);
mergesort(m, m, to);
/* Then merge */
Array b = new Array(to - from);
int i = from;
int j = m;
int ptr = 0;
while (i < m || j < to) {
if (i == m || a[j] < a[i]) {
b[ptr] = a[j++];
} else {
b[ptr] = a[i++];
}
ptr++;
}
for (i = from; i < to; i++)
a[i] = b[i - from];
}
Because we are not interested in the actual result but the complexity, we change the routine so that it actually returns the number of units of work carried out:
int mergesort(Array a, int from, int to) {
if (from >= to - 1) return 1;
int m = (from + to) / 2;
/* Recursively sort halves */
int count = 0;
count += mergesort(a, from, m);
count += mergesort(m, m, to);
/* Then merge */
Array b = new Array(to - from);
int i = from;
int j = m;
int ptr = 0;
while (i < m || j < to) {
if (i == m || a[j] < a[i]) {
b[ptr] = a[j++];
} else {
b[ptr] = a[i++];
}
ptr++;
count++;
}
for (i = from; i < to; i++) {
count++;
a[i] = b[i - from];
}
return count;
}
Then we remove those lines that do not actually impact the counts and simplify:
int mergesort(Array a, int from, int to) {
if (from >= to - 1) return 1;
int m = (from + to) / 2;
/* Recursively sort halves */
int count = 0;
count += mergesort(a, from, m);
count += mergesort(m, m, to);
/* Then merge */
count += to - from;
/* Copy the array */
count += to - from;
return count;
}
Still simplifying a bit:
int mergesort(Array a, int from, int to) {
if (from >= to - 1) return 1;
int m = (from + to) / 2;
int count = 0;
count += mergesort(a, from, m);
count += mergesort(m, m, to);
count += (to - from) * 2;
return count;
}
We can now actually dispense with the array:
int mergesort(int from, int to) {
if (from >= to - 1) return 1;
int m = (from + to) / 2;
int count = 0;
count += mergesort(from, m);
count += mergesort(m, to);
count += (to - from) * 2;
return count;
}
We can now see that actually the absolute values of from and to do not matter any more, but only their distance, so we modify this to:
int mergesort(int d) {
if (d <= 1) return 1;
int count = 0;
count += mergesort(d / 2);
count += mergesort(d / 2);
count += d * 2;
return count;
}
And then we get to:
int mergesort(int d) {
if (d <= 1) return 1;
return 2 * mergesort(d / 2) + d * 2;
}
Here obviously d on the first call is the size of the array to be sorted, so you have the recurrence for the complexity M(x) (this is in plain sight on the second line :)
M(x) = 2(M(x/2) + x)
and this you need to solve in order to get to a closed form solution. This you do easiest by guessing the solution M(x) = x log x, and verify for the right side:
2 (x/2 log x/2 + x)
= x log x/2 + 2x
= x (log x - log 2 + 2)
= x (log x - C)
and verify it is asymptotically equivalent to the left side:
x log x - Cx
------------ = 1 - [Cx / (x log x)] = 1 - [C / log x] --> 1 - 0 = 1.
x log x
Even though this is an over generalization, I like to think of Big-O in terms of lists, where the length of the list is N items.
Thus, if you have a for-loop that iterates over everything in the list, it is O(N). In your code, you have one line that (in isolation all by itself) is 0(N).
for (int i = 0; i < n; i++) {
If you have a for loop nested inside another for loop, and you perform an operation on each item in the list that requires you to look at every item in the list, then you are doing an operation N times for each of N items, thus O(N^2). In your example above you do in fact, have another for loop nested inside your for loop. So you can think about it as if each for loop is 0(N), and then because they are nested, multiply them together for a total value of 0(N^2).
Conversely, if you are just doing a quick operation on a single item then that would be O(1). There is no 'list of length n' to go over, just a single one time operation.To put this in context, in your example above, the operation:
if (i % 2 ==0)
is 0(1). What is important isn't the 'if', but the fact that checking to see if a single item is equal to another item is a quick operation on a single item. Like before, the if statement is nested inside your external for loop. However, because it is 0(1), then you are multiplying everything by '1', and so there is no 'noticeable' affect in your final calculation for the run time of the entire function.
For logs, and dealing with more complex situations (like this business of counting up to j or i, and not just n again), I would point you towards a more elegant explanation here.
I like to use two things for Big-O notation: standard Big-O, which is worst case scenario, and average Big-O, which is what normally ends up happening. It also helps me to remember that Big-O notation is trying to approximate run-time as a function of N, the number of inputs.
The TA explained this with something like combinations. Like this is n choose 2 = (n(n-1))/2 = n^2 + 0.5, then remove the constant so it becomes n^2. I can put int test values and try but how does this combination thing come in?
As I said, normal big-O is worst case scenario. You can try to count the number of times that each line gets executed, but it is simpler to just look at the first example and say that there are two loops over the length of n, one embedded in the other, so it is n * n. If they were one after another, it'd be n + n, equaling 2n. Since its an approximation, you just say n or linear.
What if theres an if statement? How is the complexity determined?
This is where for me having average case and best case helps a lot for organizing my thoughts. In worst case, you ignore the if and say n^2. In average case, for your example, you have a loop over n, with another loop over part of n that happens half of the time. This gives you n * n/x/2 (the x is whatever fraction of n gets looped over in your embedded loops. This gives you n^2/(2x), so you'd get n^2 just the same. This is because its an approximation.
I know this isn't a complete answer to your question, but hopefully it sheds some kind of light on approximating complexities in code.
As has been said in the answers above mine, it is clearly not possible to determine this for all snippets of code; I just wanted to add the idea of using average case Big-O to the discussion.
For the first snippet, it's just n^2 because you perform n operations n times. If j was initialized to i, or went up to i, the explanation you posted would be more appropriate but as it stands it is not.
For the second snippet, you can easily see that half of the time the first one will be executed, and the second will be executed the other half of the time. Depending on what's in there (hopefully it's dependent on n), you can rewrite the equation as a recursive one.
The recursive equations (including the third snippet) can be written as such: the third one would appear as
T(n) = T(n-1) + 1
Which we can easily see is O(n).
Big-O is just an approximation, it doesn't say how long an algorithm takes to execute, it just says something about how much longer it takes when the size of its input grows.
So if the input is size N and the algorithm evaluates an expression of constant complexity: O(1) N times, the complexity of the algorithm is linear: O(N). If the expression has linear complexity, the algorithm has quadratic complexity: O(N*N).
Some expressions have exponential complexity: O(N^N) or logarithmic complexity: O(log N). For an algorithm with loops and recursion, multiply the complexities of each level of loop and/or recursion. In terms of complexity, looping and recursion are equivalent. An algorithm that has different complexities at different stages in the algorithm, choose the highest complexity and ignore the rest. And finally, all constant complexities are considered equivalent: O(5) is the same as O(1), O(5*N) is the same as O(N).
This question already has answers here:
nth fibonacci number in sublinear time
(16 answers)
Closed 6 years ago.
I am a CSE student and preparing myself for programming contest.Now I am working on Fibonacci series. I have a input file of size about some Kilo bytes containing positive integers. Input formate looks like
3 5 6 7 8 0
A zero means the end of file. Output should like
2
5
8
13
21
my code is
#include<stdio.h>
int fibonacci(int n) {
if (n==1 || n==2)
return 1;
else
return fibonacci(n-1) +fibonacci(n-2);
}
int main() {
int z;
FILE * fp;
fp = fopen ("input.txt","r");
while(fscanf(fp,"%d", &z) && z)
printf("%d \n",fibonacci(z));
return 0;
}
The code works fine for sample input and provide accurate result but problem is for my real input set it is taking more time than my time limit. Can anyone help me out.
You could simply use a tail recursion version of a function that returns the two last fibonacci numbers if you have a limit on the memory.
int fib(int n)
{
int a = 0;
int b = 1;
while (n-- > 1) {
int t = a;
a = b;
b += t;
}
return b;
}
This is O(n) and needs a constant space.
You should probably look into memoization.
http://en.wikipedia.org/wiki/Memoization
It has an explanation and a fib example right there
You can do this by matrix multiplictation, raising the matrix to power n and then multiply it by an vector. You can raise it to power in logaritmic time.
I think you can find the problem here. It's in romanian but you can translate it with google translate. It's exactly what you want, and the solution it's listed there.
Your algorithm is recursive, and approximately has O(2^N) complexity.
This issue has been discussed on stackoverflow before:
Computational complexity of Fibonacci Sequence
There is also a faster implementation posted in that particular discussion.
Look in Wikipedia, there is a formula that gives the number in the Fibonacci sequence with no recursion at all
Use memoization. That is, you cache the answers to avoid unnecessary recursive calls.
Here's a code example:
#include <stdio.h>
int memo[10000]; // adjust to however big you need, but the result must fit in an int
// and keep in mind that fibonacci values grow rapidly :)
int fibonacci(int n) {
if (memo[n] != -1)
return memo[n];
if (n==1 || n==2)
return 1;
else
return memo[n] = fibonacci(n-1) +fibonacci(n-2);
}
int main() {
for(int i = 0; i < 10000; ++i)
memo[i] = -1;
fibonacci(50);
}
Nobody mentioned the 2 value stack array version, so I'll just do it for completeness.
// do not call with i == 0
uint64_t Fibonacci(uint64_t i)
{
// we'll only use two values on stack,
// initialized with F(1) and F(2)
uint64_t a[2] = {1, 1};
// We do not enter loop if initial i was 1 or 2
while (i-- > 2)
// A bitwise AND allows switching the storing of the new value
// from index 0 to index 1.
a[i & 1] = a[0] + a[1];
// since the last value of i was 0 (decrementing i),
// the return value is always in a[0 & 1] => a[0].
return a[0];
}
This is a O(n) constant stack space solution that will perform slightly the same than memoization when compiled with optimization.
// Calc of fibonacci f(99), gcc -O2
Benchmark Time(ns) CPU(ns) Iterations
BM_2stack/99 2 2 416666667
BM_memoization/99 2 2 318181818
The BM_memoization used here will initialize the array only once and reuse it for every other call.
The 2 value stack array version performs identically as a version with a temporary variable when optimized.
You can also use the fast doubling method of generating Fibonacci series
Link: fastest-way-to-compute-fibonacci-number
It is actually derived from the results of the matrix exponentiation method.
Use the golden-ratio
Build an array Answer[100] in which you cache the results of fibonacci(n).
Check in your fibonacci code to see if you have precomputed the answer, and
use that result. The results will astonish you.
Are you guaranteed that, as in your example, the input will be given to you in ascending order? If so, you don't even need memoization; just keep track of the last two results, start generating the sequence but only display the Nth number in the sequence if N is the next index in your input. Stop when you hit index 0.
Something like this:
int i = 0;
while ( true ) {
i++; //increment index
fib_at_i = generate_next_fib()
while ( next_input_index() == i ) {
println fib_at_i
}
I leave exit conditions and actually generating the sequence to you.
In C#:
static int fib(int n)
{
if (n < 2) return n;
if (n == 2) return 1;
int k = n / 2;
int a = fib(k + 1);
int b = fib(k);
if (n % 2 == 1)
return a * a + b * b;
else
return b * (2 * a - b);
}
Matrix multiplication, no float arithmetic, O(log N) time complexity assuming integer multiplication/addition is done in constant time.
Here goes python code
def fib(n):
x,y = 1,1
mat = [1,1,1,0]
n -= 1
while n>0:
if n&1==1:
x,y = x*mat[0]+y*mat[1], x*mat[2]+y*mat[3]
n >>= 1
mat[0], mat[1], mat[2], mat[3] = mat[0]*mat[0]+mat[1]*mat[2], mat[0]*mat[1]+mat[1]*mat[3], mat[0]*mat[2]+mat[2]*mat[3], mat[1]*mat[2]+mat[3]*mat[3]
return x
You can reduce the overhead of the if statement: Calculating Fibonacci Numbers Recursively in C
First of all, you can use memoization or an iterative implementation of the same algorithm.
Consider the number of recursive calls your algorithm makes:
fibonacci(n) calls fibonacci(n-1) and fibonacci(n-2)
fibonacci(n-1) calls fibonacci(n-2) and fibonacci(n-3)
fibonacci(n-2) calls fibonacci(n-3) and fibonacci(n-4)
Notice a pattern? You are computing the same function a lot more times than needed.
An iterative implementation would use an array:
int fibonacci(int n) {
int arr[maxSize + 1];
arr[1] = arr[2] = 1; // ideally you would use 0-indexing, but I'm just trying to get a point across
for ( int i = 3; i <= n; ++i )
arr[i] = arr[i - 1] + arr[i - 2];
return arr[n];
}
This is already much faster than your approach. You can do it faster on the same principle by only building the array once up until the maximum value of n, then just print the correct number in a single operation by printing an element of your array. This way you don't call the function for every query.
If you can't afford the initial precomputation time (but this usually only happens if you're asked for the result modulo something, otherwise they probably don't expect you to implement big number arithmetic and precomputation is the best solution), read the fibonacci wiki page for other methods. Focus on the matrix approach, that one is very good to know in a contest.
#include<stdio.h>
int g(int n,int x,int y)
{
return n==0 ? x : g(n-1,y,x+y);}
int f(int n)
{
return g(n,0,1);}
int main (void)
{
int i;
for(i=1; i<=10 ; i++)
printf("%d\n",f(i)
return 0;
}
In the functional programming there is a special algorithm for counting fibonacci. The algorithm uses accumulative recursion. Accumulative recursion are used to minimize the stack size used by algorithms. I think it will help you to minimize the time. You can try it if you want.
int ackFib (int n, int m, int count){
if (count == 0)
return m;
else
return ackFib(n+m, n, count-1);
}
int fib(int n)
{
return ackFib (0, 1, n+1);
}
use any of these: Two Examples of recursion, One with for Loop O(n) time and one with golden ratio O(1) time:
private static long fibonacciWithLoop(int input) {
long prev = 0, curr = 1, next = 0;
for(int i = 1; i < input; i++){
next = curr + prev;
prev = curr;
curr = next;
}
return curr;
}
public static long fibonacciGoldenRatio(int input) {
double termA = Math.pow(((1 + Math.sqrt(5))/2), input);
double termB = Math.pow(((1 - Math.sqrt(5))/2), input);
double factor = 1/Math.sqrt(5);
return Math.round(factor * (termA - termB));
}
public static long fibonacciRecursive(int input) {
if (input <= 1) return input;
return fibonacciRecursive(input - 1) + fibonacciRecursive(input - 2);
}
public static long fibonacciRecursiveImproved(int input) {
if (input == 0) return 0;
if (input == 1) return 1;
if (input == 2) return 1;
if (input >= 93) throw new RuntimeException("Input out of bounds");
// n is odd
if (input % 2 != 0) {
long a = fibonacciRecursiveImproved((input+1)/2);
long b = fibonacciRecursiveImproved((input-1)/2);
return a*a + b*b;
}
// n is even
long a = fibonacciRecursiveImproved(input/2 + 1);
long b = fibonacciRecursiveImproved(input/2 - 1);
return a*a - b*b;
}
using namespace std;
void mult(LL A[ 3 ][ 3 ], LL B[ 3 ][ 3 ]) {
int i,
j,
z;
LL C[ 3 ][ 3 ];
memset(C, 0, sizeof( C ));
for(i = 1; i <= N; i++)
for(j = 1; j <= N; j++) {
for(z = 1; z <= N; z++)
C[ i ][ j ] = (C[ i ][ j ] + A[ i ][ z ] * B[ z ][ j ] % mod ) % mod;
}
memcpy(A, C, sizeof(C));
};
void readAndsolve() {
int i;
LL k;
ifstream I(FIN);
ofstream O(FOUT);
I>>k;
LL A[3][3];
LL B[3][3];
A[1][1] = 1; A[1][2] = 0;
A[2][1] = 0; A[2][2] = 1;
B[1][1] = 0; B[1][2] = 1;
B[2][1] = 1; B[2][2] = 1;
for(i = 0; ((1<<i) <= k); i++) {
if( k & (1<<i) ) mult(A, B);
mult(B, B);
}
O<<A[2][1];
}
//1,1,2,3,5,8,13,21,33,...
int main() {
readAndsolve();
return(0);
}
public static int GetNthFibonacci(int n)
{
var previous = -1;
var current = 1;
int element = 0;
while (1 <= n--)
{
element = previous + current;
previous = current;
current = element;
}
return element;
}
This is similar to answers given before, but with some modifications. Memorization, as stated in other answers, is another way to do this, but I dislike code that doesn't scale as technology changes (size of an unsigned int varies depending on the platform) so the highest value in the sequence that can be reached may also vary, and memorization is ugly in my opinion.
#include <iostream>
using namespace std;
void fibonacci(unsigned int count) {
unsigned int x=0,y=1,z=0;
while(count--!=0) {
cout << x << endl; // you can put x in an array or whatever
z = x;
x = y;
y += z;
}
}
int main() {
fibonacci(48);// 48 values in the sequence is the maximum for a 32-bit unsigend int
return 0;
}
Additionally, if you use <limits> its possible to write a compile-time constant expression that would give you the largest index within the sequence that can be reached for any integral data type.
#include<stdio.h>
main()
{
int a,b=2,c=5,d;
printf("%d %d ");
do
{
d=b+c;
b=c;
c=d;
rintf("%d ");
}