I'm having hard time trying to figure out how to calculate the time complexity of some code. I know the basics of Big O, although I can't fully understand how to calculate in general.
Here is an example to something I couldn't solve. Hopefully you can:
void f(int n) {
int j, s;
for (j = 0, s = 1; s < n; j++, s*=2)
printf(“!”);
double values[j];
for (int k = 0; k < j; k++)
values[k] = 0;
while (j--)
for (int k = 1; k < j; k++)
values[k] += 1.0 / k;
}
What's the run time? I would love an explanation :)
The first loop iterates log2(n) times, computing j, the order of the highest bit of n. Complexity O(log(n)).
The second loop initializes an array of size j: time and space complexity O(log(n)).
The third loop is a nested loop iterating j times with the nested loop iterating j to 1 times, for a total of j * (j - 1) / 2 times. The time complexity of this is O(log(n)^2), and dominates the previous phases.
The overall time complexity of this function is O(log(n)^2), while the space complexity is O(log(n)).
Related
This question already has answers here:
Why is the runtime of this code O(n^5)?
(3 answers)
Closed 3 years ago.
Find a tight upper bound on the complexity of this program.
I tried. I think the time complexity of this code O(n2).
void function(int n)
{
int count = 0;
for (int i=0; i<n; i++)
for (int j=i; j< i*i; j++)
if (j%i == 0)
{
for (int k=0; k<j; k++)
printf("*");
}
}
But the answer given is O(n5). How?
EDIT: Yikes, after spending the time to answer this question, I discovered that this is a duplicate of a previous question that I had also answered three years ago. Oops!
The tightest bound you can get on this function's runtime is Θ(n4). Here's the derivation.
This is a great place to illustrate a great general strategy for determining the big-O of a piece of code:
"When in doubt, work inside out!"
Let's take your code:
for (int i=0; i<n; i++)
{
for (int j=i; j< i*i; j++)
{
if (j%i == 0)
{
for (int k=0; k<j; k++)
{
printf("*");
}
}
}
}
Our approach for analyzing the runtime complexity will be to repeatedly take the innermost loop and replace it with the amount of work that it does. When we're done, we'll have our final time complexity.
Let's begin with this innermost loop:
for (int k=0; k<j; k++)
{
printf("*");
}
The amount of work done here is Θ(j), since the number of loop iterations is directly proportional to j and we do a constant amount of work per loop iteration. So let's replace this loop with the simpler "do Θ(j) work," giving us this simplified loop nest:
for (int i=0; i<n; i++)
{
for (int j=i; j< i*i; j++)
{
if (j%i == 0)
{
do Θ(j) work
}
}
}
Now, let's take aim at what's now the innermost loop:
for (int j=i; j < i*i; j++)
{
if (j%i == 0)
{
do Θ(j) work
}
}
This loop is unusual in that the amount of work that it does varies pretty dramatically from one iteration to the next. Specifically:
most iterations will do only O(1) work, but
one out of every i iterations will do Θ(j) work.
To analyze this loop, we'll therefore split the work apart into these two constituent pieces and see how much each contributes to the total.
First, let's look at the "easy" iterations of the loop, which do only O(1) work. There are a total of Θ(i2) iterations of the loop (the loop starts counting at j = i and stops when j = i2 and i2 - i = Θ(i2). We can therefore bound the contribution of the of these "easy" loop iterations at O(i2) work.
Now, what about the "hard" loop iterations? These iterations occur when j = i, when j = 2i, when j = 3i, j = 4i, etc. Moreover, each of these "hard" iterations do work directly proportional to j during the iteration. This means that, if we add up the work across all of these iterations, the total work done is given by
i + 2i + 3i + 4i + ... + (i - 1)i.
We can simplify this as follows:
i + 2i + 3i + 4i + ... + (i - 1)i
= i(1 + 2 + 3 + ... + i-1)
= i · Θ(i2)
= Θ(i3).
This uses the fact that 1 + 2 + 3 + ... + k = k(k + 1) / 2 = Θ(k2), which is Gauss's famous sum.
So now we see that the work done by the inner loop here is given by
O(i2) work for the easy iterations, and
Θ(i3) work for the hard iterations.
Summing this up, we see that the total work done by this inner loop is Θ(i3). Continuing our process of working inside out, we can replace this inner loop with "do Θ(i3) work" to get the following:
for (int i=0; i<n; i++)
{
do Θ(i^3) work
}
From here, we see that the work done is
13 + 23 + 33 + ... + (n - 1)3,
and that sum is Θ(n4). (Specifically, it's n2(n - 1)2 / 4.)
So overall, the theory predicts that the runtime should be Θ(n4), which is a factor of n lower than the O(n5) bound you mentioned above. How does the theory match the practice?
I ran this code on a variety of values of n and counted how many times that a star was printed. Here's the values I got back:
n = 500: 7760510375
n = 1000: 124583708250
n = 1500: 631407093625
n = 2000: 1996668166500
n = 2500: 4876304426875
n = 3000: 10113753374750
n = 3500: 18739952510125
n = 4000: 31973339333000
n = 4500: 51219851343375
If the runtime is Θ(n4), then if we double the size of the input, we should scale the output by a factor of 16. If the runtime is Θ(n5), then doubling the input size should scale the output by a factor of 32. Here's what I found:
Ratio of n = 1000 to n = 500: 16.0535
Ratio of n = 2000 to n = 1000: 16.0267
Ratio of n = 3000 to n = 1500: 16.0178
Ratio of n = 4000 to n = 2000: 16.0133
This strongly suggests that the runtime of this function is indeed Θ(n4) rather than Θ(n5).
Hope this helps!
I agree with the other poster, except the innermost loop is O(n^2), as k spans from 0 to j, which itself spans up to n^2. This gives us the desired answer of O(n^5).
The first loop
for (int i=0; i<n; i++)
Gives your first O(n) multiplier.
Next, the second loop
for (int j=i; j< i*i; j++)
Itself multiplies by O(n^2) complexity because i is "like" n here.
The third loop
for (int k=0; k<j; k++)
Multiplies by another O(n^2) because j is "like" n^2 here.
So you get complexity O(n^5).
int f1(int N) {
int Sum, i, j, k;
Sum = 0;
for (i = 0; i < N; i++)
for (j = 0; j < i * i; j++)
for (k = 0; k < j; k++)
Sum++;
return Sum;
}
int f2(int N) {
int Sum, i, j;
Sum = 0;
for (i = 0; i < 10; i++)
for (j = 0; j < i; j++)
Sum += j * N;
return Sum;
}
What are the complexities of f1 and f2?
I have no idea about the complexity of f1 and I think the complexity of f2 should be O(1) since the number of iterations is constant. It is correct?
Your first function has the complexity O(N^(1+2+2)) = O(N^5).
In the first loop i goes from 0 on N, the second one j loops over a limit that depends on N^2 and in the 3rd one k loops on an interval whose size depends on N^2 as well.
The function F2 is constant time, so O(1) because the loops do not have any degree of liberty.
This kind of stuff is studied in the courses of algorithms at the topic "complexity".
There is also another kind of measurement of complexity of algorithms, based on omega-notation.
The complexity of f1 is in O(n^5) since
for(i=0; i<N; i++) //i has a upper bound of n
for(j=0; j<i*i; j++) //j has a upper bound of i^2, which itself has the upper bound of n, so this is n^2
for(k=0; k<j; k++) //k has a upper bound of j, which is n^2
Sum++; //constant
So the complete upper bound is n * n^2 * n^2 which is n^5 so f1 is in O(n^5).
for (i = 0; i < 10; i++) //upper bound of 10 so in O(10) which is in O(1)
for (j = 0; j < i; j++) //upper bound of i so also O(1)
Sum += j * N; //N is just a integer here, and multiplication is a constant operation independent of the size of N, so also O(1)
So f2 is in O(1*1*1) which is simply O(1).
Note all assignments and declarations are also constant.
BTW since Sum++ has no side effects and with the according loops develops a series we know a solution for (math yay), a programmer or optimal compiler optimiser could reduce f1 to a constant program using the gaussian sum formula (n*n+n) / 2, so sum could be just calculated by something like (N*N + N ) / 2 * (N*N*N*N + N*N) / 2) * 2 , however my formula does not consider starting at 0.
Using sigma notation:
f1:
The outer loop runs from 0 to N, the one inside it runs from 0 to i^2 and the last one runs from 0 to j, and inside we only have one operation so we are summing 1. Thus we get:
1+1+1... j times gives 1*j=j, thus we get:
Using the rule of the summation of natural numbers but we replace n (in the Wikipedia article) with i^2 so we get:
The reason for the approximation is because when finding the time complexity of a function and we have the addition of multiple powers we take the highest one. This just makes the math simpler. For example f(n)=(n^3+n^2+n)=O(n^3) (supposing that f(n) represents the maximal running time required by the given algorithm depending on the input size n) .
And using the formula for the summation of the first N numbers to 4th power we get (look at the note in the end):
Thus the time complexity for f1 is O(n^5).
f2:
Using the same method we get:
But this just gives a constant which doesn't depend on n thus the time complexity for f2 is O(1).
note:
When we have a summation of the first N numbers that are to the K power, the time complexity of it would be N^(K+1), so you obviously don't need to remember the formula. For example:
I understand how to use summation to extract the time complexity (and Big O) from linear for-loops using summation, but how would you use it for multiplication incremental loops to get O(logn). For example, the code below is O(nlogn), but I don't know why.
for (i = 0; i < n; i++)
for (j = 1; j < n; j*7)
/*some O(1) operations*/
Also, why is a while loop O(logn) and a do-while loop O(n^2).
At each iteration of the inner loop you perform j = j * 7 (I assume this is what you meant)
That is, at each iteration j = 7j
After n iterations, j = j*7*7*7*7*...*7*7 = j*(7 ^ n)
Let n be the number we want to reach and m the number of iterations, so:
n = j*7*7*7*...7 = j*(7 ^ m)
Let's take a log from both sides:
log(n) = log(j * (7 ^ m)) ~= m*log(7) = O(m)
So, as we can see - the inner loop runs O(log(n)) times.
#include <stdio.h>
int main() {
int N = 8; /* for example */
int sum = 0;
for (int i = 1; i <= N; i++)
for (int j = 1; j <= i*i; j++)
sum++;
printf("Sum = %d\n", sum);
return 0;
}
for each n value (i variable), j values will be n^2. So the complexity will be n . n^2 = n^3. Is that correct?
If problem becomes:
#include <stdio.h>
int main() {
int N = 8; /* for example */
int sum = 0;
for (int i = 1; i <= N; i++)
for (int j = 1; j <= i*i; j++)
for (int k = 1; k <= j*j; k++)
sum++;
printf("Sum = %d\n", sum);
return 0;
}
Then you use existing n^3 . n^2 = n^5 ? Is that correct?
We have i and j < i*i and k < j*j which is x^1 * x^2 * (x^2)^2 = x^3 * x^4 = x^7 by my count.
In particular, since 1 < i < N we have O(N) for the i loop. Since 1 < j <= i^2 <= N^2 we have O(n^2) for the second loop. Extending the logic, we have 1 < k <= j^2 <= (i^2)^2 <= N^4 for the third loop.
Inner to Outer loops, we execute up to N^4 times for each j loop, and up to N^2 times for each i loop, and up to N times over the i loop, making the total be of order N^4 * N^2 * N = N^7 = O(N^7).
I think the complexity is actually O(n^7).
The first loop executes N steps.
The second loop executes N^2 steps.
In the third loop, j*j can reach N^4, so it has O(N^4) complexity.
Overall, N * N^2 * N^4 = O(N^7)
For i = 1 inner loop runs 1^1 times, for i = 2inner loop runs 2^2 times .... and for i = N inner loop runs N^N times. Its complexity is (1^1 + 2^2 + 3^3 + ...... + N^N) of order O(N^3).
In second case, for i = N first inner loop iterates N^N times and hence the second inner loop(inner most) will iterate up to N * (N^N) * (N^N) times. Hence the complexity is of order N * N^2 * N^4, i.e, O(N^7).
Yes. In the first example, the i loop runs N times, and the inner j loop tuns i*i times, which is O(N^2). So the whole thing is O(N^3).
In the second example there is an additional O(N^4) loop (loop to j*j), so it is O(N^5) overall.
For a more formal proof, work out how many times sum++ is executed in terms of N, and look at the highest polynomial order of N. In the first example it will be a(N^3)+b(N^2)+c(N)+d (for some values of a, b, c and d), so the answer is 3.
NB: Edited re example 2 to say it's O(N^4): misread i*i for j*j.
Consider the number of times all loops will be called.
int main() {
int N = 8; /* for example */
int sum = 0;
for (int i = 1; i <= N; i++) /* Called N times */
for (int j = 1; j <= i*i; j++) /* Called N*N times for i=0..N times */
for (int k = 1; k <= j*j; k++) /* Called N^2*N^2 times for j=0..N^2 times and i=0..N times */
sum++;
printf("Sum = %d\n", sum);
return 0;
}
Thus sum++ statement is called O(N^4)*O(N^2)*O(N) times = O(N^7) and this the overall complexity of the program.
The incorrect way to solve this (although common, and often gives the correct answer) is to approximate the average number of iterations of an inner loop with its worst-case. Here, the inner loop loops at worst O(N^4), the middle loop loops at worst O(N^2) times and the outer loop loops O(N) times, giving the (by chance correct) solution of O(N^7) by multiplying these together.
The right way is to work from the inside out, being careful to be explicit about what's being approximated.
The total number of iterations, T, of the increment instruction is the same as your code. Just writing it out:
T = sum(i=1..N)sum(j=1..i^2)sum(k=1..j^2)1.
The innermost sum is just j^2, giving:
T = sum(i=1..N)sum(j=1..i^2)j^2
The sum indexed by j is a sum of squares of consecutive integers. We can calculate that exactly: sum(j=1..n)j^2 is n*(n+1)*(2n+1)/6. Setting n=i^2, we get
T = sum(i=1..N)i^2*(i^2+1)*(2i^2+1)/6
We could continue to compute the exact answer, by using the formula for sums of 6th, 4th and 2nd powers of consecutive integers, but it's a pain, and for complexity we only care about the highest power of i. So we can approximate.
T = sum(i=1..N)(i^6/3 + o(i^5))
We can now use that sum(i=1..N)i^p = Theta(N^{p+1}) to get the final result:
T = Theta(N^7)
I am trying to figure out the complexity of a for loop using Big O notation. I have done this before in my other classes, but this one is more rigorous than the others because it is on the actual algorithm. The code is as follows:
for(i=n ; i>1 ; i/=2) //for any size n
{
for(j = 1; j < i; j++)
{
x+=a
}
}
and
for(i=1 ; i<=n;i++,x=1) //for any size n
{
for(j = 1; j <= i; j++)
{
for(k = 1; k <= j; x+=a,k*=a)
{
}
}
}
I have arrived that the first loop is of O(n) complexity because it is going through the list n times. As for the second loop I am a little lost!
Thank you for the help in the analysis. Each loop is in its own space, they are not together.
Consider the first code fragment,
for(i=n ; i>1 ; i/=2) //for any size n
{
for(j = 1; j < i; j++)
{
x+=a
}
}
The instruction x+=a is executed for a total of n + n/2 + n/4 + ... + 1 times.
Sum of the first log2n terms of a G.P. with starting term n and common ratio 1/2 is, (n (1-(1/2)log2n))/(1/2). Thus the complexity of the first code fragment is O(n).
Now consider the second code fragment,
for(i=1 ; i<=n; i++,x=1)
{
for(j = 1; j <= i; j++)
{
for(k = 1; k <= j; x+=a,k*=a)
{
}
}
}
The two outer loops together call the innermost loop a total of n(n+1)/2 times. The innermost loop is executed at most log<sub>a</sub>n times. Thus the total time complexity of the second code fragment is O(n2logan).
You may formally proceed like the following:
Fragment 1:
Fragment 2 (Pochhammer, G-Function, and Stirling's Approximation):
With log(G(n)).
[UPDATE of Fragment 2]:
With some enhancements from "DISCRETE LOOPS AND WORST CASE PERFORMANCE" publication, by Dr. Johann Blieberger (All cases verified for a = 2):
Where:
Therefore,
EDIT: I agree the first code block is O( n )
You decrement the outer loop i by diving by 2, and in the inner loop you run i times, so the number of iterations will be a sum over all the powers of two less than or equal to N but greater than 0, which is nlog(n)+1 - 1, so O(n).
The second code block is O(loga(n)n2) assuming a is a constant.
The two outermost loops equate to a sum of all the numbers less than or equal to n, which is n(n-1)/2, so O(n2). Finally the inner loop is the powers of a less than an upper bound of n, which is O(logan).