I have a problem with a loop that requires a decreasing number of operations each time the loop executes. Here's the code:
for (int i = 1; i < n; i++) {
...code that takes at most 100/i operations to execute...
}
I need to find a big O that describes the number of operations. I think what's tripping me up here is that bigger n = more operations, but the growth is smaller.
Thanks for your help!
Harmonic number 1 + 1/2 + 1/3 + ... + 1/n is O(log n)
Also, what if n > 100? For instance: Is 100/12345 operations well defined?
Related
When looking at this code for example :
for (int i = 1; i < n; i*=2)
for (int j = 0; j < i; j +=2)
{
// some contstant time operations
}
Is it as simple as saying that because the outer loop is log and and inner loop is n , that combined the result is big(O) of nlogn ?
Here is the analysis of the example in the question. For simplicity I will neglect the increment of 2 in the inner loop and will consider it as 1, because in terms of complexity it does not matter - the inner loop is linear in i and the constant factor of 2 does not matter.
So we can notice, that the outer loop is producing is of values which are powers of 2 capped by n, that is:
1, 2, 4, 8, ... , 2^(log2 n)
these numbers are also the numbers that the "constant time operation" in the inner loop is running for each i.
So all we have to do is to sum up the above series. It is easy to see that these are geometric series:
2^0 + 2^1 + 2^2 + ... + 2^(log2 n)
and it has a well known solution:
(from Wiki )
We have a=1, r=2, and... well n_from_the_image =log n. We have a same name for different variables here, so it is a bit of a problem.
Now let's substitute and get that the sum equals
(1-2^((log2 n) + 1) / (1 - 2) = (1 - 2*n) / (1-2) = 2n-1
Which is a linear O(n) complexity.
Generally, we take the O time complexity to be the number of times the innermost loop is executed (and here we assume the innermost loop consists of statements of O(1) time complexity).
Consider your example. The first loop executes O(log N) times, and the second innermost loop executes O(N) times. If something O(N) is being executed O(log N) times, then yes, the final time complexity is just them multiplied: O(N log N).
Generally, this holds true with most nested loops: you can assume their big-O time complexity to be the time complexity of each loop, multiplied.
However, there are exceptions to this rule when you can consider the break statement. If the loop has the possibility of breaking out early, the time complexity will be different.
Take a look at this example I just came up with:
for(int i = 1; i <= n; ++i) {
int x = i;
while(true) {
x = x/2;
if(x == 0) break;
}
}
Well, the innermost loop is O(infinity), so can we say that the total time complexity is O(N) * O(infinity) = O(infinity)? No. In this case we know the innermost loop will always break in O(log N), giving a total O(N log N) time complexity.
What is the time comlexity of this function(f1)?
as I can see that the first loop(i=0)-> (n/4 times) the second one(i=3)->(n/4 - 3 times).... etc, the result is: (n/3)*(n/4 + (n-3)/4 + (n-6)/4 + (n-9)/4 ....
And I stop here, how to continue?
int f1(int n){
int s=0;
for(int i=0; i<n; i+=3)
for (int j=n; j>i; j-=4)
s+=j+i;
return s;
}
The important thing about Big(O) notation is that it eliminates 'constants'. The objective is to determine trend as input size grows without concern for specific numbers.
Think of it as determining the curve on a graph where you don't know the number ranges of the x and y axes.
So in your code, even though you skip most of the values in the range of n for each iteration of each loop, this is done at a constant rate. So regardless of how many you actually skip, this still scales relative to n^2.
It wouldn't matter if you calculated any of the following:
1/4 * n^2
0.0000001 * n^2
(1/4 * n)^2
(0.0000001 * n)^2
1000000 + n^2
n^2 + 10000000 * n
In Big O, these are all equivalent to O(n^2). The point being that once n gets big enough (whatever that may be), all the lower order terms and constant factors become irrelevant in the 'big picture'.
(It's worth emphasising that this is why on small inputs you should be wary of relying too heavily on Big O. That's when constant overheads can still have a big impact.)
Key observation: The inner loop executes (n-i)/4 times in step i, hencei/4 in step n-i.
Now sum all these quantities for i = 3k, 3(k-1), 3(k-2), ..., 9, 6, 3, 0, where 3k is the largest multiple of 3 before n (i.e., 3k <= n < 3(k+1)):
3k/4 + 3(k-1)/4 + ... + 6/4 + 3/4 + 0/4 = 3/4(k + (k-1) + ... + 2 + 1)
= 3/4(k(k+1))/2
= O(k^2)
= O(n^2)
because k <= n/3 <= k+1 and therefore k^2 <= n^2/9 <= (k+1)^2 <= 4k^2
In theory it's "O(n*n)", but...
What if the compiler felt like optimising it into this:
int f1(int n){
int s=0;
for(int i=0; i<n; i+=3)
s += table[i];
return s;
}
Or even this:
int f1(int n){
if(n <= 0) return 0;
return table[n];
}
Then it could also be "O(n)" or "O(1)".
Note that on the surface these kinds of optimisations seem impractical (due to worst case memory costs); but with a sufficiently advanced compiler (e.g. using "whole program optimisation" to examine all callers and determine that n is always within a certain range) it's not inconceivable. In a similar way it's not impossible for all of the callers to be using a constant (e.g. where a sufficiently advanced compiler can replace things like x = f1(123); with x = constant_calculated_at_compile_time).
In other words; in practice, the time complexity of the original function depends on how the function is used and how good/bad the compiler is.
I would like to know exactly how to compute the big O of the second while when the number of repetitions keeps going down over time.
int duplicate_check(int a[], int n)
{
int i = n;
while (i > 0)
{
i--;
int j = i - 1;
while (j >= 0)
{
if (a[i] == a[j])
{
return 1;
}
j--;
}
}
return 0;
}
Still O(n^2) regardless of the smaller repetition.
The value you are computing is Sum of (n-k) for k = 0 to n.
This equates to (n^2 + n) / 2 which since O() ignores constants and minor terms is O(n^2).
Note you can solve this problem more efficiently by sorting the array O(nlogn) and then searching for two consecutive numbers that are the same O(n) so total O(nlogn)
Big O is an estimate/theoretical speed, it's not the exact calculation.
Like twain249 said, regardless, the time complexity is O(n^2)
BigO shows the worst case time complexity of an algorithm that means the maximum time an algorithm can take ever.It shows upper bound which indicates that whatever the input is time complexity will always be under that bound.
In your case the worst case will when i will iterate until 0 then complexity will be like:
for i=n j will run n-1 times for i=n-1 j will run n-2 times and so on.
adding all (n-1)+(n-2)+(n-3)+............(n-n)=(n-1)*(n)/2=n^2/2-n/2
after ignoring lower term that is n and constant that is 1/2 it becomes n^2.
So O(n^2) that's how it is computed.
I had to determinate big O complexity of this piece of code.
I thought the answer is nlogn but apparently its n. Can anyone help explain why that is so?
void funct(int n)
{
for (int i = n; i > 0; i /= 2)
for(int j = 0; j < i; j++)
printf("%d\n", j%2);
}
That's geometric progression
The first time the inner loop is executed n times.
The second time it is executed n/2 times.
etc...
So we have the sequence:
n + n/2 + n/4 + ... + 1
so the final formula is:
n*(1 - (1/2)^(log n))/(1/2)
which is equivalent to n
Look these can be solved using Double Sigma's :
Let $ represents sigma.
so this problem is :
$(i=n downto 0 by a factor of 2 )$(j=0 to i-1) 1
where 1 represent a unit cost
now for inner sigma its sum of 1 i times that is = i
now problem is
$(i=n downto 1 by a factor of 2 ) i
which is sum of i values i.e. n+n/2+n/4+...+1(when n/2^x=1 or after log(n) terms)+0
or
n*(1+1/2+.....log(n) terms)
which is a convergent Geometric progression. and the result will be n*(1 - (1/2)^(log n))/(1/2) i.e O(n)
The outer loop, as I'm sure you can see is executed log(n) times. The inner loop is executed on average log(n)/2 times. So the printf statement is executed log(n) * (log(n) / 2) times which equals n / 2. So the complexity of the code is O(n).
So I had to insert N elements in random order into a size-N array, but I am not sure about the time complexity of the program
the program is basically:
for (i = 0 -> n-1){
index = random (0, n); (n is exclusive)
while (array[index] != null)
index = random (0, n);
array[index] = n
}
Here is my assumption: a normal insertion of N numbers is of course strictly N, but how much cost will the collision from random positions cost? For each n, its collision rate increases like 0, 1/n, 2/n .... n-1/n, so expected number of insertions attempts will be 1, 2, 3 .. n-1, this is O(n), so total time complexity will be O(n^2), so is this the average cost? but wow this is really bad, am I right?
So what will happen if I do a linear search instead of keep trying to generate random numbers? Its worst case will obviously be O(n^2>, but I don't know how to analyze its average case, which depends on average input distribution?
First consider the inner loop. When do we expect to have our first success (find an open position) when there are i values already in the array? For this we use the geometric distribution:
Pr(X = k) = (1-p)^{k-1} p
Where p is the probability of success for an attempt.
Here p is the probability that the array index is not already filled.
There are i filled positions so p = (1 - (i/n)) = ((n - i)/n).
From the wiki, the expectation for the geometric distribution is 1/p = 1 / ((n-i)/n) = n/(n-i).
Therefore, we should expect to make (n / (n - i)) attempts in the inner loop when there are i items in the array.
To fill the array, we insert a new value when the array has i=0..n-1 items in it. The amount of attempts we expect to make overall is the sum:
sum_{i=0,n-1} n/(n-i)
= n * sum_{i=0,n-1}(1/(n-i))
= n * sum_{i=0,n-1}(1/(n-i))
= n * (1/n + 1/(n-1) + ... + 1/1)
= n * (1/1 + ... + 1/(n-1) + 1/n)
= n * sum_{i=1,n}(1/i)
Which is n times the nth harmonic number and is approximately ln(n) + gamma, where gamma is a constant. So overall, the number of attempts is approximately n * (ln(n) + gamma), which is O(nlog n). Remember that this is only the expectation and there is no true upper bound since the inner loop is random; it may never find an open spot.
The expected number of insertions attempt at step i is
sum_{t=0}^infinity (1-i/n)^t * (n-i)/n * t
= (n-i)/n * i/n * (1-i/n)^{-2}
= i/(n-i)
Summing over i you get
sum_{i=0}^{n-1} i/(n-1)
>= sum_{i=n/2}^n i / (n-i)
>= n/2 sum_{x=1}^n/2 1/x
>= n/2 * log(n) + O(n)
And
sum_{i=0}^{n-1} i/(n-i)
<= n * sum _{x=1}^n 1/x
<= n * log(n) + O(n)
So you get exactly n*log(n) as an asymptotic complexity. Which is not as bad as you feared.
About doing a linear search, I don't know how you would do it while keeping the array random. If you really want an efficient algorithm to shuffle your array, you should check out Fisher-Yates shuffle.