Big O - Why is this algorithm O(AxB)? - loops

I am unsure why this code evaluates to O(A*B)?
void printUnorderedPairs(int[] arrayA, int[] arrayB) {
for (int i= 0; i < arrayA.length; i++) {
for (int j = 0; j < arrayB.length; j++) {
for (int k= 0; k < 100000; k++) {
System.out.println(arrayA[i] + "," + arrayB[j]);
}
}
}
}
Sure, more precisely its O(1000*AB) and we would drop the 1000 making it O(AB). But what if array A had a length of 2? wouldn't the 1000 iterations be more significant? Is it just because we know the final loop is constant (and its value is shown) that we don't count it? what if we knew all of the arrays sizes?
Can anyone explain why we would not say its O(ABC)? What would be the runtime if I made the code this:
int[] arrayA = new int[20];
int[] arrayB = new int[500];
int[] arrayC = new int[100000];
void printUnorderedPairs(int[] arrayA, int[] arrayB) {
for (int i= 0; i < arrayA.length; i++) {
for (int j = 0; j < arrayB.length; j++) {
for (int k= 0; k < arrayC.length; k++) {
System.out.println(arrayA[i] + "," + arrayB[j]);
}
}
}
}

If the running time (or number of execution steps, or number of times println gets called, or whatever you are assessing with your Big O notation) is O(AB), it means that the running time approaches being linearly proportional to AB as AB grows without bound (approaches infinity). It is literally a limit to infinity, in terms of calculus.
Big O is not concerned with what happens for any finite number of iterations. It's about what the limiting behaviour of the function is as its free variables approach infinity. Sure, for small values of A there could very well be a constant term that dominates execution time. But as A approaches infinity, all those other factors becomes insignificant.
Consider a polynomial like Ax^3 + Bx^2 + Cn + D. It will be proportional to x^3 as x grows to infinity - regardless of the magnitude of A, B, C, or D. B can be Grahams number for all Big O cares; infinity is still way bigger than any big finite number you pick and therefore the x^3 term dominates.
So first, considering what if A were 2 is not really in the spirit of AB approaching infinity. Any number you can fit on a whiteboard basically rounds down to zero..
And second, remember that proportional to AB means equal to AB times some constant; and it doesn't matter what that constant is. It is fine if the constant happens to be 10000. Saying something is proportional to 2N is the same as saying it is proportional to N, or any other number times N. So O(2N) is the same as O(N). By convention we always simplify when using Big-O notation to drop any constant factors. So we would always write O(N), and never O(2N). And for that same reason, we would write O(AB) and not O(10000AB).
And finally we don't say O(ABC) only because "C" (the number of iterations of your inner loop in your question) happens to be a constant; which also happens to equal 10000. That's why we say it's O(AB) and not O(ABC) because C is not a free variable; it's hard-coded to 10000. If the size of B were not expected to change (were to be constant for whatever reason) then you could say that it is simply O(A). But if you allow B to grow without bound, then the limit is O(AB) and if you also allow C to grow without bound then the limit is O(ABC). You get to decide which numbers are constant and which variables are free variables depending on the context of your analysis.
You can read more about Big O notation at Wikipedia.

Appreciate that the for loops in i and j are independent of each other, so their running time is O(A*B). The inner loop in k is a fixed number of iterations, 100000, and also is independent of the two outer loops, so we get O(100000*A*B). But, since the k loop is just a constant (non variable) penalty, with are still left with O(A*B) for the overall complexity.
If you were to write the inner loop in k from 0 to C, then you could write O(A*B*C) for the complexity, and that would be valid as well.

Generally the A*B doesn't matter, and it's just considered O(N).
If there was some knowledge that A and B were always somewhat the same length, then one could argue that it's really O(N^2).
Any sort of constant doesn't really matter in order-notation, because for really really large numbers of A/B, the constant becomes of negligible importance.

void printUnorderedPairs(int[] arrayA, int[] arrayB) {
for (int i= 0; i < arrayA.length; i++) {
for (int j = 0; j < arrayB.length; j++) {
for (int k= 0; k < 100000; k++) {
System.out.println(arrayA[i] + "," + arrayB[j]);
}
}
}
}
This code is evaluated to O(AB), because arrayC has constant length. Of course, its run time is proportional to AB*100000. Here, we never care about constant values, because when the variables get higher and higher like 10^10000, the constants can be easily ignored.
In the second code, we say its O(1), because all arrays have constant length and we can calculate its run time without any variable.

Related

How is this loop's time complexity O(n^2)?

How is this loop's time complexity O(n^2)?
for (int i = n; i > 0; i -= c)
{
for (int j = i+1; j <=n; j += c)
{
// some O(1) expressions
}
}
Can anyone explain?
Assumption
n > 0
c > 0
First loop
The first loop start with i=n and at each step, it substracts c from i. On one hand, if c is big, then the first loop will be iterated only a few times. (Try with n=50, c=20, you will see). On the other hand, if c is small (let say c=1), then it will iterate n times.
Second loop
The second loop is the same reasoning. If c is big, then it will be iterated only a few times, if c is small, many times and at the worst case n times.
Combined / Big O
Big O notation gives you the upper bound for time complexity of an algorithm. In your case, first and second loop upper bound combined, it gives you a O(n*n)=O(n^2).

What is the time complexity \big(O) of this specific function?

What is the time comlexity of this function(f1)?
as I can see that the first loop(i=0)-> (n/4 times) the second one(i=3)->(n/4 - 3 times).... etc, the result is: (n/3)*(n/4 + (n-3)/4 + (n-6)/4 + (n-9)/4 ....
And I stop here, how to continue?
int f1(int n){
int s=0;
for(int i=0; i<n; i+=3)
for (int j=n; j>i; j-=4)
s+=j+i;
return s;
}
The important thing about Big(O) notation is that it eliminates 'constants'. The objective is to determine trend as input size grows without concern for specific numbers.
Think of it as determining the curve on a graph where you don't know the number ranges of the x and y axes.
So in your code, even though you skip most of the values in the range of n for each iteration of each loop, this is done at a constant rate. So regardless of how many you actually skip, this still scales relative to n^2.
It wouldn't matter if you calculated any of the following:
1/4 * n^2
0.0000001 * n^2
(1/4 * n)^2
(0.0000001 * n)^2
1000000 + n^2
n^2 + 10000000 * n
In Big O, these are all equivalent to O(n^2). The point being that once n gets big enough (whatever that may be), all the lower order terms and constant factors become irrelevant in the 'big picture'.
(It's worth emphasising that this is why on small inputs you should be wary of relying too heavily on Big O. That's when constant overheads can still have a big impact.)
Key observation: The inner loop executes (n-i)/4 times in step i, hencei/4 in step n-i.
Now sum all these quantities for i = 3k, 3(k-1), 3(k-2), ..., 9, 6, 3, 0, where 3k is the largest multiple of 3 before n (i.e., 3k <= n < 3(k+1)):
3k/4 + 3(k-1)/4 + ... + 6/4 + 3/4 + 0/4 = 3/4(k + (k-1) + ... + 2 + 1)
= 3/4(k(k+1))/2
= O(k^2)
= O(n^2)
because k <= n/3 <= k+1 and therefore k^2 <= n^2/9 <= (k+1)^2 <= 4k^2
In theory it's "O(n*n)", but...
What if the compiler felt like optimising it into this:
int f1(int n){
int s=0;
for(int i=0; i<n; i+=3)
s += table[i];
return s;
}
Or even this:
int f1(int n){
if(n <= 0) return 0;
return table[n];
}
Then it could also be "O(n)" or "O(1)".
Note that on the surface these kinds of optimisations seem impractical (due to worst case memory costs); but with a sufficiently advanced compiler (e.g. using "whole program optimisation" to examine all callers and determine that n is always within a certain range) it's not inconceivable. In a similar way it's not impossible for all of the callers to be using a constant (e.g. where a sufficiently advanced compiler can replace things like x = f1(123); with x = constant_calculated_at_compile_time).
In other words; in practice, the time complexity of the original function depends on how the function is used and how good/bad the compiler is.

Time Complexity of an Algorithm (Nested Loops)

I'm trying to figure out the time complexity of this pseudocode given algorithm:
sum = 0;
for (i = 1; i <= n; i++)
for (j = 1; j <= n / 6; j++)
sum = sum + 1;
I know that the first line runs
n times
But I'm not sure about the second line.
Using Sigma notation, we can find the asymptotic bounds of your algorithm as follows:
Here you have a simple double loop:
for i=1;i<=n;i++
for j=1; j<=n/6; j++
so if you count how many times the body of the loop will be executed (i.e. how many times this line of code sum = sum + 1; will be executed), you will see it's:
n*n/6 = n²/6
which in terms of big-O notation is:
O(n²)
because we do not really care for the constant term, because as n grows, the constant term makes no (big) difference if it's there or not!
When and only when you fully realize what I am saying, you can go deeper with this nice question: Big O, how do you calculate/approximate it?
However, please notice that such questions are more appropriate for the Theoretical Computer Science, rather than SO.
You make n*n/6 operations, thus, the time complexity is O(n^2/6) = O(n^2).

Confused with Big O Notation

So I get that the first for loop runs O(n) times, then inside that it runs 3 times, then 3 times again. How do I express this at big O notation though? Then do the 2 print statements matter? How do I add them to my big-o expression? Thanks, really confused and appreciate any help.
for (int x = 0; x < n; x++) {
for (int j = 0; j < 3; j++) {
for (int k = 0; k < 3; k++) {
printf("%d", arr[x]);
}
printf("\n");
}
}
O(n) is linear time, so any k * O(n) where k is a constant (like in your example) is also linear time and is just expressed as O(n). Your example has O(n) time complexity.
Big O notation are always defined as a function of input size - n. Big O gives the upper limit of total time taken to run that module. Because your inner "for" loops are always run 3*3 =9 times irrespective of the input size of n - there are still considered as constant time in Big O calculations
Time Complexity = O(n+9+constantTimeToPrint) = O(n)
The two inner loops are constant, so it's still O(n). constant factors don't matter, the runtime varies only with the input size.

Does "Find all triplets whose sum is less than some number" have any solution better than O(n3) runtime? [duplicate]

This question already has answers here:
Find all triplets in array with sum less than or equal to given sum
(5 answers)
Closed 8 years ago.
I got asked this on an interview.
Given an array of ints, find all triplets whose sum is less than some number
After some scrambling I told the interviewer that the best solution would still lead to worst-case runtime O(n3) and possibly would need O(n3).
The interviewer blatantly disagreed with me and told me "you need to go back to your algorithms...".
Am I missing something?
A possible optimization will be:
Remove all elements in the array that bigger than sum;
Sort the array;
Run O(N^2) to pick up a[i] + a[j], then binary search for sum - a[i] - a[j] in the range of [j + 1, N], the index is the number of possible candidates, but you should subtract j since they have been covered.
The complexity will be O(N^2 log N), slightly better.
You can solve this O(n^2) time:
First, sort the array.
Then, loop over the array with the first pointer i.
Now, use a second pointer j to loop up from there and a third pointer k to simultaneously loop down from the end.
Whenever you're in a situation where A[i]+A[j]+A[k] < X, you know that the same holds for all j<k'<k so increment your count with k-j and increment j. I keep the hidden invariant that A[i]+A[j]+A[k+1] >= X, so incrementing j only makes that statement stronger.
Otherwise, decrement k. When j and k meet, increment i.
You will only increment j and decrement k, so they need O(n) amortized time to meet.
In pseudocode:
count= 0
for i = 0; i < N; i++
j = i+1
k = N-1
while j < k
if A[i] + A[j] + A[k] < X
count += k-j
j++
else
k--
I see that you ask for all triplets. It is quite obvious that there can be O(n^3) triplets, so if you want them all you will need as much time, worst case.
This is an example of a problem where the output size matters. For example, if the array contains just 1, 2, 3, 4, 5, ..., n and the maximum value is set at 3n then every single triplet will be an answer, and you have to do Ω(n3) work just to list them all. On the other hand, if the maximum value had been 0, it would be nice to finish in O(n) time after confirming all the items are too large.
Basically, we want an output-sensitive algorithm with a running time that's something like O(f(n) + t) where t is the output size and n is the input size.
An O(n2 + t) algorithm would work by essentially tracking the transition points where triplets transitioned from being over the limit to under the limit. Then it would yield everything under that surface. The space is three-dimensional so the surface is two-dimensional, and you can track along it from point to point in aggregate constant time.
Here's some python code (untested!):
def findTripletsBelow(items, limit):
surfaceCoords = []
s = sorted(items)
for i in range(len(s)):
k = len(s)-1
for j in range(i, len(s))
while k >= 0 and s[i]+s[j]+s[k] > limit:
k -= 1
if k < 0: break
surfaceCoords.append((i,j,k))
results = []
for (i,j,k) in surfaceCoords:
for k2 in range(k+1):
results.append((s[i], s[j], s[k2]))
return results
O(n2) algorithm.
Sort the list.
For every element ai, this is how you calculate the number of combinations:
Binary search and find maximum aj such that j < i and ai+aj <= total.
Binary search and find maximum ak such that k < j and ai+aj+ak <= total
For this particular combination of (ai, aj), k is the number of sums that is less than or equal to total.
Now decrement j and increment k as much as possible (but ai+aj+ak <= total )
The total number of increments and decrements is less than i. So for a particular i the complexity is O(i). Therefore overall complexity is O(n2).
I am leaving out many corner conditions, but this should give you an idea.
Edit:
In the worst case there are O(n3) solutions. So outputting them explicitly would certainly require O(n3) time. There is no way around it.
But if you want to return a implicit list (i.e. a compressed list of combinations) this would still work. An example of compressed output would be (ai, aj, ak) for k in 1:p.

Resources