I'm trying to obtain the complexity of a particular divide and conquer algorithm so transpose a given matrix.
From what I've been reading, I got that the recursion should start as follows:
C(1) = 1
C(n) = 4C(n/2) + O(n)
I know how to solve the recursion but I'm not sure if it's right. Everytime the function is called, the problem is divided by 2 (vars fIni and fEnd), and then another 4 functions are called. Also, at the end, swap is called with a complexity of O(n²) so I'm pretty sure I'm not taking that into account in the above recursion.
The code, as follows:
void transposeDyC(int **m,int f,int c, int fIni, int fEnd, int cIni, int cEnd){
if(fIni < fEnd){
int fMed = (fIni+fFin)/2;
int cMed = (cIni+cFin)/2;
transposeDyC(m,f,c, fIni, fMed, cIni, cMed);
transposeDyC(m,f,c, fIni, fMed, cMed+1, cEnd);
transposeDyC(m,f,c, fMed+1, fFin, cIni, cMed);
transposeDyC(m,f,c, fMed+1, fFin, cMed+1, cEnd);
swap(m,f,c, fMed+1, cIni, fIni, cMed+1, fEnd-fMed);
}
}
void swap (int **m,int f, int c,int fIniA, int cIniA, int fIniB, int cIniB, int dimen){
for (int i=0; i<=dimen-1; i++){
for (int j=0; j<=dimen-1; j++) {
int aux = m[fIniA+i][cIniA+j];
m[fIniA+i][cIniA+j] = m[fIniB+i][cIniB+j];
m[fIniB+i][cIniB+j] = aux;
}
}
}
I'm really stuck in this complexity with recursion and divide and conquer. I don't know how to continue.
You got the recursion wrong. It is 4C(n/2) + O(n2), because when joining the matrix back, for a size n, there are total n2 elements.
Two ways:
Master Theorem
Here we have a = 4, b = 2, c = 2, Logba = 2
Since, Logba == c, this falls under the case 2, resulting in the complexity of O(ncLog n) = O(n2 Log n).
Recurrence tree visualization
If you'd try to unfold your recurrence, you can see that you are solving the problem of size n by breaking it down into 4 problems of size n/2 and then doing a work of size n2 (at each level).
Total work done at each level = 4 * Work (n/2) + n2
Total number of levels will be equal to the number of times you'd have to divide the n sized problem until you come to a problem of size 1. That will be simply equal to Log2n.
Therefore, total work = Log(n) (4*(n / 2) + n2), which is O(n2 Log n).
Each recursive step reduces the number of elements by a factor of 4, so the number of levels of recursion will be on the order O(log n). At each level, the swap has order O(n^2), so the algorithm has complexity O((n^2)(log n)).
Related
When looking at this code for example :
for (int i = 1; i < n; i*=2)
for (int j = 0; j < i; j +=2)
{
// some contstant time operations
}
Is it as simple as saying that because the outer loop is log and and inner loop is n , that combined the result is big(O) of nlogn ?
Here is the analysis of the example in the question. For simplicity I will neglect the increment of 2 in the inner loop and will consider it as 1, because in terms of complexity it does not matter - the inner loop is linear in i and the constant factor of 2 does not matter.
So we can notice, that the outer loop is producing is of values which are powers of 2 capped by n, that is:
1, 2, 4, 8, ... , 2^(log2 n)
these numbers are also the numbers that the "constant time operation" in the inner loop is running for each i.
So all we have to do is to sum up the above series. It is easy to see that these are geometric series:
2^0 + 2^1 + 2^2 + ... + 2^(log2 n)
and it has a well known solution:
(from Wiki )
We have a=1, r=2, and... well n_from_the_image =log n. We have a same name for different variables here, so it is a bit of a problem.
Now let's substitute and get that the sum equals
(1-2^((log2 n) + 1) / (1 - 2) = (1 - 2*n) / (1-2) = 2n-1
Which is a linear O(n) complexity.
Generally, we take the O time complexity to be the number of times the innermost loop is executed (and here we assume the innermost loop consists of statements of O(1) time complexity).
Consider your example. The first loop executes O(log N) times, and the second innermost loop executes O(N) times. If something O(N) is being executed O(log N) times, then yes, the final time complexity is just them multiplied: O(N log N).
Generally, this holds true with most nested loops: you can assume their big-O time complexity to be the time complexity of each loop, multiplied.
However, there are exceptions to this rule when you can consider the break statement. If the loop has the possibility of breaking out early, the time complexity will be different.
Take a look at this example I just came up with:
for(int i = 1; i <= n; ++i) {
int x = i;
while(true) {
x = x/2;
if(x == 0) break;
}
}
Well, the innermost loop is O(infinity), so can we say that the total time complexity is O(N) * O(infinity) = O(infinity)? No. In this case we know the innermost loop will always break in O(log N), giving a total O(N log N) time complexity.
In a book about data structure and algorithms, there is the following implementation of the insertion sort:
int insertionSort(void *data, int size, int esize, int(*compare)(const void *key1, const void *key2)){
int i,j;
void *key,
char *a = data;
key = (char *) malloc(esize);
if(key == NULL){
return -1;
}
for(j=1; j<size; j++){
memcpy(key, &a[j*esize], esize);
i = j-1;
while(i>=0 && compare(&a[i*esize], key)>0){
memcpy(&a[(i+1)*esize],&a[i*esize], esize);
i--;
}
memcpy(&a[(i+1)*esize], key, esize);
}
}
So, in the section about the complexity it states the following:
The runtime complexity of insertion sort focuses on its nested loops.
With this in mind, the outer loop has a running time of T(n) = n – 1,
times some constant amount of time, where n is the number of elements
being sorted. Examining the inner loop in the worst case, we assume
that we will have to go all the way to the left end of the array
before inserting each element into the sorted set. Therefore, the
inner loop could iterate once for the first element, twice for the
second, and so forth until the outer loop terminates. The running time
of the nested loop is repre- sented as a summation from 1 to n – 1,
which results in a running time of T (n) = (n (n + 1)/2) – n, times
some constant amount of time. (This is from the well- known formula
for summing a series from 1 to n.) Using the rules of O-notation, this simplifies to O(n^2)
So, I googled the summation formula and understand why it is (n(n+1)/2). There are also a bunch of youtube videos about this formula which I also watched.
But I could not understand the last part of T(n) in the book which is the "-n" part. Why minus n at the end ?
For instance: I have an unsorted list A of 10 elements. I need the sublist of k consecutive elements from i through i+k-1 of the sorted version of A.
Example:
Input: A { 1, 6, 13, 2, 8, 0, 100, 3, -4, 10 }
k = 3
i = 4
Output: sublist B { 2, 3, 6 }
If i and k are specified, you can use a specialized version of quicksort where you stop recursion on parts of the array that fall outside of the i .. i+k range. If the array can be modified, perform this partial sort in place, if the array cannot be modified, you will need to make a copy.
Here is an example:
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
// Partial Quick Sort using Hoare's original partition scheme
void partial_quick_sort(int *a, int lo, int hi, int c, int d) {
if (lo < d && hi > c && hi - lo > 1) {
int x, pivot = a[lo];
int i = lo - 1;
int j = hi;
for (;;) {
while (a[++i] < pivot)
continue;
while (a[--j] > pivot)
continue;
if (i >= j)
break;
x = a[i];
a[i] = a[j];
a[j] = x;
}
partial_quick_sort(a, lo, j + 1, c, d);
partial_quick_sort(a, j + 1, hi, c, d);
}
}
void print_array(const char *msg, int a[], int count) {
printf("%s: ", msg);
for (int i = 0; i < count; i++) {
printf("%d%c", a[i], " \n"[i == count - 1]);
}
}
int int_cmp(const void *p1, const void *p2) {
int i1 = *(const int *)p1;
int i2 = *(const int *)p2;
return (i1 > i2) - (i1 < i2);
}
#define MAX 1000000
int main(void) {
int *a = malloc(MAX * sizeof(*a));
clock_t t;
int i, k;
srand((unsigned int)time(NULL));
for (i = 0; i < MAX; i++) {
a[i] = rand();
}
i = 20;
k = 10;
printf("extracting %d elements at %d from %d total elements\n",
k, i, MAX);
t = clock();
partial_quick_sort(a, 0, MAX, i, i + k);
t = clock() - t;
print_array("partial qsort", a + i, k);
printf("elapsed time: %.3fms\n", t * 1000.0 / CLOCKS_PER_SEC);
t = clock();
qsort(a, MAX, sizeof *a, int_cmp);
t = clock() - t;
print_array("complete qsort", a + i, k);
printf("elapsed time: %.3fms\n", t * 1000.0 / CLOCKS_PER_SEC);
return 0;
}
Running this program with an array of 1 million random integers, extracting the 10 entries of the sorted array starting at offset 20 gives this output:
extracting 10 elements at 20 from 1000000 total elements
partial qsort: 33269 38347 39390 45413 49479 50180 54389 55880 55927 62158
elapsed time: 3.408ms
complete qsort: 33269 38347 39390 45413 49479 50180 54389 55880 55927 62158
elapsed time: 149.101ms
It is indeed much faster (20x to 50x) than sorting the whole array, even with a simplistic choice of pivot. Try multiple runs and see how the timings change.
An idea could be to scan your array for bigger or equal numbers of i and smaller or equal numbers of i+k and add them to another list/container.
This will take you O(n) and give an unordered list of the numbers you need. Then you sort that list O(nlogn) and you are done.
For really big arrays the advantage of this method is that you will sort a smaller list of numbers. (given that the k is relatively small).
You can use Quickselect, or a heap selection algorithm to get the i+k smallest items. Quickselect works in-place, but it modifies the original array. It also won't work if the list of items is larger than will fit in memory. Quickselect is O(n), but with a fairly high constant. When the number of items you are selecting is a very small fraction of the total number of items, the heap selection algorithm is faster.
The idea behind the heap selection algorithm is that you initialize a max-heap with the first i+k items. Then, iterate through the rest of the items. If an item is smaller than the largest item on the max-heap, remove the largest item from the max-heap and replace it with the new, smaller item. When you're done, you have the first i+k items on the heap, with the largest k items at the top.
The code is pretty simple:
heap = new max_heap();
Add first `i+k` items from a[] to heap
for all remaining items in a[]
if item < heap.peek()
heap.pop()
heap.push(item)
end-if
end-for
// at this point the smallest i+k items are on the heap
This requires O(i+k) extra memory, and worst case running time is O(n log(i+k)). When (i+k) is less than about 2% of n, it will usually outperform Quickselect.
For much more information about this, see my blog post When theory meets practice.
By the way, you can optimize your memory usage somewhat based on i. That is, if there are a billion items in the array and you want items 999,999,000 through 999,999,910, the standard method above would require a huge heap. But you can re-cast that problem to one in which you need to select the smallest of the last 1,000 items. Your heap then becomes a min-heap of 1,000 items. It just takes a little math to determine which way will require the smallest heap.
That doesn't help much, of course, if you want items 600,000,000 through 600,000,010, because your heap still has 400 million items in it.
It occurs to me, though, that if time isn't a huge issue, you can just build the heap in the array in-place using Floyd's algorithm, pop the first i items like you would with heap sort, and the next k items are what you're looking for. This would require constant extra space and O(n + (i+k)*log(n)) time.
Come to think of it, you could implement the heap selection logic with a heap of (i+k) items (as described above) in-place, as well. It would be a little tricky to implement, but it wouldn't require any extra space and would have the same running time O(n*log(i+k)).
Note that both would modify the original array.
One thing you could do is modify heapsort, such that you will first create the heap, but then pop the first i elements. The next k elements you pop form the heap will be your result. Discarding the n - i - k elements remaining let's the algorithm terminate early.
The result will be in O((i + k) log n) which is in O(n log n), but is significantly faster with relative low values for i and k.
Is O(log(log(n))) actually just O(log(n)) when it comes to time complexity?
Do you agree that this function g() has a time complexity of O(log(log(n)))?
int f(int n) {
if (n <= 1)
return 0;
return f(n/2) + 1;
}
int g(int n) {
int m = f(f(n));
int i;
int x = 0;
for (i = 0; i < m; i++) {
x += i * i;
}
return m;
}
function f(n) computes the logarithm in base 2 of n by repeatedly dividing by 2. It iterates log2(n) times.
Calling it on its own result will indeed return log2(log2(n)) for an additional log2(log2(n)) iterations.
So far the complexity is O(log(N)) + O(log(log(N)). The first term dominates the second, overall complexity is O(log(N)).
The final loop iterates log2(log2(n)) times, time complexity of this last phase is O(log(log(N)), negligible in front of the initial phase.
Note that since x is not used before the end of function g, computing it is not needed and the compiler may well optimize this loop to nothing.
Overall time complexity comes out as O(log(N)), which is not the same as O(log(log(N)).
Looks like it is log(n) + log(log n) + log(log n).
In order: the first recursion of f(), plus the second recursion of f(), and the for loop, so the final complexity is O(log n), because lower order terms are ignored.
int f(int n) {
if (n<=1)
return 0;
return f(n/2) + 1;
}
Has Time Complexity of Order O(log2(n)). Here 2 is base of logrithm.
int g(int n) {
int m = f(f(n)); // O(log2(log2(n))
int i, x=0;
for( i = 0; i < m; i++) {
x += i*i;
}
// This for loop will take O(log2(log2(n))
return m;
}
Hence overall time complexity of given function is :
T(n) = t1 + t2 + t3
But here O(log2(n)) dominates over O(log2(log2(n)).
Hence time complexity of given function is log2(n).
Please read What is a plain English explanation of "Big O" notation? once.
The time consumed by O(log n) algorithms depends only linearly on the number of digits of n. So it is very easy to scale it.
Say you want to compute F(100000000), the 10^8th F....ascci number. For a O(log n) algorithm it is only going to take 4x the time consumed by computing F(100).
O(log log n) terms can show up in a variety of different places, but there are typically two main routes that will arrive at this runtime. Reference link enter code here here.
#include<stdio.h>
#include<math.h>
void printboard(int n);
void fourQueen(int k,int n);
int place(int k,int i);
int x[100];
void NQueen(int k,int n)
{
int i;
for(i=1;i<=n;i++)
{
if(place(k,i)==1)
{ x[k]=i;
if(k==n)
{
printf("Solution\n");
printboard(n);
}
else
NQueen(k+1,n);
}
}
}
int place(int k,int i)
{
int j;
for(j=1;j<k;j++)
{
if((x[j]==i)||abs(x[j]-i)==abs(j-k))
return 0;
}
return 1;
}
void printboard(int n)
{
int i;
for(i=1;i<=n;i++)
printf("%d ",x[i]);
}
void main()
{
int n;
printf("Enter Value of N:");
scanf("%d",&n);
NQueen(1,n);
}
I think it has time complexity: O(n^n), As NQueen function is recursively calling, but is there is any tighter bound possible for this program? what about best case, and worst case time complexity. I am also confused about the place() function which is O(k) and calling from NQueen().
There are a lot of optimizations than can improve the time complexity of the algorithm.
There is more information in these links:
https://sites.google.com/site/nqueensolver/home/algorithm-results
https://sites.google.com/site/nqueensolver/home/algorithms/2backtracking-algorithm
For Your function T(n) = n*T(n-1) + O(n^2) which translates to O(N!) time complexity approximately.
TIME COMPLEXITY OF N-QUEEN PROBLEM IS
> O(N!)
Explanation:
If we add all this up and define the run time as T(N). Then T(N) = O(N2) + N*T(N-1). If you draw a recursion tree using this recurrence, the final term will be something like n3+ n!O(1). By the definition of Big O, this can be reduced to O(n!) running time.
O(n^n) is definitely an upper bound on solving n-queens using backtracking.
I'm assuming that you are solving this by assigning a queen column-wise.
However, consider this - when you assign a location of the queen in the first column, you have n options, after that, you only have n-1 options as you can't place the queen in the same row as the first queen, then n-2 and so on. Thus, the worst-case complexity is still upper bounded by O(n!).
Hope this answers your question even though I'm almost 4 years late!
Let us consider that our queen is a rook, meaning we need not take care of diagonal conflicts.
Time complexity in this case will be O(N!) in the worst case, supposed if we were on a hunt to check if any solution exists or not. Here is a simple explanation.
Let us take an example where N=4.
Supposed we are want to fill the 2-D matrix. X represents a vacant position while 1 represents a taken position.
In the starting, the answer matrix (which we need to fill) looks like,
X X X X
X X X X
X X X X
X X X X
Let us fill this row-wise, meaning will select one location in each row then move forward to the next row.
For the first row, since none is filled in the matrix as a whole, we have 4 options.
For the second row, we have 3 options as one row has already been taken off.
Similarly, for the third row, we have 2 options and for the final row, we are left with just 1 option.
Total options = 4*3*2*1 = 24 (4!)
Now, this was the case had our queen were a rook but since we have more constraints in case of a queen. Complexity should be less than O(N!) in terms of the actual number of operations.
The complexity is n^n and here is the explanation
Here n represent the number of of queens and will remain same for every function call.
K is the row number and function will be called times till k reaches the n.There if n=8,we have n rows and n queens.
T(n)=n(n+t(max of k - 1))=n^max of k=n^n as the max of k is n.
Note:The function has two parameters.In loop, n is not decreasing ,For every function call it remains same.But for number of times the function is called it is decreasing so that recursion could terminate.
The complexity is (n+1)!n^n begin with T(i)=O(niT(i+1)), T(n)=n^3.
So, T(1)=nT(2)=2n^2T(3)=...=(n-1)n^(n-1)!T(n)=(n+1)