Related
Talking about Big O notations, if one algorithm time complexity is O(N) and other's is O(2N), which one is faster?
The definition of big O is:
O(f(n)) = { g | there exist N and c > 0 such that g(n) < c * f(n) for all n > N }
In English, O(f(n)) is the set of all functions that have an eventual growth rate less than or equal to that of f.
So O(n) = O(2n). Neither is "faster" than the other in terms of asymptotic complexity. They represent the same growth rates - namely, the "linear" growth rate.
Proof:
O(n) is a subset of O(2n): Let g be a function in O(n). Then there are N and c > 0 such that g(n) < c * n for all n > N. So g(n) < (c / 2) * 2n for all n > N. Thus g is in O(2n).
O(2n) is a subset of O(n): Let g be a function in O(2n). Then there are N and c > 0 such that g(n) < c * 2n for all n > N. So g(n) < 2c * n for all n > N. Thus g is in O(n).
Typically, when people refer to an asymptotic complexity ("big O"), they refer to the canonical forms. For example:
logarithmic: O(log n)
linear: O(n)
linearithmic: O(n log n)
quadratic: O(n2)
exponential: O(cn) for some fixed c > 1
(Here's a fuller list: Table of common time complexities)
So usually you would write O(n), not O(2n); O(n log n), not O(3 n log n + 15 n + 5 log n).
Timothy Shield's answer is absolutely correct, that O(n) and O(2n) refer to the same set of functions, and so one is not "faster" than the other. It's important to note, though, that faster isn't a great term to apply here.
Wikipedia's article on "Big O notation" uses the term "slower-growing" where you might have used "faster", which is better practice. These algorithms are defined by how they grow as n increases.
One could easily imagine a O(n^2) function that is faster than O(n) in practice, particularly when n is small or if the O(n) function requires a complex transformation. The notation indicates that for twice as much input, one can expect the O(n^2) function to take roughly 4 times as long as it had before, where the O(n) function would take roughly twice as long as it had before.
It depends on the constants hidden by the asymptotic notation. For example, an algorithm that takes 3n + 5 steps is in the class O(n). So is an algorithm that takes 2 + n/1000 steps. But 2n is less than 3n + 5 and more than 2 + n/1000...
It's a bit like asking if 5 is less than some unspecified number between 1 and 10. It depends on the unspecified number. Just knowing that an algorithm runs in O(n) steps is not enough information to decide if an algorithm that takes 2n steps will complete faster or not.
Actually, it's even worse than that: you're asking if some unspecified number between 1 and 10 is larger than some other unspecified number between 1 and 10. The sets you pick from being the same doesn't mean the numbers you happen to pick will be equal! O(n) and O(2n) are sets of algorithms, and because the definition of Big-O cancels out multiplicative factors they are the same set. Individual members of the sets may be faster or slower than other members, but the sets are the same.
Theoretically O(N) and O(2N) are the same.
But practically, O(N) will definitely have a shorter running time, but not significant. When N is large enough, the running time of both will be identical.
O(N) and O(2N) will show significant difference in growth for small numbers of N, But as N value increases O(N) will dominate the growth and coefficient 2 becomes insignificant. So we can say algorithm complexity as O(N).
Example:
Let's take this function
T(n) = 3n^2 + 8n + 2089
For n= 1 or 2, the constant 2089 seems to be the dominant part of function but for larger values of n, we can ignore the constants and 8n and can just concentrate on 3n^2 as it will contribute more to the growth, If the n value still increases the coefficient 3 also seems insignificant and we can say complexity is O(n^2).
For detailed explanation refer here
O(n) is faster however you need to understand that when we talk about Big O, we are measuring the complexity of a function/algorithm, not its speed. And we measure this complexity asymptotically. In lay man terms, when we talk about asymptotic analysis, we take immensely huge values for n. So if you plot the graph for O(n) and O(2n), the values will stay in some particular range from each other for any value of n. They are much closer compared to the other canonical forms like O(nlogn) or O(1), so by convention we approximate the complexity to the canonical form O(n).
I've been trying to implement SVD in C for the past few weeks now, and currently I've been using the algorithm 6 found here, and from my understanding this algorithm will run in time O(n^5) because there are two loops (One of the loops does not go from 0 to n, I know but n^5 works as a crude bound), and inside the inner loop matrix multiplication has to be done which is an n^3 process.
However, according this website, for an n by n matrix, SVD can be calculated in O(2n^3). Does anyone know where I can find an algorithm for that time complexity?
In the event anyone is looking for an answer to this in the future, the algorithm to calculate SVD in O(n^3), if the matrix is a square matrix, is the method of Jacobi Rotations.
For more information on the specific algorithm you can look at Algorithm 7 on this website.
The notation on the website is a little confusing, because of the typos, but in the step when determining the values of d1, d2, c, and č (sorry that's the closest I could get to c with the hat on top), what they mean is that c = cos(theta), s = sin(theta), č = cos(phi) and š = sin(phi).
You can calculate these values of theta and phi by elimination and substitution or you can check out this StackExchange post to see how to calculate them.
After that it is just following that algorithm.
Computing the SVD of an m × n matrix has complexity O(mn min(n, m)). Since this is super-linear in the size of the data, it becomes computationally expensive for large data sets. However, if we have a low rank matrix, we would need only k basis vectors, where k << m, n. One way of computing the rank k approximation is to compute the SVD of the full matrix and retain only the k largest singular values and vectors.
https://arxiv.org/pdf/1710.02812.pdf
Consider a directed bipartite graph where both vertex sets A and B have m weighted vertices. Edges go only from A to B, and All vertices in A have the same degree which we denote n. The vertex weights are upper bounded by their degree.
As an example, consider m = 4 and n = 2, so we have A and B with 4 vertices each, and two edges from each vertex of A going to B. All weights on vertices in A are upper bounded by 2.
I am interested to loop over all possible edge flows from A to B, and in particular the resulting weights of vertices in B. I want to do this as efficiently as possible in C, and in particular with little memory, as I will be using this as a sub-routine in a depth-first-search.
I really hope for your clever inputs :)
Edit: All edges have a capacity of 1
The question has already been answered, but the main problem I am facing is in understanding one of the answers..
From
https://stackoverflow.com/a/1621913/2673063
How is the following algorithm O(n) ?
It states as
By first sorting the points / computing the convex hull (in O(n log n) time) if necessary, we can assume we have the convex polygon/hull with the points cyclically sorted in the order they appear in the polygon. Call the points 1, 2, 3, … , n. Let (variable) points A, B, and C, start as 1, 2, and 3 respectively (in the cyclic order). We will move A, B, C until ABC is the maximum-area triangle. (The idea is similar to the rotating calipers method, as used when computing the diameter (farthest pair).)
With A and B fixed, advance C (e.g. initially, with A=1, B=2, C is advanced through C=3, C=4, …) as long as the area of the triangle increases, i.e., as long as Area(A,B,C) ≤ Area(A,B,C+1). This point C will be the one that maximizes Area(ABC) for those fixed A and B. (In other words, the function Area(ABC) is unimodal as a function of C.)
Next, advance B (without changing A and C) if that increases the area. If so, again advance C as above. Then advance B again if possible, etc. This will give the maximum area triangle with A as one of the vertices. (The part up to here should be easy to prove, and simply doing this separately for each A would give O(n2). But read on.) Now advance A again, if it improves the area, etc.
Although this has three "nested" loops, note that B and C always advance "forward", and they advance at most 2n times in total (similarly A advances at most n times), so the whole thing runs in O(n) time.
As the author of the answer that is the subject of the question, I feel obliged to give a more detailed explanation of the O(n) runtime.
Firstly, just as an example, here is a figure from the paper, showing the first few steps of the algorithm, for a particular sample input (a 12-gon). First we start with A, B, C as three consecutive vertices (step 1 in the figure), advance C as long as area increases (steps 2 to 6), then advance B, and so on.
The triangles with asterisks above them are the "anchored local maxima", i.e., the ones that are best for a given A (i.e., advancing either C or B would decrease the area).
As far as the runtime being O(n): Let the "actual" value of B, in terms of the number of times it's been incremented and ignoring the wrap around, be nB, and similarly for C be nC. (In other words, B = nB % n and C = nC % n.) Now, note that,
("B is ahead of A") whatever the value of A, we have A ≤ nB < A + n
nB is always increasing
So, as A varies from 0 to n, we know that nB only varies between 0 and 2n: it can be incremented at most 2n times. Similarly nC. This shows that the running time of the algorithm, which is proportional to the total number of times A, B and C are incremented, is bounded by O(n) + O(2n) + O(2n), which is O(n).
Think about it like this: each of A, B, C are pointers that, at any given moment, point towards one of the elements of the convex hull. Due to the way the algorithm increments them, each one of them will point to each element of the convex hull at most once. Therefore, each one will iterate over a collection of O(n) elements. They will never be reset, once one of them has passed an element, it will not pass that element ever again.
Since there are 3 pointers (A, B, C), we have time complexity 3 * O(n) = O(n).
Edit:
As the code is presented in the provided link, it sounds possible that it is not O(n), since B and C wrap around the array. However, according to the description, this wrapping around does not sound necessary: before seeing the code, I imagined the method stopping the advancement of B and C past n. In that case, it would definitely be O(n). As the code is presented however, I'm not sure.
It might still be that, for some mathematical reason, B and C still iterate only O(n) times in the entirety of the algorithm, but I can't prove that. Neither can I prove that it is correct to not wrap around (as long as you take care of index out of bounds errors).
I have a connected directed weighted graph. The edge weights represent probabilities of moving between vertices; weights for all edges emanating from a vertex sum up to one. The graph contains two sinks: A and B. For each vertex in the graph, I want to know the probability that a walk originating there will reach A and the same for B. What kind of problem is this? How do I solve it?
This problem is of the algebra kind. For a path starting at a vertex, the probability of reaching A is the average of probabilities of reaching A from each of its neighbouring vertices, weighted by the edge weights. Let's put this into more concrete terms.
Let P be the adjacency matrix for the graph. That is, Pi,j is the probability of moving from vertex i to vertex j. Set PA,A = 1. If we take a vector of probabilities assigned to each vertex and multiply it by P, then the resulting vector contains a weighted average of each vertex's neighbours. What we are looking for is a vector v, such that P v = v and vA = 1.
This vector v is the eigenvector of P corresponding to the eigenvalue of 1. Does P always have such an eigenvalue? Fortunately, the Perron-Frobenius theorem tells us that it does, and that this is the largest eigenvalue of P. The solution is then to form the adjacency matrix P and find the eigenvector corresponding to its largest eigenvalue.
There is also an approximate solution. If we take a vector x of vertex probabilities, with xA = 1, and the other elements set to 0, then Pk x will converge to v as k goes to infinity. Pk might be easier to compute for small values of k than the eigenvector.
Example
Let's look at the following simple graph:
If we order the vertices alphabetically, then the matrix P corresponding to the graph is:
This matrix has an eigenvalue equal to 1, and the corresponding eigenvector is: [1 0 70/79 49/79]. That is, the exact probability of reaching A from C is 70/79, and from D it is 49/79. If you work out the answer for B, it comes out to 9/79 and 30/79, which is exactly what we expect.
The value of P16 [1 0 0 0] is approximately [1 0 0.886 0.62] and is correct to 6 decimal places.