I am trying to find a pxq matrix multiplication implementation in c using the Strassen algorithm. I am not sure it is possible.
In wikipedia the algorithm is written like this:
M1 = (A + D) * (E + H) = AE + AH + DE + DH
M2 = (A + B) * H = AH + BH
but that will not work for a pxq matrix. This approach is defined for 2x2 matrix, not a pxq matrix. For that a loop of some kind is necessary, but I am not sure what kind? It may not be possible?
UPDATE:
WHY I AM ASKING THIS ?
I am pretty sure that the Strassen algorithm is a super fast one (no personal experaince. just reviews) even I have to write some code for large matrix multiplications (contains number of rows and columns), and then number of rows and columns which will keep changing every time. So, I need to write one for pxq matrix. So, I want to confirm whether it is possible...
Strassen_algorithm: http://en.wikipedia.org/wiki/Strassen_algorithm
Related
can we calculate the inverse of a matrix in codesys?
I am trying to write a code for the following equation.
For Codesys there is a paid matrix library. For TwinCAT there is the free and open source TcMatrix.
This is a case where you don't need to calculate the inverse explicitly and indeed are better not to, both for performance and accuracy.
Changing notation, so that it's easier to type here, you want
V = inv( I + A'*A) *A'*B
This can be calculated like this:
compute
b = A'*B
C = I + A'*A
solve
C = L*L' for lower triangular L
This is cholesky decomposition, which can be done in place, and you should read up on
solve
L*x = b for x
L'*v = x for v
Note that because L is lower triangular (and so L' upper triangular) each of the above solutions can be done in O(dim*dim) and can be done in place if desired.
V is what you wanted.
Most implementations of binomial coefficient computation using dynamic programming makes use of 2-dimensional arrays, as in these examples:
http://www.csl.mtu.edu/cs4321/www/Lectures/Lecture%2015%20-%20Dynamic%20Programming%20Binomial%20Coefficients.htm
http://www.geeksforgeeks.org/dynamic-programming-set-9-binomial-coefficient/
My question is, why not just compute it using a single dimensional array like this:
def C(n, r):
memo = list()
if (r > int(n/2)):
r = n - r
memo.append(1.0)
for i in range(1,r+1):
now = ((n-i+1)*memo[i-1])/i
memo.append(now)
return memo[r]
Basically using the recursive formula:
C(n,r) = ((n-r+1)/r) * C(n,r-1)
This has a O(r) complexity, while the 2 dimensional logic has a O(nr) complexity.
Am I missing something here?
If you want all of the values, then the 2D logic is certainly more efficient. The 2D logic may be more efficient for some parameters on some hardware that, e.g., lacks hardware multiply and divide. You have to be careful about integer overflow when multiplying before dividing, whereas the integer addition in the 2D recurrence is always fine. Other than that, no, the 1D recurrence is better.
I've been trying to implement SVD in C for the past few weeks now, and currently I've been using the algorithm 6 found here, and from my understanding this algorithm will run in time O(n^5) because there are two loops (One of the loops does not go from 0 to n, I know but n^5 works as a crude bound), and inside the inner loop matrix multiplication has to be done which is an n^3 process.
However, according this website, for an n by n matrix, SVD can be calculated in O(2n^3). Does anyone know where I can find an algorithm for that time complexity?
In the event anyone is looking for an answer to this in the future, the algorithm to calculate SVD in O(n^3), if the matrix is a square matrix, is the method of Jacobi Rotations.
For more information on the specific algorithm you can look at Algorithm 7 on this website.
The notation on the website is a little confusing, because of the typos, but in the step when determining the values of d1, d2, c, and č (sorry that's the closest I could get to c with the hat on top), what they mean is that c = cos(theta), s = sin(theta), č = cos(phi) and š = sin(phi).
You can calculate these values of theta and phi by elimination and substitution or you can check out this StackExchange post to see how to calculate them.
After that it is just following that algorithm.
Computing the SVD of an m × n matrix has complexity O(mn min(n, m)). Since this is super-linear in the size of the data, it becomes computationally expensive for large data sets. However, if we have a low rank matrix, we would need only k basis vectors, where k << m, n. One way of computing the rank k approximation is to compute the SVD of the full matrix and retain only the k largest singular values and vectors.
https://arxiv.org/pdf/1710.02812.pdf
I am writing a function to take the Mahalanobis distance between two vectors. I understand that this is achieved using the equation a'*C^-1*b, where a and b are vectors and C is the covariance matrix. My question is, is there an efficient way to find the inverse of the matrix without using Gauss-Jordan elimination, or is there no way around this? I'm looking for a way to do this myself, not with any predefined functions.
I know that C is a Hermitian, positive definite matrix, so is there any way that I can algorithmically take advantage of this fact? Or is there some clever way compute the Mahalanobis distance without calculating the inverse of the covariance at all? Any help would be appreciated.
***Edit: The Mahalanobis distance equation above is incorrect. It should be
x'*C^-1*x where x = (b-a), and b and a are the two vectors whose distance we are trying to find (thanks LRPurser). The solution posited in the selected answer is therefore as follows:
d=x'*b, where b = C^-1*x
C*b = x, so solve for b using LU factorization or LDL' factorization.
You can (and should!) use LU decomposition instead of explicitly inverting the matrix: solving C x = b using a decomposition has better numeric properties than computing C^-1 and multiplying the vector b.
Since your matrix is symmetric, an LU decomposition is effectively equivalent to an LDL* decomposition, which is what you should actually use in your case. Since your matrix is also positive-definite, you should be able to perform this decomposition without pivoting.
Edit: note that, for this application, you don't need to solve the full C x = b problem.
Instead, given C = L D L* and difference vector v = a-b, solve L* y = v for y (which is half as much work as the full LU solver).
Then, y^t D^-1 y = v^t C^-1 v can be computed in linear time.
First Mahalanobis Distance (MD) is the normed distance with respect to uncertainty in the measurement of two vectors. When C=Indentity matrix, MD reduces to the Euclidean distance and thus the product reduces to the vector norm. Also MD is always positive definite or greater than zero for all non-zero vectors. By your formulation with the appropriate choice of the vectors a and b, a*C^-1*b can be less than zero. Hopefully the difference of vectors you are looking for is x=(a-b) which makes the calculation x^t*C^-1*x where x^t is the transpose of the vector x. Also note that MD=sqrt(x^t*C^-1*x)
Since your matrix is symmetric and positive definite then you can utilize the Cholesky decomposition (MatLab-chol) which uses half of the operations as LU and is numerically more stable. chol(C)=L where C=L*L^t where L is a lower triangular matrix and L^t is the transpose of L which make it upper triangular. Your algorithm should go something like this
(Matlab)
x=a-b;
L=chol(C);
z=L\x;
MD=z'*z;
MD=sqrt(MD);
# Mahalanobis Distance Matrix
import numpy as np
from scipy.spatial import distance
from scipy.spatial.distance import mahalanobis
from scipy.spatial.distance import pdist
from scipy.spatial.distance import squareform
# Example
Data = np.array([[1,2],[3,2],[4,3]])
Cov = np.cov(np.transpose([[1,2],[3,2],[4,3]]))
invCov = np.linalg.inv(Cov)
Y = pdist(Data, 'mahalanobis', invCov)
MD = squareform(Y)
In GSL a real n * m matrix M is represented internally as an array of size n*m. To access the (i,j) element of M, internally GSL has to access the (i-1) * n + j - 1 location of the array, which involves integer multiplications and additions.
In Numerical Recipes for C, they recommend the alternative method of declaring an array of n pointers, each pointing to an array of m numbers. Then to access the (i,j) element, one puts M[i-1][j-1]. They claim that this is more efficient because it avoids the integer multiplication. The downside is that one has to initialize each pointer separately.
I am wondering, what are the advantages/disadvantages of each approach?
In C:
#define n 2
#define m 3
int M[n*m];
is the same as
int M[n][m];
in C matrices are said to be stored in row-major order
http://en.wikipedia.org/wiki/Row-major_order
In C,
M[1][2]
is the same as
*(M + 1*m + 2) // if M is define as M[n][m]
You could define M as an array of n pointers, but you still have to put the data somewhere and the best place is probably a 2D array. I would suggest:
int M[n][m];
int* Mrows[n] = {M[0], M[1]};
You can then do a direct offset into rows to get to the row you want. Then:
Mrows[1][2]
is the same as
*((*(Mrows + 1)) + 2)
Its more work for the programmer and probably only worth it if you want to go really fast. In that case you may want to look into more optimizations such as specific machine instructions. Also, depending on your algorithm, you may be able to just use + operations (like if you are iterating over the matrix)