I am trying to calculate the Mahalanobis distance between two vectors a and b. Eventually, I will be using this as a distance measure in statistical algorithms. I am using gsl to implement them. The formula for the mahalanobis distance is sqrt((a-b)'c^-1(a-b)), where c is the covariance matrix. According to this gsl documentation, it takes in two data sets and returns one covariance value. I am not sure how to calculate the covariance matrix using that.
Any help is appreciated.
Thanks.
I think you need to understand the calcuation of a covariance matrix first, second heres a sample code to get you started
for (i = 0; i < A->size1; i++) {
for (j = i; j < A->size2; j++) {
a = gsl_matrix_column (A, i);
b = gsl_matrix_column (A, j);
double cov = gsl_stats_covariance(a.vector.data, a.vector.stride,b.vector.data, b.vector.stride, a.vector.size);
gsl_matrix_set (C, i, j, cov);
}
}
Related
//Cannot understand use of this function
for(int i=0; i<n; i++) {
for(int j=0; j<n; j++) {
double sum = 0;
for(int k=0; k<n; k++) {
//Why is i*n+k used here?
sum += A[i*n+k]*A[j*n+k];
}
C[i*n+j] = sum;
int main() {
double *m4 = (double*)malloc(sizeof(double)*n*n);
//Why was gemm_ATA function used here?
gemm_ATA(m3, m4, n); //make a positive-definite matrix
printf("\n");
//show_matrix(m4,n);
}
I am making a project for parallelizing Cholesky method and found a useful code. In the given, project this function is used and I have no idea why is it used.
Also, can someone help me understand the code and its function used in the code given in the link:-
http://coliru.stacked-crooked.com/a/6f5750c20d456da9
The function gemm_ATA takes an input matrix A and calculates C = A^T * A, which is positive semi-definite by definition (note the semi-definiteness, depending on the properties of the input matrix).
Mathematically, calculating this matrix would be:
c_i,j = sum_k a_k,i * a_k,j
c_i,j is the entry of C in the i-th row and j-th column. The expressions i*n+k and j*n+k transform these 2D indices (row and column) to a 1D index of the underlying array.
gemm_ATA calculate AA^T and stores it in C. A^T is A but flipped over its diagonal. So A*A^T multiply each row of matrix A (call A[-,i]) with each column of the matrix A (call A^T[j,-]) which is also the row of A (A[-,j]).
Also, if you flatten an n*n 2D matrix to 1D matrix, you can access the first element of each i-th row by i*n+0. So if you want k-th column of ith row you have it with A[i*n+k].
Note that since you pass C by reference to your function, after calling the function, m4 is your positive definite matrix created from m3.
I have a matrix S(n x m) and a vector Sigma(n), and I would like to multiply each row S(i) by Sigma(i).
I have thought of 3 things :
-> Convert Sigma to a square diagonal matrix and compute S = Sigma * S, but it seems the functions exist only for general or triangular matrix...
-> Multiply each line by a scalar Sigma[i] using a DSCAL, in a loop
-> mkl_ddiamm, but it seems kinda obscure to me.
Any advices on how I should implement that ? Thank you !
It is a very simple operation that MKL/BLAS does not provide a function for it. You could implement it by yourself with for loops.
for(int i=0; i<nrow; ++i) {
for(int j=0; j<ncols; ++j) {
s[i][j] += sigma[i];
}
}
I am using the functions gsl_eigen_nonsymm and/or gsl_eigen_symm from the GSL library to find the eigenvalues of an L x L matrix M[i][j] which is also a function of time t = 1,....,N so i have M[i][j][t] to get the eigenvalues for every t i allocate an L x L matrix E[i][j] = M[i][j][t] and diagonalize it for every t.
The problem is that the program gives the eigenvalues at different order after some iteration. For example (L = 3) if at t = 0 i get eigen[t = 0] = {l1,l2,l3}(0) at t = 1 i may get eigen[t = 1] = {l3,l2,l1}(1) while i need to always have {l1,l2,l3}(t)
To be more concrete: consider the matrix M (t) ) = {{0,t,t},{t,0,2t},{t,t,0}} the eigenvalues will always be (approximatevly) l1 = -1.3 t , l2 = -t , l3 = 2.3 t When i tried to diagonalize it (with the code below) i got several times a swap in the result of the eigenvalues. Is there a way to prevent it? I can't just sort them by magnitude i need them to be always in the same order (whatever it is) a priori. (the code below is just an example to elucidate my problem)
EDIT: I can't just sort them because a priori I don't know their value nor if they reliably have a structure like l1<l2<l3 at every time due to statistical fluctuations, that's why i wanted to know if there is a way to make the algorithm behave always in the same way so that the order of the eigenvalues is always the same or if there is some trick to make it happen.
Just to be clearer I'll try to re-describe the toy problem I presented here. We have a matrix that depends on time, I, maybe naively, expected to just get lambda_1(t).....lambda_N(t), instead what I see is that the algorithm often swaps the eigenvalues at different times, so if at t = 1 I've got ( lambda_1,lambda_2,lambda_3 )(1) at time t = 2 (lambda_2,lambda_1,lambda_3)(2) so if for instance I wanted to see how lambda_1 evolves in time I can't because the algorithm mixes the eigenvalues at different times. The program below is just an analytical-toy example of my problem: The eigenvalues of the matrix below are l1 = -1.3 t , l2 = -t , l3 = 2.3 t but the program may give me as an output(-1.3,-1,2.3)(1), (-2,-2.6,4.6)(2), etc As previously stated, I was wondering then if there is a way to make the program order the eigenvalues always in the same way despite of their actual numerical value, so that i always get the (l1,l2,l3) combination. I hope it is more clear now, please tell me if it is not.
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <gsl/gsl_linalg.h>
#include <gsl/gsl_eigen.h>
#include <gsl/gsl_sort_vector.h>
main() {
int L = 3, i, j, t;
int N = 10;
double M[L][L][N];
gsl_matrix *E = gsl_matrix_alloc(L, L);
gsl_vector_complex *eigen = gsl_vector_complex_alloc(L);
gsl_eigen_nonsymm_workspace * w = gsl_eigen_nonsymm_alloc(L);
for(t = 1; t <= N; t++) {
M[0][0][t-1] = 0;
M[0][1][t-1] = t;
M[0][2][t-1] = t;
M[1][0][t-1] = t;
M[1][1][t-1] = 0;
M[1][2][t-1] = 2.0 * t;
M[2][1][t-1] = t;
M[2][0][t-1] = t;
M[2][2][t-1] = 0;
for(i = 0; i < L; i++) {
for(j = 0; j < L; j++) {
gsl_matrix_set(E, i, j, M[i][j][t - 1]);
}
}
gsl_eigen_nonsymm(E, eigen, w); /*diagonalize E which is M at t fixed*/
printf("#%d\n\n", t);
for(i = 0; i < L; i++) {
printf("%d\t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)))
}
printf("\n");
}
}
Your question makes zero sense. Eigenvalues do not have any inherent order to them. It sounds to me like you want to define eigenvalues of M_t something akin to L_1(M_t),..., L_n(M_t) and then track how they change in time. Assuming your process driving M_t is continuous, then so will your eigenvalues be. In other words they will not significantly change when you make small changes to M_t. So if you define an ordering by enforcing L_1 < L_2... < L_n, then this ordering will not change for small changes in t. When you have two eigenvalues cross, you'll need to make a decision about how to assign changes. If you have "random fluctuations" which are larger than the typical distance between your eigenvalues, then this becomes essentially impossible.
Here's another way of tracking eigenvectors, which might prove better. To do this, suppose that your eigenvectors are v_i, with components v_ij . What you do is first "normalize" your eigenvectors such that v_i1 is nonnegative, i.e. just flip the sign of each eigenvector appropriately. This will define an ordering on your eigenvalues through an ordering on v_i1, the first component of each eigenvector. This way you can still keep track of eigenvalues that cross over each other. However if your eigenvectors cross on the first component, you're in trouble.
I don't think you can do what you want. As t changes the output changes.
My original answer mentioned ordering on the pointers but looking at the data structure it won't help. When the eigenvalues have been computed the values are stored in E. You can see them as follows.
gsl_eigen_nonsymm(E, eigen, w);
double *mdata = (double*)E->data;
printf("mdata[%i] \t%lf\n", 0, mdata[0]);
printf("mdata[%i] \t%lf\n", 4, mdata[4]);
printf("mdata[%i] \t%lf\n", 8, mdata[8]);
The following code is how the the data in the eigenvector is layed out...
double *data = (double*)eigen->data;
for(i = 0; i < L; i++) {
printf("%d n \t%zu\n", i, eigen->size);
printf("%d \t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)));
printf("%d r \t%lf\n", i, data[0]);
printf("%d i \t%lf\n", i, data[1]);
printf("%d r \t%lf\n", i, data[2]);
printf("%d i \t%lf\n", i, data[3]);
printf("%d r \t%lf\n", i, data[4]);
printf("%d i \t%lf\n", i, data[5]);
}
If, and you can check this when you see the order change, the order of the data in mdata changes AND the order in data changes then the algorithm does not have a fixed order ie you cannot do what you're asking it to do. If the order does not change in mdata and it changes in data then you have a solution but I really doubt that will be the case.
According to the docs, those functions return unordered:
https://www.gnu.org/software/gsl/manual/html_node/Real-Symmetric-Matrices.html
This function computes the eigenvalues of the real symmetric matrix A. Additional workspace of the appropriate size must be provided in w. The diagonal and lower triangular part of A are destroyed during the computation, but the strict upper triangular part is not referenced. The eigenvalues are stored in the vector eval and are unordered.
Even the functions that return ordered results, do so by simple ascending/descending magnitude:
https://www.gnu.org/software/gsl/manual/html_node/Sorting-Eigenvalues-and-Eigenvectors.html
This function simultaneously sorts the eigenvalues stored in the vector eval and the corresponding real eigenvectors stored in the columns of the matrix evec into ascending or descending order according to the value of the parameter sort_type as shown above.
If you're looking for the time evolution of the eigenvalues, just do like you have been doing and solve for the time-dependent representations, e.g.:
lambda_1(t).....lambda_N(t)
For your simple time-as-scalar example,
l1 = -1.3 t , l2 = -t , l3 = 2.3 t
You literally have a parameterization of all possible solutions and because you've assigned them identifiers ln you don't run into the issue of degeneracy. Even if any M[i][j] are nonlinear functions of t, it shouldn't matter because the system itself is linear and solutions are computed purely by the characteristic equation (which will hold t constant while solving for lambda).
I'm working on a demo that requires a lot of vector math, and in profiling, I've found that it spends the most time finding the distances between given vectors.
Right now, it loops through an array of X^2 vectors, and finds the distance between each one, meaning it runs the distance function X^4 times, even though (I think) there are only (X^2)/2 unique distances.
It works something like this: (pseudo c)
#define MATRIX_WIDTH 8
typedef float vec2_t[2];
vec2_t matrix[MATRIX_WIDTH * MATRIX_WIDTH];
...
for(int i = 0; i < MATRIX_WIDTH; i++)
{
for(int j = 0; j < MATRIX_WIDTH; j++)
{
float xd, yd;
float distance;
for(int k = 0; k < MATRIX_WIDTH; k++)
{
for(int l = 0; l < MATRIX_WIDTH; l++)
{
int index_a = (i * MATRIX_LENGTH) + j;
int index_b = (k * MATRIX_LENGTH) + l;
xd = matrix[index_a][0] - matrix[index_b][0];
yd = matrix[index_a][1] - matrix[index_b][1];
distance = sqrtf(powf(xd, 2) + powf(yd, 2));
}
}
// More code that uses the distances between each vector
}
}
What I'd like to do is create and populate an array of (X^2) / 2 distances without redundancy, then reference that array when I finally need it. However, I'm drawing a blank on how to index this array in a way that would work. A hash table would do it, but I think it's much too complicated and slow for a problem that seems like it could be solved by a clever indexing method.
EDIT: This is for a flocking simulation.
performance ideas:
a) if possible work with the squared distance, to avoid root calculation
b) never use pow for constant, integer powers - instead use xd*xd
I would consider changing your algorithm - O(n^4) is really bad. When dealing with interactions in physics (also O(n^4) for distances in 2d field) one would implement b-trees etc and neglect particle interactions with a low impact. But it will depend on what "more code that uses the distance..." really does.
just did some considerations: the number of unique distances is 0.5*n*n(+1) with n = w*h.
If you write down when unique distances occur, you will see that both inner loops can be reduced, by starting at i and j.
Additionally if you only need to access those distances via the matrix index, you can set up a 4D-distance matrix.
If memory is limited we can save up nearly 50%, as mentioned above, with a lookup function that will access a triangluar matrix, as Code-Guru said. We would probably precalculate the line index to avoid summing up on access
float distanceArray[(H*W+1)*H*W/2];
int lineIndices[H];
searchDistance(int i, int j)
{
return i<j?distanceArray[i+lineIndices[j]]:distanceArray[j+lineIndices[i]];
}
I am writing an algorithm in C that requires Matrix and Vector multiplications. I have a matrix Q (W x W) which is created by multiplying the transpose of a vector J(1 x W) with itself and adding Identity matrix I, scaled using scalar a.
Q = [(J^T) * J + aI].
I then have to multiply the inverse of Q with vector G to get vector M.
M = (Q^(-1)) * G.
I am using cblas and clapack to develop my algorithm. When matrix Q is populated using random numbers (type float) and inverted using the routines sgetrf_ and sgetri_ , the calculated inverse is correct.
But when matrix Q is symmetrical, which is the case when you multiply (J^T) x J, the calculated inverse is wrong!!.
I am aware of the row-major (in C) and column-major (in FORTRAN) format of arrays while calling lapack routines from C, but for a symmetrical matrix this should not be a problem as A^T = A.
I have attached my C function code for matrix inversion below.
I am sure there is a better way to solve this. Can anyone help me with this?
A solution using cblas would be great...
Thanks.
void InverseMatrix_R(float *Matrix, int W)
{
int LDA = W;
int IPIV[W];
int ERR_INFO;
int LWORK = W * W;
float Workspace[LWORK];
// - Compute the LU factorization of a M by N matrix A
sgetrf_(&W, &W, Matrix, &LDA, IPIV, &ERR_INFO);
// - Generate inverse of the matrix given its LU decompsotion
sgetri_(&W, Matrix, &LDA, IPIV, Workspace, &LWORK, &ERR_INFO);
// - Display the Inverted matrix
PrintMatrix(Matrix, W, W);
}
void PrintMatrix(float* Matrix, int row, int colm)
{
int i,k;
for (i =0; i < row; i++)
{
for (k = 0; k < colm; k++)
{
printf("%g, ",Matrix[i*colm + k]);
}
printf("\n");
}
}
I don't know BLAS or LAPACK, so I have no idea what may cause this behaviour.
But, for matrices of the given form, calculating the inverse is quite easy. The important fact for this is
(J^T*J)^2 = (J^T*J)*(J^T*J) = J^T*(J*J^T)*J = <J|J> * (J^T*J)
where <u|v> denotes the inner product (if the components are real - the canonical bilinear form for complex components, but then you'd probably consider not the transpose but the conjugate transpose, and you'd be back at the inner product).
Generalising,
(J^T*J)^n = (<J|J>)^(n-1) * (J^T*J), for n >= 1.
Let us denote the symmetric square matrix (J^T*J) by S and the scalar <J|J> by q. Then, for general a != 0 of sufficiently large absolute value (|a| > q):
(a*I + S)^(-1) = 1/a * (I + a^(-1)*S)^(-1)
= 1/a * (I + ∑ (-1)^k * a^(-k) * S^k)
k>0
= 1/a * (I + (∑ (-1)^k * a^(-k) * q^(k-1)) * S)
k>0
= 1/a * (I - 1/(a+q)*S)
= 1/a*I - 1/(a*(a+q))*S
That formula holds (by analyticity) for all a except a = 0 and a = -q, as can be verified by calculating
(a*I + S) * (1/a*I - 1/(a*(a+q))*S) = I + 1/a*S - 1/(a+q)*S - 1/(a*(a+q))*S^2
= I + 1/a*S - 1/(a+q)*S - q/(a*(a+q))*S
= I + ((a+q) - a - q)/(a*(a+q))*S
= I
using S^2 = q*S.
That calculation is also much simpler and more efficient than first finding the LU decomposition.
You may want to try Armadillo, which is an easy to use C++ wrapper for LAPACK. It provides several inverse related functions:
inv(), general inverse, with an optional speedup for symmetric positive definite matrices
pinv(), pseudo-inverse
solve(), solve a system of linear equations (that can be over- or under-determined), without doing the actual inverse
Example for 3x3 matrix inversion, visit sgetri.f for more
//__CLPK_integer is typedef of int
//__CLPK_real is typedef of float
__CLPK_integer ipiv[3];
{
//Compute LU lower upper factorization of matrix
__CLPK_integer m=3;
__CLPK_integer n=3;
__CLPK_real *a=(float *)this->m1;
__CLPK_integer lda=3;
__CLPK_integer info;
sgetrf_(&m, &n, a, &lda, ipiv, &info);
}
{
//compute inverse of a matrix
__CLPK_integer n=3;
__CLPK_real *a=(float *)this->m1;
__CLPK_integer lda=3;
__CLPK_real work[3];
__CLPK_integer lwork=3;
__CLPK_integer info;
sgetri_(&n, a, &lda, ipiv, work, &lwork, &info);
}