Related
I am working on C, using GNU library for scientific computing. Essentially, I need to do the equivalent of the following MATLAB code:
x=x.*(A*x);
where x is a gsl_vector, and A is a gsl_matrix.
I managed to do (A*x) with the following command:
gsl_blas_dgemv(CblasNoTrans, 1.0, A, x, 1.0, res);
where res is an another gsl_vector, which stores the result. If the matrix A has size m * m, and vector x has size m * 1, then vector res will have size m * 1.
Now, what remains to be done is the elementwise product of vectors x and res (the results should be a vector). Unfortunately, I am stuck on this and cannot find the function which does that.
If anyone can help me on that, I would be very grateful. In addition, does anyone know if there is some better documentation of GNU rather than https://www.gnu.org/software/gsl/manual/html_node/GSL-BLAS-Interface.html#GSL-BLAS-Interface which so far is confusing me.
Finally, would I lose in time performance if I do this step by simply using a for loop (the size of the vector is around 11000 and this step will be repeated 500-5000 times)?
for (i = 0; i < m; i++)
gsl_vector_set(res, i, gsl_vector_get(x, i) * gsl_vector_get(res, i));
Thanks!
The function you want is:
gsl_vector_mul(res, x)
I have used Intel's MKL, and I like the documentation on their website for these BLAS routines.
The for-loop is ok if GSL is well designed. For example gsl_vector_set() and gsl_vector_get() can be inlined. You could compare the running time with gsl_blas_daxpy. The for-loop is well optimized if the timing result is similar.
On the other hand, you may want to try a much better matrix library Eigen, with which you can implement your operation with the code similar to this
x = x.array() * (A * x).array();
I am using the functions gsl_eigen_nonsymm and/or gsl_eigen_symm from the GSL library to find the eigenvalues of an L x L matrix M[i][j] which is also a function of time t = 1,....,N so i have M[i][j][t] to get the eigenvalues for every t i allocate an L x L matrix E[i][j] = M[i][j][t] and diagonalize it for every t.
The problem is that the program gives the eigenvalues at different order after some iteration. For example (L = 3) if at t = 0 i get eigen[t = 0] = {l1,l2,l3}(0) at t = 1 i may get eigen[t = 1] = {l3,l2,l1}(1) while i need to always have {l1,l2,l3}(t)
To be more concrete: consider the matrix M (t) ) = {{0,t,t},{t,0,2t},{t,t,0}} the eigenvalues will always be (approximatevly) l1 = -1.3 t , l2 = -t , l3 = 2.3 t When i tried to diagonalize it (with the code below) i got several times a swap in the result of the eigenvalues. Is there a way to prevent it? I can't just sort them by magnitude i need them to be always in the same order (whatever it is) a priori. (the code below is just an example to elucidate my problem)
EDIT: I can't just sort them because a priori I don't know their value nor if they reliably have a structure like l1<l2<l3 at every time due to statistical fluctuations, that's why i wanted to know if there is a way to make the algorithm behave always in the same way so that the order of the eigenvalues is always the same or if there is some trick to make it happen.
Just to be clearer I'll try to re-describe the toy problem I presented here. We have a matrix that depends on time, I, maybe naively, expected to just get lambda_1(t).....lambda_N(t), instead what I see is that the algorithm often swaps the eigenvalues at different times, so if at t = 1 I've got ( lambda_1,lambda_2,lambda_3 )(1) at time t = 2 (lambda_2,lambda_1,lambda_3)(2) so if for instance I wanted to see how lambda_1 evolves in time I can't because the algorithm mixes the eigenvalues at different times. The program below is just an analytical-toy example of my problem: The eigenvalues of the matrix below are l1 = -1.3 t , l2 = -t , l3 = 2.3 t but the program may give me as an output(-1.3,-1,2.3)(1), (-2,-2.6,4.6)(2), etc As previously stated, I was wondering then if there is a way to make the program order the eigenvalues always in the same way despite of their actual numerical value, so that i always get the (l1,l2,l3) combination. I hope it is more clear now, please tell me if it is not.
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <gsl/gsl_linalg.h>
#include <gsl/gsl_eigen.h>
#include <gsl/gsl_sort_vector.h>
main() {
int L = 3, i, j, t;
int N = 10;
double M[L][L][N];
gsl_matrix *E = gsl_matrix_alloc(L, L);
gsl_vector_complex *eigen = gsl_vector_complex_alloc(L);
gsl_eigen_nonsymm_workspace * w = gsl_eigen_nonsymm_alloc(L);
for(t = 1; t <= N; t++) {
M[0][0][t-1] = 0;
M[0][1][t-1] = t;
M[0][2][t-1] = t;
M[1][0][t-1] = t;
M[1][1][t-1] = 0;
M[1][2][t-1] = 2.0 * t;
M[2][1][t-1] = t;
M[2][0][t-1] = t;
M[2][2][t-1] = 0;
for(i = 0; i < L; i++) {
for(j = 0; j < L; j++) {
gsl_matrix_set(E, i, j, M[i][j][t - 1]);
}
}
gsl_eigen_nonsymm(E, eigen, w); /*diagonalize E which is M at t fixed*/
printf("#%d\n\n", t);
for(i = 0; i < L; i++) {
printf("%d\t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)))
}
printf("\n");
}
}
Your question makes zero sense. Eigenvalues do not have any inherent order to them. It sounds to me like you want to define eigenvalues of M_t something akin to L_1(M_t),..., L_n(M_t) and then track how they change in time. Assuming your process driving M_t is continuous, then so will your eigenvalues be. In other words they will not significantly change when you make small changes to M_t. So if you define an ordering by enforcing L_1 < L_2... < L_n, then this ordering will not change for small changes in t. When you have two eigenvalues cross, you'll need to make a decision about how to assign changes. If you have "random fluctuations" which are larger than the typical distance between your eigenvalues, then this becomes essentially impossible.
Here's another way of tracking eigenvectors, which might prove better. To do this, suppose that your eigenvectors are v_i, with components v_ij . What you do is first "normalize" your eigenvectors such that v_i1 is nonnegative, i.e. just flip the sign of each eigenvector appropriately. This will define an ordering on your eigenvalues through an ordering on v_i1, the first component of each eigenvector. This way you can still keep track of eigenvalues that cross over each other. However if your eigenvectors cross on the first component, you're in trouble.
I don't think you can do what you want. As t changes the output changes.
My original answer mentioned ordering on the pointers but looking at the data structure it won't help. When the eigenvalues have been computed the values are stored in E. You can see them as follows.
gsl_eigen_nonsymm(E, eigen, w);
double *mdata = (double*)E->data;
printf("mdata[%i] \t%lf\n", 0, mdata[0]);
printf("mdata[%i] \t%lf\n", 4, mdata[4]);
printf("mdata[%i] \t%lf\n", 8, mdata[8]);
The following code is how the the data in the eigenvector is layed out...
double *data = (double*)eigen->data;
for(i = 0; i < L; i++) {
printf("%d n \t%zu\n", i, eigen->size);
printf("%d \t%lf\n", i, GSL_REAL(gsl_vector_complex_get(eigen, i)));
printf("%d r \t%lf\n", i, data[0]);
printf("%d i \t%lf\n", i, data[1]);
printf("%d r \t%lf\n", i, data[2]);
printf("%d i \t%lf\n", i, data[3]);
printf("%d r \t%lf\n", i, data[4]);
printf("%d i \t%lf\n", i, data[5]);
}
If, and you can check this when you see the order change, the order of the data in mdata changes AND the order in data changes then the algorithm does not have a fixed order ie you cannot do what you're asking it to do. If the order does not change in mdata and it changes in data then you have a solution but I really doubt that will be the case.
According to the docs, those functions return unordered:
https://www.gnu.org/software/gsl/manual/html_node/Real-Symmetric-Matrices.html
This function computes the eigenvalues of the real symmetric matrix A. Additional workspace of the appropriate size must be provided in w. The diagonal and lower triangular part of A are destroyed during the computation, but the strict upper triangular part is not referenced. The eigenvalues are stored in the vector eval and are unordered.
Even the functions that return ordered results, do so by simple ascending/descending magnitude:
https://www.gnu.org/software/gsl/manual/html_node/Sorting-Eigenvalues-and-Eigenvectors.html
This function simultaneously sorts the eigenvalues stored in the vector eval and the corresponding real eigenvectors stored in the columns of the matrix evec into ascending or descending order according to the value of the parameter sort_type as shown above.
If you're looking for the time evolution of the eigenvalues, just do like you have been doing and solve for the time-dependent representations, e.g.:
lambda_1(t).....lambda_N(t)
For your simple time-as-scalar example,
l1 = -1.3 t , l2 = -t , l3 = 2.3 t
You literally have a parameterization of all possible solutions and because you've assigned them identifiers ln you don't run into the issue of degeneracy. Even if any M[i][j] are nonlinear functions of t, it shouldn't matter because the system itself is linear and solutions are computed purely by the characteristic equation (which will hold t constant while solving for lambda).
I am trying to implement a wavelet transform in C and I have never done it before. I have read some about Wavelets, and understand the 'growing subspaces' idea, and how Mallat's one sided filter bank is essentially the same idea.
However, I am stuck on how to actually implement Mallat's fast wavelet transform. This is what I understand so far:
The high pass filter, h(t), gives you the detail coefficients. For a given scale j, it is a reflected, dilated, and normed version of the mother wavelet W(t).
g(t) is then the low pass filter that makes up the difference. It is supposed to be the quadrature mirror of h(t)
To get the detail coefficients, or the approximation coefficients for the jth level, you need to convolve your signal block with h(t) or g(t) respectively, and downsample the signal by 2^{j} (ie take every 2^{j} value)
However these are my questions:
How can I find g(t) when I know h(t)?
How can I compute the inverse of this transform?
Do you have any C code that I can reference? (Yes I found the one on wiki but it doesn't help)
What I would like some code to say is:
A. Here is the filter
B. Here is the transform (very explicitly)
C.) Here is the inverse transform (again for dummies)
Thanks for your patience, but there doesn't seem to be a Step1 - Step2 - Step3 -- etc guide out there with explicit examples (that aren't HAAR because all the coefficients are 1s and that makes things confusing).
the Mallat recipe for the fwt is really simple. If you look at the matlab code, eg the script by Jeffrey Kantor, all the steps are obvious.
In C it is a bit more work but that is mainly because you need to take care of your own declarations and allocations.
Firstly, about your summary:
usually the filter h is a lowpass filter, representing the scaling function (father)
likewise, g is usually the highpass filter representing the wavelet (mother)
you cannot perform a J-level decomposition in 1 filtering+downsampling step. At each level, you create an approximation signal c by filtering with h and downsampling, and a detail signal d by filtering with g and downsampling, and repeat this at the next level (using the current c)
About your questions:
for a filter h of an an orthogonal wavelet basis, [h_1 h_2 .. h_m h_n], the QMF is [h_n -h_m .. h_2 -h_1], where n is an even number and m==n-1
the inverse transform does the opposite of the fwt: at each level it upsamples detail d and approximation c, convolves d with g and c with h, and adds the signals together -- see the corresponding matlab script.
Using this information, and given a signal x of len points of type double, scaling h and wavelet g filters of f coefficients (also of type double), and a decomposition level lev, this piece of code implements the Mallat fwt:
double *t=calloc(len+f-1, sizeof(double));
memcpy(t, x, len*sizeof(double));
for (int i=0; i<lev; i++) {
memset(y, 0, len*sizeof(double));
int len2=len/2;
for (int j=0; j<len2; j++)
for (int k=0; k<f; k++) {
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
}
len=len2;
memcpy(t, y, len*sizeof(double));
}
free(t);
It uses one extra array: a 'workspace' t to copy the approximation c (the input signal x to start with) for the next iteration.
See this example C program, which you can compile with gcc -std=c99 -fpermissive main.cpp and run with ./a.out.
The inverse should also be something along these lines. Good luck!
The only thing that is missing is some padding for the filter operation.
The lines
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
exceed the boundaries of the t-array during first iteration and exceed the approximation part of the array during the following iterations. One must add (f-1) elements at the beginning of the t-array.
double *t=calloc(len+f-1, sizeof(double));
memcpy(&t[f], x, len*sizeof(double));
for (int i=0; i<lev; i++) {
memset(t, 0, (f-1)*sizeof(double));
memset(y, 0, len*sizeof(double));
int len2=len/2;
for (int j=0; j<len2; j++)
for (int k=0; k<f; k++) {
y[j] +=t[2*j+k]*h[k];
y[j+len2]+=t[2*j+k]*g[k];
}
len=len2;
memcpy(&t[f], y, len*sizeof(double));
}
I have a task in which I have to solve a system of linear equations Ax =B, where A is a sparse matrix of the order of 10000. I am using csparse to solve it. In my initial implementation, for demo purposes A is 3*3 order identity matrix and B ={1,2,3}. Below is the code snippet, which is returning 0 in the status which means there is some error in my implementation. What is that I am doing wrong ?
cs A;
int N = 3;
double b[]={1,2,3};
double data[]={1,1,1};
int columnIndices[]={0,1,2};
int rowIndices[]={0,1,2};
A.nzmax =3;
A.m = N;
A.n = N;
A.p = &columnIndices[0];
A.i = &rowIndices[0];
A.x = &data[0];
A.nz = 3;
int status = cs_cholsol(0,&A,&b[0]);
NSLog(#"status=%d",status); // status always returns 0, which means error
You have to convert your input matrix to CSC. The solver functions check for the matrix format and return 0 if it's coordinate form.
Btw. with a similar example I couldn't get "cholsol" to work, but "lsolve" works fine (after conversion to CSC).
Matrix A above is in matrix tripplet form. First we need to convert it to compressed column format (B) and then apply the cs_chsol function to get the results.
// Convert matrix tripplet to column compressed form
cs *B = cs_compress(&A);
int status = cs_cholsol(0,&B,&b[0]);
I implemented it in my program and everything is running perfectly fine now.
I have this matrix A, representing similarities of pixel intensities of an image. For example: Consider a 10 x 10 image. Matrix A in this case would be of dimension 100 x 100, and element A(i,j) would have a value in the range 0 to 1, representing the similarity of pixel i to j in terms of intensity.
I am using OpenCV for image processing and the development environment is C on Linux.
Objective is to compute the Eigenvectors of matrix A and I have used the following approach:
static CvMat mat, *eigenVec, *eigenVal;
static double A[100][100]={}, Ain1D[10000]={};
int cnt=0;
//Converting matrix A into a one dimensional array
//Reason: That is how cvMat requires it
for(i = 0;i < affnDim;i++){
for(j = 0;j < affnDim;j++){
Ain1D[cnt++] = A[i][j];
}
}
mat = cvMat(100, 100, CV_32FC1, Ain1D);
cvEigenVV(&mat, eigenVec, eigenVal, 1e-300);
for(i=0;i < 100;i++){
val1 = cvmGet(eigenVal,i,0); //Fetching Eigen Value
for(j=0;j < 100;j++){
matX[i][j] = cvmGet(eigenVec,i,j); //Fetching each component of Eigenvector i
}
}
Problem: After execution I get nearly all components of all the Eigenvectors to be zero. I tried different images and also tried populating A with random values between 0 and 1, but the same result.
Few of the top eigenvalues returned look like the following:
9805401476911479666115491135488.000000
-9805401476911479666115491135488.000000
-89222871725331592641813413888.000000
89222862280598626902522986496.000000
5255391142666987110400.000000
I am now thinking on the lines of using cvSVD() which performs singular value decomposition of real floating-point matrix and might yield me the eigenvectors. But before that I thought of asking it here. Is there anything absurd in my current approach? Am I using the right API i.e. cvEigenVV() for the right input matrix (my matrix A is a floating point matrix)?
cheers
Note to readers: This post at first may seem unrelated to the topic, but please refer to the discussion in the comments above.
The following is my attempt at implementing the Spectral Clustering algorithm applied to image pixels in MATLAB. I followed exactly the paper mentioned by #Andriyev:
Andrew Ng, Michael Jordan, and Yair Weiss (2002).
On spectral clustering: analysis and an algorithm.
In T. Dietterich, S. Becker, and Z. Ghahramani (Eds.),
Advances in Neural Information Processing Systems 14. MIT Press
The code:
%# parameters to tune
SIGMA = 2e-3; %# controls Gaussian kernel width
NUM_CLUSTERS = 4; %# specify number of clusters
%% Loading and preparing a sample image
%# read RGB image, and make it smaller for fast processing
I0 = im2double(imread('house.png'));
I0 = imresize(I0, 0.1);
[r,c,~] = size(I0);
%# reshape into one row per-pixel: r*c-by-3
%# (with pixels traversed in columwise-order)
I = reshape(I0, [r*c 3]);
%% 1) Compute affinity matrix
%# for each pair of pixels, apply a Gaussian kernel
%# to obtain a measure of similarity
A = exp(-SIGMA * squareform(pdist(I,'euclidean')).^2);
%# and we plot the matrix obtained
imagesc(A)
axis xy; colorbar; colormap(hot)
%% 2) Compute the Laplacian matrix L
D = diag( 1 ./ sqrt(sum(A,2)) );
L = D*A*D;
%% 3) perform an eigen decomposition of the laplacian marix L
[V,d] = eig(L);
%# Sort the eigenvalues and the eigenvectors in descending order.
[d,order] = sort(real(diag(d)), 'descend');
V = V(:,order);
%# kepp only the largest k eigenvectors
%# In this case 4 vectors are enough to explain 99.999% of the variance
NUM_VECTORS = sum(cumsum(d)./sum(d) < 0.99999) + 1;
V = V(:, 1:NUM_VECTORS);
%% 4) renormalize rows of V to unit length
VV = bsxfun(#rdivide, V, sqrt(sum(V.^2,2)));
%% 5) cluster rows of VV using K-Means
opts = statset('MaxIter',100, 'Display','iter');
[clustIDX,clusters] = kmeans(VV, NUM_CLUSTERS, 'options',opts, ...
'distance','sqEuclidean', 'EmptyAction','singleton');
%% 6) assign pixels to cluster and show the results
%# assign for each pixel the color of the cluster it belongs to
clr = lines(NUM_CLUSTERS);
J = reshape(clr(clustIDX,:), [r c 3]);
%# show results
figure('Name',sprintf('Clustering into K=%d clusters',NUM_CLUSTERS))
subplot(121), imshow(I0), title('original image')
subplot(122), imshow(J), title({'clustered pixels' '(color-coded classes)'})
... and using a simple house image I drew in Paint, the results were:
and by the way, the first 4 eigenvalues used were:
1.0000
0.0014
0.0004
0.0002
and the corresponding eigenvectors [columns of length r*c=400]:
-0.0500 0.0572 -0.0112 -0.0200
-0.0500 0.0553 0.0275 0.0135
-0.0500 0.0560 0.0130 0.0009
-0.0500 0.0572 -0.0122 -0.0209
-0.0500 0.0570 -0.0101 -0.0191
-0.0500 0.0562 -0.0094 -0.0184
......
Note that there are step performed above which you didn't mention in your question (Laplacian matrix, and normalizing its rows)
I would recommend this article. The author implements Eigenfaces for face recognition. On page 4 you can see that he uses cvCalcEigenObjects to generate the eigenvectors from an image. In the article the whole pre processing step necessary for this computations are shown.
Here's a not very helpful answer:
What does theory (or maths scribbled on a piece of paper) tell you the eigenvectors ought to be ? Approximately.
What does another library tell you the eigenvectors ought to be ? Ideally what does a system such as Mathematica or Maple (which can be persuaded to compute to arbitrary precision) tell you the eigenvectors ought to be ? If not for a production-sixed problem at least for a test-sized problem.
I'm not an expert with image processing so I can't be much more helpful, but I spend a lot of time with scientists and experience has taught me that a lot of tears and anger can be avoided by doing some maths first and forming an expectation of what results you ought to get before wondering why you got 0s all over the place. Sure it might be an error in the implementation of an algorithm, it might be loss of precision or some other numerical problem. But you don't know and shouldn't follow up those lines of inquiry yet.
Regards
Mark