There is sparse version of BLAS in intel MKL, for example doing multiplication of matrix and vector, we could use
mkl_?coogemv
Computes matrix-vector product of a sparse general matrix stored in
the coordinate format with one-based indexing
But I heard that multiplication is much more efficient when doing in CSR storage.
However to generate CSR storage sparse matrix is a headache, it is much easy and understandable to generate a COO storage sparse matrix.
What I want to know is that: Will MKL change the COO storage to CSR storage internally when using coogemv? The documentation didn't say.
Related
Does the Gnu Scientific Library (GSL) support finding eigenvectors/values of a sparse matrix (complex or otherwise) directly?
The C library GSL supports sparse matrices (highly compressed matrices, used for whenever you have a matrix which is mostly 0), example a complex matrix:
gsl_spmatrix_complex* H_sparse = gsl_spmatrix_complex_alloc (N_states, N_states);
And GSL has operations for finding eigenvalues/eigenvectors of a dense (not sparse) matrix, so this for instance works if I inflate the matrix H beforehand (skipping any allocations/freeing of workspace, vectors, matrices etc.):
gsl_spmatrix_complex_sp2d(H_dense, H_sparse);
/*...Allocate workspace w, dense matrix evec for eigenvectors, and vector eval for eigenvalues*/
gsl_eigen_hermv(H_dense, eval,evec , w);//hermv = hermitan complex matrix, and get vectors
gsl_eigen_hermv_sort ( eval,evec,GSL_EIGEN_SORT_VAL_ASC);
Inflating the sparse matrix before doing the one operation I need to do largely defeats the purpose of having a sparse matrix in the first place.
But I can't find functions for getting the eigenvalues and eigenvectors of a sparse matrix directly. I have tried looking through the header files gsl_eigen.h, gsl_splinalg.h, and gsl_spmatrix.h and the manuals, but I can't seem to find the functions I am looking for.
So does GSL support finding eigenvectors/values of a sparse matrix directly?
I have a large sparse matrix, for example, 128000×128000 SparseMatrixCSC{Complex{Float64},Int64} with 1376000 stored entries.
How to quickly get all eigenvalues of the sparse matrix ? Is it possible ?
I tried eigs for 128000×128000 with 1376000 stored entries but the kernel was dead.
I use a mac book pro with 16GB memory and Julia 1.3.1 on jupyter notebook.
As far as I'm aware (and I would love to be proven wrong) there is no efficient way to get all the eigenvalues of a general sparse matrix.
The main algorithm to compute the eigenvalues of a matrix is the QR algorithm. The first step of the QR algorithm is to reduce the matrix to a Hessenberg form (in order to do the QR factorisations in O(n) time). The problem is that reducing a matrix to Hessenberg form destroys the sparsity and you just end up with a dense matrix.
There are also other methods to compute the eigenvalues of a matrix like the (inverse) power iteration, that only require matrix vector products and solving linear systems, but these only give you the largest or smallest eigenvalues, and they become expensive when you want to compute all the eigenvalues (they require storing the eigenvectors for the "deflation").
So that was in general, now if your matrix has some special structure there may some better alternatives. For example, if your matrix is symmetric, then its Hessenberg form is tridiagonal and you can compute all the eigenvalues pretty fast.
TLDR: Is it possible ? — in general, no.
P.S: I tried to keep this short but if you're interested I can give you more details on any part of the answer.
I'm currently working with sparse matrices, and I have to compare the computation time of sparse matrix-matrix multiplication with full matrix-matrix multiplication. The issue is that sparse matrix computation is waaaaay slower than full matrix computation.
I'm compressing my matrices with the Compressed Row Storage, and multiplicating 2 matrices is very time consuming (quadruple for loop), so I'm wondering if there is a better compression format more suitable for matrix-matrix operation (CRS is very handy with matrix-vector computation).
Thanks in advance!
It's usually referred to as "Compressed Sparse Rows" (CSR), not CRS. The transpose, Compressed Sparse Columns (CSC) is also commonly used, including by the CSparse package that ends up being the backend of quite a lot of systems including MatLAB and SciPy (I think).
There is also a less-common Doubly-Compressed Sparse Columns (DCSC) format used by the Combinatorial BLAS. It compresses the column index again, and is useful for cases where the matrix is hypersparse. A hypersparse matrix has most columns empty, something that happens with 2D matrix decomposition.
That said, yes there is more overhead. However your operations are now dominated by the number of nonzeros, not the dimensions. So your FLOPS might be less but you still get your answer quicker.
You might look at the paper EFFICIENT SPARSE MATRIX-MATRIX PRODUCTS USING COLORINGS http://www.mcs.anl.gov/papers/P5007-0813_1.pdf for a discussion of how to achieve high performance with sparse matrix matrix products.
I am working on Fermion and Boson Hubbard Model, in which dimension of Hilbert Space are quite large (~50k). I am currently using the Lapack routine DSYEV to determine the eigenvalues & eigenfunctions of the large (50k x 50k) Hamiltonian matrix, but this takes a long time, about 8 hours on a Xeon workstation.
I would like to reduce this run time on this particular machine. I am looking at the Lanczos method and wondering if this is the best option, or if there is another choice.
Lanczos (or other iterative) method is used to compute extreme (small/big) eigenvalues. It is better than direct DSYEV, if you need eigenvalues and eigenfunctions much less than the system size (50k). Especially, if the matrix you have is sparse then the acceleration you will get is much better.
If you are looking for all eigenvalues and your matrix is dense then the better method is direct DSYEV.
I'm doing some calculations, and doing some analysis on the forces and weakness of different BLAS implementations. however I have come across a problem.
I'm testing cuBlas, doing linAlg on the GPU would seem like a good idea, but there is one problem.
The cuBlas implementation using column-major format, and since this is not what I need in the end, I'm curious if there is a way in with one can make BLAS do matrix-transpose?
BLAS doesn't have a matrix transpose routine built in. The CUDA SDK includes a matrix transpose example with a paper which discusses optimal strategy for performing a transpose. Your best strategy is probably to use row major inputs to CUBLAS with the transpose input version of the calls, then perform the intermediate calculations in column major, and lastly perform a transpose operation afterwards using the SDK transpose kernel.
Edited to add that CUBLAS added a transpose routine in CUBLAS version 5, geam, which can performed matrix transposition in GPU memory and should be regarded as optimal for whatever architecture you are using.