dgesvd seems to be a LAPACKE_dgesvd with the layout LAPACK_COL_MAJOR, but looking at the examples for dgesvd and LAPACKE_dgesvd it seems there is an extra step in the dgesvd example where an optimal workspace is queried and allocated.
Is it correct to assume that this step is to figure out if the input matrix is COL_MAJOR or ROW_MAJOR?
Is it correct to assume that once the optimal workspace is figured out, 'dgesvd' internally calls LAPACKE_dgesvd with the appropriate layout?
If I already know the matrix layout to be COL_MAJOR is using LAPACKE_dgesvd better (faster/less expensive) than dgesvd?
We have two functions here which refer to two different interfaces:
i. dgesvd : calls fortran interface
ii. LAPACKE_dgesvd : calls C interface
For detail see this.
No it is not correct. As you will notice in the first call to dgesvd the value of lwork is set to -1 which as documented here is used for just calculating the size of lwork. So if you already know about the size of lwork you don't need to call it twice. Input matrix has to be LAPACK_COL_MAJOR for dgesvd as this is default for fortran. Also there is no way to calculate if the matrix is row major or column major.
No it's not true. dgesvd is the fortran interface which was implemented first.
This would depend on the compiler optimization. If the matrix is small it probably wouldn't matter. For me if its column major I would use fortran interface.
For matrix layout infotmation see this. Here is the technical paper for C interface
I know that Intel Fortran has libraries with functions and subroutines for working with sparse matricies, but I'm wondering if there is also some sort of data type or automated method for creating the sparse matricies in the first place.
BACKGROUND: I have a program that uses some 3 & 4 dimensional arrays that can be very large in the first 2 dimensions (~10k to ~100k elements in each dimension, maybe more). In the first 2 dimensions, each array is mostly (95% or so) populated w/ zeroes. To make the program friendly to machines with a "normal" amount of RAM available, I'd like to convert to sparse matricies. The manner in which the current conventional arrays are handled & updated throughout the code is pretty dependent on the code application, so I'm looking for a way to convert to sparse matrix storage without significant modification to the code. Basically, I'm lazy, and I don't want to revise the entire memory management implementation or write an entire new module where my arrays live and are managed. Is there a library or something else for Fortran that would implement a data type or something so that I can use sparse matrix storage without re-engineering each array and how it is handled? Thanks for the help. Cheers.
There are many different sparse formats and many different libraries for handling sparse matrices in Fortran (e.g. sparskit, petsc, ...) However, none of them can offer that compact array handling formalism, which is available in Fortran for intrinsic dense arrays (especially the subarray notation). So, you'll have to touch your code at several places, when you want to change it to use sparse matrices.
I need to solve a large, sparse system of linear equations from a program written in D. Ideally I'd like a library with a D-style interface, but I doubt one exists. However, D can directly access C APIs. Therefore, please suggest some libraries that solve large, sparse systems of linear equations with the following characteristics:
Exposes a C API.
Free/open source. Preferably non-copyleft, too, but this is not a hard requirement.
Well-tested and debugged. Easy to set up and use. Not written by academics just to get a paper on their method and then completely unmaintained.
The classical library for sparse problem is suite-sparse. You have packages on many systems. Matlab uses it internally.
My bad, I tangle LAPACK that I used old time ago and ARPACK that I used more time ago.
Here is link to arpack http://www.caam.rice.edu/~kristyn/parpack_home.html:
The package is designed to compute a few eigenvalues and corresponding eigenvectors of a general n by n matrix A. It is most appropriate for large sparse or structured matrices.
And here link with comparison of libaries for Linear Algebra:
http://www.netlib.org/utk/people/JackDongarra/la-sw.html
you can find there SparseLib++, mentioned here arpack and much more libaries in matrix form.
There is a dedicated package called CSPARSE, and it's written in C. It seems that the implemention is based on [david2006direct].
https://people.sc.fsu.edu/~jburkardt/c_src/csparse/csparse.html
Davis, T. A. (2006). Direct methods for sparse linear systems. Society for Industrial and Applied Mathematics.
I want to tackle some image-processing problems in Haskell. I'm working with both bitonal (bitmap) and color images with millions of pixels. I have a number of questions:
On what basis should I choose between Vector.Unboxed and UArray? They are both unboxed arrays, but the Vector abstraction seems heavily advertised, particular around loop fusion. Is Vector always better? If not, when should I use which representation?
For color images I will wish to store triples of 16-bit integers or triples of single-precision floating-point numbers. For this purpose, is either Vector or UArray easier to use? More performant?
For bitonal images I will need to store only 1 bit per pixel. Is there a predefined datatype that can help me here by packing multiple pixels into a word, or am I on my own?
Finally, my arrays are two-dimensional. I suppose I could deal with the extra indirection imposed by a representation as "array of arrays" (or vector of vectors), but I'd prefer an abstraction that has index-mapping support. Can anyone recommend anything from a standard library or from Hackage?
I am a functional programmer and have no need for mutation :-)
For multi-dimensional arrays, the current best option in Haskell, in my view, is repa.
Repa provides high performance, regular, multi-dimensional, shape polymorphic parallel arrays. All numeric data is stored unboxed. Functions written with the Repa combinators are automatically parallel provided you supply +RTS -Nwhatever on the command line when running the program.
Recently, it has been used for some image processing problems:
Real time edge detection
Efficient Parallel Stencil Convolution in Haskell
I've started writing a tutorial on the use of repa, which is a good place to start if you already know Haskell arrays, or the vector library. The key stepping stone is the use of shape types instead of simple index types, to address multidimensional indices (and even stencils).
The repa-io package includes support for reading and writing .bmp image files, though support for more formats is needed.
Addressing your specific questions, here is a graphic, with discussion:
On what basis should I choose between Vector.Unboxed and UArray?
They have approximately the same underlying representation, however, the primary difference is the breadth of the API for working with vectors: they have almost all the operations you'd normally associate with lists (with a fusion-driven optimization framework), while UArray have almost no API.
For color images I will wish to store triples of 16-bit integers or triples of single-precision floating-point numbers.
UArray has better support for multi-dimensional data, as it can use arbitrary data types for indexing. While this is possible in Vector (by writing an instance of UA for your element type), it isn't the primary goal of Vector -- instead, this is where Repa steps in, making it very easy to use custom data types stored in an efficient manner, thanks to the shape indexing.
In Repa, your triple of shorts would have the type:
Array DIM3 Word16
That is, a 3D array of Word16s.
For bitonal images I will need to store only 1 bit per pixel.
UArrays pack Bools as bits, Vector uses the instance for Bool which does do bit packing, instead using a representation based on Word8. Howver, it is easy to write a bit-packing implementation for vectors -- here is one, from the (obsolete) uvector library. Under the hood, Repa uses Vectors, so I think it inherits that libraries representation choices.
Is there a predefined datatype that can help me here by packing multiple pixels into a word
You can use the existing instances for any of the libraries, for different word types, but you may need to write a few helpers using Data.Bits to roll and unroll packed data.
Finally, my arrays are two-dimensional
UArray and Repa support efficient multi-dimensional arrays. Repa also has a rich interface for doing so. Vector on its own does not.
Notable mentions:
hmatrix, a custom array type with extensive bindings to linear algebra packages. Should be bound to use the vector or repa types.
ix-shapeable, getting more flexible indexing from regular arrays
chalkboard, Andy Gill's library for manipulating 2D images
codec-image-devil, read and write various image formats to UArray
Once I reviewed the features of Haskell array libraries which matter for me, and compiled a comparison table (only spreadsheet: direct link). So I'll try to answer.
On what basis should I choose between Vector.Unboxed and UArray? They are both unboxed arrays, but the Vector abstraction seems heavily advertised, particular around loop fusion. Is Vector always better? If not, when should I use which representation?
UArray may be preferred over Vector if one needs two-dimensional or multi-dimensional arrays. But Vector has nicer API for manipulating, well, vectors. In general, Vector is not well suited for simulating multi-dimensional arrays.
Vector.Unboxed cannot be used with parallel strategies. I suspect that UArray cannot be used neither, but at least it is very easy to switch from UArray to boxed Array and see if parallelization benefits outweight the boxing costs.
For color images I will wish to store triples of 16-bit integers or triples of single-precision floating-point numbers. For this purpose, is either Vector or UArray easier to use? More performant?
I tried using Arrays to represent images (though I needed only grayscale images). For color images I used Codec-Image-DevIL library to read/write images (bindings to DevIL library), for grayscale images I used pgm library (pure Haskell).
My major problem with Array was that it provides only random access storage, but it doesn't provide many means of building Array algorithms nor doesn't come with ready to use libraries of array routines (doesn't interface with linear algebra libs, doesn't allow to express convolutions, fft and other transforms).
Almost every time a new Array has to be built from the existing one, an intermediate list of values has to be constructed (like in matrix multiplication from the Gentle Introduction). The cost of array construction often out-weights the benefits of faster random access, to the point that a list-based representation is faster in some of my use cases.
STUArray could have helped me, but I didn't like fighting with cryptic type errors and efforts necessary to write polymorphic code with STUArray.
So the problem with Arrays is that they are not well suited for numerical computations. Hmatrix' Data.Packed.Vector and Data.Packed.Matrix are better in this respect, because they come along with a solid matrix library (attention: GPL license). Performance-wise, on matrix multiplication, hmatrix was sufficiently fast (only slightly slower than Octave), but very memory-hungry (consumed several times more than Python/SciPy).
There is also blas library for matrices, but it doesn't build on GHC7.
I didn't have much experience with Repa yet, and I don't understand repa code well. From what I see it has very limited range of ready to use matrix and array algorithms written on top of it, but at least it is possible to express important algorithms by the means of the library. For example, there are already routines for matrix multiplication and for convolution in repa-algorithms. Unfortunately, it seems that convolution is now limited to 7×7 kernels (it's not enough for me, but should suffice for many uses).
I didn't try Haskell OpenCV bindings. They should be fast, because OpenCV is really fast, but I am not sure if the bindings are complete and good enough to be usable. Also, OpenCV by its nature is very imperative, full of destructive updates. I suppose it's hard to design a nice and efficient functional interface on top of it. If one goes OpenCV way, he is likely to use OpenCV image representation everywhere, and use OpenCV routines to manipulate them.
For bitonal images I will need to store only 1 bit per pixel. Is there a predefined datatype that can help me here by packing multiple pixels into a word, or am I on my own?
As far as I know, Unboxed arrays of Bools take care of packing and unpacking bit vectors. I remember looking at implementation of arrays of Bools in other libraries, and didn't see this elsewhere.
Finally, my arrays are two-dimensional. I suppose I could deal with the extra indirection imposed by a representation as "array of arrays" (or vector of vectors), but I'd prefer an abstraction that has index-mapping support. Can anyone recommend anything from a standard library or from Hackage?
Apart from Vector (and simple lists), all other array libraries are capable of representing two-dimensional arrays or matrices. I suppose they avoid unneccesary indirection.
Although, this doesn't exactly answer your question and isn't really even haskell as such, I would recommend taking a look at CV or CV-combinators libraries at hackage. They bind the many rather useful image processing and vision operators from the opencv-library and make working with machine vision problems much faster.
It would be rather great if someone figures out how repa or some such array library could be directly used with opencv.
Here is a new Haskell Image Processing library that can handle all of the tasks in question and much more. Currently it uses Repa and Vector packages for underlying representations, which consequently inherits fusion, parallel computation, mutation and most of the other goodies that come with those libraries. It provides an easy to use interface that is natural for image manipulation:
2D indexing and unboxed pixels with arbitrary precision (Double, Float, Word16, etc..)
all essential functions like map, fold, zipWith, traverse ...
support for various color spaces: RGB, HSI, gray scale, Bi-tonal, Complex, etc.
common image processing functionality:
Binary morphology
Convolution
Interpolation
Fourier transform
Histogram plotting
etc.
Ability to treat pixels and images as regular numbers.
Reading and writing common image formats through JuicyPixels library
Most importantly, it is a pure Haskell library, so it does not depend on any external programs. It is also highly extendable, new color spaces and image representations can be introduced.
One thing that it does not do is packing multiple binary pixels in a Word, instead it uses a Word per binary pixel, maybe in a future...
What do I need? [an unordered list]
VERY easy parallelization
support for map, filter etc.
ability to perform array based computations efficiently, like A=B+C, sort of like matlab arrays.
Generation of SIMD code. I guess this is out of the question in the near future for anything, but hey, I can ask :)
support for matrices should be there at a minimum, higher dimensions are lower priority right now.
ability to get a pointer to it and create one from a C pointer.
Support from other libraries. IE, bindings to popular C math packages, i/o to disk or images if the arrays are 2D
What do I see?
Array package in haskell-platform. It's the blessed one and can do parallel
Data.Vector. Has loop fusion, but not in platform, so its maturity is unknown to me.
repa package, contributed by the DPH team, but doesn't work well with any stable ghc today.
Lots of variation in the level of support for array implementations. For instance, there doesn't seem to be an easy way to dump a 2D vector to a image file. IOW, the haskell community apparently hasn't settled on an array implementation.
So please, help me choose.
EDIT A=B+C refers to element wise addition, and not list concatenation
Correct, the community hasn't settled on a good array implementation. I think it would be a good Haskell Prime submission to put forward the Vector API and remove Data.Array.
Vector is very mature! It has:
VERY easy parallelization
support for map, filter etc.
performs array based computations efficiently, like A=B+C (but I'm not in tune with how matlab does it)
vector creation from a pointer via Vector.Storable
It does not:
have enough support from other libraries. IE, bindings to popular C math packages
support matrices, but you can have vectors of vectors. If you build some vector-based matrix operations then perhaps you could upload to hackage as vector-matrix.
Generate SIMD code.
NOTE: You can turn bytestrings into vectors of whatever, so if you have an image as a bytestring then, via Vector.Storable, you might be able to do what you want with the image as a vector.
(I am not allowed to comment)
rpg: Does hmatrix accept Data.Vector? It has a Data.Packed.Vector but are they the same?
Yes. The last version of hmatrix uses by default Data.Vector.Storable for 1D vectors (previously it was optional). The dependency on vector is not shown in Hackage, probably because it is in a configuration flag.
For LAPACK compatibility matrices are not Vector or Vector t, but they can be easily converted (e.g.: Data.Vector.fromList . toRows).
If you want bindings to popular C libraries, the best options are probably hmatrix and blas. Blas is just a binding to a BLAS library, whereas hmatrix provides some higher-level operations. There are also many libraries built upon hmatrix offering further functionality. If you're doing any sort of matrix work, that's what I would start with.
The vector package is also a good choice; it's stable and provides excellent performance. The Data.Vector.Storable types are represented as C arrays, so it's trivial to interface from them to other C libraries. The biggest drawback is that there's no matrix support, so you'd have to do that yourself.
As for exporting to an image format, most haskell image libraries seem to use ByteStrings. You could either convert to a ByteString, or bind to a C library that does what you want. If you find a Haskell library that does what you want, it should be easy enough to convert hmatrix data to the proper format.