I can't find a conjugate gradient inversion in Armadillo or that works with Armadillo. CG is very useful for inversion of a sparse matrix and for large matrix inversion where you only need the pseudo-inverse. (Armadillo pinv() uses SVD, which is expensive.....)
Does this not exist? Is there interest in CG based pinv() for dense and sparse matrices?
Related
While using a computer display, polylines are used. You want to use an algorithm which will reduce points in the polyline. The polyline should be decimated within a specified tolerance. Which of the following algorithm would you use?
A) Flood fill Algorithm
B) Lee Algorithm
C) Floyd's Cycle Detection Algorithm
D) Vertex Reduction
First of all this has nothing to do with artificial intelligence. All of the mentioned algorithms are algorithms that solve some sort of graph related problem. Simplified they can be describes as follows:
Flood fill is an algorithm that "colors" an arbitrarily shaped region in a "maze" together
Lee's algorithm finds the best path between two points through a "maze" with obstacles
Cycle Detection algorithms find cycles (a non-trivial path where the start is also the end)
Vertex reduction algorithms remove vertices from a graph such that the graph still stays in it's original shape but with fewer features.
The answer to the problem should therefore be obvious ;)
I have been developing a C language control software working in real time. The software implements among others discrete state space observer of the controlled system. For implementation of the observer it is necessary to calculate inverse of the matrix with 4x4 dimensions. The inverse matrix calculation has to be done each 50 microseconds and it is worthwhile to say that during this time period also other pretty time consuming calculation will be done. So the inverse matrix calculation has to consume much less than 50 microseconds. It is also necessary to say that the DSP used does not have ALU with floating point operations support.
I have been looking for some efficient way how to do that. One idea which I have is to prepare general formula for calculation the determinant of the matrix 4x4 and general formula for calculation the adjoint matrix of the 4x4 matrix and then calculate the inverse matrix according to below given formula.
What do you think about this approach?
As I understand the consensus among those who study numerical linear algebra, the advice is to avoid computing matrix inverses unnecessarily. For example if the inverse of A appears in your controller only in expressions such as
z = inv(A)*y
then it is better (faster, more accurate) to solve for z the equation
A*z = y
than to compute inv(A) and then multiply y by inv(A).
A common method to solve such equations is to factorize A into simpler parts. For example if A is (strictly) positive definite then the cholesky factorization finds lower triangular matrix L so that
A = L*L'
Given that we can solve A*z=y for z via:
solve L*u = y for u
solve L'*z = u for z
and each of these is easy given the triangular nature of L
Another factorization (that again only applies to positive definite matrices) is the LDL which in your case may be easier as it does not involve square roots. It is described in the wiki article linked above.
More general factorizations include the LUD and QR These are more general in that they can be applied to any (invertible) matrix, but are somewhat slower than cholesky.
Such factorisations can also be used to compute inverses.
To be pedantic describing adj(A) in your post as the adjoint is, perhaps, a little old fashioned; I thing adjugate or adjunct is more modern. In any case adj(A) is not the transpose. Rather the (i,j) element of adj(A) is, up to a sign, the determinant of the matrix obtained from A by deleting the i'th row and j'th column. It is awkward to compute this efficiently.
Assume a workflow for 2D image feature extraction by using SIFT, SURF, or MSER methods followed by bag-of-words/features encoded and subsequently used to train classifiers.
I was wondering if there is an analogous approach for 3D datasets, for example, a 3D volume of MRI data. When dealing with 2D images, each image represents an entity with features to be detected and indexed. However, in a 3D dataset is it possible to extract features from the three-dimensional entity? Does this have to be done slice-by-slice, by decomposing the 3D images to multiple 2D images (slices)? Or is there a way of reducing the 3D dimensionality to 2D while retaining the 3D information?
Any pointers would be greatly appreciated.
You can perform feature extraction by passing your 3D volumes through a pre-trained 3D convolutional neural network. Because pre-trained 3D CNNs are hard to find, you could consider training your own on a similar, but distinct, dataset.
Here is a link for code for a 3D CNN in Lasagne. The authors use 3D CNN versions of VGG and Resnet.
Alternatively, you can perform 2D feature extraction on each slice of the volume and then combine the features for each slice, using PCA to reduce the dimensionality to something reasonable. For this, I recommend using ImageNet pre-trained Resnet-50 or VGG.
In Keras, these can be found here.
Assume a grey-scale 2D image which can mathematically be described as a matrix. Generalizing the concept of a matrix results in theory about tensors (informally you can think of a multidimensional array). I.e. a RGB 2D image is represented as a tensor of size [width, height, 3]. Further a RGB 3D Image is represented as a tensor of size [width, height, depth, 3]. Moreover and like in the case of matrices you can also perform tensor-tensor multiplications.
For instance consider the typical neural network with 2D images as input. Such a network does basically nothing else than matrix-matrix multiplications (despite of the elementwise non-linear operations at nodes). In the same way a neural network operates on tensors by performing tensor-tensor multiplications.
Now back to your question of feature extraction: Indeed the problem of tensors are their high dimensionality. Hence modern research problems regard the efficient decomposition of tensors retaining the initial (most meaningful) information. In order to extract features from tensors a tensor decomposition approach might be a good start in order to reduce the rank of the tensor. A few papers on tensors in machine learning are:
Tensor Decompositions for Learning Latent Variable Models
Supervised Learning With Quantum-Inspired Tensor Networks
Optimal Feature Extraction and Classification of Tensors via Matrix Product State Decomposition
Hope this helps, even though the math behind is not easy.
So basically I have a 2d array filled with 1s and 0s. There should be a linear slope associated with the 1s, and I need to find that linear slope with the best accuracy and quickness possible.
How could I do this (please note, there could be multiple lines, but they should all be of similar slope)? (picture of plot below)
I am trying to convert the following matlab code to c
cX = (fft(inData, fftSize) / fftSize);
Power_X = (cX*cX')/50;
Questions:
why divide the results of fft (array of fftSize complex elements) by
fftSize?
I'm not sure at all how to convert the complex conjugate
transform to c, I just don't understand what that line does.
Peter
1. Why divide the results of fft (array of fftSize complex elements) by fftSize?
Because the "energy" (sum of squares) of the resultant fft grows as the number of points in the fft grows. Dividing by the number of points "N" normalizes it so that the sum of squares of the fft is equal to the sum of squares of the original signal.
2. I'm not sure at all how to convert the complex conjugate transform to c, I just don't understand what that line does.
That is what is actually calculating the sum of the squares. It is easy to verify cX*cX' = sum(abs(cX)^2), where cX' is the conjugate transpose.
Ideally a Discrete Fourier Transform (DFT) is purely a rotation, in that it returns the same vector in a different coordinate system (i.e., it describes the same signal in terms of frequencies instead of in terms of sound volumes at sampling times). However, the way the DFT is usually implemented as a Fast Fourier Transform (FFT), the values are added together in various ways that require multiplying by 1/N to keep the scale unchanged.
Often, these multiplications are omitted from the FFT to save computing time and because many applications are unconcerned with scale changes. The resulting FFT data still contains the desired data and relationships regardless of scale, so omitting the multiplications does not cause any problems. Additionally, the correcting multiplications can sometimes be combined with other operations in an application, so there is no point in performing them separately. (E.g., if an application performs an FFT, does some manipulations, and performs an inverse FFT, then the combined multiplications can be performed once during the process instead of once in the FFT and once in the inverse FFT.)
I am not familiar with Matlab syntax, but, if Stuart’s answer is correct that cX*cX' is computing the sum of the squares of the magnitudes of the values in the array, then I do not see the point of performing the FFT. You should be able to calculate the total energy in the same way directly from iData; the transform is just a coordinate transform that does not change energy, except for the scaling described above.