I have 2 fixed length arrays of doubles. I want to get a similarity measure of these 2 arrays. The closer each elements in the arrays are the more similar the 2 arrays are.
What kind of metric or algorithm is there to achieve that?
Simplest and universal is "Sum of Squared Differences"
In case of scaled data can help "Normalized cross correlation"
Also you can read about different "norm" L1, L2, INF and others. All it depend from your practical task.
You can use cosine similarity, which is a measure insensitive to the magnitude of data, as it calculates the angle between the vectors.
See also: https://en.wikipedia.org/wiki/Cosine_similarity
Related
I am now interested in the bundle adjustment in SLAM, where the Rodrigues vectors $R$ of dimension 3 are used as part of variables. Assume, without loss of generality, we use Gauss-Newton method to solve it, then in each step we need to solve the following linear least square problem:
$$J(x_k)\Delta x = -F(x_k),$$
where $J$ is the Jacobi of $F$.
Here I am wondering how to calculate the derivative $\frac{\partial F}{\partial R}$. Is it just like the ordinary Jacobi in mathematic analysis? I have this wondering because when I look for papers, I find many other concepts like exponential map, quaternions, Lie group and Lie algebra. So I suspect if there is any misunderstanding.
This is not an answer, but is too long for a comment.
I think you need to give more information about how the Rodrigues vector appears in your F.
First off, is the vector assumed to be of unit length.? If so that presents some difficulties as now it doesn't have 3 independent components. If you know that the vector will lie in some region (eg that it's z component will always be positive), you can work round this.
If instead the vector is normalised before use, then while you could then compute the derivatives, the resulting Jacobian will be singular.
Another approach is to use the length of the vector as the angle through which you rotate. However this means you need a special case to get a rotation through 0, and the resulting function is not differentiable at 0. Of course if this can never occur, you may be ok.
Say I have some array of length n where arr[k] represents how much of object k I want. I also have some arbitrary number of arrays which I can sum integer multiples of in any combination - my goal being to minimise the sum of the absolute differences across each element.
So as a dumb example if my target was [2,1] and my options were A = [2,3] and B = [0,1], then I could take A - 2B and have a cost of 0
I’m wondering if there is an efficient algorithm for approximating something like this? It has a weird knapsack-y flavour to is it maybe just intractable for large n? It doesn’t seem very amenable to DP methods
This is the (NP-hard) closest vector problem. There's an algorithm due to Fincke and Pohst ("Improved methods for calculating vectors of short length in a lattice, including a complexity analysis"), but I haven't personally worked with it.
I have been developing a C language control software working in real time. The software implements among others discrete state space observer of the controlled system. For implementation of the observer it is necessary to calculate inverse of the matrix with 4x4 dimensions. The inverse matrix calculation has to be done each 50 microseconds and it is worthwhile to say that during this time period also other pretty time consuming calculation will be done. So the inverse matrix calculation has to consume much less than 50 microseconds. It is also necessary to say that the DSP used does not have ALU with floating point operations support.
I have been looking for some efficient way how to do that. One idea which I have is to prepare general formula for calculation the determinant of the matrix 4x4 and general formula for calculation the adjoint matrix of the 4x4 matrix and then calculate the inverse matrix according to below given formula.
What do you think about this approach?
As I understand the consensus among those who study numerical linear algebra, the advice is to avoid computing matrix inverses unnecessarily. For example if the inverse of A appears in your controller only in expressions such as
z = inv(A)*y
then it is better (faster, more accurate) to solve for z the equation
A*z = y
than to compute inv(A) and then multiply y by inv(A).
A common method to solve such equations is to factorize A into simpler parts. For example if A is (strictly) positive definite then the cholesky factorization finds lower triangular matrix L so that
A = L*L'
Given that we can solve A*z=y for z via:
solve L*u = y for u
solve L'*z = u for z
and each of these is easy given the triangular nature of L
Another factorization (that again only applies to positive definite matrices) is the LDL which in your case may be easier as it does not involve square roots. It is described in the wiki article linked above.
More general factorizations include the LUD and QR These are more general in that they can be applied to any (invertible) matrix, but are somewhat slower than cholesky.
Such factorisations can also be used to compute inverses.
To be pedantic describing adj(A) in your post as the adjoint is, perhaps, a little old fashioned; I thing adjugate or adjunct is more modern. In any case adj(A) is not the transpose. Rather the (i,j) element of adj(A) is, up to a sign, the determinant of the matrix obtained from A by deleting the i'th row and j'th column. It is awkward to compute this efficiently.
in a text I have found the following:
"The LASSO regerssion method offers a sparse solution and as such the interpretability of the model can be improved".
Can someone help me to understand what is meant by this? As far as I know, a sparse decomposition of a solution to a system of equation is that vector of dimension l with minimum pseudo-l norm such that the system is still satisfied. How would a sparse solution, which is setting some regression coefficients to zero, be of help in the interpretation?
Sparse matrix/array or whatever is by definition when your matrix contains mostly zeros and few non-zero entries. In the other hand, a dense matrix/array is when you have few zeros.
When you apply LASSO regression, the sparsity of your learned coefficients depends on the amount of the penalty (lambda). The higher the penalty, the more sparse coefficients you get. That is, the non-zero coefficients (selected variables). For example, if you have 100 independent variables in your regression, the LASSO may return only 10 non-zero variables. That means 10 non-zero variables and 90 zero variables. This is exactly what is the meaning of sparsity.
Having few selected variables (non-zero) means interpretable model as you can explain it with few variables (in the above example 10 variables) instead of using the 100 variables.
Lasso Regression's penalty method is different than Ridge, which is using L1 regularization. And there is "alpha" parameter which you can set it on scikit-learn. For high values of "alpha", many coefficients are exactly zeroed.
This method is also using absolute sum of the coeeficients ( |w| ). For instance, if there are high correlated features on your dataset, Lasso is doing one of the correlated predictor to largest coefficient, while the rest of them are set to 0
If there are two or more highly collinear variables then LASSO regression select one of them randomly (which is not good for the interpretation of data)
You can find more details here => https://www.geeksforgeeks.org/lasso-vs-ridge-vs-elastic-net-ml/
say there is a number of arrays with length 12, containing signed integers in a range of roughly ±100, how can i compare the 'signature' or 'harmonic content' of these arrays to each other, in a way that is more accurate than a simple weighted average?
Would i have to look into neural networks (if this even would be suitable, i don't know tbh) or is there a more straightforward way?
EDIT:
on a graph, imagine the index on the x-axis and the value on the y-axis. two arrays would be very similar if the shape of the graphs would be similar after normalization.
i find it hard to find the right words for this problem.
The arrays could be treated as signals, hence you should try to do a scaled cross correlation between the two signals, or extract frequency domain info and compare their spectrum ( FFT of both signals ).