I have a linear algebra algorithm written in Matlab and this code runs correctly.
I have implemented the same algorithm in C for obtaining better performance.
However, my C code has bugs.
In order to find bugs, I visually compare matrices generated by the two implementations at each step of the algorithm.
Is there a better way to compare output of Matlab and C codes? I have to work close to machine precision.
Related
I'm running the simulator with MATLAB.
However, it takes a few days.
Hence, I decided to change the code into C.
(First, I tried to use c-mex in MATLAB, but I think coding and debugging are very hard. mxType!!!?!?!? Thus, I decided to make C code using the visual studio 2017.)
In my MATLAB code, I used
x = [unifrnd(varmin(1),varmax(1),varnum,1),...
unifrnd(varmin(2),varmax(2),varnum,1),...
unifrnd(varmin(3),varmax(3),varnum,1)];
That is, x is the matrix of size varnum*3, whose 1st column is random numbers uniformly distributed from varmin(1) to varmax(1), 2nd column is random numbers uniformly distributed from varmin(2) to varmax(2), and 3rd column is random numbers uniformly distributed from varmin(3) to varmax(3).
When I make a matrix in C code, I will code like the follows:
srand(time(NULL));
for(j=1; j<3; j++) {
for(i=1; i<varnum; i++) {
x[i][j] = rand() % (varmax[j]-varmin[j]) + varmin[j];
}
}
I am changing my code from MATLAB to C because the running time is so long. However, I think MATLAB handles a matrix at once, but C handles a matrix of size MxN by running M*N iteration.
Nonetheless, is the code below (C) faster than the code above (MATLAB)?
Also, is there any better codes to make a matrix having random numbers?
Both MATLAB and C will run the random generation M*N times and you can't really get around that. In C it might be a bit more explicit, but in the end it's just an issue of typing rather than actual speed.
I would say that MATLAB is extremely fast if you use it the right way. That means leaning a lot on the built-in matrix operations and vectorizing the others as much as possible. The fact that it automatically parallelizes the computation for you is just topping on the cake.
If on the other hand you're doing a lot of manual for loops over large matrices you're likely to have problems. Mind you, newer versions of MATLAB know how to JIT that sort of code as well, but it's just not going to be as fast as a hyper-tuned builtin method.
So I'd avoid going outside MATLAB to C. At most do just some small functions in C where MATLAB really doesn't help you, but leave the rest in MATLAB.
Unless you really know what you're doing, you're unlikely to see a performance increase. In fact, I'll wager that you'll see a decrease in performance. And a guaranteed increase in development cost.
If resources permit I'd investigate getting a bigger machine for the team, or renting some compute resources from AWS/GCP. Though the logistics of getting MATLAB to work there might be prohibitive.
I am computing a statistical model in Matlab which has to run about 200 kalman filter per iteration, and I want to iterate the modelat least 10 000 times which suggest that I should run it at least 2 000 000 times. I am therefore searching for a way to optimize the computationnal speed of Matlab on this part. I have already gone operation per operation to try to optimize the computation in Matlab using all the tricks which could be used but I would like to go further...
I am not familiar with C/C++ but I read that mex-file could be usefull in some cases. Anyone could tell me if it would be Worth going into this direction ???
Thanks...
Writing mex files will definitely speed up the entire process, but you will not be able to use a lot of the built in MATLAB functions. You are limited to what you can do in C++ and C. Of course you can write your own functions, as long as you know how to.
The speed increase mainly comes from the fact the mex files are compiled and not interpreted line by line as are standard MATLAB scripts. Once you compile the mex you can call it the same way you do any other MATLAB function.
For a class I took in college I had to write my own image scaling function, I had initially written it in a standard script and it would take a couple seconds to complete on large images, but when I wrote it in C in a mex it would complete in less than 0.1 seconds.
MEX Files Documentation
If you you are not familiar at all with C/C++ this will be hard going. Hopefully you have some experience with another language besides Matlab? You can try learning/copying from the many included examples, but you'll really need to figure out the basics first.
One thing in particular. If you use mex you'll need some way to obtain decent random numbers for your Kalman filter noise. You might be surprised, but a very large percentage of your calculation time will likely be in generating random numbers for noise (depending on the complexity of your filter it could be > 50%).
Don't use the default random number generators in C/C++.
These are not suitable for scientific computation, especially when generating vast numbers of values as you seem to need. Your first option is to pass in a large array of random numbers generated via randn in Matlab to your mex code. Or look into including the C code Mersenne Twister algorithm itself and find/implement a scheme for generating normal random numbers from the uniform ones (log-polar is simplest, but Ziggurat will be faster). This is not too hard. I've done it myself and the Double precision SIMD-oriented Fast Mersenne Twister (dSFMT) is actually 2+ times faster than Matlab's current implementation for uniform variates.
You could use parfor loops or the parallel computing toolbox in general to speedup your calculations. Did you already checked whether MATLAB is using 100% CPU?
I would like to perform a non-linear optimization algorithm using C.
The problem is:
over the five points that are in vector X.
X, Y(X), lower and upper bounds are known.
I have found the nlopt library on C but I do not know if It is possible to perform the optimization over the five discrete points.
Anything to suggest, even another library?
Thanks!
I would suggest Octave. For nonlinear programming on Octave, refer to
Octave Optimization.
You could implement using matlab-like language.
It also has C/C++ api.
See this post: How to embed the GNU Octave in C/C++ program?.
And also, this pdf
Consider optimizing matlab code instead of reimplementing algorithm in another language - matlab can be pretty fast if optimized properly (avoid using for loop, use vectorized computations, pre-allocate memory).
Take a look at http://www.mathworks.com/company/newsletters/news_notes/june07/patterns.html
For these days I was working on C-mex code in order to improve speed in DBSCAN matlab code. In fact, at the moment I finished a DBSCAN on C-mex. But instead, it takes more time (14.64 seconds in matlab, 53.39 seconds in C-Mex) with my test data which is a matrix 3 x 14414. I think this is due to the use of mxRealloc function in several parts of my code. Would be great that someone give me some suggestion with the aim to get better results.
Here is the code DBSCAN1.c:
https://www.dropbox.com/sh/mxn757a2qmniy06/PmromUQCbO
Using mxRealloc in every iteration of a loop is indeed a performance killer. You can use vector or similar class instead. Dynamic allocation is not needed at all in your distance function.
If your goal is not to implement DBSCAN as a mex but to speed it up, I will offer you a different solution.
I don't know which Matlab implementation are you using, but you won't make a trivial n^2 implementation much faster by just rewriting it to C in the same way. Most of the time is spent calculating the nearest neighbors which won't be faster in C than it is in Matlab. DBSCAN can run in nlogn time by using an index structure to get the nearest neighbors.
For my application, I am using this implementation of dbscan, but I have changed the calculation of nearest neighbors to use a KD-tree (available here). The speedup was sufficient for my application and no reimplementation was required. I think this will be faster than any n^2 c implementation no matter how good you write it.
My question may seem primitive or dumb because, I've just switched to C.
I have been working with MATLAB for several years and I've learned that any computation should be vectorized in MATLAB and I should avoid any for loop to get an acceptable performance.
It seems that if I want to add two vectors, or multiply matrices, or do any other matrix computation, I should use a for loop.
It is appreciated if you let me know whether or not there is any way to do the computations in a vectorized sense, e.g. reading all elements of a vector using only one command and adding those elements to another vector using one command.
Thanks
MATLAB suggests you to avoid any for loop because most of the operations available on vectors and matrices are already implements in its API and ready to be used. They are probably optimized and they work directly on underlying data instead that working at MATLAB language level, a sort of opaque implementation I guess.
Even MATLAB uses for loops underneath to implement most of its magic (or delegates them to highly specialized assembly instructions or through CUDA to the GPU).
What you are asking is not directly possible, you will need to use loops to work on vectors and matrices, actually you would search for a library which allows you to do most of the work without directly using a for loop but by using functions already defined that wraps them.
As it was mentioned, it is not possible to hide the for loops. However, I doubt that the code MATLAB produces is in any way faster the the one produced by C. If you compile your C code with the -O3 it will try to use every hardware feature your computer has available, such as SIMD extensions and multiple issue. Moreover, if your code is good and it doesn't cause too many pipeline stalls and you use the cache, it will be really fast.
But i think what you are looking for are some libraries, search google for LAPACK or BLAS, they might be what you are looking for.
In C there is no way to perform operations in a vectorized way. You can use structures and functions to abstract away the details of operations but in the end you will always be using fors to process your data.
As for speed C is a compiled language and you will not get a performance hit from using for loops in C. C has the benefit (compared to MATLAB) that it does not hide anything from you, so you can always see where your time is being used. On the downside you will notice that things that MATLAB makes trivial (svd,cholesky,inv,cond,imread,etc) are challenging in C.