I have to use MPI API for send/receive matrices in my programs. To send a matrix I used the below syntax:
MPI_Send(matrix, ...) <- USE THIS
MPI_Send(&matrix, ...)
MPI_Send(&matrix[0][0], ...)
Similar to the last one, but untested: MPI_Send(&matrix[0],....
I saw in different examples the use of the above and I wonder if there is a difference between them or is any one going to cause errors on big data? I'm asking because I've heard a person who knows a little bit of HPC talking about using only the first syntax (enumerated above) but I didn't have time to ask him why and I cannot find the reason on the Internet, or I don't know where to look. All of the above works just find on little examples that I compiled but I have only 2 cores on my processor so I may not be able to see the problem.
I my understanding those examples point to the same memory address so what will be the problem ? Is there a MPI API related issue?
Thanks in advance
All the matrix references in your question point to the same address, albeit with different types. MPI_Send takes a void*, so it looses the type anyway. So all calls are practically equivalent.
If the type of matrix is changed from a 2D array to a pointer, then the above are no longer equivalent. E.g., if matrix is an int*, only the first and last option both compile and produce the required address. Therefore, I too would recommend using the first option - it's safer should the type of matrix changes.
With MPI_Send() you only specify the pointer of the data the matrix. It is recommended to store the matrix as a 1-dimentional array :
int *data = (int*)calloc(cols*rows,sizeof(int));
MPI_Send(data,cols*rows, ...);
When you use MPI_Recv() remember that you sent an array and you may for sure want to work with a matrix.
Related
I would like an opinion on a situation in which I find myself in my project. i basically have a structure composed of an int, an array of ints and an array of chars. I need to send this structure to another process. the problem is that these two arrays in the structure are dynamically allocated (practically they are pointers). What's the best way to do this?
I have two main ideas:
Use Pack and Unpack but this is very heavy (this sends in repeat often).
Concatene all three data in one single array and send it with a simple send.
Use MPI_Type_struct but I don't know the specific size of this arrays (it change often in all send).
Someone can help me please. I don't upload code because is very complex and long.
Of those three options Pack/Unpack is probably the best choice. (2) is basically what Pack/Unpack does. MPI_Type_struct won't work if elements inside the struct are dynamically allocated.
Another option is to simply send the arrays in separate messages. Avoids the extra buffer and the packing/unpacking but of course sends more messages, so it may be better or worse performance-wise. If the arrays are very small, it will probably be worse, otherwise it might not make much of a difference or even be faster. Try it out and measure what's best.
I am currently trying to write a linux driver and for this it is necessary that I understand some apis to make best use of them. Often I come across a pattern where I start digging into a funciton and end up at a point where the function reads:
returnType OperationX(args...) {
...
struct operations_t operations = get_operations();
if(operations->X)
return operations->X(args...)
}
Basically get_operations() returns a pointer to a global struct, which holds a pointer to the actual function running the operation.
I find it very tedious using the linux cross reference to dig into the different places and then actually understand which assignement actually takes place. Is there a better faster way?
AN example would be dma mapping.
git grep and cscope are your best friends.
By the way, DMA operations are filled either by platform code or in IOMMU implementations. I bet most probably you have lib/swiotlb.c in use for that.
I would like to implement some R package, written in C code.
The C code must:
take an array (of any type) as input.
produce array as output (of unpredictable size).
What is the best practice of implementing array passing?
At the moment C code is called with .C(). It accesses array directly from R, through pointer. Unfortunately same can't be done for output, as output dimensions need to be known in advance which is not true in my case.
Would it make sense to pass array from C to R through a file? For example, in ramfs, if using linux?
Update 1:
Here the exact same problem was discussed.
There, possible option of returning array with unknown dimensions was mentioned:
Splitting external function into two, at the point before calculating the array but after dimensions are known. First part would return dimensions, then empty array would be prepared, then second part would run and populate the array in R.
In my case full dimensions are known only once whole code is executed, so this method would mean running C code twice. Taking a guess on maximal array size isn't optional either.
Update 2: It seems only way to do this is to use .Call() instead, as power suggested. Here are few good examples: http://www.sfu.ca/~sblay/R-C-interface.ppt.
Thanks.
What is the best practice of implementing array passing?
Is the package already written in ANSI C? .C() would then be quick and easy.
If you are writing from scratch, I suggest .Call() and Rcpp. In this way, you can pass R objects to your C/C++ code.
Would it make sense to pass array through a file?
No
Read "Writing R Extensions".
So, i've been commissioned to translate some fortran subroutines into C. These subroutines are being called as part of the control flow of a large porgram based primarily in C.
I am translating the functions one at a time, starting with the functions that are found at the top of call stacks.
The problem I am facing is the hand-off of array data from C to fortran.
Suppose we have declared an array in c as
int* someCArray = (int*)malloc( 50 * 4 * sizeof(int) );
Now, this array needs to be passed down into a fortran subroutine to be filled with data
someFortranFunc( someCArray, someOtherParams );
when the array arrives in fortran land, it is declared as a variable sized matrix as such:
subroutine somefortranfunc(somecarray,someotherparams)
integer somefarray(50,*)
The problem is that fortran doesn't seem to size the array correctly, becuase the program seg-faults. When I debug the program, I find that indexing to
somefarray(1,2)
reports that this is an invalid index. Any references to any items in the first column work fine, but there is only one available column in the array when it arrives in fortran.
I can't really change the fact that this is a variable sized array in fortran. Can anyone explain what is happening here, and is there a way that I can mitigate the problem from the C side of things?
[edit]
By the way, the fortran subroutine is being called from the replaced fortran code as
integer somedatastorage(plentybignumber)
integer someindex
...
call somefarray(somedatastorage(someindex))
where the data storage is a large 1d array. There isn't a problem with overrunning the size of the data storage. Somehow, though, the difference between passing the C array and the fortran (sub)array is causing a difference in the fortran subroutine.
Thanks!
Have you considered the Fortran ISO C Binding? I've had very good results with it to interface Fortran and C in both directions. My preference is to avoid rewriting existing, tested code. There are a few types that can't be transferred with the current version of the ISO C Binding, so a translation might be necessary.
What it shouldn't be that others suggested:
1. Size of int vs. size of Integer. Since the first column has the right values.
2. Row vs. column ordering. Would just get values in wrong order not segmentation faulted.
3. Reference vs value passing. Since the first column has the right values. Unless the compiler is doing something evil behind your back.
Are you sure you don't do this in some secret way?:
someCArray++
print out the value of the someCArray pointer right after you make it and right before you pass it. You also should print it out using the debugger in the fortran code just to verify that the compiler is not generating some temporary copies to help you.
i recently came across a problem with my parallel program. Each process has several glib hashtables that need to be exchanged with other processes, these hashtables may be quite large. What is the best approach to achieve that?
create derived datatype
use mpi pack and unpack
send key & value as arrays (problem, since amount of elements is not known at compile time)
I haven't used 1 & 2 before and don't even know if thats possible, that's why i am asking you guys..
Pack/unpack creates a copy of your data: if your maps are large, you'll want to avoid that. This also rules out your 3rd option.
You can indeed define a custom datatype, but it'll be a little tricky. See the end of this answer for an example (replacing "graph" with "map" and "node" with "pair" as you read). I suggest you read up on these topics to get a firm understanding of what you need to do.
That the number of elements is not known at compile time shouldn't be a real issue. You can just send a message containing the payload size before sending the map contents. This will let the receiving process allocate just enough memory for the receive buffer.
You may also want to consider simply printing the contents of your maps to files, and then having the processes read each others' ouput. This is much more straightforward, but also less elegant and much slower than message passing.