Recently, I tried to write mexfunctions using structure variables.
I watched the tutorial but got confused because of how the variable values are passed.
The following example (mexfunction_using_ex_wrong.m & mexfunction_using_ex_wrong.cpp) demonstrates how to fetch the variables passed from matlab in mexfunction.
However, in this case, the result is:
address i_c1=2067094464 i_c2=2067094464
i_c1=10 i_c2=10
address i_c1=1327990656 i_c2=2067100736
i_c1=2 i_c2=20
address i_c1=2067101056 i_c2=2067063424
i_c1=3 i_c2=30
As can be seen, the 1st element of the c1 & c2 array of a structure variable is accidentally the same.
But, in another example (mexfunction_using_ex_correct.m & mexfunction_using_ex_correct.cpp), the elements of array 1 (b1) and array 2(b2) of a structure variable are unrelated as I expect.
The result is:
address i_b1=1978456576 i_b2=1326968576
i_b1=1 i_b2=10
address i_b1=1978456584 i_b2=1326968584
i_b1=2 i_b2=20
address i_b1=1978456592 i_b2=1326968592
i_b1=3 i_b2=30
However, it's more common to use the 1st example in programming. so could anybody explain why in the 1st example the addresses of i_c1 & i_c2 are the same?
The following code is mexfunction_using_ex_wrong.m
clc
clear all
close all
mex mexfunction_using_ex_c_wrong.cpp;
a.b(1).c1=double(1);
a.b(2).c1=double(2);
a.b(3).c1=double(3);
a.b(1).c2=double(1);
a.b(2).c2=double(2);
a.b(3).c2=double(3);
mexfunction_using_ex_c_wrong(a);
The following code is mexfunction_using_ex_c_wrong.cpp
#include "mex.h"
void mexFunction(int nlhs,mxArray *plhs[],int nrhs,const mxArray *prhs[])
{
int i, j, k;
double *i_c1;
double *i_c2;
// for struct variables(pointers) inside fcwcontext
mxArray *mx_b, *mx_c1, *mx_c2;
mx_b=mxGetField(prhs[0], 0, "b");
for(i = 0;i < 3;i=i+1)
{
mx_c1=mxGetField(mx_b, i, "c1");
mx_c2=mxGetField(mx_b, i, "c2");
i_c1=mxGetPr(mx_c1);
i_c2=mxGetPr(mx_c2);
*i_c2=(*i_c2)*10;
printf("address i_c1=%d i_c2=%d\n", i_c1, i_c2);
printf(" i_c1=%g i_c2=%g\n", *i_c1, *i_c2);
}
}
The following code is mexfunction_using_ex_c_correct.m
clc
clear all
close all
mex mexfunction_using_ex_correct.cpp;
a.b1(1)=double(1);
a.b1(2)=double(2);
a.b1(3)=double(3);
a.b2(1)=double(1);
a.b2(2)=double(2);
a.b2(3)=double(3);
mexfunction_using_ex_correct(a);
The following code is mexfunction_using_ex_c_correct.cpp
#include "mex.h"
void mexFunction(int nlhs,mxArray *plhs[],int nrhs,const mxArray *prhs[])
{
int i, j, k;
double *i_b1;
double *i_b2;
mxArray *mx_b1, *mx_b2;
mx_b1=mxGetField(prhs[0], 0, "b1");
mx_b2=mxGetField(prhs[0], 0, "b2");
for(i = 0;i < 3;i=i+1)
{
i_b1=mxGetPr(mx_b1);
i_b2=mxGetPr(mx_b2);
i_b2[i]=i_b2[i]*10;
printf("address i_b1=%d i_b2=%d\n", &i_b1[i], &i_b2[i]);
printf(" i_b1=%g i_b2=%g\n", i_b1[i], i_b2[i]);
}
}
The addresses are not "accidentally the same" - they're intentionally the same, due to MATLAB's internal copy-on-write optimisations. If you look at the MEX documentation, you'll see warnings scattered around...
Do not modify any prhs values in your MEX-file. Changing the data in these read-only mxArrays can produce undesired side effects.
...in various forms...
Note Inputs to a MEX-file are constant read-only mxArrays. Do not modify the inputs. Using mxSetCell* or mxSetField* functions to modify the cells or fields of a MATLAB® argument causes unpredictable results.
...trying to make it very clear that you should absolutely not modify anything you recieve as an input. By calling mxGetPr() on input data and writing back to that pointer as you do with i_b2 and i_c2, you're getting right into that "unpredictable results" territory - if you look at a.b(1).c1 in the MATLAB workspace after the call, it'll really be 10 even though you "only" changed c2.
From MEX, you're looking at the raw data storage without any knowledge of, or access to, MATLAB's internal housekeeping, so the only safe way to modify anything is to use the mxCreate* or mxDuplicate* functions to get your own safe arrays you can then do whatever you want with, and pass back to MATLAB via plhs.
That said, I will admit to having abused in-place modification for a significant performance gain in one instance where I could guarantee my data was unique and unshared, but it's at best unsupported and at worst downright perilous.
Related
I am using the legacy_code tool in MATLAB, to generate some S Functions, then I want the S Functions to be under analysis by the simulink coverage toolbox.
I am asking also here because maybe this is a C issue and not MATLAB related.
I am setting to true the flag to use the coverage toolbox when generating the S functions using the legacy_code tool.
def.Options.supportCoverage = true;
But I get the following error at compilation, I am using the MinGW 64 bits compiler for MATLAB in windows.
“lib_control.c", line 254: error: bad expr node kind (b:\matlab\polyspace\src\shared\cxx_front_end_kernel\edg\src\cp_gen_be.c, line 14084)
Warning: File "lib_control.c" not instrumented for coverage because of previous error
In codeinstrum.internal.LCInstrumenter/instrumentAllFiles
In codeinstrum.internal.SFcnInstrumenter/instrument
In slcovmexImpl
In slcovmex (line 48)
In legacycode.LCT/compile
In legacycode.LCT.legacyCodeImpl
In legacy_code (line 101)
In generate_sfun (line 70)
In the C code I have the following kind of functions:
void controller( int n_var,
double my_input,
double my_output )
{
double my_var[n_var];
for ( int i=0; i<n_var; i++ )
{
my_output = my_input + my_var[i];
}
}
The compiler is complaining about this line:
double my_var[n_var];
Do I have to do something to declare variables like this, so they can be included in the coverage analysis?
Is this error from MATLAB or is it a C error for instrumentation of files?
If I compile without the coverage flag there is no problems and the S Functions is generated without warnings.
Seems your code won't work because of issues.
First try to declare my_var like this
double *my_var = malloc(n_var * sizeof(double));
memset(my_var, 0, n_var * sizeof(double));
This is the correct way to allocate memory according to function parameter.
And there is also an issue.
my_output = my_input + my_var[i];
So it is correct solution.
*my_output = *my_input + my_var[i];
You are going to change value of parameter which is stack register variable
In C language, parameters are saved in to stack register so it will be freed after function ends.
so it won't reflect any changes
To do this, you need to send pointer of variable as parameter
void controller( int n_var,
double *my_input,
double *my_output ) {
*my_output = ....; // like this
}
and in caller side, you can do like this.
double a, b;
controller(10, &a, &b);
Hope this helps you
I am using FreeRtos and have a Task that I am using multiple times. I am calling static functions within the Task. During the multiple uses of this task, will the static functions be overwritten with one another's data or cause some type of issue? Eg...
static int addSomeNumbers(int x)
{
int num1 = 4;
int num2 = 83;
int num3 = 65;
return x+num1+num2+num3;
}
void SomeTask(void *pvParameters)
{
int localInt = *(int *) pvParameters;
addSomeNumbers(localInt);
//continue
}
I use this Task three times
xTaskCreate(SomeTask, "SomeTask0", stackDepth, someNumber0, ...)
xTaskCreate(SomeTask, "SomeTask1", stackDepth, someNumber1, ...)
xTaskCreate(SomeTask, "SomeTask2", stackDepth, someNumber2, ...)
The problem I am running into, is that the data I see coming out of each Task is not consistent. It seems that the second Task sometimes is using the first task's data within addSomeNumbers and so on. Any way to prevent this?
I don't believe this is the case.
Static functions
are only about the visibility scope of the function, but not about the storage class of variables used therein.
If this is your actual code (and you haven't declared num1 etc. as static int, the nums will be re-allocated on each task stack, and shall not see "each other" (unless you managed to corrupt the kernel by some unrelated error behaviour...).
I would like to be able to use my own memory allocation function for certain data structures (real valued vectors and arrays) in R. The reason for this is that I need my data to be 64bit aligned and I would like to use the numa library for having control over which memory node is used (I'm working on compute nodes with four 12-core AMD Opteron 6174 CPUs).
Now I have two functions for allocating and freeing memory: numa_alloc_onnode and numa_free (courtesy of this thread). I'm using R version 3.1.1, so I have access to the function allocVector3 (src/main/memory.c), which seems to me as the intended way of adding a custom memory allocator. I also found the struct R_allocator in src/include/R_ext
However it is not clear to me how to put these pieces together. Let's say, in R, I want the result res of an evaluation such as
res <- Y - mean(Y)
to be saved in a memory area allocated with my own function, how would I do this? Can I integrate allocVector3 directly at the R level? I assume I have to go through the R-C interface. As far as I know, I cannot just return a pointer to the allocated area, but have to pass the result as an argument. So in R I call something like
n <- length(Y)
res <- numeric(length=1)
.Call("R_allocate_using_myalloc", n, res)
res <- Y - mean(Y)
and in C
#include <R.h>
#include <Rinternals.h>
#include <numa.h>
SEXP R_allocate_using_myalloc(SEXP R_n, SEXP R_res){
PROTECT(R_n = coerceVector(R_n, INTSXP));
PROTECT(R_res = coerceVector(R_res, REALSXP));
int *restrict n = INTEGER(R_n);
R_allocator_t myAllocator;
myAllocator.mem_alloc = numa_alloc_onnode;
myAllocator.mem_free = numa_free;
myAllocator.res = NULL;
myAllocator.data = ???;
R_res = allocVector3(REALSXP, n, myAllocator);
UNPROTECT(2);
}
Unfortunately I cannot get beyond a variable has incomplete type 'R_allocator_t' compilation error (I had to remove the .data line since I have no clue as to what I should put there). Does any of the above code make sense? Is there an easier way of achieving what I want to? It seems a bit odd to have to allocate a small vector in R and the change its location in C just to be able to both control the memory allocation and have the vector available in R...
I'm trying to avoid using Rcpp, as I'm modifying a fairly large package and do not want to convert all C calls and thought that mixing different C interfaces could perform sub-optimally.
Any help is greatly appreciated.
I made some progress in solving my problem and I would like to share in case anyone else encounters a similar situation. Thanks to Kevin for his comment. I was missing the include statement he mentions. Unfortunately this was only one among many problems.
dyn.load("myAlloc.so")
size <- 3e9
myBigmat <- .Call("myAllocC", size)
print(object.size(myBigmat), units = "auto")
rm(myBigmat)
#include <R.h>
#include <Rinternals.h>
#include <R_ext/Rallocators.h>
#include <numa.h>
typedef struct allocator_data {
size_t size;
} allocator_data;
void* my_alloc(R_allocator_t *allocator, size_t size) {
((allocator_data*)allocator->data)->size = size;
return (void*) numa_alloc_local(size);
}
void my_free(R_allocator_t *allocator, void * addr) {
size_t size = ((allocator_data*)allocator->data)->size;
numa_free(addr, size);
}
SEXP myAllocC(SEXP a) {
allocator_data* my_allocator_data = malloc(sizeof(allocator_data));
my_allocator_data->size = 0;
R_allocator_t* my_allocator = malloc(sizeof(R_allocator_t));
my_allocator->mem_alloc = &my_alloc;
my_allocator->mem_free = &my_free;
my_allocator->res = NULL;
my_allocator->data = my_allocator_data;
R_xlen_t n = asReal(a);
SEXP result = PROTECT(allocVector3(REALSXP, n, my_allocator));
UNPROTECT(1);
return result;
}
For compiling the c code, I use R CMD SHLIB -std=c99 -L/usr/lib64 -lnuma myAlloc.c. As far as I can tell, this works fine. If anyone has improvements/corrections to offer, I'd be happy to include them.
One requirement from the original question that remains unresolved is the alignment issue. The block of memory returned by numa_alloc_local is correctly aligned, but other fields of the new VECTOR_SEXPREC (eg. the sxpinfo_struct header) push back the start of the data array. Is it somehow possible to align this starting point (the address returned by REAL())?
R has, in memory.c:
main/memory.c
84:#include <R_ext/Rallocators.h> /* for R_allocator_t structure */
so I think you need to include that header as well to get the custom allocator (RInternals.h merely declares it, without defining the struct or including that header)
I have implemented a facade pattern that uses C functions underneath and I would like to test it properly.
I do not really have control over these C functions. They are implemented in a header. Right now I #ifdef to use the real headers in production and my mock headers in tests. Is there a way in C to exchange the C functions at runtime by overwriting the C function address or something? I would like to get rid of the #ifdef in my code.
To expand on Bart's answer, consider the following trivial example.
#include <stdio.h>
#include <stdlib.h>
int (*functionPtr)(const char *format, ...);
int myPrintf(const char *fmt, ...)
{
char *tmpFmt = strdup(fmt);
int i;
for (i=0; i<strlen(tmpFmt); i++)
tmpFmt[i] = toupper(tmpFmt[i]);
// notice - we only print an upper case version of the format
// we totally disregard all but the first parameter to the function
printf(tmpFmt);
free(tmpFmt);
}
int main()
{
functionPtr = printf;
functionPtr("Hello world! - %d\n", 2013);
functionPtr = myPrintf;
functionPtr("Hello world! - %d\n", 2013);
return 0;
}
Output
Hello World! - 2013
HELLO WORLD! - %D
It is strange that you even need an ifdef-selected header. The code-to-test and your mocks should have the exact same function signatures in order to be a correct mock of the module-to-test. The only thing that then changes between a production-compilation and a test-compilation would be which .o files you give to the linker.
It is possible With Typemock Isolator++ without creating unnecessary new levels of indirection. It can be done inside the test without altering your production code. Consider the following example:
You have the Sum function in your code:
int Sum(int a, int b)
{
return a+b;
}
And you want to replace it with Sigma for your test:
int Sigma(int a, int b)
{
int sum = 0;
for( ; 0<a ; a--)
{
sum += b;
}
return sum;
}
In your test, mock Sum before using it:
WHEN_CALLED: call the method you want to fake.
ANY_VAL: specify the args values for which the mock will apply. in this case any 2 integers.
*DoStaticOrGlobalInstead: The alternative behavior you want for Sum.
In this example we call Sigma instead.
TEST_CLASS(C_Function_Tests)
{
public:
TEST_METHOD(Exchange_a_C_function_implementation_at_run_time_is_Possible)
{
void* context = NULL; //since Sum global it has no context
WHEN_CALLED(Sum (ANY_VAL(int), ANY_VAL(int))).DoStaticOrGlobalInstead(Sigma, context);
Assert::AreEqual(2, Sum(1,2));
}
};
*DoStaticOrGlobalInstead
It is possible to set other types of behaviors instead of calling an alternative method. You can throw an exception, return a value, ignore the method etc...
For instance:
TEST_METHOD(Alter_C_Function_Return_Value)
{
WHEN_CALLED(Sum (ANY_VAL(int), ANY_VAL(int))).Return(10);
Assert::AreEqual(10, Sum(1,2));
}
I don't think it's a good idea to overwrite functions at runtime. For one thing, the executable segment may be set as read-only and even if it wasn't you could end up stepping on another function's code if your assembly is too large.
I think you should create something like a function pointer collection for the one and the other set of implementations you want to use. Every time you want to call a function, you'll be calling from the selected function pointer collection. Having done that, you may also have proxy functions (that simply call from the selected set) to hide the function pointer syntax.
I have a legacy C Linux application that I need to reuse . This application uses a lot of global variables. I want to reuse this application's main method and invoke that in a loop. I have found that when I call the main method( renamed to callableMain) in a loop , the application behavior is not consistent as the values of global variables set in previous iteration impact the program flow in the new iteration.
What I would like to do is to reset all the global variables to the default value before the execution of the the new iteration.
for example , the original program is like this
OriginalMain.C
#include <stdio.h>
int global = 3; /* This is the global variable. */
void doSomething(){
global++; /* Reference to global variable in a function. */
}
// i want to rename this main method to callableMain() and
// invoke it in a loop
int main(void){
if(global==3) {
printf(" All Is Well \n");
doSomething() ;
}
else{
printf(" Noooo\n");
doNothing() ;
}
return 0;
}
I want to change this program as follows:
I changed the above file to rename the main() to callableMain()
And my new main methods is as follows:
int main(){
for(int i=0;i<20;i++){
callableMain();
// this is where I need to reset the value of global vaiables
// otherwise the execution flow changes
}
}
Is this possible to reset all the global variables to the values before main() was invoked ?
The short answer is that there is no magical api call that would reset global variables. The global variables would have to be cached and reused.
I would invoke it as a subprocess, modifying its input and output as needed. Let the operating system do the dirty work for you.
The idea is to isolate the legacy program from your new program by relegating it to its own process. Then you have a clean separation between the two. Also, the legacy program is reset to a clean state every time you run it.
First, modify the program so that it reads the input data from a file, and writes its output in a machine-readable format to another file, with the files being given on the command line.
You can then create named pipes (using the mkfifo call) and invoke the legacy program using system, passing it the named pipes on the command line. Then you feed it its input and read back its output.
I am not an expert on these matters; there is probably a better way of doing the IPC. Others here have mentioned fork. However, the basic idea of separating out the legacy code and invoking it as a subprocess is probably the best approach here.
fork() early?
You could fork(2) at some early point when you think the globals are in a good state, and then have the child wait on a pipe or something for some work to do. This would require writing any changed state or at least the results back to the parent process but would decouple your worker from your primary control process.
In fact, it might make sense to fork() at least twice, once to set up a worker controller and save the initialized (but not too initialized :-) global state, and then have this worker controller fork() again for each loop you need run.
A simpler variation might be to just modify the code so that the process can start in a "worker mode", and then use fork() or system() to start the application at the top, but with an argument that puts it in to the slave mode.
There is a way to do this on certain platforms / compilers, you'd basically be performing the same initialization your compiler performs before calling main().
I have done this for a TI DSP, in that case I had the section with globals mapped to a specific section of memory and there were linker directives available that declared variables pointing to the start and end of this section (so you can memset() the whole area to zero before starting initialization). Then, the compiler provided a list of records, each of which comprised of an address, data length and the actual data to be copied into the address location. So you'd just loop through the records and do memcpy() into the target address to initialize all globals.
Very compiler specific, so hopefully the compiler you're using allows you to do something similar.
In short, no. What I would do in this instance is create definitions, constants if you will, and then use those to reset the global variables with.
Basically
#define var1 10
int vara = 10
etc... basic C right?
You can then go ahead and wrap the reinitialization in a handy function =)
I think you must change the way you see the problem.
Declare all the variables used by callableMain() inside callableMain()'s body, so they are not global anymore and are destroyed after the function is executed and created once again with the default values when you call callableMain() on the next iteration.
EDIT:
Ok, here's what you could do if you have the source code for callableMain(): in the beginning of the function, add a check to verify if its the first time the function its being called. Inside this check you will copy the values of all global variables used to another set of static variables (name them as you like). Then, on the function's body replace all occurences of the global variables by the static variables you created.
This way you will preserve the initial values of all the global variables and use them on every iteration of callableMain(). Does it makes sense to you?
void callableMain()
{
static bool first_iter = true;
if (first_iter)
{
first_iter = false;
static int my_global_var1 = global_var1;
static float my_global_var2 = global_var2;
..
}
// perform operations on my_global_var1 and my_global_var2,
// which store the default values of the original global variables.
}
for (int i = 0; i < 20; i++) {
int saved_var1 = global_var1;
char saved_var2 = global_var2;
double saved_var3 = global_var3;
callableMain();
global_var1 = saved_var1;
global_var2 = saved_var2;
global_var3 = saved_var2;
}
Or maybe you can find out where global variables start memcpy them. But I would always cringe when starting a loop ...
for (int i = 0; i < 20; i++) {
static unsigned char global_copy[SIZEOFGLOBALDATA];
memcpy(global_copy, STARTOFGLOBALDATA, SIZEOFGLOBALDATA);
callableMain();
memcpy(STARTOFGLOBALDATA, global_copy, SIZEOFGLOBALDATA);
}
If you don't want to refactor the code and encapsulate these global variables, I think the best you can do is define a reset function and then call it within the loop.
Assuming we are dealing with ELF on Linux, then the following function to reset the variables works
// these extern variables come from glibc
// https://github.com/ysbaddaden/gc/blob/master/include/config.h
extern char __data_start[];
extern char __bss_start[];
extern char _end[];
#define DATA_START ((char *)&__data_start)
#define DATA_END ((char *)&__bss_start)
#define BSS_START ((char *)&__bss_start)
#define BSS_END ((char *)&_end)
/// first call saves globals, subsequent calls restore
void reset_static_data();
// variable for quick check
static int pepa = 42;
// writes to memory between global variables are reported as buffer overflows by asan
ATTRIBUTE_NO_SANITIZE_ADDRESS
void reset_static_data()
{
// global variable, ok to leak it
static char * x;
size_t s = BSS_END - DATA_START;
// memcpy is always sanitized, so access memory as chars in a loop
if (x == NULL) { // store current static variables
x = (char *) malloc(s);
for (size_t i = 0; i < s; i++) {
*(x+i) = *(DATA_START + i);
}
} else { // restore previously saved static variables
for (size_t i = 0; i < s; i++) {
*(DATA_START + i) = *(x+i);
}
}
// quick check, see that pepa does not grow in stderr output
fprintf(stderr, "pepa: %d\n", pepa++);
}
The general approach is based on answer in How to get the data and bss address space in run time (In Unix C program), see the linked ysbaddaden/gc GitHub repo for macOS version of the macros.
To test the above code, just call it a few times and note that the incremented global variable pepa still keeps the value of 42.
reset_static_data();
reset_static_data();
reset_static_data();
Saving current state of the globals is convenient in that it does not require rerunning __attribute__((constructor)) functions which would be necessary if I set everything in .bss to zero (which is easy) and everything in .data to the initial values (which is not so easy). For example, if you load libpython3.so in your program, it does do run-time initialization which is lost by zeroing .bss. Calling into Python then crashes.
Sanitizers
Writing into areas of memory immediately before or after a static variable will trigger buffer-overflow warning from Address Sanitizer. To prevent this, use the ATTRIBUTE_NO_SANITIZE_ADDRESS macro the way the code above does. The macro is defined in sanitizer/asan_interface.h.
Code coverage
Code coverage counters are implemented as global variables. Therefore, resetting globals will cause coverage information to be forgotten. To solve this, always dump the coverage-to-date before restoring the globals. There does not seem to be a macro to detect whether code coverage is enabled or not in the compiler, so use your build system (CMake, ...) to define suitable macro yourself, such as QD_COVERAGE below.
// The __gcov_dump function writes the coverage counters to gcda files
// and the __gcov_reset function resets them to zero.
// The interface is defined at https://github.com/gcc-mirror/gcc/blob/7501eec65c60701f72621d04eeb5342bad2fe4fb/libgcc/libgcov-interface.c
extern "C" void __gcov_reset();
extern "C" void __gcov_dump();
void flush_coverage() {
#if defined(QD_COVERAGE)
__gcov_dump();
__gcov_reset();
#endif
}