Access violation when trying to populate an array of struct - c

Original code comment specifying the core question:
The error I am getting is while iterating through the while loop,
memory out of range or something... resizing to 300 ... Access
violation writing location that's the exact Fraze...
I'm trying to implement a faster .Net List<T> equivalent in C.
I'm using blittable data types in C#.
In the code below I've moved a function body to the main function just for testing after I have failed to understand where am I wrong.
The problem seems to be that inside the while loop UntArr does not increment.
What am I doing wrong?
typedef struct {
int Id;
char *StrVal;
}Unit; // a data unit as element of an array
unsigned int startTimer(unsigned int start);
unsigned int stopTimer(unsigned int start);
int main(){
Unit *UntArr= {NULL};
//Unit test[30000];
//decelerations comes first..
char *dummyStringDataObject;
int adummyNum,requestedMasterArrLength,requestedStringSize,MasterDataArrObjectMemorySize,elmsz;
int TestsTotalRounds, TestRoundsCounter,ccountr;
unsigned int start, stop, mar;
//Data Settings (manually for now)
requestedMasterArrLength=300;
requestedStringSize = 15;
//timings
start=0;stop=0;
//data sizes varies (x86/x64) compilation according to fastest results
MasterDataArrObjectMemorySize = sizeof(UntArr);
elmsz= sizeof(UntArr[0]);
TestRoundsCounter=-1;
start = startTimer(start);
while(++TestRoundsCounter<requestedMasterArrLength){
int count;
count=-1;
//allocate memory for the "Master Arr"
UntArr = (Unit *)malloc(sizeof(Unit)*requestedMasterArrLength);
dummyStringDataObject = (char*)malloc(sizeof(char)*requestedStringSize);
dummyStringDataObject = "abcdefgHijkLmNo";
while (++count<requestedMasterArrLength)
{
dummyStringDataObject[requestedStringSize-1]=count+'0';
puts(dummyStringDataObject);
ccountr=-1;
// tried
UntArr[count].Id = count;
UntArr[count].StrVal = (char*)malloc(sizeof(char)*requestedStringSize);
UntArr[count].StrVal = dummyStringDataObject;// as a whole
//while(++ccountr<15)// one by one cause a whole won't work ?
//UntArr[count].StrVal[ccountr] = dummyStringDataObject[ccountr];
}
free(UntArr);free(dummyStringDataObject);
}
stop = startTimer(start);
mar = stop - start;
MasterDataArrObjectMemorySize = sizeof(UntArr)/1024;
printf("Time taken in millisecond: %d ( %d sec)\r\n size: %d kb\r\n", mar,(mar/1000),MasterDataArrObjectMemorySize);
printf("UntArr.StrVal: %s",UntArr[7].StrVal);
getchar();
return 0;
}
unsigned int startTimer(unsigned int start){
start = clock();
return start;
}
unsigned int stopTimer(unsigned int start){
start = clock()-start;
return start;
}
testing the code one by one instead of within a while loop work as expected
//allocate memory for the "Master Arr"
UntArr = (Unit *)malloc(sizeof(Unit)*requestedMasterArrLength);
UntArr[0].Id = 0;
dummyStringDataObject = (char*)malloc(sizeof(char)*requestedStringSize);
dummyStringDataObject = "abcdefgHijkLmNo";
////allocate memory for the string object
UntArr[0].StrVal = (char*)malloc(sizeof(char)*requestedStringSize);
////test string manipulation
adummyNum=5;
UntArr[0].StrVal= dummyStringDataObject;
//
UntArr[0].StrVal[14] = adummyNum+'0';
////test is fine

as it happens and as i am new to pointers i have not realize that when debugging
i will not see the elements of given pointer to an array as i am used to
with normal Array[] but looping through result which i did not even try as when i was hovering above the Array* within the while loop expecting to see the elements as in a normal array:
Data[] DataArr = new Data[1000] <- i have expected to actually see the body of the array while looping and populating the Data* and did not realize it is not an actual array but a pointer to one so you can not see the elements/body.
the solution is via a function now as planed originally :
void dodata(int requestedMasterArrLength,int requestedStringSize){
int ccountr,count;
count=0;
UntArr=NULL;
UntArr = (Unit *)malloc(sizeof(Unit)*requestedMasterArrLength);
while(count!=requestedMasterArrLength)
{
char dummyStringDataObject[]= "abcdefgHi";
UntArr[count].StrVal=NULL;
dummyStringDataObject[requestedStringSize-1] = count+'0';
UntArr[count].Id= count;
ccountr=0;
UntArr[count].StrVal= (char*)malloc(sizeof(char)*requestedStringSize);
while(ccountr!=requestedStringSize){
UntArr[count].StrVal[ccountr] = dummyStringDataObject[ccountr];
++ccountr;
}
++count;
}
}
generaly speaking, x86 compilation would get better performance for this current task , populating an array of a struct.
so i have compiled it also in c++ and c#.
executing similar code in C# and C++
minimum time measured in c# ~ 3,100 ms.
minimum time measured in this code - C ~ 1700 ms.
minimum time measured in C++ ~ 900 ms.
i was surprised to see this last result c++ is the winner but why.
i thought c is closer to the system level, CPU, Memory...

Related

Using a c-program to read an NMEA-string

I am trying to make a c-program that will will a string, but I want it only to read a very small part of it.
The NMEA-telegram that I try to read is $WIXDR, and do receive the necessary strings.
Here's 2 examples of strings that I get into the CPU:
$WIXDR,C,1.9,C,0,H,83.2,P,0,P,1023.9,H,0*46
$WIXDR,V,0.01,M,0,Z,10,s,0,R,0.8,M,0,V,0.0,M,1,Z,0,s,1,R,0.0,M,1,R,89.9,M,2,R,0.0,M,3*60
If it were only 1 string (not both C and V), this would not be a problem for me.
The problem here is that it's 2 seperate strings. One with the temperature, and one with rain-info.
The only thing that I'm interested in is the value "1.9" from
$WIXDR,C,1.9,C,0......
Here's what I have so far:
void ProcessXDR(char* buffPtr)
{
char valueBuff[10];
int result, x;
float OutSideTemp;
USHORT uOutSideTemp;
// char charTemperature, charRain
IODBerr eCode;
//Outside Temperature
result = ReadAsciiVariable(buffPtr, &valueBuff[0], &buffPtr, sizeof(valueBuff));
sscanf(&valueBuff[0],"%f",&OutSideTemp);
OutSideTemp *= 10;
uOutSideTemp = (USHORT)OutSideTemp;
eCode = IODBWrite(ANALOG_IN,REG_COM_XDR,1,&uOutSideTemp,NULL);
}
// XDR ...
if(!strcmp(&nmeaHeader[0],"$WIXDR"))
{
if(PrintoutEnable)printf("XDR\n");
ProcessXDR(buffPtr);
Timer[TIMER_XDR] = 1200; // Update every minute
ComStateXDR = 1;
eCode = IODBWrite(DISCRETE_IN,REG_COM_STATE_XDR,1,&ComStateXDR,NULL);
}
There's more, but this is the main part that I have.
I have found the answer to my own question. The code that would do as I intented is as follows:
What my little code does, is to look for the letter C, and if the C is found, it will take the value after it and put it into "OutSideTemp". The reason I had to look for C is that there is also a similar string received with the letter V (Rain).
If someone have any input in a way it could be better, I don't mind, but this little piece here does what I need it to do.
Here's to example telegrams I receive (I wanted the value 3.0 to be put into "OutSideTemp"):
$WIXDR,C,3.0,C,0,H,59.2,P,0,P,1026.9,H,04F
$WIXDR,V,0.00,M,0,Z,0,s,0,R,0.0,M,0,V,0.0,M,1,Z,0,s,1,R,0.0,M,1,R,89.9,M,2,R,0.0,M,358
void ProcessXDR(char* buffPtr)
{
char valueBuff[10];
int result, x;
float OutSideTemp;
USHORT uOutSideTemp;
// char charTemperature, charRain
IODBerr eCode;
// Look for "C"
result = ReadAsciiVariable(buffPtr, &valueBuff[0], &buffPtr, sizeof(valueBuff));
// sscanf(&valueBuff[0],"%f",&charTemperature);
if (valueBuff[0] == 'C')
//Outside Temperature
result = ReadAsciiVariable(buffPtr, &valueBuff[0], &buffPtr, sizeof(valueBuff));
sscanf(&valueBuff[0],"%f",&OutSideTemp);
OutSideTemp *= 10;
uOutSideTemp = (USHORT)OutSideTemp;
eCode = IODBWrite(ANALOG_IN,REG_COM_XDR,1,&uOutSideTemp,NULL);
}

fprintf starts printing with strange characters then proceeds to print normally

I have run into a problem when trying to write to file using fprintf(). When I start writing into file the first few lines are composed of semi-random invalid characters and after that the rest is printed normally. I have no idea what can cause the problem. Below is the code that produces the problem.
functions called:
double calc_time(clock_t s, clock_t e){
return ((double)(e-s) / (sysconf(_SC_CLK_TCK)));
}
void print_times(FILE *f, char *operation, clock_t s, clock_t e, struct tms *st, struct tms *et){
printf("%s", operation);
printf("\nREAL TIME: %f\n", calc_time(s,e));
printf("USER TIME: %f\n", calc_time(st->tms_utime,et->tms_utime));
printf("SYS TIME: %f\n", calc_time(st->tms_stime,et->tms_stime));
printf("HERE");
fprintf(f,"%s", operation);
fprintf(f,"\nREAL TIME: %fl", calc_time(s,e));
fprintf(f,"\nUSER TIME: %fl", calc_time(st->tms_utime,et->tms_utime));
fprintf(f,"\nSYS TIME: %fl", calc_time(st->tms_stime,et->tms_stime));
}
and the printing takes place here:
f = fopen("raport2.txt","a");
clock_t r_times[2];
struct tms* t_times[2];
t_times[0] = calloc(1,sizeof(struct tms));
t_times[1] = calloc(1,sizeof(struct tms));
r_times[0] = times(t_times[0]);
struct block_array* array = create(4);
r_times[1] = times(t_times[1]);
print_times(f, "\nCreating array", r_times[0], r_times[1], t_times[0], t_times[1]);
r_times[0] = times(t_times[0]);
struct file_sequence seq = seq_def("t1.txt t2.txt b1.txt b2.txt");
char *tmp = compare(seq);
create_blocks(array,tmp,4);
r_times[1] = times(t_times[1]);
print_times(f,"\nCreating blocks",r_times[0],r_times[1],t_times[0],t_times[1]);
r_times[0] = times(t_times[0]);
delete_block(array,1);
delete_block(array,2);
delete_block(array,3);
delete_block(array,4);
r_times[1] = times(t_times[1]);
print_times(f, "\nDeleting blocks", r_times[0], r_times[1], t_times[0], t_times[1]);
The error was caused by a different part of code than provided in the snippets (although the github source will still reproduce it). It turns out that in a different part of program I was writing to a file without erasing it's previous contents (I thought that touch does that), then data retrieved from the file was causing memory leaks and all kinds allocation problems.
Use %lf to print doubles, rather than the typo %fl you're using.

Increase performance in Lua get_table slow

I would like to use the lua script do to some mathematic precalculations in my application i don't want to hardcode it. I use the LUA as a DLL linked libary. Caller program code languange is not C-based language.
The application is handling pretty big array. The array is normaly (25k-65k) * 8 double number array.
My target is:
put this array into the lua script using global variable
read back this array from the lua script
i would like to reach this action is less than 100ms.
Currently i tested with 28000 x 6 array but the time is 5 sec.
I am using lua_gettable function and iterating across the array, it is a huge amount of stack write and read.
My question is no have any other solution for that? I checked the API but maybe i skipped some function. Any possibilities to ask lua to put array subset into the stack? And of course the opposite way.
Thank you so much for any help and suggestion!
As suggested by DarkWiiPlayer, I believe the best way to achieve this in a reasonably fast speed would be to use Lua's userdata. I did an example using a class with a double matrix with [65536][65536][8] dimensions, as you said yours would be:
class MatrixHolder {
public:
double matrix[65536][65536][8];
};
Then, I created a method to create a new MatrixHolder and another one to perform an operation in one of the positions of the matrix (passing I, J and K as parameters).
static int newMatrixHolder(lua_State *lua) {
MatrixHolder* object;
size_t nbytes = sizeof(MatrixHolder);
object = static_cast<MatrixHolder*>(lua_newuserdata(lua, nbytes));
return 1;
}
static int performOperation(lua_State *lua) {
MatrixHolder* object = static_cast<MatrixHolder*>(lua_touserdata(lua, 1));
int i = luaL_checkinteger(lua, -3);
int j = luaL_checkinteger(lua, -2);
int k = luaL_checkinteger(lua, -1);
object->matrix[i][j][k] += 1.0;
lua_pushinteger(lua, object->matrix[i][j][k]);
return 1;
}
static const struct luaL_Reg matrixHolderLib [] = {
{"new", newMatrixHolder},
{"performOperation", performOperation},
{NULL, NULL} // - signals the end of the registry
};
In my computer, it executed the given Lua scripts in the following times:
m = matrixHolder.new()
i = matrixHolder.performOperation(m, 1,1,1);
j = matrixHolder.performOperation(m, 1,2,1);
i = matrixHolder.performOperation(m, 1,1,1);
~845 microseconds
for i = 1, 1000
do
m = matrixHolder.new()
i = matrixHolder.performOperation(m, 1,1,1);
j = matrixHolder.performOperation(m, 1,2,1);
i = matrixHolder.performOperation(m, 1,1,1);
end
~617 milliseconds
I'm unsure if it will serve your purpose, but it seems already way faster than the 5 seconds you mentioned. My computer is a 2,3 GHz 8-Core Intel Core i9 16 GB RAM, for comparison.

Can gcc/clang optimize initialization computing?

I recently wrote a parser generator tool that takes a BNF grammar (as a string) and a set of actions (as a function pointer array) and output a parser (= a state automaton, allocated on the heap). I then use another function to use that parser on my input data and generates a abstract syntax tree.
In the initial parser generation, there is quite a lot of steps, and i was wondering if gcc or clang are able to optimize this, given constant inputs to the parser generation function (and never using the pointers values, only dereferencing them) ? Is is possible to run the function at compile time, and embed the result (aka, the allocated memory) in the executable ?
(obviously, that would be using link time optimization, since the compiler would need to be able to check that the whole function does indeed have the same result with the same parameters)
What you could do in this case is have code that generates code.
Have your initial parser generator as a separate piece of code that runs independently. The output of this code would be a header file containing a set of variable definitions initialized to the proper values. You then use this file in your main code.
As an example, suppose you have a program that needs to know the number of bits that are set in a given byte. You could do this manually whenever you need:
int count_bits(uint8_t b)
{
int count = 0;
while (b) {
count += b & 1;
b >>= 1;
}
return count;
}
Or you can generate the table in a separate program:
int main()
{
FILE *header = fopen("bitcount.h", "w");
if (!header) {
perror("fopen failed");
exit(1);
}
fprintf(header, "int bit_counts[256] = {\n");
int count;
unsigned v;
for (v=0,count=0; v<256; v++) {
uint8_t b = v;
while (b) {
count += b & 1;
b >>= 1;
}
fprintf(header, " %d,\n" count);
}
fprintf(header, "};\n");
fclose(header);
return 0;
}
This create a file called bitcount.h that looks like this:
int bit_counts[256] = {
0,
1,
1,
2,
...
7,
};
That you can include in your "real" code.

External Functions and Parameter Size Limitation (C)

I am very much stuck in the following issue. Any help is very much appreciated!
Basically I have a program wich contains an array of structs and I am getting a segmentation error when I call an external function. The error only happens when I have more than 170 items on the array being passed.
Nothing on the function is processed. The program stops exactly when accessing the function.
Is there a limit for the size of the parameters that are passed to external functions?
Main.c
struct ratingObj {
int uid;
int mid;
double rating;
};
void *FunctionLib; /* Handle to shared lib file */
void (*Function)(); /* Pointer to loaded routine */
const char *dlError; /* Pointer to error string */
int main( int argc, char * argv[]){
// ... some code ...
asprintf(&query, "select mid, rating "
"from %s "
"where uid=%d "
"order by rand()", itable, uid);
if (mysql_query(conn2, query)) {
fprintf(stderr, "%s\n", mysql_error(conn2));
exit(1);
}
res2 = mysql_store_result(conn2);
int movieCount = mysql_num_rows(res2);
// withhold is a variable that defines a percentage of the entries
// to be used for calculations (generally 20%)
int listSize = round((movieCount * ((double)withhold/100)));
struct ratingObj moviesToRate[listSize];
int mvCount = 0;
int count =0;
while ((row2 = mysql_fetch_row(res2)) != NULL){
if(count<(movieCount-listSize)){
// adds to another table
}else{
moviesToRate[mvCount].uid = uid;
moviesToRate[mvCount].mid = atoi(row2[0]);
moviesToRate[mvCount].rating = 0.0;
mvCount++;
}
count++;
}
// ... more code ...
FunctionLib = dlopen("library.so", RTLD_LAZY);
dlError = dlerror();
if( dlError ) exit(1);
Function = dlsym( FunctionLib, "getResults");
dlError = dlerror();
(*Function)( moviesToRate, listSize );
// .. more code
}
library.c
struct ratingObj {
int uid;
int mid;
double rating;
};
typedef struct ratingObj ratingObj;
void getResults(struct ratingObj *moviesToRate, int listSize);
void getResults(struct ratingObj *moviesToRate, int listSize){
// ... more code
}
You are likely blowing up the stack. Move the array to outside of the function, i.e. from auto to static land.
Another option is that the // ... more code - array gets populated... part is corrupting the stack.
Edit 0:
After you posted more code - you are using C99 variable sized array on the stack - Bad IdeaTM. Think what happens when your data set grows to thousands, or millions, of records. Switch to dynamic memory allocation, see malloc(3).
You don't show us what listsize is, but I suppose it is a variable and not a constant.
What you are using are variable length arrays, VLA. These are a bit dangerous if they are too large since they usually allocated on the stack.
To work around that you can allocate such a beast dynamically
struct ratingObj (*movies)[listSize] = malloc(sizeof(*movies));
// ...
free(movies);
You'd then have in mind though that movies then is a pointer to array, so you have to reference with one * more than before.
Another, more classical C version would be
struct ratingObj * movies = malloc(sizeof(*movies)*listsize);
// ...
free(movies);

Resources