While freeing some pointers, I get an access violation.
In order to know what's going on, I've decided to ask to free the pointers at an earlier stage in the code, even directly after memory has been allocated, and still it crashes.
It means that something is seriously wrong in the way my structures are handled in memory.
I know that in a previous version of the code, there was a keyword before the definition of some variables, but that keyword is lost (it was part of a #define clause I can't find back).
Does anybody know what's wrong in this piece of code or what the mentioned keyword should be?
typedef unsigned long longword;
typedef struct part_tag { struct part_tag *next;
__int64 fileptr;
word needcount;
byte loadflag,lock;
byte partdat[8192];
} part;
static longword *partptrs;
<keyword> part *freepart;
<keyword> part *firstpart;
void alloc_parts (void) {
part *ps;
int i;
partptrs = (longword*)malloc (number_of_parts * sizeof(longword)); // number... = 50
ps = (part*)&freepart;
for (i=0; i<number_of_parts; i++) {
ps->next = (struct part_tag*)malloc(sizeof(part));
partptrs[i] = (longword)ps->next;
ps = ps->next;
ps->fileptr = 0; ps->loadflag = 0; ps->lock = 0; ps->needcount = 0; // fill in "ps" structure
};
ps->next = nil;
firstpart = nil;
for (i=0; i<number_of_parts; i++) {
ps = (part*)partptrs[i];
free(ps); <-- here it already crashes at the first occurence (i=0)
};
}
Thanks in advance
In the comments somebody asks why I'm freeing pointers directly after allocating them. This is not how the program originally was written, but in order to know what's causing the access violation I've rewritten in that style.
Originally:
alloc_parts();
<do the whole processing>
free_parts();
In order to analyse the access violation I've adapted the alloc_parts() function into the source code excerpt I've written there. The point is that even directly after allocating memory, the freeing is going wrong. How is that even possible?
In the meanwhile I've observed another weird phenomena:
While allocating the memory, the values of ps seem to be "complete" address values. While trying to free the memory, the values of ps only contain the last digits of the memory addresses.
Example of complete address : 0x00000216eeed6150
Example of address in freeing loop : 0x00000000eeed6150 // terminating digits are equal,
// so at least something is right :-)
This problem was caused by the longword type: it seems that this type was too small to hold entire memory addresses. I've replaced this by another type (unsigned long long) but the problem still persists.
Finally, after a long time of misery, the problem is solved:
The program was originally meant as a 32-bit application, which means that the original type unsigned long was sufficient to keep memory addresses.
However, this program gets compiled now as a 64-bit application, hence the mentioned type is not sufficiently large anymore to keep 64-bit memory addresses, hence another type has been used for solving this issue:
typedef intptr_t longword;
This solves the issue.
#Andrew Henle: sorry, I didn't realise that your comment contained the actual solution to this problem.
Related
I have a big C-project and thanks to valgrind I cleaned up some mess I was doing with a memory management. I cleaned up everything except for one thing, and after a week of analysis I'm starting to think that that's valgrind's misunderstanding of my code, rather than my mistake. The program is running for good, but that means nothing (I've seen the case when the program runs for good for weeks and then stucks because of 31th flipped bit in an int).
My code uses the next idea: there's the a storage (in terms of my project a "warehouse"), which holds all the kinds of structures I desired to build. I use the trick from the Xlib to keep this as small as possible in memory:
typedef struct
{
// data
} TypeA
typedef struct
{
// data
} TypeB
typedef struct
{
// data
} TypeC
typedef union
{
TypeA typea;
TypeB typeb;
TypeC typec;
} UniType;
typedef struct
{
int type;
UniType data;
} Element;
Then I create an element:
SmlErrors SmlWhsAdd(SmlElement element, SmlIndex * index)
{
SML_CHECKPTR(index);
SmlElement * ptrold = warehouse.elem;
warehouse.elem = realloc(warehouse.elem,
(++warehouse.elemcount) * sizeof(SmlElement));
if (!(warehouse.elem))
{
warehouse.elem = ptrold;
*index = 0;
warehouse.elemcount--;
return SML_ERR_BADALLOC;
}
warehouse.elem[warehouse.elemcount - 1] = element;
*index = (warehouse.elemcount - 1);
return SML_ERR_SUCCESS;
}
For those, who thinks that ptr = realloc(ptr... is bad - look closer, I save the old one and restore it after. I'm planning to replace all alloc tools with myalloc to crash the program there instead of continuing work
This code is clear, valgrind is silent. Except for one case. One of my "TypeX" structures (if to be exact, child-of-child-of-TypeX) contains an array:
SmlIndex sprite[SML_THEMEBLOCK_SIZE];
Every sprite is also an index from warehouse, so it's mobiles in mobile, as that array is a part of one of the element of that warehouse (in the house that Jack built).
I use the aforementioned function to write the value inside of one of the sprites:
SML_CHECKLOC(SmlImageCreate(&(widget->sprite[i]),
widget->geometry.size));
// Which calls `WhsAdd` with `&(widget->sprite[i]` as `index`-parameter.
And every time I call it this way the valgrind is whining about Invalid write of size 4. Every time I'm trying to use the value from sprite[x] after that - Invalid read of size 4. If to be exact, it's whining about the following line:
*index = (warehouse.elemcount - 1);
My system is 32bit, SmlIndex is uint32_t
Please, give me a clue about where to dig. After a week of research I'm out of ideas. That's why I'm started to think that that might be valgrind's bug - I also 've heard that it's working strange with unions and structs in it.
One more thing.
widget->sprite[i] = 0; // No complainings.
SmlImageCreate(&(widget->sprite[i], ...) // Complainings.
Can someone give me a hand, please? I'm drowning in that swamp. Any suggestions about where to look. Anything.
UPD:
MCVE: http://pastebin.com/r5T5ZBPC
Regards,
Alex.
(Edit history note: the MCVE originally used uninitialized variables, however after initializing all of those, the problem persists)
In the MCVE, the issue comes from:
SmlWhsAdd(sprite, &(warehouse.elem[window].data.wdg.sprite[0]));
The second argument is a pointer into the space allocated by an earlier call to realloc.
However, inside the SmlWhsAdd function, realloc is called on this space, which allocates a new block and frees the old. This leaves the second argument pointing into freed space.
To fix this, my suggestion would be to review all uses of SmlWhsAdd and avoid passing a pointer which is under warehouse.elem.
One option might be to use a temporary variable and then assign the index after the call; another option might be to pass some other information which allows the SmlWhsAdd function to compute the location to write the index, after it performs realloc; or if the index is always at the end you don't even need to have that parameter at all because the caller can do warehouse.elemcount-1 after.
I have a fairly simple question on allocating memory for a struct in c and I've tried looking online, but I wasn't able to find anything. The struct is defined as:
typedef struct cblock_ {
....
} cblock, *Cblock;
When allocating memory, I've tried
Cblock new_block = malloc(sizeof(struct cblock_));
However, I keep getting getting a memory corruption error. I've tried changing the parameter of sizeof to different things, but everything is getting the same memory corruption error. What am I doing wrong?
****EDIT****
So after running valgrind, I am getting quite a bit of "invalid write of size 4" or "invalid read of size 4" throughout my code.
It's probably too much to post all of it but here's one spot where valgrinds gives me a "invalid write of size 4"
static Ablock agroup_main;
typedef struct ablock_ {
int length;
int num;
...
} ablock, *Ablock;
void init_groups() {
agroup_main = malloc(sizeof(Ablock));
agroup_main->length = DEFAULT_GROUP_LENGTH;
agroup_main->num = DEFAULT_GROUP_NUM;
}
Valgrind is saying on the line setting length that it's an invalid write of size 4, and the address is 0 bytes after a block of size 4 alloc'd. For num, it's the same except the address is 4 bytes after a block of size 4 alloc'd. I don't really understand this either, since length is just an int, and to me, the error sounds like I'm suppose to allocate memory for it...
The memory corruption is probably because of an earlier incorrect malloc (e.g. with a too small size), premature free, or genuine buffer overflow (or uninitialized pointer dereference).
Use a memory leak detector. On Linux, try valgrind
Your malloc for Cblock looks ok. The bug is probably elsewhere. Don't forget to compile with all warnings and debug info (e.g. gcc -Wall -g). Use also a debugger (e.g. gdb). You might want to disable ASLR to ease debugging (and have more reproducible runs).
You should test the result of malloc:
cblock* new_block = malloc(sizeof(struct cblock_));
if (!new_block) {
perror("malloc cblock");
exit(EXIT_FAILURE);
}
and I think you should mark clearly all the pointers in your code (so I prefer cblock* instead of Cblock which I find confusing, and which should at least be named CblockPtr).
BTW, you have been bitten by that confusion:
agroup_main = malloc(sizeof(Ablock)); // WRONG!
is incorrect, since Ablock is a pointer type (on Linux/PC, all pointers have the same size, which is 8 bytes on x86-64). You mean
agroup_main = malloc(sizeof(ablock));
if (!agroup_main)
{ perror("malloc ablock"); exit (EXIT_FAILURE); };
You really should loose the bad habit of naming pointer types in a way hiding their pointerness (and with a name similar to the pointed data).
This:
agroup_main = malloc(sizeof(Ablock));
is clearly wrong, since the type Ablock is struct ablock_ *, i.e. a pointer. You're just allocating the size of the pointer, not the size of the object being pointed at.
You should always write it like this:
agroup_main = malloc(sizeof *agroup_main);
that uses sizeof on the proper type, the type that the pointer receiving the return value points at.
I also stronly recommend against typedefing away the pointer like that, since it adds confusion. Pointers are important in C, hiding them often makes things harder to understand and follow, thus increasing the risk of error. The question "is this an object, or a pointer to an object?" is very important, and you must be able to answer it quickly and correctly, all the time.
I need to allocate all the memory my application will use up front. And then whenever needed overwrite that memory with data I need to do computations on. The memory has to be allocated first before any computations because I'm trying to run a multi-threaded CUDA algorithm in parallel as explained in my question here (Multi-Threaded CPU CUDA application not asynchronous when calling CudaFree).
I thought I could allocate all the memory needed as a byte pointer and then store that pointer as a void pointer:
void * allocateMemory()
{
byte *mem;
int nbytes = 13107200;
mem = (byte *) malloc(nbytes);
return mem;
}
Later in my program I want to use the memory that's already allocated to store data. I don't know ahead of time what type the data will be but I know it's size won't go over the allocated limit.
void doSomething(void * mem)
{
int *a = (int*) mem;
for (int i = 0; i < 100; i++)
{
a[i] = i;
}
//do stuff
}
There are many other functions like doSomething(void * mem) above but that use type double or type float or maybe even type byte. I need to be able to overwrite the orignally allocated memory with whatever data type I need. The above code does not work because it says I can't deference a void pointer. It also says I attempted to read or write protected memory.
What is the proper way to do this? What is the best way to accomplish my goal of having all my memory allocated at the beginning and then used however necessary throughout? Thanks!
It sounds like you have two problems.
Cannot dereference a void pointer. Somewhere in your code you have used the result from allocateMemory() without a cast. The code you give is OK, but whatever line the compiler is flagging as wrong is not OK. For example, maybe you have:
void *foo = allocateMemory();
foo[42]; // compiler doesn't have a real type here - error
((int*)foo)[42]; // compiler happy
Attempted to access protected memory. Somewhere in your code you have an invalid pointer. The most likely cause is that allocateMemory() is returning NULL (which you are not checking for).
Your general approach seems OK to me; the issues you describe are related to details in your code, not the overall idea.
EDIT: Updated code with new Pastebin link but it's still stopping at the info->citizens[x]->name while loop. Added realloc to loops and tidied up the code. Any more comments would be greatly appreciated
I'm having a few problems with memory allocation overflowing
http://pastebin.com/vukRGkq9 (v2)
No matter what I try, simply not enough memory is being allocated for info->citizens and gdb is often saying that it cannot access info->citizens[x]->name.
On occasion, I'll even get KERN_INVALID_ADDRESS errors directly after printf statements for strlen (Strlen is not used in the code at the point where gdb halts due to the error, but I'm assuming printf uses strlen in some way). I think it's something to do with how the structure is being allocated memory. So I was wondering if anyone could take a look?
You shouldn't do malloc(sizeof(PEOPLE*)), because it allocates exactly amount of bytes for pointer (4 bytes on 32bit arch).
Seems the thing you want to do is malloc(sizeof(PEOPLE) * N) where N is the max. number of PEOPLE you want to put into that memory chunk.
Clearly the problem lies with:
info->citizens = malloc(sizeof(PEOPLE *));
info->citizens[0] = malloc(sizeof(PEOPLE *));
info->citizens[1] = malloc(sizeof(PEOPLE *));
Think about it logically what you are trying to do here.
Your structs should almost certainly not contains members such as:
time_t *modtimes;
mode_t *modes;
bool *exists;
Instead you should simply use:
time_t modtimes;
mode_t modes;
bool exists;
In that way you do not need to dynamically allocate them, or subsequently release them. The reasons are that a) they're small and b) their size is known in advance. You would use:
char *name;
for a string field because it's not small and you don't know in advance how large it is.
Elsewhere in the code, you have the folllowing:
if(top)
{
PEOPLE *info;
info = malloc(sizeof(PEOPLE *));
}
If top is true then this code allocates a pointer and then immediately leaks it -- the scope of the second info is limited to the if statement so you can neither use it later nor can you release it later. You would need to do something like this:
PEOPLE *process(PEOPLE *info, ...)
{
if (top)
{
info = malloc(sizeof(PEOPLE));
}
info->name = strdup("Henry James");
info->exists = true;
return info;
}
It seems you have one too many levels of indirection. Why are you using **citizens instead of *?
Also, apart from the fact that you are allocating the space for a pointer, not the struct, there are a couple of weird things, such as the local variable info on line 31 means the initial allocation is out of scope once the block closes at line 34.
You need to think more clearly about what data is where.
Lots of memory allocation issues with this code. Those mentioned above plus numerous others, for example:
info->citizens[masterX]->name = malloc(sizeof(char)*strlen(dp->d_name)+1);
info->citizens[masterX]->name = dp->d_name;
You cannot copy strings in C through assignment (using =). You can write this as:
info->citizens[masterX]->name = malloc(strlen(dp->d_name)+1);
strcpy(info->citizens[masterX]->name, dp->d_name);
Or you could condense the whole allocate & copy as follows:
info->citizens[masterX]->name = strdup(dp->d_name);
Similarly at lines 143/147 (except in that case you have also allocated one byte too few in your malloc call).
OK, I hope I explain this one correctly.
I have a struct:
typedef struct _MyData
{
char Data[256];
int Index;
} MyData;
Now, I run into a problem. Most of the time MyData.Data is OK with 256, but in some cases I need to expand the amount of chars it can hold to different sizes.
I can't use a pointer.
Is there any way to resize Data at run time? How?
Code is appreciated.
EDIT 1:
While I am very thankful for all the comments, the "maybe try this..." or "do that", or "what you are dong is wrong..." comments are not helping. Code is the help here. Please, if you know the answer post the code.
Please note that:
I cannot use pointers. Please don't try to figure out why, I just can't.
The struct is being injected into another program's memory that's why no pointers can be used.
Sorry for being a bit rough here but I asked the question here because I already tried all the different approaches that thought might work.
Again, I am looking for code. At this point I am not interested in "might work..." or " have you considered this..."
Thank you and my apologies again.
EDIT 2
Why was this set as answered?
You can use a flexible array member
typedef struct _MyData
{
int Index;
char Data[];
} MyData;
So that you can then allocate the right amount of space
MyData *d = malloc(sizeof *d + sizeof(char[100]));
d->Data[0..99] = ...;
Later, you can free, and allocate another chunk of memory and make a pointer to MyData point to it, at which time you will have more / less elements in the flexible array member (realloc). Note that you will have to save the length somewhere, too.
In Pre-C99 times, there isn't a flexible array member: char Data[] is simply regarded as an array with incomplete type, and the compiler would moan about that. Here i recommend you two possible ways out there
Using a pointer: char *Data and make it point to the allocated memory. This won't be as convenient as using the embedded array, because you will possibly need to have two allocations: One for the struct, and one for the memory pointed to by the pointer. You can also have the struct allocated on the stack instead, if the situation in your program allows this.
Using a char Data[1] instead, but treat it as if it were bigger, so that it overlays the whole allocated object. This is formally undefined behavior, but is a common technique, so it's probably safe to use with your compiler.
The problem here is your statement "I can't use a pointer". You will have to, and it will make everything much easier. Hey, realloc even copies your existing data, what do you want more?
So why do you think you can't use a pointer? Better try to fix that.
You would re-arrange the structure like that
typedef struct _MyData
{
int Index;
char Data[256];
} MyData;
And allocate instances with malloc/realloc like that:
my_data = (MyData*) malloc ( sizeof(MyData) + extra_space_needed );
This is an ugly approach and I would not recommend it (I would use pointers), but is an answer to your question how to do it without a pointer.
A limitation is that it allows for only one variable size member per struct, and has to be at the end.
Let me sum up two important points I see in this thread:
The structure is used to interact between two programs through some IPC mechanism
The destination program cannot be changed
You cannot therefore change that structure in any way, because the destination program is stuck trying to read it as currently defined. I'm afraid you are stuck.
You can try to find ways to get the equivalent behavior, or find some evil hack to force the destination program to read a new structure (e.g., modifying the binary offsets in the executable). That's all pretty application specific so I can't give much better guidance than that.
You might consider writing a third program to act as an interface between the two. It can take the "long" messages and do something with them, and pass the "short" messages onward to the old program. You can inject that in between the IPC mechanisms fairly easily.
You may be able to do this like this, without allocating a pointer for the array:
typedef struct _MyData
{
int Index;
char Data[1];
} MyData;
Later, you allocate like this:
int bcount = 256;
MyData *foo;
foo = (MyData *)malloc(sizeof(*foo) + bcount);
realloc:
int newbcount = 512;
MyData *resized_foo;
resized_foo = realloc((void *)foo, sizeof(*foo) + newbcount);
It looks like from what you're saying that you definitely have to keep MyData as a static block of data. In which case I think the only option open to you is to somehow (optionally) chain these data structures together in a way that can be re-assembled be the other process.
You'd need and additional member in MyData, eg.
typedef struct _MyData
{
int Sequence;
char Data[256];
int Index;
} MyData;
Where Sequence identifies the descending sequence in which to re-assemble the data (a sequence number of zero would indicate the final data buffer).
The problem is in the way you're putting the question. Don't think about C semantics: instead, think like a hacker. Explain exactly how you are currently getting your data into the other process at the right time, and also how the other program knows where the data begins and ends. Is the other program expecting a null-terminated string? If you declare your struct with a char[300] does the other program crash?
You see, when you say "passing data" to the other program, you might be [a] tricking the other process into copying what you put in front of it, [b] tricking the other program into letting you overwrite its normally 'private' memory, or [c] some other approach. No matter which is the case, if the other program can take your larger data, there is a way to get it to them.
I find KIV's trick quite usable. Though, I would suggest investigating the pointer issue first.
If you look at the malloc implementations
(check this IBM article, Listing 5: Pseudo-code for the main allocator),
When you allocate, the memory manager allocates a control header and
then free space following it based on your requested size.
This is very much like saying,
typedef struct _MyData
{
int size;
char Data[1]; // we are going to break the array-bound up-to size length
} MyData;
Now, your problem is,
How do you pass such a (mis-sized?) structure to this other process?
That brings us the the question,
How does the other process figure out the size of this data?
I would expect a length field as part of the communication.
If you have all that, whats wrong with passing a pointer to the other process?
Will the other process identify the difference between a pointer to a
structure and that to a allocated memory?
You cant reacolate manualy.
You can do some tricks wich i was uning when i was working aon simple data holding sistem. (very simple filesystem).
typedef struct
{
int index ;
char x[250];
} data_ztorage_250_char;
typedef struct
{
int index;
char x[1000];
} data_ztorage_1000_char;
int main(void)
{
char just_raw_data[sizeof(data_ztorage_1000_char)];
data_ztorage_1000_char* big_struct;
data_ztorage_250_char* small_struct;
big_struct = (data_ztorage_1000_char*)big_struct; //now you have bigg struct
// notice that upper line is same as writing
// big_struct = (data_ztorage_1000_char*)(&just_raw_data[0]);
small_struct = (data_ztorage_250_char*)just_raw_data;//now you have small struct
//both structs starts at same locations and they share same memory
//addresing data is
small_struct -> index = 250;
}
You don't state what the Index value is for.
As I understand it you are passing data to another program using the structure shown.
Is there a reason why you can't break your data to send into chunks of 256bytes and then set the index value accordingly? e.g.
Data is 512 bytes so you send one struct with the first 256 bytes and index=0, then another with the next 256 bytes in your array and Index=1.
How about a really, really simple solution? Could you do:
typedef struct _MyData
{
char Data[1024];
int Index;
} MyData;
I have a feeling I know your response will be "No, because the other program I don't have control over expects 256 bytes"... And if that is indeed your answer to my answer, then my answer becomes: this is impossible.