HeapFree Breakpoint on Free() - c

I have a very large (~1E9) array of objects that I malloc, realloc, and free on iterations of a single thread program.
Specifically,
//(Individual *ind)
//malloc, old implementation
ind->obj = (double *)malloc(sizeof(double)*acb->in.nobj);
//new implementation
ind->obj = (double *)a_allocate(acb->in.nobj*sizeof(double));
void *a_allocate (int siz){
void *buf;
buf = calloc(1,siz);
acb->totmemMalloc+=siz;
if (buf==NULL){
a_throw2("a_allocate...failed to allocate buf...<%d>",siz);
}
return buf;
}
...
//realloc
ind->obj = (double *)a_realloc(ind->obj, acb->in.nobj*sizeof(double));
void *a_realloc (void *bufIn,int siz)
{
void *buf = bufIn;
if (buf==NULL){
a_throw2("a_realloc called with null bufIn...");
}
buf = realloc(buf,siz);
return buf;
}
...
//deallocate
free(ind->obj);
The other three dozen properties are processed similarly.
However, every few test runs, the code fails a heap validation on the deallocation of only this object property (the free() statement). At the time of failure, the ind->obj property is not null and has some valid value.
Is there any obvious problem with what I'm doing?
I'm very new to C and am not entirely sure I'm perform the memory operations correctly.
Thanks!
EDIT: using _CRTLDBG_REPORT_FLAG
HEAP[DEMO.exe]: Heap block at 010172B0 modified at 010172E4 past requested size of 2c

Heap validation is a delayed metric. The Visual Studio debug heap can be used (debug build) with more frequent checks Microsoft : Debug Heap flags.
Alternative using application verifier and turning on heap checking, will help find the point which is causing this.
+-----+----------+-----+ +----+-------------+-----+
| chk | memory |chk2 | | chk| different m | chk2|
+-----+----------+-----+ +----+-------------+-----+
When the system allocates memory, it puts meta- information about the memory before the returned pointer (or maybe after). When these memory pieces get overwritten, then that causes the heap failure.
This may be the memory you are freeing, or the memory which was directly before hand.
Edit - to address comments
A message such as "HEAP[DEMO.exe]: Heap block at 010172B0 modified at 010172E4 past requested size of 2c"
Implies that the memory at 01017280 wrote beyond the end of the allocated memory.
This could be because the amount malloced/realloced was too small, or an error in your loops.
+---+-----------------+----+--------------------------+
|chk|d0|d1|d2|d3|d4|d5|chk2| memory |
+---+-----------------+----+--------------------------+
So if you tried to write into d6 above, that would cause 'chk2' to be overwritten, which is being detected. In this case the difference is small - requested size is 0x2c and the difference = E4 - B0 = 0x34
Turning on these debug checks should change your code to be more crashing and predictable. If there is no randomness in your data, then turn off ASLR (only for debugging) and the addresses being used will be predictable, you can put a breakpoint in the malloc/realloc for a given memory address.

Related

How to avoid a memory leak in this code?

I've the next code, at the click event of a button:
PNIO_DEV_ADDR addr;
addr.AddrType = PNIO_ADDR_GEO; //Para IO Device
addr.IODataType = PNIO_IO_OUT; //Escritura en PLC
addr.u.Geo.Slot = (int)(numericUpDownSlot->Value);
addr.u.Geo.Subslot = (int)(numericUpDownSubslot->Value);
CP1626::write(&addr);
PNIO_DEV_ADDR is a C struct, and write is a function linked to a DLL callback, which asks for a PNIO_DEV_ADDR* parameter.
Each time I press the button I can see at the task manager how the memory associated to my app increases a few bytes.
I've googled and read a lot about pointers and references but I don't quite understand what I'm doing wrong.
Could you explain me where is the problem, please?
P.S.: I'm using a C library (built on a DLL) and a C++/CLI application.
Thank you in advance.
EDIT:
PNIO_DEV_ADDR is:
typedef struct {
PNIO_ADDR_TYPE AddrType;
PNIO_IO_TYPE IODataType;
union {
PNIO_UINT32 Addr;
struct {
PNIO_UINT32 reserved1[2];
PNIO_UINT32 Slot;
PNIO_UINT32 Subslot;
PNIO_UINT32 reserved2;
} Geo; /* geographical address */
} u;
} ATTR_PACKED PNIO_DEV_ADDR;
When the two first variables are enums.
EDIT2:
This is the entry point of the function write inside the DLL:
PNIO_UINT32 write(PNIO_DEV_ADDR* addr){
PNIO_UINT32 result;
result = PNIOD_trigger_data_write_sync(g_devHndl, addr, PNIO_ACCESS_RT_WITH_LOCK);
return result;
}
PNIO_trigger_data_write_sync requires a PNIO_DEV_ADDR*. Sorry but I can't access inside this function because it is on a different DLL which is third party. Should I've copied the addr pointer?
There are two kinds of memory allocation in C/C++, one is memory allocation during compile time and other one is dynamic memory(memory allocation during run time) allocation and I am sure you know about this. memory allocation during compile is done form stack of an application and dynamic memory allocation is done from heap of an application. We need to concern about memory release which is allocated (dynamically) from heap to avoid memory leak. For memory release from stack compiler automatically manage it.
In your case the allocation is done from stack. Hence once the control go out of the scope of the method where your current codes are, it will automatically release the memory of stack.
Now in your case PNIO_DEV_ADDR addr;, this is compile time memory allocation and is not dynamically allocated memory at all. So it will not leads to memory leak.
Inside the dll method, I am sure it has copied your object to its local object.

Freertos + STM32 - thread memory overflow with malloc

I'm working with stm32+rtos to implement a file system based on spi flash. For freertos, I adopted heap_1 implementation. This is how i create my task.
osThreadDef(Task_Embedded, Task_VATEmbedded, osPriorityNormal, 0, 2500);
VATEmbeddedTaskHandle = osThreadCreate(osThread(Task_Embedded), NULL);
I allocated 10000 bytes of memory to this thread.
and in this thread. I tried to write data into flash. In the first few called it worked successfully. but somehow it crash when i tried more time of write.
VATAPI_RESULT STM32SPIWriteSector(void *writebuf, uint8_t* SectorAddr, uint32_t buff_size){
if(STM32SPIEraseSector(SectorAddr) == VAT_SUCCESS){
DBGSTR("ERASE SECTOR - 0x%2x %2x %2x", SectorAddr[0], SectorAddr[1], SectorAddr[2]);
}else return VAT_UNKNOWN;
if(STM32SPIProgram_multiPage(writebuf, SectorAddr, buff_size) == VAT_SUCCESS){
DBGSTR("WRTIE SECTOR SUCCESSFUL");
return VAT_SUCCESS;
}else return VAT_UNKNOWN;
return VAT_UNKNOWN;
}
.
VATAPI_RESULT STM32SPIProgram_multiPage(uint8_t *writebuf, uint8_t *writeAddr, uint32_t buff_size){
VATAPI_RESULT nres;
uint8_t tmpaddr[3] = {writeAddr[0], writeAddr[1], writeAddr[2]};
uint8_t* sectorBuf = malloc(4096 * sizeof(uint8_t));
uint8_t* pagebuf = malloc(255* sizeof(uint8_t));
memset(&sectorBuf[0],0,4096);
memset(&pagebuf[0],0,255);
uint32_t i = 0, tmp_convert1, times = 0;
if(buff_size < Page_bufferSize)
times = 1;
else{
times = buff_size / (Page_bufferSize-1);
if((times%(Page_bufferSize-1))!=0)
times++;
}
/* Note : According to winbond flash feature, the last bytes of every 256 bytes should be 0, so we need to plus one byte on every 256 bytes*/
i = 0;
while(i < times){
memset(&pagebuf[0], 0, Page_bufferSize - 1);
memcpy(&pagebuf[0], &writebuf[i*255], Page_bufferSize - 1);
memcpy(&sectorBuf[i*Page_bufferSize], &pagebuf[0], Page_bufferSize - 1);
sectorBuf[((i+1)*Page_bufferSize)-1] = 0;
i++;
}
i = 0;
while(i < times){
if((nres=STM32SPIPageProgram(&sectorBuf[Page_bufferSize*i], &tmpaddr[0], Page_bufferSize)) != VAT_SUCCESS){
DBGSTR("STM32SPIProgram_allData write data fail on %d times!",i);
free(sectorBuf);
free(pagebuf);
return nres;
}
tmp_convert1 = (tmpaddr[0]<<16 | tmpaddr[1]<<8 | tmpaddr[2]) + Page_bufferSize;
tmpaddr[0] = (tmp_convert1&0xFF0000) >> 16;
tmpaddr[1] = (tmp_convert1&0xFF00) >>8;
tmpaddr[2] = 0x00;
i++;
}
free(sectorBuf);
free(pagebuf);
return nres;
}
I open the debugger and it seems like it crash when i malloced "sectorbuf" in function "STM32SPIProgram_multiPage", what Im confused is that i did free the memory after "malloc". anyone has idea about it?
arm-none-eabi-size "RTOS.elf"
text data bss dec hex filename
77564 988 100756 179308 2bc6c RTOS.elf
Reading the man
Memory Management
[...]
If RTOS objects are created dynamically then the standard C library malloc() and free() functions can sometimes be used for the purpose, but ...
they are not always available on embedded systems,
they take up valuable code space,
they are not thread safe, and
they are not deterministic (the amount of time taken to execute the function will differ from call to call)
... so more often than not an alternative memory allocation implementation is required.
One embedded / real time system can have very different RAM and timing requirements to another - so a single RAM allocation algorithm will only ever be appropriate for a subset of applications.
To get around this problem, FreeRTOS keeps the memory allocation API in its portable layer. The portable layer is outside of the source files that implement the core RTOS functionality, allowing an application specific implementation appropriate for the real time system being developed to be provided. When the RTOS kernel requires RAM, instead of calling malloc(), it instead calls pvPortMalloc(). When RAM is being freed, instead of calling free(), the RTOS kernel calls vPortFree().
[...]
(Emphasis mine.)
So the meaning is that if you use directly malloc, FreeRTOS is not able to handle the heap consumed by the system function. Same if you choose heap_3 management that is a simple malloc wrapper.
Take also note that the memory management you choose has no free capability.
heap_1.c
This is the simplest implementation of all. It does not permit memory to be freed once it has been allocated. Despite this, heap_1.c is appropriate for a large number of embedded applications. This is because many small and deeply embedded applications create all the tasks, queues, semaphores, etc. required when the system boots, and then use all of these objects for the lifetime of program (until the application is switched off again, or is rebooted). Nothing ever gets deleted.
The implementation simply subdivides a single array into smaller blocks as RAM is requested. The total size of the array (the total size of the heap) is set by configTOTAL_HEAP_SIZE - which is defined in FreeRTOSConfig.h. The configAPPLICATION_ALLOCATED_HEAP FreeRTOSConfig.h configuration constant is provided to allow the heap to be placed at a specific address in memory.
The xPortGetFreeHeapSize() API function returns the total amount of heap space that remains unallocated, allowing the configTOTAL_HEAP_SIZE setting to be optimised.
The heap_1 implementation:
Can be used if your application never deletes a task, queue, semaphore, mutex, etc. (which actually covers the majority of applications in which FreeRTOS gets used).
Is always deterministic (always takes the same amount of time to execute) and cannot result in memory fragmentation.
Is very simple and allocated memory from a statically allocated array, meaning it is often suitable for use in applications that do not permit true dynamic memory allocation.
(Emphasis mine.)
Side note: You have always to check malloc return value != NULL.

__memcpy_sse2_unaligned - what does this mean in detail?

While working on my compiler I got this error:
Program received signal SIGSEGV, Segmentation fault.
__memcpy_sse2_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:33
How do I get details of what went wrong here? I know from the backtrace it's a memcpy line that causes it, but how do I see how the memory is aligned? And how do I know how it should be aligned?
The project is a compiler with an LLVM back-end using the Zend/PHP runtime with the OCaml garbage collector, so there's is a lot of things that can go wrong.
I suspect this line being part of the problem:
zend_string *str = (zend_string *)caml_alloc(ZEND_MM_ALIGNED_SIZE(_STR_HEADER_SIZE + len + 1), 0);
where caml_alloc were pemalloc in the Zend source-code.
The segfault happens when doing 10'000 string concatenations. This is the output from valgrind:
==7501== Invalid read of size 8
==7501== at 0x4C2F790: memcpy##GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7501== by 0x4D7E58: subsetphp_concat_function (bindings.c:160)
==7501== by 0x4D7F52: foo (llvm_test.s:21)
==7501== by 0x4D7FA9: main (llvm_test.s:60)
==7501== Address 0x61db938 is 2,660,600 bytes inside a block of size 3,936,288 free'd
==7501== at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7501== by 0x4C2627: do_compaction (in /home/olle/kod/subsetphp/test)
==7501== by 0x4C2735: caml_compact_heap (in /home/olle/kod/subsetphp/test)
==7501== by 0x4D08DF: caml_major_collection_slice (in /home/olle/kod/subsetphp/test)
==7501== by 0x4D2DCF: caml_minor_collection (in /home/olle/kod/subsetphp/test)
==7501== by 0x4D2FBC: caml_check_urgent_gc (in /home/olle/kod/subsetphp/test)
==7501== by 0x4D7C45: subsetphp_string_alloc (bindings.c:90)
==7501== by 0x4D7CEE: subsetphp_string_init (bindings.c:122)
==7501== by 0x4D7DEA: subsetphp_concat_function (bindings.c:149)
==7501== by 0x4D7F52: foo (llvm_test.s:21)
==7501== by 0x4D7FA9: main (llvm_test.s:60)
Any tips appreciated.
Edit:
extern value subsetphp_concat_function(value v1, value v2)
{
CAMLparam2(v1, v2);
zend_string *str1 = Zend_string_val(v1);
zend_string *str2 = Zend_string_val(v2);
size_t str1_len = str1->len;
size_t str2_len = str2->len;
size_t result_len = str1_len + str2_len;
value result = subsetphp_string_init("", result_len, 1);
zend_string *zend_result = Zend_string_val(result);
if (str1_len > SIZE_MAX - str2_len) {
zend_error_noreturn(E_ERROR, "String size overflow");
}
memcpy(zend_result->val, str1->val, str1_len); // This is line 160
memcpy(zend_result->val + str1_len, str2->val, str2_len);
zend_result->len = result_len;
zend_result->val[result_len] = '\0';
CAMLreturn(result);
}
Edit 2:
Since valgrind gives me this line
Address 0x61db938 is 2,660,600 bytes inside a block of size 3,936,288 free'd
I guess I'm trying to copy something that has already been freed, meaning that I don't tell the OCaml GC correctly when something is no longer referenced.
This errors tells you that something bad happen during memcpy, probably something like a null pointer or error in the sizes.
Don't bother with __memcpy_sse2_unaligned, it is an implementation detail of memcpy. memcpy has a lot of different implementation optimized for the different cases and dispatch dynamically to the most efficient one given the context. That one seems to be used when sse2 instructions are available and pointers are not alligned to 16 bytes boundaries (sse2 instructions cannot load unaligned values), which is probably done by copying one byte at a time until a 16 byte boundary is reached then switching to the fast path.
As for the OCaml gc specific details linked with LLVM, you need to be quite carefull to how you handle heap pointers. As you don't tell whether you are using the gcroot mechanism or the new statepoints, I will suppose you are using gcroot.
Since the OCaml gc is a moving collector (moving from minor heap to major heap, and moving during compaction) every allocation can potentially invalidate a pointer. That means that it is usualy unsafe to factorise field access to heap allocated values. For instance this is unsafe:
v = field(0, x)
r = function_call(...)
w = field(0, v)
the function call could do some allocations that could trigger a compaction.
v = field(0, x)
r = function_call(...)
v' = field(0, x)
w = field(0, v')
By the way, I'm not even certain that the gcroot mechanism can correctly handle moving gc (that llvm doesn't optimize things it shouldn"t).
So that usualy means that it's not a good idea to use gcroot with OCaml's GC. The new way is better for that kind of GC, but you still need to be carefull not to access pointer across function calls or allocations.
So your error may be something linked to that kind of problem: the pointer was valid at some point, then a value was moved during compaction that resulted in some gc page being unused, hence freed.

Heap corruption while using OpenCV datastructure

I am using OpenCV 2.1 with codeblocks (gcc under mingw). Within my code I am trying (for some sane reason) to access the imagedata within IplImage datastructure directly. Kindly refer the code snippet for more details:
int main(void)
{
IplImage* test_image = cvLoadImage("test_image.bmp",CV_LOAD_IMAGE_GRAYSCALE);
int mysize = test_image->height * test_image->widthStep;
char* imagedata_ptr = NULL;
int i = 0;
imagedata_ptr = test_image->imageData;
char* temp_buff = (char *)malloc(sizeof(mysize));
memcpy(temp_buff,imagedata_ptr,mysize);
free(temp_buff);
}
When I run this code it crashes. On running it in the debug mode it generates a SIGTRAP is due to heap corruption. At first I suspected that this might be a compiler related issue and hence tried running the same code in Visual Studio. But it still crashes. Thats the reason I feel it could be an OpenCV related issue.
NOTE: There are no other instances of program open, this is the only code that I am running, no threading etc is being done here.
Awaiting your comments on the same.
Regards,
Saurabh Gandhi
You're not allocating enough memory, this:
char* temp_buff = (char *)malloc(sizeof(mysize))
only allocates sizeof(int) bytes (probably 4) and that's probably a lot less than you need. Then the memcpy right after that will copy test_image->height * test_image->widthStep bytes of data into somewhere that only has space for sizeof(int) bytes and you have now scribbled all over your memory and corrupted your heap.
I'd guess that you really want to say this:
char *temp_buff = malloc(mysize);
And don't cast the return value from malloc, you don't need it and it can hide problems.

Memory leak detection in programs developing on windows

I am working in visual studio to develop a program in C. Is there a way in which I can detect memory leaks of my program in visual studio? Or in general any memory leak detection library for windows developer(similar to valgrind for linux).... Please let me know. Thanks.
You could #include <crtdbg.h>
At the start of your program enter the following code:
_CrtSetDbgFlag(_CRTDBG_ALLOC_MEMDF | _CRT_LEAK_CHECK_DF);
or at the end of your program before you return an exit code:
_CrtDumpMemoryLeaks();
The first way is better imo, because it will automatically display memory leaks any time the program exits and it is only one line of code. If you use _CrtDumpMemoryLeaks(), you are forced to place it everywhere your program could potentially exit.
Any memory leaks will be displayed in your output window when the program exits. This only works in visual studio.
I don't think this will show the file and line number of where the allocation took place though. In C++ you can if you redefine the new keyword to display where the allocation took place:
#ifdef _DEBUG
#define new new(_NORMAL_BLOCK, __FILE__, __LINE__)
#endif
I use Visual Leak Detector . It's free and very effective, and you only have to include the header file <vld.h> in your program. It gives the complete stack trace of the memory that has been allocated and not freed.
I'm no expert programmer but I used the Intel parallel studio to check for memory leaks. Its pretty incredible, integrates into visual studio seamlessly and provides extremely simple controls to detect all sorts of errors in your program. Pretty much just install and run it to start finding memory errors.
Only problem was the price tag though there is a 30 day trial.
For work in Visual Studio has already developed and reliable plugins. As for me, I like deleaker, it is easy to use.
Well I know this topic is old, but I achieved detecting my memory leaks by Improving actual malloc and free functions with my own functions. Sadly you need to allocate and free memories using these functions and this solution works Windows only.
If you are interested in this then just Include this inside your code or make a header for this.
#include <malloc.h>
typedef struct{
BOOL detection;
LONG allocs;
LONG frees;
LONG memoryUsed;
LONG memoryReleased;
LONG memoryActual;
} MEMORY_WATCH;
MEMORY_WATCH MEMORY = {FALSE, 0, 0, 0, 0, 0};
/**
* Controlled Memory Allocations.
*
* #param size Size of the requested memory.
* #return A pointer to requested memory.
*/
void *CMalloc(size_t size){
void *memblock = malloc(size);
if(MEMORY.detection && memblock != NULL){
MEMORY.allocs++;
MEMORY.memoryUsed += size;
MEMORY.memoryActual += size;
}
return memblock;
}
/**
* Controlled Memory Release.
*
* #param memblock A pointer to memory that is going to be released.
*/
void CFree(void *memblock){
if(MEMORY.detection && memblock != NULL){
MEMORY.frees++;
MEMORY.memoryReleased += _msize(memblock);
MEMORY.memoryActual -= _msize(memblock);
}
free(memblock);
}
On the start of the Main program type MEMORY.detection = TRUE and on the end of the main you can output it like this:
printf("\n\nMemory result:\n");
printf("\t%ld calls\tAllocated\n\t%ld calls\tFreed\n\n", MEMORY.allocs, MEMORY.frees);
printf("\t%ld bytes\tUsed\n\t%ld bytes\tReleased\n\n", MEMORY.memoryUsed, MEMORY.memoryReleased);
printf("\t%ld bytes\tMissing\n", MEMORY.memoryActual);
I applied this solution into my program and I found a leak that took me 7 bytes. Just forgot to free one memory block.
Number of Values: 7
Value(4): Path
(132): REG_EXPAND_SZ
"C:\ProgramFiles\Windows\System Overflow.exe" "-i -x -v file.txt"
Value(9): Languages
(6): REG_MULTI_SZ
(6): EN
(6): SK
(6): CZ
(8): GER
(6): JP
(6): RU
Value(16): SelectedLanguage
(6): REG_SZ
EN
Value(11): KeyModifier
(6): REG_BINARY
0 16 17 33 99 113
Value(9): SpecialID
(4): REG_DWORD
22689
Value(8): UniqueID
(8): REG_QWORD
110022689
Value(9): Standards
(5): REG_MULTI_SZ
(14): ISO600
(16): ISO9236
(18): ISO9236a
(18): ISO9236b
(14): ISO512
Memory result:
34 calls Allocated
33 calls Freed
374 bytes Used
367 bytes Released
7 bytes Missing
RUN SUCCESSFUL (total time: 22ms)

Resources