Need a way to optimize heap usage C - Freebsd/glibc - c

Question

I have an application that reads an external xml file, parses the content and creates a list.
The file keeps getting updated frequently by external vendors. So i do not have any control on the number of entries/contents of the file. My problem is that my application is using an inordiante amount of memory as evidenced in the ps Aux output (RSS coloumn). I traced it to the fact that while parsing the file, my application links to libxml2.so.3.
It does it for parsing the xml file and the outline of the culprit function is as below:
processxml_func( ) {
1) xmlReadFile(app_xml, NULL, 0); --> This internally allocated around 7MB of data and is a purely libxml call. So i have no control on its memory allocations.
and then code to
2) parse the xml contents --> again using libxml calls and create the list(using malloc) . List has members like name(variable length), description (again variable length etc). This list is long living and needs to live until the program terminates.
3) Free the memory allocated in 1.
processxml_func_end
Using pmap i traced that after the call to processxml_func, my programs heap memory has incresed by ~10 MB. Single tracing through gdb, i noticed that :
step1: allocated 7MB --> usage of the heap is 7MB
step2: allocates 3 MB --> usage of the heap is 7+3 MB
step3: free memory in step1 --> usage is still 10MB
The reason i feel for the heap usage to be still be 10MB after 3 is because , during 2 we allocated some of the memory for step2 from the fragmented blocks/remnants in step1 and thus
we are not able to free the entire 7MB even after step3. I commented out step2 and checked that
the heap usage is close to zero after we exit from the function.
Now given the above, i need suggestions/tweaks to reduce the heap foot print of my application.
One approach i was thinking was to create a new process (fork), for processxml_func, then
use IPC to transfer over the list to the parent process(reconstruction of the list again needed in the parent process) and kill off the child process.
Just wondering, a) if there is a better wa of doing this
b) some flags to control malloc behaviour?
Thanks for your time.
My environment:
C language/ FreeBsd , /usr/lib/libc.so.6

Answer 1

The general solution is not to read the complete file to memory. Instead, read a chunk, extract what you need for this, read some more, extract some more. Your XML parser must support this mode.
Googling "streaming XML parser" gives many results, which seems to indicate that such are not uncommon.
P.S. 10MB looks like almost nothing. I wouldn't worry, unless the file is expected to grow to the GB area.

Answer 2

Mini-XML (http://www.msweet.org/projects.php?Z3)
I recommend you this xml parser, i use this xml-lib in my docsis-NMS application (under Debian 7.0) almost since 2 years. I think its documentation is very good, easy to learn, and the API is stable.