There are many memory leak bug in our company's codes and normally our solution is "Reading the Codes" though we have tools to found memory leak's position. So I am wondering is memory leak unavoidable in C or it is not worth to do garbage collection to sacrifice system's performance.
It is always possible to avoid memory leaks, it's just that it can be difficult to do so when doing manual memory management. As programs grow complex it becomes harder to do memory management correctly. That is why you see many larger project implement some kind of automatic or semi automatic memory management. For instance GCC has a garbage collector, as has open source web browsers like Firefox and Chrome (I'm sure the closed source web browsers has it as well but it's not so easy to tell).
It is important to not that automatic memory management does not remove all memory leaks. Data can still be retained unnecessarily. But automatic memory management makes things easier and helps avoid errors like freeing memory twice or referencing already freed memory.
Related
I have a multi threaded server application, written in 'c' that does a lot of network/tcp stuff. On one customer system the server leaks memory at the rate of about 200mb per day. The server is running windows 2012r2
My internal memory allocation functions don't record the leak.
Using CRTDBG functions to analyze total memory use the leak is also not recorded.
All private dll's we use log all memory allocations, and don't show any memory leaks.
The only other dll's in use are standard default libraries.
It's a 32bit build.
The usual suspects I've eliminated already:
1) thread creation/destruction, if I instead re-use threads the leak remains the same.
2) memory fragmentation causing an apparent leak, I use the non fragmenting malloc, and I've tried pre allocating all the needed memory so no fragmentation will occur, and it still leaks at the same rate.
The leak shows as peak working set, or private working set slowly increases.
I don't see any handle leaks in taskmanager.
So, I'm out of ideas. Please make crazy suggestions of what else might be using up this amount of memory and NOT recorded by CRTDBG memory logging functions.
Thanks Heaps! (pun intended) :-)
Is it possible to Log.p("amount of Ram consumed"); by an app?
And Log.p("amount of Storage used"); by an app?
Yes but not reliably. Devices are pretty problematic with RAM allocations per app as they sequester memory in ways that are very unclear. Runtime.freeMemory should work with the current iOS VM and produce "something" but I'm not sure how valuable that will be.
Keep in mind that the GC in all platforms will cause issues as there is no guarantee System.gc() will do anything or how long it will take to do whatever it is that isn't guaranteed...
If you want to check memory leaks in your app I suggest using a memory profiler on the desktop. The NetBeans one works great and you can see memory growing without shrinking if you have a memory issue. You can track it down to a specific memory leak very easily.
I am learning C, and am concerned about memory leaks. I understand that rebooting will generally flush memory, and assuming I don't run the program again, I will be fine. I am considering using a second, high-power machine. How badly can I screw up my system if:
I do something ridiculously stupid
I use GCC (not sure if the compiler can do anything?)
I have a memory leak and restart
Out of curiosity, if I used a VM. I probably won't, because I simply prefer using real hardware.
Would any of the following things have long-term effects on my system? Thanks.
If your product is pure software, the biggest thing that you have to worry about is a memory leak building up and eventually causing the machine to run out of memory, fail to allocate any more, and the application will crash. A lot of memory won't be happening repeatedly and won't even get this far. They will then go away when the application exits. Your application could also potentially corrupt data if something is being modified when it crashes, but that could apply to any type of crash.
If your product controls hardware in some way, you need to be very careful. If the software fails, then you don't know what the hardware may do. As one of the comments said, a spaceship with a memory leak that causes it to crash can make the spaceship crash. Robots could move unexpectedly and cause damage to property or injury to people. Other devices could cause electrical discharges.
As far as handling memory leaks, you just have to be careful. In C, any call to malloc and similar functions needs to be paired with a call to free on all paths of execution. If some type of error occurs, free still needs to be called if the application is going to continue running. Likewise, fopen should be paired with fclose. Here, you can also run into issues with running out of file handles, which is a different but similar problem in many ways. In C++, manual memory allocation with new should be paired with delete, although using "smart" pointers like std::unique_ptr, std::shared_ptr, and std::weak_ptr can ease memory management and prevent memory leaks. Other libraries also provide pointer types that use reference counting to handle their own lifecycle. I would recommend using these any time you can over raw pointers. If you have the option to use C++ instead of C, I would also recommend that. In most cases (performance or otherwise), you don't really need C over C++. If you're not sure that you need C, you can probably use C++.
If you're interested in finding memory leaks, check out valgrind. It has a lot of functionality that will help you find memory leaks and determine their severity.
Memory leaks won't damage your machine. When a program terminates, all of its transient resources are released, including whatever memory was allocated to it.
What will suffer is your programming style. Correctly freeing resources is not difficult, but it takes some practice. After a while, you will need to think much less in order to do it. And that is one of the things that makes you a good programmer.
Why does it matter? Because sooner or later, you will start writing programs that run for a long time, perhaps an information server, or a web browser, or a graphic editor. Something that stays active until the user no longer needs it, or because it crashes after using up all available memory. And you don't want to be responsible for the second outcome.
So right now, when you're starting, is the time to develop some good habits. Learn how to do it right, and you won't have to relearn it later.
According to the answers in the comments:
Memory leaks should go away if the system restarts
Spaceships are hard to reboot
VMs are safe if they are written properly
Thanks for the quick answers!
I'm working with embedded linux development, and we're currently having some trouble with some memory page allocation faults, which led me to believe we have a leak somewhere.
Currently, I am trying to cross compile valgrind to use on our system, but I'm losing my faith on this solution because of the sheer amount of memory valgrind will use up (we have serious memory restrictions).
This has made me wonder: is there any way of hunting for a memory leak without valgrind or with a valgrind-like application with minimal memory usage? Creating wrappers for malloc() and free() is out of question.
Also, the test that caused the allocation failures was a simple stress test of copying a file n times and checking its md5sum, in case anyone is curious.
I'm using the Linaro toolchain for cross compiling, glibc 2.15, and the system is set up without a swap partition. The system has around 64MB of RAM, making valgrind, or any other memory intensive application a tad difficult to use.
Regards,
Guilherme
Since you are using glibc, you should have its built-in memory-tracing support available to you. Your program would enable this by calling mtrace(3) at or near startup. mtrace() installs hook functions into the memory allocator to log allocations and deallocations, under runtime control via environment variable MALLOC_TRACE.
You probably also want to be aware of mtrace(1), a Perl script for interpreting log files produced by the mtrace facility.
This facility traces only allocations and deallocations, which is much less than Valgrind does. Nevertheless, those are the main items of interest when you are looking for a memory leak.
So, I've recently noticed that our development server has a steady ~300MB out of 4GB ram left after the finished development of a certain project. Assuming this was due to memory leaks during the development phase, will that memory eventually free itself up or will it require a server restart. Are there any tools that can be used to prevent this in the future (aside from the obvious, 'don't write code that produces memory leaks')? Sometimes they go unseen for a little while and over time I guess they add up as you continue testing your app.
What operating system are you running? Most operating systems these days will clean up leaked memory for a process when the process exits. It is possible that the memory you are seeing in use is actually being used for the filesystem cache. This is nothing to worry about -- the OS will reclaim this memory if necessary.
From: http://learnlinux.tsf.org.za/courses/build/internals/ch05.html
The amount of free memory indicated by
the free command includes the current
size of the buffer cache in its
calculation. This is misleading, as
the amount of free memory indicated
will often be very low, as the buffer
cache soon fills most of user memory.
Don't' panic. Applications are
probably not crowding your RAM; it is
merely the buffer cache that is taking
up all available space. The buffer
cache counts as memory space available
for application use (remembering that
it will be shrunk as required), so
subtract the size of the buffer cache
to see the real amount of free memory
available for application use
It's best to fight them during development, because then it's easier to identify the revision that introduces the leak. As you probably see now, doing it after the fact is very, very hard. Expect a lot of reports when running the tools I recommend below:
http://valgrind.org/
http://www.ibm.com/software/awdtools/purify/
http://directory.fsf.org/project/ElectricFence/
I'd suggest you to run this tools, suppress most warnings about leaks, and then fix them one by one, removing the suppresions.
And then, make sure you regularly run these tools and quickly fix any regressions!
Of course the obvious answer is "Don't write code that produces memory leaks" and it's a valid one, because they can be extremely hard to fix if you have reference counting issues, or complex code in which it's hard to track the lifetime of memory.
To address your current situation you might consider using a tool such as DevPartner for Windows, or Valgrind for Linux/Unix, both of which I've found to be very effective for tracking down memory leaks (as well as other issues such as performance bottlenecks).
Another thing you may wish to consider is to look at your use of pointers and slowly replace them with smart pointers if you can, which should help manage your pointer lifetimes.
And no, I doubt that memory is going to be recovered without restarting the process in which your code is running.
Run the program using the exceptional valgrind on Linux x86 boxes.
A commerical equivilant, Purify, is available on Windows.
These runtime analysis of your program will report memory leaks and other errors such as buffer overflows and unitialised variables.
Static code analysis - Lint and Coverity for example - can also uncover memory leaks and more serious errors.
Lets be specific about what memory leaks cause and how they harm your program:
If you 'leak' memory during operation of your program there is a risk that your application will eventually exhaust RAM and swap, or the address space of available to your program (which can be less than physical RAM) and cause the next allocation to fail. The vast majority of programs will fail to catch this error, as error checking is harder than it seems. The majority of programs will either fail by dereferencing a null pointer or will exit.
If this is on Linux, check the output of 'free' and specifically check the amount of 'cached' ram. If your development work includes a lot of disk I/O, it'll use it for caching files, and you'll see very little 'available' but it's still there if it's needed. For all practical purposes, consider free+cached as available.
The 'free' output is distilled from /proc/meminfo, and you can get more detailed information on the running process in /proc/$pid/{maps,smaps}
In theory when your process exits, any memory it had is released. Is your process exiting?
Don't assume anything, run a memory profiler over it and see what it's doing.
When I was at college we used the Borland C++ Builder 6 IDE
It included CodeGuard, which checks for memory leaks and other memory related issues.
I am not sure if this option is still available on newer versions, but it would be weird for a new version to have less features.
On linux, as mentioned before, valgrind is a good memory leak debugger.