In C is it mandatory to check if the realloc function made it?
void *tmp = realloc(data, new_size);
if (tmp == NULL) return 1;
data = tmp;
In C is it mandatory to check if the realloc function made it?
The quick answer is: NO! checking for failure is not mandatory. If realloc fails, it returns a null pointer, storing it into the original pointer overwrites the previous value, potentially making the block unreachable for later freeing. Dereferencing this null pointer has undefined behavior, a crash on architectures with protected virtual memory. In practice, on these systems with virtual memory, unless you pass an insanely large number for the new size, the call will not fail so you wont get hit by this sloppy code:
data = realloc(data, new_size); // assume realloc succeeds
If you care to be friendly to the next guy trying to debug the program in more stressed environments, you could add:
data = realloc(data, new_size); // assume realloc succeeds
assert(data);
The long answer is: YES you should check for realloc failure in a reliable production program and handle the failure gracefully.
Obviously realloc can fail if you the requested amount of memory is too large for the heap to honor, but it can also fail for internal reasons for requests for smaller amounts, even amounts smaller than the size of the allocated block passed as an argument, even in the absence of heap corruption caused by undefined behavior. There is just no reason to assume that realloc() will always succeed.
If you know the current size allocated for the object you mean to realloc, you can ignore a realloc failure to shrink the object.
For other requests, you should handle handle the error gracefully. If the error causes the function to abort its operation, any memory allocated for the current task should be free'd to avoid memory leaks if the calling program continues. This is a recommendation to avoid memory or resource leaks and allow the program to run reliably for a long time, but depending on your local constraints, you may get away with slack.
To summarize: depending on local constraints (from quick and dirty throw away code to reliable sturdy production code running indefinitely in a life support system), you might not care about the potential improbable failures or it could be mandatory for you to detect, handle and document any unexpected situations.
Detecting and reporting less improbable errors such as fopen() failure to open files, fgets() failure at end of file or scanf() conversion errors from invalid user input is recommended to avoid wasting hours trying to make sense of unexpected behavior, or worse relying on corrupted data that did not produce obviously incorrect results.
Is it mandatory to check if realloc worked?
Yes, or at least, as mandatory as is to check any other success/failure return.
realloc can certainly fail, just as malloc can. In fact, realloc can fail even if you're trying to make a block smaller.
Now, if you're working on the assumption that malloc on a virtual memory system can never fail, and if you're not checking the return value from malloc, either, then no, with the same justification, you could skip checking realloc's return value also. (It'd be silly to check realloc's return value but not malloc's.)
In practice, the question of whether to check for memory allocation failure (malloc and/or realloc) on a modern, virtual-memory system is the subject of some debate. There are plenty of programmers who believe that checking the return values is a waste of time, since "malloc never fails". Me, I always check (except in the very most rushed, throwaway code), because even after all these years, the number of times I've said "I'm getting so tired of typing if( … != NULL) all the time!" is still zero, while the number of times that a checked and caught NULL return has alerted me promptly to a bug in my code, and saved me hours of pointless debugging, is countless.
So in summary, and at the risk of repeating myself, I'd say that, yes, it's mandatory. See also Always check malloc'ed memory?
It is a return value from a library function. The language has no knowledge of the semantics of any function and it is not syntactically mandatory to check or use any return value. That is to say it will compile.
However it may be a semantic error not to check, and an unchecked failure may cause unintended runtime behaviour - i.e. a bug. The language does not prevent you from writing bugs.
It may be "mandatory" in a "local" sense, in cases where development is subjected a coding standard that makes it so. Any reasonable coding standard would likely impose such a requirement, and that might be enforced by static analysis tools or peer review used by the development team. But not universally mandatory. You are free to write bad code if such standards are not applied to your project.
If you use realloc() on a pointer that has not been initialized with either malloc() or calloc(), as far as I know there's no retrospective way to check realloc()'s return value since it will throw an error:
realloc(): invalid pointer
Aborted (core dumped)
Example:
int myint = 10;
int * ptr = &myint;
ptr = (int*) realloc(ptr,(2 * sizeof(int)));
Related
Is always protecting mallocs important? By protecting I mean:
char *test_malloc = malloc(sizeof(char) * 10000);
if (!test_malloc)
exit(EXIT_FAILURE);
I mean, in electronic devices, I don't doubt it's essential.
But in programs I'm running on my own machine that I'm sure my allocation size will be positive and that the size will not be astronomical.
Some people say, "Ah imagine there’s not enough memory in your computer at this moment."
There's always the possibility that the system can run out of memory and not have any more to allocate, so it's always a good idea to do so.
Given that there's almost no way to recover from a failed malloc, I prefer to use a wrapper function to do the allocation and checking. That makes the code more readable.
void *safe_malloc(size_t size)
{
void *p = malloc(size);
if (p == NULL) {
perror("malloc failed!");
exit(EXIT_FAILURE);
}
return p;
}
The other reason it's a good idea to check for malloc failures (and why I always do) is: to catch programming mistakes.
It's true, memory is effectively infinite on many machines today, so malloc almost never fails because it ran out of memory. But it often fails (for me, at least) because I screwed up and overwrote memory in some way, screwing up the heap, and the next call to malloc often returns NULL to let you know it noticed the corruption. And that's something I really want to know about, and fix.
Over the years, probably 10% of the time malloc has returned NULL on me was because it was out of memory, and the other 90% was because of those other problems, my problems, that I really needed to know about.
So, to this day, I still maintain a pretty religious habit of always checking malloc.
It is important to check for error conditions? Absolutely!
Now when it comes to malloc you might find, that some implementations of the C standard library – the glibc for example – will never return NULL and instead abort, due to the assumption that if allocating memory fails, a program won't be able to recover from that. IMHO that assumption is ill founded. But it places malloc in that weird groups of function where, even if you properly implement out-of-memory condition handling that works without requesting any more memory, your program is still being crashed.
That being said: You should definitely check for error conditions, for the sole reason you might run your program in an environment that gives you a chance to properly react to them.
The Standard makes no distinction between a number of kinds of implementations:
Those where a program a program could allocate as much memory as it malloc() will supply, without affecting system stability, and where a program that could behave usefully without needing the storage from the failed allocation may run normally.
Those where system memory exhaustion may cause programs to terminate unexpectedly even without malloc() ever indicating a failure, but where attempting to dereference a pointer returned via malloc() will either access a valid memory region or force program termination.
Those where malloc() might return null, and dereferencing a null pointer might have unwanted effects beyond abnormal program termination, but where programs might be unexpectedly terminated due to a lack of storage even when malloc() reports success.
If there's no way a program would be able to usefully continue operation if malloc() fails, it's probably best to allocate memory via wrapper function that calls malloc() and forcibly terminates the program if it returns null.
Suppose I have the following line in my code:
struct info *pinfo = malloc(sizeof(struct info));
Usually there is another line of code like this one:
if (!pinfo)
<handle this error>
But does it really worth it? especially if the object is so small that the code generated to check it might need more memory than the object itself.
It's true that running out of memory is rare, especially for little test programs that are only allocating tens of bytes of memory, especially on modern systems that have many gigabytes of memory available.
Yet malloc failures are very common, especially for little test programs.
malloc can fail for two reasons:
There's not enough memory to allocate.
malloc detects that the memory-allocation heap is messed up, perhaps because you did something wrong with one of your previous memory allocations.
Now, it turns out that #2 happens all the time.
And, it turns out that #1 is pretty common, too, although not because there's not enough memory to satisfy the allocation the programmer meant to do, but because the programmer accidentally passed a preposterously huge number to malloc, accidentally asking for more memory than there is in the known universe.
So, yes, it turns out that checking for malloc failure is a really good idea, even though it seems like malloc "can't fail".
The other thing to think about is, what if you take the shortcut and don't check for malloc failure? If you sail along and use the null pointer that malloc gave you instead, that'll cause your program to immediately crash, and that'll alert you to your problem just as well as an "out of memory" message would have, without your having to wear your fingers to the bone typing if(!pinfo) and fprintf(stderr, "out of memory\n"), right?
Well, no.
Depending on what your program accidentally does with the null pointer, it's possible it won't crash right away. Anyway, the crash you get, with a message like "Segmentation violation - core dumped" doesn't tell you much, doesn't tell you where your problem is. You can get segmentation violations for all sorts of reasons (especially in little test programs, especially if you're a beginner not quite sure what you're doing). You can spend hours in a futile effort to figure out why your program is crashing, without realizing it's because malloc is returning a null pointer. So, definitely, you should always check for malloc failure, even in the tiniest test programs.
Deciding which errors to test for, versus those that "can't happen" or for whatever reason aren't worth catching, is a hard problem in general. It can take a fair amount of experience to know what is and isn't worth checking for. But, truly, anybody who's programmed in C for very long can tell you emphatically: malloc failure is definitely worth checking for.
If your program is calling malloc all over the place, checking each and every call can be a real nuisance. So a popular strategy is to use a malloc wrapper:
void *my_malloc(size_t n)
{
void *ret = malloc(n);
if(ret == NULL) {
fprintf(stderr, "malloc failed (%s)\n", strerror(errno));
exit(1);
}
return ret;
}
There are three ways of thinking about this function:
Whenever you have some processing that you're doing repetitively, all over the place (in this case, checking for malloc failure), see if you can move it off to (centralize it in) a single function, like this.
Unlike malloc, my_malloc can't fail. It never returns a null pointer. It's almost magic. You can call it whenever and wherever you want, and you never have to check its return value. It lets you pretend that you never have to worry about running out of memory (which was sort of the goal all along).
Like any magical result, my_malloc's benefit — that it never seems to fail — comes at a price. If the underlying malloc fails, my_malloc summarily exits (since it can't return in that case), meaning that the rest of your program doesn't get a chance to clean up. If the program were, say, a text editor, and whenever it had a little error it printed "out of memory" and then basically threw away the file the user had been editing for the last hour, the user might not be too pleased. So you can't use the simple my_malloc trick in production programs that might lose data. But it's a huge convenience for programs that don't have to worry about that sort of thing.
If malloc fails then chances are the system is out of memory or it's something else your program can't handle. It should abort immediately and at most log some diagnostics. Not handling NULL from malloc will make you end up in undefined behavior land. One might argue that having to abort because of a failure of malloc is already catastrophic but just letting it exhibit UB falls under a worse category.
But what if the malloc fails? You will dereference the NULL pointer, which is UB (undefined behaviour) and your program will (probably) fail!
Sometimes code which checks the correctness of the data is longer than the code which does something with it :).
This is very simply, if you won't check for NULL you might end up with runtime error. Checking for NULL will help you to avoid program from unexpected crash and gracefully handle the error case.
If you just want to quickly test some algorithm, then fine, but know it can fail. For example run it in the debugger.
When you include it in your Real World Program, then add all the error checking and handling needed.
By the worry of my last days/weeks when I figured out that much of my code does break c99 rules, what is leading into undefined behaviour, I started explicitly reading the ISO/IEC 9899:TC3 draft paper.
Especially the Appendix "J.2 Undefined behaviour"
The most of it was logical to me why it could be hard to compile code that's breaking those rules, or at some cases I could at least think "well, I don't get it, what's the problem with that, but I 'shall' do it that way"
but there is one point...
"A non-null pointer returned by a call to the calloc, malloc, or realloc function with a zero requested size is used to access an object (7.20.3)."
(for all who haven't read ISO/IEC 9899:TC3 "J.2 Undefined behaviour", this section just explains which cases will run into undefined behaviour).
So there are so many Questions in my head about that case.
First of all:
Why should I want to allocate a memory-block of zero size?
And when I have got such a block, what can I do with it?
With the aim of avoiding undefined behaviours, probably I don't want to access the memory its pointing to...
So I did a bit more research.... looked for some different malloc() man pages.
and found out in the Linux malloc(3) man:
"If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free()."
Well the only thing this helped me with was: I got now additional questions to you and myself.
The case that function call with identical parameters under identical conditions may return different results isn't that hard to imagine, OK... but those different results mostly don't have to be tread that different.
Thats what lets me suggesting, a non-null pointer to a requested zero size block could just be a unwanted side-effect.
Does this mean
if ((void *ptr = malloc (0)) == NULL)
{
/*...*/
}
this isn't enough?
do I have to handle *alloc calls like this?
if (X <= 0)
{
if ((*ptr = malloc (X)) != NULL)
{
exit (*);
}
else
{
/*...*/
}
}
else
{
if ((*ptr = malloc (X)) == NULL)
{
/*...*/
}
}
But even if its expected, to get such an
"unique pointer value that can later be successfully passed to free()"
,
how to work with it?
I could change it around OK...
I'm even allowed to free it (BTW does it mean I HAVE to free it as I should do with every other allocated memory too, or is it just an >you are allowed to, to don't break your code flow
what would be the difference to just make any pointer like this?
void *X = (void *)"1234abc";
I hope any one can help me with that philosophy of science or is even better as interested as I'm, in it.
C does not support zero-sized objects, but the argument to malloc() is of type size_t, and there's no good way to prevent a program from calling malloc(0). It may not be the programmer's intent, but it's not necessarily literally malloc(0); it's more likely to be malloc(count), where count is the result of some computation.
As for the standard permitting two different behaviors, that's simply because existing implementations (at the time the original standard was being written) did different things, and the authors wanted to avoid breaking existing code. Such code was arguably already broken, or at least non-portable, but by permitting either behavior a program that made assumptions about how malloc(0) behaves could continue to work on the system for which it was written.
If you're looking for a coherent explanation, you're not going to find one. If C were being designed from scratch today, the behavior of malloc(0) almost certainly would have been nailed down, one way or the other. Either that, or the behavior would have been made undefined, but making it implementation-defined means that code doesn't have to check quite as carefully that it's not passing zero to malloc().
And in fact the committee's decision is documented in the C99 Rationale, section 7.20.3, pages 160-161.
It does mean that:
void *ptr = malloc(0);
free(ptr);
will work correctly; free() does nothing if its argument is a null pointer.
What can you do with the result of malloc(0)? Well, if malloc(1024) is successful, you can store 1024 bytes in the allocated space. You can store no bytes in the space allocated by malloc(0) -- which is exactly what you asked for.
Many memory allocation routines have a minimum-sized allocation block which they can support, and will expand any allocation request to that minimum size. While they could have had malloc check whether the size is zero and return null if it is, it was simpler and easier to have all malloc requests less than the minimum size (including those for exactly zero bytes) get padded out to the minimum size. Since the only code which would be likely to request a zero-byte malloc would be asking for a buffer whose size was determined at runtime, and since any such code would expect to free any buffer it had asked for without regard for its size, such behavior caused no difficulty.
Another thing to consider is that if realloc is called on a block multiple times, it may be desirable to keep on using the same memory block if it has adequate space available. If a block which has e.g. 256 bytes allocated to it gets sized to zero, it may be desirable to hold off for awhile on using the formerly-occupied 256 bytes for any purpose other than re-expanding that memory block. If calling realloc on a memory block with a requested size of zero returns a pointer to that same block, then the system will be able to recognize a subsequent request to re-expand that block and use the formerly-occupied space to satisfy it. If the zero-byte realloc returned a null pointer, a request to expand that block to a non-zero size would be indistinguishable from a request to allocate a new one.
Code which uses realloc (or anything malloc related, for that matter) will generally not have any control over memory allocation and recycling policy, but some implementations of the "malloc library" may benefit from having code which doesn't need something at the moment but will likely need it in future reallocate it to size zero rather than freeing it. Note that while the usage pattern which is best for one library may be sub-optimal in another, code which is uses "malloc library" functions in a standards-compliant fashion should work, even if not optimally, on any machine which implements them standards-compliant fashion. If one knows that for the next year or two a program is likely to be run on a machine with a particular allocator, it may be helpful to write the code in the way that's optimal for that allocator. The program might have in future have to run on a machine for which its allocation pattern is no longer optimal, but if that machine is faster and has more RAM (as will likely be the case), the suboptimal performance probably won't pattern.
Your code can be more explicit, as in:
if (X <= 0)
{
ptr = NULL;
}
else
{
ptr = malloc(X);
if (ptr == NULL)
{
/*...*/
}
}
and this can be distilled to:
ptr = (X <= 0) ? NULL : malloc(X);
if (ptr == NULL) { /* ... */}
Today, in C, if malloc cannot acquire the requested storage it returns NULL. Also, this code completely avoids the question and more importantly avoids a potential pitfall.
Some time ago a friend of mine told me not to use realloc because it's unsafe, but he couldn't tell me why, so I made some research on the subject and the nearest references to my doubt were:
First
Second
I want to know if I can continue to use realloc in my code or if it's unsafe is there any other way to reallocate memory?
It's perfectly safe to use realloc. It is the way to reallocate memory in a C program.
However you should always check the return value for an error condition. Don't fall into this common trap:
p = realloc(p, new_size); // don't do this!
If this fails, realloc returns NULL and you have lost access to p. Instead do this:
new_p = realloc(p, new_size);
if (new_p == NULL)
...handle error
p = new_p;
The first of the two linked article raises two complaints above and beyond the "check the call succeeded" points already raised here.
When this is done, the old contents are discarded and left in memory somewhere. For secure memory applications where it is important to erase all traces of data, this behavior is inappropriate.
This is a valid point if you happen to be storing sensitive data (e.g. private keys, unhashed(!) passwords etc.) and want to make it harder for exploits to recover the data or other processes on the system to steal the data.
Since it moves memory around, any old pointers to that memory become invalid and could cause the program to crash or otherwise misbehave.
This point seems like nonsense to me. Their proposed solution is no better, they malloc(), copy and then free() the original which has the same net effect - the address has changed. If you wanted to avoid moving the memory you might be able to use some platform specific calls to do that, if you arranged for there to be sufficient free address space near them. If you knew a priori how much address space to reserve then you'd probably not be thinking of calling realloc() in the first place though!
If you're gambling on realloc() never moving, always growing then you've probably got bigger problems to worry about anyway and switching to malloc() + copy + free() can't possibly solve that.
Besides the "check your return value properly point", the most interesting point from the second article is a warning about:
Do not realloc your buffer by 1 byte at a time.
they warn:
This is guaranteed to churn your memory heap
This is a potentially valid point, but it's not a criticism of realloc() itself; the same would happen if you used malloc()+copy+free(). The real fix is to grow buffers sensibly regardless of how you grow them or better yet allocate in correct sized chunks up front.
They also have a point about
Using realloc to return memory to the system.
They're correct here in that using any size other than 0 might not actually make a return. It probably makes things no worse, but this usage still seems like an example of premature "optimisation". The fix again is to use sensible sized allocations to begin with.
Sort answer: it's not unsafe, but it's not a magical solution to all your problems either.
realloc is safe in itself, but using it safely is a bit tricky -- to the point that I'd say roughly 85-90% of the code I've seen that uses it does not do so safely. The problem is that realloc returns NULL to indicate failure -- but when it does so, the pointer you supplied as input is still valid (provided you didn't resize its allocation to 0).
Therefore, you have to assign the return from realloc to the pointer you supplied as input if and only if realloc returned a non-null pointer. If it returns a null pointer, your previous pointer is valid, but the allocation has not be resized.
Also note that many people assume realloc can only fail and/or move the allocation when you enlarge the allocation. In reality, it can fail (though that's unlikely) or move the data to a different location (much more likely) even when you're reducing the allocation size.
Like everything in C, as long as you know what do you do, it's fine.
(Knowing what do you do includes checking for errors, don't use the old pointer,etc)
I read that on Unix systems, malloc can return a non-NULL pointer even if the memory is not actually available, and trying to use the memory later on will trigger an error. Since I cannot catch such an error by checking for NULL, I wonder how useful it is to check for NULL at all?
On a related note, Herb Sutter says that handling C++ memory errors is futile, because the system will go into spasms of paging long before an exception will actually occur. Does this apply to malloc as well?
Quoting Linux manuals:
By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no
guarantee that
the memory really is available. This is a really bad bug. In case it turns out that the system is out of memory, one or more
processes will be
killed by the infamous OOM killer. In case Linux is employed under circumstances where it would be less desirable to suddenly lose
some randomly
picked processes, and moreover the kernel version is sufficiently recent, one can switch off this overcommitting behavior
using a command like:
# echo 2 > /proc/sys/vm/overcommit_memory
You ought to check for NULL return, especially on 32-bit systems, as the process address space could be exhausted far before the RAM: on 32-bit Linux for example, user processes might have usable address space of 2G - 3G as opposed to over 4G of total RAM. On 64-bit systems it might be useless to check the malloc return code, but might be considered good practice anyway, and it does make your program more portable. And, remember, dereferencing the null pointer kills your process certainly; some swapping might not hurt much compared to that.
If malloc happens to return NULL when one tries to allocate only a small amount of memory, then one must be cautious when trying to recover from the error condition as any subsequent malloc can fail too, until enough memory is available.
The default C++ operator new is often a wrapper over the same allocation mechanisms employed by malloc().
On Linux, you can indeed not rely on malloc returning NULL if sufficient memory is not available due to the kernel's overallocation strategy, but you should still check for it because in some circumstances malloc will return NULL, e.g. when you ask for more memory than is available in the machine in total. The Linux malloc(3) manpage calls the overallocation "a really bad bug" and contains advice on how to turn it off.
I've never heard about this behavior also occurring in other Unix variants.
As for the "spasms of paging", that depends on the machine setup. E.g., I tend not to setup a swap partition on laptop Linux installations, since the exact behavior you fear might kill the hard disk. I would still like the C/C++ programs that I run to check malloc return values, give appropriate error messages and when possible clean up after themselves.
Checking for the return of malloc doesn't help you much by its own to make your allocations safer or less error prone. It can even be a trap if this is the only test that you implement.
When called with an argument of 0 the standard allows malloc to return a sort of unique address, which is not a null pointer and which you don't have the right to access, nevertheless. So if you just test if the return is 0 but don't test the arguments to malloc, calloc or realloc you might encounter a segfault much later.
This error condition (memory exhausted) is quite rare in "hosted" environments. Usually you are in trouble long before you hassle with this kind of error. (But if you are writing runtime libraries, are a kernel hacker or rocket builder this is different, and there the test makes perfect sense.)
People then tend to decorate their code with complicated captures of that error condition that span several lines, doing perror and stuff like that, that can have an impact on the readability of the code.
I think that this "check the return of malloc" is much overestimated, sometimes even defended quite dogmatically. Other things are much more important:
always initialize variables, always. for pointer variables this is crucial,
let the program crash nicely before things get too bad. uninitialized pointer members in structs are an important cause of errors that are difficult to find.
always check the argument to malloc and Co. if this is a compile
time constant like sizof toto there can't be a problem, but
always ensure that your vector allocation handles the zero case properly.
An easy thing to check for return of malloc is to wrap it up with something like memset(malloc(n), 0, 1). This just writes a 0 in the first byte and crashes nicely if malloc had an error or n was 0 to start with.
To view this from an alternative point of view:
"malloc can return a non-NULL pointer even if the memory is not actually available" does not mean that it always returns non-NULL. There might (and will) be cases where NULL is returned (as others already said), so this check is necessary nevertheless.