What is more effective?
Declaring array e.g. char buffer[4096] before loop and in each iteration clear its content by strlcpy or declare array inside the loop with initial "" value?
char buffer[4096] = "";
while(sth)
{
strlcpy(buffer,"",sizeof(buffer));
//some action;
}
vs
while(sth)
{
char buffer[4096] = "";
//some action;
}
There's probably no significant difference between the two. The only way to be sure would be to benchmark or look at the assembly generated by the compiler.
The second example is clearer and would be preferred.
Neither, there is no reason to clear the entire buffer if it's merely to be used as a string buffer. All you need to do is ensure that a NUL byte \0 is appended to the end of the relevant string.
If you want to guarantee that the str* functions start at the beginning of buffer, a simple *buffer = '\0' is all that is needed for the reason mentioned above.
The only possible difference in performance is the assignment to the buffer, and this will probably translated by your compiler to a single statement which will be identical to this:
*buffer = 0;
Compilers will not perform any stack management when you enter the loop. Whether you define buffer inside or outside the loop doesn't make any difference for the stack. Place for buffer will be reserved on the stack in the beginning of the routine, regardless of where you define your buffer variable (which doesn't mean that the compiler will allow you to access buffer outside the scope where it is defined).
Even the single assignment shown above will probably be optimized by your compiler if it sees that it doesn't make sense to reinitialize buffer every time over and over again.
To quote Knuth, "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."
Focus on readability. Optimize when you need to.
I suggest the first version. In the first version the array is allocated only once. In the second version, the array is allocated every iteration of the loop.
Although compilers may optimize the allocation (such as performing only once), I believe factoring it out indicates to the reader that the memory is allocated only once.
This may be nothing to worry about since many platforms allocate on the stack and allocation consists of changing the contents of the stack pointer.
When in doubt profile.
When considering optimizations, don't. Spend the time making the program correct and robust.
Related
I have a function that reads from a socket, it returns a char** where packets are stored and my intention is to use a NULL unsigned int pointer where I store the length of single packet.
char** readPackets(int numToRead,unsigned int **lens,int socket){
char** packets=(char**)malloc(numToRead);
int *len=(int*)malloc(sizeof(int)*numToRead);
*(lens)=len;
for(int i=0;i<numToRead;i++){
//read
packets[i]=(char*)malloc(MAX_ETH_LEN);
register int pack_len=read(socket,packets[i],MAX_ETH_LEN);
//TODO handler error in case of freezing
if(pack_len<=0){
i--;
continue;
}
len[i]=pack_len;
}
return packets;
}
I use it in this way:
unsigned int *lens_out=NULL;
char **packets=readPackets(N_PACK,&lens,sniff_sock[handler]);
where N_PACK is a constant defined previously.
Now the problem is that when I am inside the function everything works, in fact *(lens) points to the same memory area of len and outside the function lens_out points to the same area too. Inside the function len[i] equals to *(lens[i]) (I checked it with gdb).
The problem is that outside the function even if lens_out points to the same area of len elements with same index are different for example
len[0]=46
lens_out[0]=4026546640
Can anyone explain where I made the mistake?
Your statement char** packets=(char**)malloc(numToRead) for sure does not reserve enough memory. Note that an element of packets-array is of type char*, and that sizeof(char*) is probably 8 (eventually 4), but very very unlikely 1. So you should write
char** packets = malloc(sizeof(char*) * numToRead)
Otherwise, you write out of the bounds of reserved memory, thereby yielding undefined behaviour (probably the one you explained).
Note further that with i--; continue;, you get memory leaks since you assign a new memory block to the ith element, but you lose reference to the memory reserved right before. Write free(packets[i]);i--;continue; instead.
Further, len[0] is an integral type, whereas lens[0] refers to a pointer to int. Comparing these two does not make sense.
Firstly, I want to put it out there that you should write clear code for the sake of future maintenance, and not for what you think is optimal (yet). This entire function should merely be replaced with read. That's the crux of my post.
Now the problem is that when I am inside the function everything works
I disagree. On a slightly broader topic, the biggest problem here is that you've posted a question containing code which doesn't compile when copied and pasted unmodified, and the question isn't about the error messages, so we can't answer the question without guessing.
My guess is that you haven't noticed these error messages; you're running a stale binary which we don't have the source code for, we can't reproduce the issue and we can't see the old source code, so we can't help you. It is as valid as any other guess. For example, there's another answer which speculates:
Your statement char** packets=(char**)malloc(numToRead) for sure does not reserve enough memory.
The malloc manual doesn't guarantee that precisely numToRead bytes will be allocated; in fact, allocations to processes tend to be performed in pages and just as the sleep manual doesn't guarantee a precise number of milliseconds/microseconds, it may allocate more or it may allocate less; in the latter case, malloc must return NULL which your code needs to check.
It's extremely common for implementations to seem to behave correctly when a buffer is overflowed anyway. Nonetheless, it'd be best if you fixed that buffer overflow. malloc doesn't know about the type you're allocating; you need to tell it everything about the size, not just the number of elements.
P.S. You probably want select or sleep within your loop, you know, to "handle error in case of freezing" or whatever. Generally, OSes will switch context to another program when you call one of those, and only switch back when there's data ready to process. By calling sleep after sending or receiving, you give the OS a heads up that it needs to perform some I/O. The ability to choose that timing can be beneficial, when you're optimising. Not at the moment, though.
Inside the function len[i] equals to *(lens[i]) (I checked it with gdb).
I'm fairly sure you've misunderstood that. Perhaps gdb is implicitly dereferencing your pointers, for you; that's really irrelevant to C (so don't confuse anything you learn from gdb with anything C-related).
In fact, I strongly recommend learning a little bit less about gdb and a lot more about assert, because the former won't help you document your code for future maintenance from other people, including us, those who you ask questions to, where-as the latter will. If you include assert in your code, you're almost certainly strengthening your question (and code) much more than including gdb into your question would.
The types of len[i] and *(len[i]) are different, and their values are affected by the way types are interpreted. These values can only be considered equal When they're converted to the same type. We can see this through C11/3.19p1 (the definition of "value", where the standard establishes it is dependant upon type). len[i] is an int * value, where-as *(len[i]) is an int value. The two categories of values might have different alignment, representation and... well, they have different semantics entirely. One is for integral data, and the other is a reference to an object or array. You shouldn't be comparing them, no matter how equal they may seem; the information you obtain from such a comparison is virtually useless.
You can't use len[i] in a multiplication expression, for example. They're certainly not equal in that respect. They might compare equal (as a side-effect of comparison introducing implicit conversions), which is useless information for you to have, and that is a different story.
memcmp((int[]){0}, (unsigned char[]){ [sizeof int] = 42 }, sizeof int) may return 0 indicating that they're equal, but you know that array of characters contains an extra byte, right? Yeh... they're equal...
You must check the return value of malloc (and don't cast the return value), if you're using it, though I really think you should reconsider your options with that regard.
The fact that you use malloc means everyone who uses your function must then use free; it's locking down-stream programmers into an anti-pattern that can tear the architecture of software apart. You should separate categories of allocation logic and user interface logic from processing logic.
For example, you use read which gives you the opportunity to choose whatever storage duration you like. This means you have an immense number of optimisation opportunities. It gives you, the downstream programmer, the opportunity to write flexible code which assigns whatever storage duration you like to the memory used. Imagine if, on the other hand, you had to free every return value from every function... That's the mess you're encouraging.
This is especially a poor, inefficient design when constants are involved (i.e. your usecase), because you could just use an automatic array and get rid of the calls to malloc and free altogether... Your downstream programmers code could be:
char packet[size][count];
int n_read = read(fd, packet, size * count);
Perhaps you think using malloc to allocate (and later read) n spaces for packets is faster than using something else to allocate n spaces. You should test that theory, because from my experience computers tend to be optimised for simpler, shorter, more concise logic.
In anticipation:
But I can't return packet; like that!
True. You can't return packet; to your downstream programmer, so you modify an object pointed at by an argument. That doesn't mean you should use malloc, though.
Unfortunately, too many programs are adopting this "use malloc everywhere" mentality. It's reminiscent of the "don't use goto" crap that we've been fed. Rather than listening to cargo cult propaganda, I recommend thinking critically about what you hear, because your peers are in the same position as you; they don't necessarily know what they're talking about.
I have format string and I'm parsing it than replacing format specifiers with input arguments. Now I consider how to allocate memory for such result string after substitution of arguments. I could allocate this string as long as format string, but than substitution of other string in the place of
%s of any long will need to reallocate this string in some indeterministic fusion, which making it necessary to do some inelegant calculations in the code.
So I thought I could allocate this string created from format string just char by char, reallocating it each time like:
/*** for loop traversing next chars in format string ***/
// if new char
str = realloc(str, sizeof(*str) +1);
// if %s
str = realloc(str, sizeof(*str) + strlen(in_str));
// if %d
str = realloc(str, sizeof(*str) + strlen(d_str));
Usually internal library code which deals with length of mutable strings/array/lists/whatever else does it in 2^n steps - i.e. when you have 4 bytes of memory and need to allocate 5, it actually allocates 8. This will reduce number of realloc() calls, which is expensive, to ~log(n) operations.
However there could be other optimisations, depending on library.
I won't comment about issues with the code that are adequately addressed in other answers.
Calling realloc for every individual act of extension of string isn't necessarily a poor practice. It looks as if it might perform badly, and to fix that, you could implement a scheme which grows the string in larger increments, less frequently. However, how do you know that realloc doesn't already do something of that sort internally? For instance you might think you're being clever by growing a 128 byte string to 256 bytes and then to 512, rather than one character at a time. But if those happen to be the only available malloc internal block sizes, then realloc cannot help but also step through the same sizes. Okay, what about the saving in the raw number of realloc calls that are made? But those are just replaced by invocations of the more clever string growing logic.
If the performance of this format-string-building loop matters, then profile it. Make a version that reduces the realloc operations and profile that, too. Compare it on all the target platforms you write for and on which the performance matters.
If the performance isn't critical, then optimize for properties like good structure, maintainability and code reuse. A function which builds a format string doesn't have to know about string memory management. It needs an abstract interface for working with dynamic strings. That interface can provide a nice function for changing a string in-place by adding a copy of another string at its tail. Functions other than just the format-string producer can use these string operations.
The string management code can decide, in one place, whether it calls realloc every time the length of a string changes, or whether it tracks the storage size separately from the length, and consequently reduces the number of realloc calls.
Code like this:
str = realloc(str, sizeof(*str) +1);
is bad. If realloc fails it will return NULL but it will not free(str). In other words - memory leak. You need to assign the result of realloc to another pointer, then check for NULL and act accordingly.
Whether or not it is good or bad practice to use many realloc depends on what you are trying to obtain, i.e. performance, maintainability, clearness, etc. The best advice is: Write the code as you like it to be. Then profile it. Performance issues? No -> Be happy. Yes -> Rewrite the code with focus on performance.
I'm working in a small C program and I need to embed binary data into an exe file. The method I'm using is converting that binary data into a char[] array... but I'm not including directly that array as a global variable; instead, I copy that array inside a function (LoadResource) that dynamically creates an array on heap, where I copy my original data. That's what I mean:
char *dataPntr;
void LoadResource()
{
char data[2048] = {/*my binary data */};
dataPntr = malloc(2048);
for (int i = 0; i < 2048; i++) dataPntr [i] = data[i];
}
That way, if my understanding is correct, when calling LoadResource() data[] will be placed in stack, copied to heap and finally data[] will be automatically deallocated from stack; heap copy should be manually deallocated with free().
I'm doing it this way because the resource is only used in some situations, not always... and I prefer to avoid a large global variable.
My questions:
When running the program, is data[] array placed somewhere in memory? text segment maybe? or is it just loaded into stack when calling LoadResource()?
Is my solution the proper one (in terms of memory management) or would it be better to just declare a global data array?
Thanks for your answers!
Generally it is a good idea to avoid global variables. I won't say you never need them, but they can be a pain to debug. The problem is that it can be difficult to follow who changed it last. And if you ever do any multi-threading then you will never want to see a global again!
I include your char *dataPntr in those comments - why is that global? It might be better to return the pointer instead.
Not sure why you are using an array on the stack (data), my guess is so that you can use the {...} initialisation syntax. Can you avoid that? It might not be a big deal, 2k is not a large overhead, but maybe it might grow?
Personally I would copy the data using memcpy()
You have a couple of "magic numbers" in your code, 2048 and 2018. Maybe one is a typo? To avoid this kind of issue, most will use a pre-processor macro. For example:
#include <string.h> /* for memcpy() */
#define DATA_SIZE 2048
char * LoadResource(void)
{
char data[DATA_SIZE] = {/*my binary data */};
char * dataPntr = malloc(DATA_SIZE);
if (dataPntr)
memcpy(dataPntr, data, DATA_SIZE);
return dataPntr;
}
By the way, notice the prototype for LoadResource as void. In C (not C++) an empty parameter list means no parameter checking, not no parameters. Also note that I check the returned value from malloc. This means that the function will return NULL on error.
Another strategy might be to make the data array static instead, however exactly when that gets initialised is compiler dependant, and you might find that you incur the memory overhead even if you don't use it.
While I agree with #cdarke in general, it sounds to me that you are creating a constant array (i. e. is never modified at run-time). If this is true, I would not hesitate to make it a global const array. Most compilers will simply place the array in text memory at link time and there will not be any run-time overhead for initialization.
If, on the other hand, you need to modify the data at run-time, I'd follow #cdarke's example, except to make your data array static const. This way, again, most compilers will place the preinitialized array in the text segment and you will avoid the run-time overhead for initializing the data array.
For example, suppose I have a buffer called char journal_name[25] which I use to store the journal name. Now suppose a few lines later in the code I want to store someone's name into a buffer. Should I go char person_name[25] or just reuse the journal_name[25]?
The trouble is that everyone who reads the code (and me too after a few weeks) has to understand journal_name is now actually person_name.
But the counter argument is that having two buffers increases space usage. So better to use one.
What do you think about this problem?
Thanks, Boda Cydo.
The way to solve this in the C way, if you really don't want to waste memory, is to use blocks to scope the buffers:
int main()
{
{
char journal_name[26];
// use journal name
}
{
char person_name[26];
// use person name
}
}
The compiler will reuse the same memory location for both, while giving you a perfectly legible name.
As an alternative, call it name and use it for both <.<
Some code is really in order here. However a couple of points to note:
Keep the identifier decoupled from your objects. Call it scratchpad or anything. Also, by the looks of it, this character array is not dynamically allocated. Which means you have to allocate a large enough scratch-pad to be able to reuse them.
An even better approach is to probably make your functions shorter: One function should ideally do one thing at a time. See if you can break up and still face the issue.
As an alternative to the previous (good) answers, what about
char buffer[25];
char* journal_name = buffer;
then later
char* person_name = buffer;
Would it be OK ?
If both buffers are automatic why not use this?
Most of compilers will handle this correctly with memory reused.
But you keep readability.
{
char journal_name[25];
/*
Your code which uses journal_name..
*/
}
{
char person_name[25];
/*
Your code which uses person_name...
*/
}
By the way even if your compiler is stupid and you are very low of memory you can use union but keep different names for readability. Usage of the same variable is worst way.
Please use person_name[25]. No body likes hard to read code. Its not going to do much if anything at all for your program in terms of memory. Please, just do it the readable way.
You should always (well unless you're very tight on memory) go for readability and maintainability when writing code for the very reason you mentioned in your question.
25 characters (unless this is just an example) isn't going to "break the bank", but if memory is at a premium you could dynamically allocate the storage for journal_name and then free it when you've finished with it before dynamically allocating the storage for person_name. Though there is the "overhead" of the pointer to the array.
Another way would be to use local scoping on the arrays:
void myMethod()
{
... some code
{
char journal_name[25];
... some more code
}
... even more code
{
char person_name[25];
... yet more code
}
}
Though even with this pseudo code the method is getting quite long and would benefit from refactoring into subroutines which wouldn't have this problem.
If you are worried about memory, and I doubt 25 bytes will be an issue, but then you can just use malloc and free and then you just have an extra 4-8 bytes being used for the pointer.
But, as others mentioned, readability is important, and you may want to decompose your function so that the two buffers are used in functions that actually give more indication as to their uses.
UPDATE:
Now, I have had a buffer called buffer that I would use for reading from a file, for example, and then I would use a function pointer that was passed, to parse the results so that the function reads the file, and handles it appropriately, so that the buffer isn't filled in and then I have to remember that it shouldn't be overwritten yet.
So, yet, reusing a buffer can be useful, when reading from sockets or files, but you want to localize the usage of this buffer otherwise you may have race conditions.
gcc 4.4.4 C89
I am just wondering what most C programmers do when they want to zero out memory.
For example, I have a buffer of 1024 bytes. Sometimes I do this:
char buffer[1024] = {0};
Which will zero all bytes.
However, should I declare it like this and use memset?
char buffer[1024];
.
.
memset(buffer, 0, sizeof(buffer));
Is there any real reason you have to zero the memory? What is the worst that can happen by not doing it?
The worst that can happen? You end up (unwittingly) with a string that is not NULL terminated, or an integer that inherits whatever happened to be to the right of it after you printed to part of the buffer. Yet, unterminated strings can happen other ways, too, even if you initialized the buffer.
Edit (from comments) The end of the world is also a remote possibility, depending on what you are doing.
Either is undesirable. However, unless completely eschewing dynamically allocated memory, most statically allocated buffers are typically rather small, which makes memset() relatively cheap. In fact, much cheaper than most calls to calloc() for dynamic blocks, which tend to be bigger than ~2k.
c99 contains language regarding default initialization values, I can't, however, seem to make gcc -std=c99 agree with that, using any kind of storage.
Still, with a lot of older compilers (and compilers that aren't quite c99) still in use, I prefer to just use memset()
I vastly prefer
char buffer[1024] = { 0 };
It's shorter, easier to read, and less error-prone. Only use memset on dynamically-allocated buffers, and then prefer calloc.
When you define char buffer[1024] without initializing, you're going to get undefined data in it. For instance, Visual C++ in debug mode will initialize with 0xcd. In Release mode, it will simply allocate the memory and not care what happens to be in that block from previous use.
Also, your examples demonstrate runtime vs. compile time initialization. If your char buffer[1024] = { 0 } is a global or static declaration, it will be stored in the binary's data segment with its initialized data, thus increasing your binary size by about 1024 bytes (in this case). If the definition is in a function, it's stored on the stack and is allocated at runtime and not stored in the binary. If you provide an initializer in this case, the initializer is stored in the binary and an equivalent of a memcpy() is done to initialize buffer at runtime.
Hopefully, this helps you decide which method works best for you.
In this particular case, there's not much difference. I prefer = { 0 } over memset because memset is more error-prone:
It provides an opportunity to get the bounds wrong.
It provides an opportunity to mix up the arguments to memset (e.g. memset(buf, sizeof buf, 0) instead of memset(buf, 0, sizeof buf).
In general, = { 0 } is better for initializing structs too. It effectively initializes all members as if you had written = 0 to initialize each. This means that pointer members are guaranteed to be initialized to the null pointer (which might not be all-bits-zero, and all-bits-zero is what you'd get if you had used memset).
On the other hand, = { 0 } can leave padding bits in a struct as garbage, so it might not be appropriate if you plan to use memcmp to compare them later.
The worst that can happen by not doing it is that you write some data in character by character and later interpret it as a string (and you didn't write a null terminator). Or you end up failing to realise a section of it was uninitialised and read it as though it were valid data. Basically: all sorts of nastiness.
Memset should be fine (provided you correct the sizeof typo :-)). I prefer that to your first example because I think it's clearer.
For dynamically allocated memory, I use calloc rather than malloc and memset.
One of the things that can happen if you don't initialize is that you run the risk of leaking sensitive information.
Uninitialized memory may have something sensitive in it from a previous use of that memory. Maybe a password or crypto key or part of a private email. Your code may later transmit that buffer or struct somewhere, or write it to disk, and if you only partially filled it the rest of it still contains those previous contents. Certain secure systems require zeroizing buffers when an address space can contain sensitive information.
I prefer using memset to clear a chunk of memory, especially when working with strings. I want to know without a doubt that there will be a null delimiter after my string. Yes, I know you can append a \0 on the end of each string and some functions do this for you, but I want no doubt that this has taken place.
A function could fail when using your buffer, and the buffer remains unchanged. Would you rather have a buffer of unknown garbage, or nothing?
This post has been heavily edited to make it correct. Many thanks to Tyler McHenery for pointing out what I missed.
char buffer[1024] = {0};
Will set the first char in the buffer to null, and the compiler will then expand all non-initialized chars to 0 too. In such a case it seems that the differences between the two techniques boil down to whether the compiler generates more optimized code for array initialization or whether memset is optimized faster than the generated compiled code.
Previously I stated:
char buffer[1024] = {0};
Will set the first char in the buffer
to null. That technique is commonly
used for null terminated strings, as
all data past the first null is
ignored by subsequent (non-buggy)
functions that handle null terminated
strings.
Which is not quite true. Sorry for the miscommunication, and thanks again for the corrections.
Depends how you're filling it: if you're planning on writing to it before even potentially reading anything, then why bother? It also depends what you're going to use the buffer for: if it's going to be treated as a string, then you just need to set the first byte to \0:
char buffer[1024];
buffer[0] = '\0';
However, if you're using it as a byte stream, then the contents of the entire array are probably going to be relevant, so memseting the entire thing or setting it to { 0 } as in your example is a smart move.
I also use memset(buffer, 0, sizeof(buffer));
The risk of not using it is that there is no guarantee that the buffer you are using is completely empty, there might be garbage which may lead to unpredictable behavior.
Always memset-ing to 0 after malloc, is a very good practice.
yup, calloc() method defined in stdlib.h allocates memory initialized with zeros.
I'm not familiar with the:
char buffer[1024] = {0};
technique. But assuming it does what I think it does, there's a (potential) difference to the two techniques.
The first one is done at COMPILE time, and the buffer will be part of the static image of the executable, and thus be 0's when you load.
The latter will be done at RUN TIME.
The first may incur some load time behaviour. If you just have:
char buffer[1024];
the modern loaders may well "virtually" load that...that is, it won't take any real space in the file, it'll simply be an instruction to the loader to carve out a block when the program is loaded. I'm not comfortable enough with modern loaders say if that's true or not.
But if you pre-initialize it, then that will certainly need to be loaded from the executable.
Mind, neither of these have "real" performance impacts in the small. They may not have any in the "large". Just saying there's potential here, and the two techniques are in fact doing something quite different.