Related
So my friend had this rather, well, perverse idea on how to create a list inside of the call stack of a program. The idea being that if you can calculate the offset between the same stack variable within a recursive call, you can access an arbitrary element further up the call stack. It sounds confusing so I decided to implement it, and it works, but of course the alarm bells chime loudly:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
// Reads all of stdin and prints it in reverse order
void read(char* last)
{
char current;
if (last==NULL)
{
// null terminator
current = '\0';
read(¤t);
}
else
{
int ip = getchar();
if (ip==EOF)
{
// end of stdin found, loop back over the stack to find all read characters
// Calculate offset between the stack frames
int offset = ¤t - last;
for (char* c = last; *c != '\0'; c -= offset)
{
printf("%c", *c);
}
}
else
{
current = (char) ip;
read(¤t);
}
}
}
int main(int c, char* argv[])
{
read(NULL);
return 0;
}
The questions being:
Is this UB? If so, why?
Can something simular be used without UB?
Can something simular be used without UB?
Yes, you just need to build a linked list explicitly:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
struct chain {
struct chain *prev;
char ch;
}
// Reads all of stdin and prints it in reverse order
void read(struct chain *last)
{
int ip = getchar();
if (ip==EOF)
{
// end of stdin found, loop back over the stack to find all read characters
for (struct chain *p = last; p; p = p->prev) { {
printf("%c", p->ch);
}
}
else
{
struct chain current = { last, (char)ip };
read(¤t);
}
}
int main(int c, char* argv[])
{
read(NULL);
return 0;
}
however, there is a good chance of a stack overflow...
Is this UB?
Yes.
If so, why?
Because it's accessing memory via unrelated handle. Generally, see pointer provenance n2263.
And because standard gives no guarantee that objects will be allocated adjacent to each other and consecutive in decreasing memory addresses. There is no guarantee that the value of c pointer is valid. Because there is no such guarantee, the behavior is not defined.
Can something simular be used without UB?
No.
Some implementations document in considerable detail how they will lay out stack frames, at least when invoked with certain options [which typically limit the kinds of optimizations the compiler would perform]. If such an implementation behaves in a manner consistent with its documentation, code which relies upon compilers behaving as documented will be non-portable, but may have correct defined behavior on compilers that behave as described. If the Standard doesn't require that an implementation document something, but the implementation does so anyway, the accuracy of such documentation would be a Quality of Implementation issue outside the Standard's jurisdiction. A conforming implementation would be free to document almost anything it likes, regardless of whether its behavior bears any relationship whatsoever to what it documents.
Note that the way the Standard is written, there are very few cases where anything an otherwise-conforming implementation could do with some particular non-contrived program would render it non-conforming. It would be literally impossible to write a program for which one could not contrive a low-quality-but-conforming implementation that would behave nonsensically when fed it [indeed, one could contrive a pair of such implementations such that no source text could be meaningfully processed by both of them].
I think it is reasonable to suggest that quality implementations should only specify that they will process certain constructs a certain way in cases where the Standard would impose no requirements if they will consistently process such constructs in that fashion even when the Standard wouldn't require them to do so. An implementation that documents that it will process something a certain way, but sometimes does something else, may be conforming but should nonetheless be recognized as being of inferior quality. While it may sometimes be useful or necessary for programmers to make concessions for poor-quality implementations, maintainers of such implementations should not regard such concessions as any kind of endorsement of their implementations' quality.
There are relatively few tasks that could be performed by exploiting detailed knowledge of stack frame layout that could not be done better in ways that don't rely upon such knowledge. Further, code which relies on stack frame layout is likely to run usefully only on a rather small fraction of implementations. Nonetheless, such constructs should be expected to work usefully on quality implementations that document their stack frame layouts in sufficient detail to specify everything the programmer needs.
I am new to the C language and just learned about structs and pointers.
My question is related to the offsetof macro I recently saw. I know how it works and the logic behind that.
In the <stddef.h> file the definition is as follows:
#define offsetof(type,member) ((unsigned long) &(((type*)0)->member))
My question is, if I have a struct as shown below:
struct test {
int field1:
int field2:
};
struct test var;
Why cannot I directly get the address of field2 as:
char * p = (char *)&var;
char *addressofField2 = p + sizeof(int);
Rather than writing something like this
field2Offset = offsetof (struct test, field2);
and then adding offset value to var's starting address?
Is there any difference? Is using offsetof more efficient?
The C compiler will often add extra padding bits or bytes between members of a struct in order to improve efficiency and keep integers word-aligned (which in some architectures is required to avoid bus errors and in some architectures is required to avoid efficiency problems). For example, in many compilers, if you have this struct:
struct ImLikelyPadded {
int x;
char y;
int z;
};
you might find that sizeof(struct ImLikelyPadded) is 12, not 9, because the compiler will insert three extra padding bytes between the end of the one-byte char y and the word-sized int z. This is why offsetof is so useful - it lets you determine where things really are even factoring in padding bytes and is highly portable.
Unlike arrays, memory layout of struct is not always contiguous. Compiler may add extra bytes, in order to align the memory. This is called padding.
Because of padding, it us difficult to find location of member manually. This is also why we always use sizeof to find struct size.
Offsetof , macro let you find out the distance,offset, of a member of struct from the strating position of the struct.
One intelligent use if offsetof is seen in Linux kernel's container_of macro. This macro let you find out starting position of node given the address of member in a generic inclusive doubly linked list
As already mentioned in the other answers, padding is one of the reasons. I won't repeat what was already said about it.
Another good reason to use the offsetof macro and not manually compute offsets is that you only have to write it once. Imagine what happens if you need to change the type of field1 or insert or remove one or more fields in front of field2. Using your hand-crafted calculation you have to find and change all its occurrences. Missing one of them will produce mysterious bugs that are difficult to find.
The code written using offsetof doesn't need any update in this situation. The compiler takes care of everything on the next compilation.
Even more, the code that uses offsetof is more clear. The macro is standard, its functionality is documented. A fellow programmer that reads the code understands it immediately. It's not that easy to understand what the hand-crafted code attempts.
I know a pointer to one type may be converted to a pointer of another type. I have three questions:
What should kept in mind while typecasting pointers?
What are the exceptions/error may come in resulting pointer?
What are best practices to avoid exceptions/errors?
A program well written usually does not use much pointer typecasting. There could be a need to use ptr typecast for malloc for instance (declared (void *)malloc(...)), but it is not even necessary in C (while a few compilers may complain).
int *p = malloc(sizeof(int)); // no need of (int *)malloc(...)
However in system applications, sometimes you want to use a trick to perform binary or specific operation - and C, a language close to the machine structure, is convenient for that. For instance say you want to analyze the binary structure of a double (that follows thee IEEE 754 implementation), and working with binary elements is simpler, you may declare
typedef unsigned char byte;
double d = 0.9;
byte *p = (byte *)&d;
int i;
for (i=0 ; i<sizeof(double) ; i++) { ... work with b ... }
You may also use an union, this is an exemple.
A more complex utilisation could be the simulation of the C++ polymorphism, that requires to store the "classes" (structures) hierarchy somewhere to remember what is what, and perform pointer typecasting to have, for instance, a parent "class" pointer variable to point at some time to a derived class (see the C++ link also)
CRectangle rect;
CPolygon *p = (CPolygon *)▭
p->whatami = POLY_RECTANGLE; // a way to simulate polymorphism ...
process_poly ( p );
But in this case, maybe it's better to directly use C++!
Pointer typecast is to be used carefully for well determined situations that are part of the program analysis - before development starts.
Pointer typecast potential dangers
use them when it's not necessary - that is error prone and complexifies the program
pointing to an object of different size that may lead to an access overflow, wrong result...
pointer to two different structures like s1 *p = (s1 *)&s2; : relying on their size and alignment may lead to an error
(But to be fair, a skilled C programmer wouldn't commit the above mistakes...)
Best practice
use them only if you do need them, and comment the part well that explains why it is necessary
know what you are doing - again a skilled programmer may use tons of pointer typecasts without fail, i.e. don't try and see, it may work on such system / version / OS, and may not work on another one
In plain C you can cast any pointer type to any other pointer type. If you cast a pointer to or from an uncompatible type, and incorrectly write the memory, you may get a segmentation fault or unexpected results from your application.
Here is a sample code of casting structure pointers:
struct Entity {
int type;
}
struct DetailedEntity1 {
int type;
short val1;
}
struct DetailedEntity2 {
int type;
long val;
long val2;
}
// random code:
struct Entity* ent = (struct Entity*)ptr;
//bad:
struct DetailedEntity1* ent1 = (struct DetailedEntity1*)ent;
int a = ent->val; // may be an error here, invalid read
ent->val = 117; // possible invali write
//OK:
if (ent->type == DETAILED_ENTITY_1) {
((struct DetailedEntity1*)ent)->val1;
} else if (ent->type == DETAILED_ENTITY_2) {
((struct DetailedEntity2*)ent)->val2;
}
As for function pointers - you should always use functions which exactly fit the declaration. Otherwise you may get unexpected results or segfaults.
When casting from pointer to pointer (structure or not) you must ensure that the memory is aligned in the exact same way. When casting entire structures the best way to ensure it is to use the same order of the same variables at the start, and differentiating structures only after the "common header". Also remember, that memory alignment may differ from machine to machine, so you can't just send a struct pointer as a byte array and receive it as byte array. You may experience unexpected behaviour or even segfaults.
When casting smaller to larger variable pointers, you must be very careful. Consider this code:
char* ptr = malloc (16);
ptr++;
uint64_t* uintPtr = ptr; // may cause an error, memory is not properly aligned
And also, there is the strict aliasing rule that you should follow.
You probably need a look at ... the C-faq maintained by Steve Summit (which used to be posted in the newsgroups, which means it was read and updated by a lot of the best programmers at the time, sometimes the conceptors of the langage itself).
There is an abridged version too, which is maybe more palatable and still very, very, very, very useful. Reading the whole abridged is, I believe, mandatory if you use C.
I have a char pointer which would be used to store a string. It is used later in the program.
I have declared and initialized like this:
char * p = NULL;
I am just wondering if this is good practice. I'm using gcc 4.3.3.
Yes, it's good idea.
Google Code Style recommends:
To initialize all your variables even if you don't need them right now.
Initialize pointers by NULL, int's by 0 and float's by 0.0 -- just for better readability.
int i = 0;
double x = 0.0;
char* c = NULL;
You cannot store a string in a pointer.
Your definition of mgt_dev_name is good, but you need to point it somewhere with space for your string. Either malloc() that space or use a previously defined array of characters.
char *mgt_dev_name = NULL;
char data[4200];
/* ... */
mgt_dev_name = data; /* use array */
/* ... */
mgt_dev_name = malloc(4200);
if (mgt_dev_name != NULL) {
/* use malloc'd space */
free(mgt_dev_name);
} else {
/* error: not enough memory */
}
It is good practice to initialize all variables.
If you're asking whether it's necessary, or whether it's a good idea to initialize the variable to NULL before you set it to something else later on: It's not necessary to initialize it to NULL, it won't make any difference for the functionality of your program.
Note that in programming, it's important to understand every line of code - why it's there and what exactly it's doing. Don't do things without knowing what they mean or without understanding why you're doing them.
Another option is to not define the variable until the place in your code where you have access to it's initial value. So rather then doing:
char *name = NULL;
...
name = initial_value;
I would change that to:
...
char *name = initial_value;
The compiler will then prevent you from referencing the variable in the part of the code where it has no value. Depending on the specifics of your code this may not always be possible (for example, the initial value is set in an inner scope but the variable has a different lifetime), moving the definition as late as possible in the code prevents errors.
That said, this is only allowed starting with the c99 standard (it's also valid C++). To enable c99 features in gcc, you'll need to either do:
gcc -std=gnu99
or if you don't want gcc extensions to the standard:
gcc -std=c99
No, it is not a good practice, if I understood your context correctly.
If your code actually depends on the mgt_dev_name having the initial value of a null-pointer, then, of course, including the initializer into the declaration is a very good idea. I.e. if you'd have to do this anyway
char *mgt_dev_name;
/* ... and soon after */
mgt_dev_name = NULL;
then it is always a better idea to use initialization instead of assignment
char *mgt_dev_name = NULL;
However, initialization is only good when you can initialize your object with a meaningful useful value. A value that you will actually need. In general case, this is only possible in languages that allow declarations at any point in the code, C99 and C++ being good examples of such languages. By the time you need your object, you usually already know the appropriate initializer for that object, and so can easily come up with an elegant declaration with a good initializer.
In C89/90 on the other hand, declarations can only be placed at the beginning of the block. At that point, in general case, you won't have meaningful initializers for all of your objects. Should you just initialize them with something, anything (like 0 or NULL) just to have them initialized? No!!! Never do meaningless things in your code. It will not improve anything, regardless of what various "style guides" might tell you. In reality, meaningless initialization might actually cover bugs in your code, making it the harder to discover and fix them.
Note, that even in C89/90 it is always beneficial to strive for better locality of declarations. I.e. a well-known good practice guideline states: always make your variables as local as they can be. Don't pile up all your local object declarations at the very beginning of the function, but rather move them to the beginning of the smallest block that envelopes the entire lifetime of the object as tightly as possible. Sometimes it might even be a good idea to introduce a fictive, otherwise unnecessary block just to improve the locality of declarations. Following this practice will help you to provide good useful initializers to your objects in many (if not most) cases. But some objects will remain uninitialized in C89/90 just because you won't have a good initializer for them at the point of declaration. Don't try to initialize them with "something" just for the sake of having them initialized. This will achieve absolutely nothing good, and might actually have negative consequences.
Note that some modern development tools (like MS Visual Studio 2005, for example) will catch run-time access to uninitialized variables in debug version of the code. I.e these tools can help you to detect situations when you access a variable before it had a chance to acquire a meaningful value, indicating a bug in the code. But performing unconditional premature initialization of your variables you essentially kill that capability of the tool and sweep these bugs under the carpet.
This topic has already been discussed here:
http://www.velocityreviews.com/forums/t282290-how-to-initialize-a-char.html
It refers to C++, but it might be useful for you, too.
There are several good answers to this question, one of them has been accepted. I'm going to answer anyway in order to expand on practicalities.
Yes, it is good practice to initialize pointers to NULL, as well as set pointers to NULL after they are no longer needed (i.e. freed).
In either case, its very practical to be able to test a pointer prior to dereferencing it. Lets say you have a structure that looks like this:
struct foo {
int counter;
unsigned char ch;
char *context;
};
You then write an application that spawns several threads, all of which operate on a single allocated foo structure (safely) through the use of mutual exclusion.
Thread A gets a lock on foo, increments counter and checks for a value in ch. It does not find one, so it does not allocate (or modify) context. Instead, it stores a value in ch so that thread B can do this work instead.
Thread B Sees that counter has been incremented, notes a value in ch but isn't sure if thread A has done anything with context. If context was initialized as NULL, thread B no longer has to care what thread A did, it knows context is safe to dereference (if not NULL) or allocate (if NULL) without leaking.
Thread B does its business, thread A reads its context, frees it, then re-initializes it to NULL.
The same reasoning applies to global variables, without the use of threads. Its good to be able to test them in various functions prior to dereferencing them (or attempting to allocate them thus causing a leak and undefined behavior in your program).
When it gets silly is when the scope of the pointer does not go beyond a single function. If you have a single function and can't keep track of the pointers within it, usually this means the function should be re-factored. However, there is nothing wrong with initializing a pointer in a single function, if only to keep uniform habits.
The only time I've ever seen an 'ugly' case of relying on an initialized pointer (before and after use) is in something like this:
void my_free(void **p)
{
if (*p != NULL) {
free(*p);
*p = NULL;
}
}
Not only is dereferencing a type punned pointer frowned upon on strict platforms, the above code makes free() even more dangerous, because callers will have some delusion of safety. You can't rely on a practice 'wholesale' unless you are sure every operation is in agreement.
Probably a lot more information than you actually wanted.
Preferred styles:
in C: char * c = NULL;
in C++: char * c = 0;
My rationale is that if you don't initialize with NULL, and then forget to initialize altogether, the kinds of bugs you will get in your code when dereferencing are much more difficult to trace due to the potential garbage held in memory at that point. On the other hand, if you do initialize to NULL, most of the time you will only get a segmentation fault, which is better, considering the alternative.
Initializing variables even when you don't need them initialized right away is a good practice. Usually, we initialize pointers to NULL, int to 0 and floats to 0.0 as a convention.
int* ptr = NULL;
int i = 0;
float r = 0.0;
It is always good to initialize pointer variables in C++ as shown below:
int *iPtr = nullptr;
char *cPtr = nullptr;
Because initializing as above will help in condition like below since nullptr is convertible to bool, else your code will end up throwing some compilation warnings or undefined behaviour:
if(iPtr){
//then do something.
}
if(cPtr){
//then do something.
}
I have a piece of code written by a very old school programmer :-) . it goes something like this
typedef struct ts_request
{
ts_request_buffer_header_def header;
char package[1];
} ts_request_def;
ts_request_def* request_buffer =
malloc(sizeof(ts_request_def) + (2 * 1024 * 1024));
the programmer basically is working on a buffer overflow concept. I know the code looks dodgy. so my questions are:
Does malloc always allocate contiguous block of memory? because in this code if the blocks are not contiguous, the code will fail big time
Doing free(request_buffer) , will it free all the bytes allocated by malloc i.e sizeof(ts_request_def) + (2 * 1024 * 1024),
or only the bytes of the size of the structure sizeof(ts_request_def)
Do you see any evident problems with this approach, I need to discuss this with my boss and would like to point out any loopholes with this approach
To answer your numbered points.
Yes.
All the bytes. Malloc/free doesn't know or care about the type of the object, just the size.
It is strictly speaking undefined behaviour, but a common trick supported by many implementations. See below for other alternatives.
The latest C standard, ISO/IEC 9899:1999 (informally C99), allows flexible array members.
An example of this would be:
int main(void)
{
struct { size_t x; char a[]; } *p;
p = malloc(sizeof *p + 100);
if (p)
{
/* You can now access up to p->a[99] safely */
}
}
This now standardized feature allowed you to avoid using the common, but non-standard, implementation extension that you describe in your question. Strictly speaking, using a non-flexible array member and accessing beyond its bounds is undefined behaviour, but many implementations document and encourage it.
Furthermore, gcc allows zero-length arrays as an extension. Zero-length arrays are illegal in standard C, but gcc introduced this feature before C99 gave us flexible array members.
In a response to a comment, I will explain why the snippet below is technically undefined behaviour. Section numbers I quote refer to C99 (ISO/IEC 9899:1999)
struct {
char arr[1];
} *x;
x = malloc(sizeof *x + 1024);
x->arr[23] = 42;
Firstly, 6.5.2.1#2 shows a[i] is identical to (*((a)+(i))), so x->arr[23] is equivalent to (*((x->arr)+(23))). Now, 6.5.6#8 (on the addition of a pointer and an integer) says:
"If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined."
For this reason, because x->arr[23] is not within the array, the behaviour is undefined. You might still think that it's okay because the malloc() implies the array has now been extended, but this is not strictly the case. Informative Annex J.2 (which lists examples of undefined behaviour) provides further clarification with an example:
An array subscript is out of range, even if an object is apparently accessible with the
given subscript (as in the lvalue expression a[1][7] given the declaration int
a[4][5]) (6.5.6).
3 - That's a pretty common C trick to allocate a dynamic array at the end of a struct. The alternative would be to put a pointer into the struct and then allocate the array separately, and not forgetting to free it too. That the size is fixed to 2mb seems a bit unusual though.
This is a standard C trick, and isn't more dangerous that any other buffer.
If you are trying to show to your boss that you are smarter than "very old school programmer", this code isn't a case for you. Old school not necessarily bad. Seems the "old school" guy knows enough about memory management ;)
1) Yes it does, or malloc will fail if there isn't a large enough contiguous block available. (A failure with malloc will return a NULL pointer)
2) Yes it will. The internal memory allocation will keep track of the amount of memory allocated with that pointer value and free all of it.
3)It's a bit of a language hack, and a bit dubious about it's use. It's still subject to buffer overflows as well, just may take attackers slightly longer to find a payload that will cause it. The cost of the 'protection' is also pretty hefty (do you really need >2mb per request buffer?). It's also very ugly, although your boss may not appreciate that argument :)
I don't think the existing answers quite get to the essence of this issue. You say the old-school programmer is doing something like this;
typedef struct ts_request
{
ts_request_buffer_header_def header;
char package[1];
} ts_request_def;
ts_request_buffer_def* request_buffer =
malloc(sizeof(ts_request_def) + (2 * 1024 * 1024));
I think it's unlikely he's doing exactly that, because if that's what he wanted to do he could do it with simplified equivalent code that doesn't need any tricks;
typedef struct ts_request
{
ts_request_buffer_header_def header;
char package[2*1024*1024 + 1];
} ts_request_def;
ts_request_buffer_def* request_buffer =
malloc(sizeof(ts_request_def));
I'll bet that what he's really doing is something like this;
typedef struct ts_request
{
ts_request_buffer_header_def header;
char package[1]; // effectively package[x]
} ts_request_def;
ts_request_buffer_def* request_buffer =
malloc( sizeof(ts_request_def) + x );
What he wants to achieve is allocation of a request with a variable package size x. It is of course illegal to declare the array's size with a variable, so he is getting around this with a trick. It looks as if he knows what he's doing to me, the trick is well towards the respectable and practical end of the C trickery scale.
As for #3, without more code it's hard to answer. I don't see anything wrong with it, unless its happening a lot. I mean, you don't want to allocate 2mb chunks of memory all the time. You also don't want to do it needlessly, e.g. if you only ever use 2k.
The fact that you don't like it for some reason isn't sufficient to object to it, or justify completely re-writing it. I would look at the usage closely, try to understand what the original programmer was thinking, look closely for buffer overflows (as workmad3 pointed out) in the code that uses this memory.
There are lots of common mistakes that you may find. For example, does the code check to make sure malloc() succeeded?
The exploit (question 3) is really up to the interface towards this structure of yours. In context this allocation might make sense, and without further information it is impossible to say if it's secure or not.
But if you mean problems with allocating memory bigger than the structure, this is by no means a bad C design (I wouldn't even say it's THAT old school... ;) )
Just a final note here - the point with having a char[1] is that the terminating NULL will always be in the declared struct, meaning there can be 2 * 1024 * 1024 characters in the buffer, and you don't have to account for the NULL by a "+1". Might look like a small feat, but I just wanted to point out.
I've seen and used this pattern frequently.
Its benefit is to simplify memory management and thus avoid risk of memory leaks. All it takes is to free the malloc'ed block. With a secondary buffer, you'll need two free. However one should define and use a destructor function to encapsulate this operation so you can always change its behavior, like switching to secondary buffer or add additional operations to be performed when deleting the structure.
Access to array elements is also slightly more efficient but that is less and less significant with modern computers.
The code will also correctly work if memory alignment changes in the structure with different compilers as it is quite frequent.
The only potential problem I see is if the compiler permutes the order of storage of the member variables because this trick requires that the package field remains last in the storage. I don't know if the C standard prohibits permutation.
Note also that the size of the allocated buffer will most probably be bigger than required, at least by one byte with the additional padding bytes if any.
Yes. malloc returns only a single pointer - how could it possibly tell a requester that it had allocated multiple discontiguous blocks to satisfy a request?
Would like to add that not is it common but I might also called it a standard practice because Windows API is full of such use.
Check the very common BITMAP header structure for example.
http://msdn.microsoft.com/en-us/library/aa921550.aspx
The last RBG quad is an array of 1 size, which depends on exactly this technique.
This common C trick is also explained in this StackOverflow question (Can someone explain this definition of the dirent struct in solaris?).
In response to your third question.
free always releases all the memory allocated at a single shot.
int* i = (int*) malloc(1024*2);
free(i+1024); // gives error because the pointer 'i' is offset
free(i); // releases all the 2KB memory
The answer to question 1 and 2 is Yes
About ugliness (ie question 3) what is the programmer trying to do with that allocated memory?
the thing to realize here is that malloc does not see the calculation being made in this
malloc(sizeof(ts_request_def) + (2 * 1024 * 1024));
Its the same as
int sz = sizeof(ts_request_def) + (2 * 1024 * 1024);
malloc(sz);
YOu might think that its allocating 2 chunks of memory , and in yr mind they are "the struct", "some buffers". But malloc doesnt see that at all.