I have a function that accepts a struct * pointer containing sensitive data (in a char array) as an argument (sort of a small library).
The two struct models are as follows:
struct struct1 {
char str[1024]; /* maybe even 4096 or 10KB+ */
size_t str_length;
}
struct struct2 {
char *str;
size_t str_length;
}
The test function is:
/* Read str_length bytes from the char array. */
void foo(struct struct1/struct2 *s) {
int i;
for (i = 0; i < s->str_length; i++) {
printf("%c\n", s->str[i]);
}
}
My concern is that, since the str_length parameter is an arbitrary value, one could intentionally set it to cause a buffer overflow (actually someone stupid enough to purposely create a security flaw in its own program, but I feel I have to take such cases into account). By using the struct1 model, however, I could simply check for a possible buffer overflow by just using:
if (s->str_length > sizeof(s->str)) {
/* ERROR */
}
The problem is that the length array is actually unknown at compile-time. So I don't know whether to use a char * pointer (struct2 style, so no overflow check) or define a very big array (struct1), which would limit the max length (something I would like to avoid) and would allocate unnecessary space most of the time (which could be problematic in embedded systems with scarce memory, I suppose). I know I have to make a compromise, I'd personally use the struct2 model, but I'm not sure if it's a good choice security-wise.
Where does the user of your library get the struct2 instance to pass to the function from? I don't think he creates it by himself and then passes its address to your function, that would be a weird way to pass arguments. It is most likely returned from another function in your library, in which case you can make struct2 an opaque data type that the user cannot alter directly (or only in hacky ways):
/* in the header file */
typedef struct2_s struct2;
/* in the implementation file, where allocation is handled as well
* so you know str_length is set to the proper value.
*/
struct struct2_s {
char *str;
size_t str_length;
};
Put the big array at the end..
struct struct1 {
anyType thisVar;
someType anotherVar
size_t str_length;
char str[10240000]; /
}
Let the user malloc it to whatever 'real' size they wish. If they set 'str_length' wrong, well, there's not much you can do about it, no matter what you do:(
Related
I want to pretty print this struct struct MyStruct { char buffer[16]; }. Depending on buffer[15] I want to print buffer as a 10 byte string or treat it as a pointer. The 10 byte case is simple and works return self.val['buffer'].string(length = 10)
The second case I can't figure out. I want to do something like (*(char**)buffer[0]). I'm not sure how to do that. I was thinking parse_and_eval could be easy even if its not optimal but I couldn't figure out how to access buffer. I also need to cast the buffer to a 32bit int (len = *(int*)(bufer+4);) I couldn't figure that out either.
If I interpret your description correctly, I think your data is actually layed out like this:
struct MyStruct {
union {
char string[10];
struct {
char *p;
int size;
} ptr;
}
int flag;
}
So I think casting the buffer to a pointer of that type, and then choosing the format based on mystruct->flag would make life easier for you.
Even if my interpretation is not correct, try to find the correct version of that struct that captures the duality of the data.
If I want to reduce malloc()s (espacially if the data is small and allocated often) I would like to allocate the pointer and pointee at once.
If you assume something like the following:
struct entry {
size_t buf_len;
char *buf;
int something;
};
I would like to allocate memory in the following way (don't care about error checking here):
size_t buf_len = 4; // size of the buffer
struct entry *e = NULL;
e = malloc( sizeof(*e) + buf_len ); // allocate struct and buffer
e->buf_len = buf_len; // set buffer size
e->buf = e + 1; // the buffer lies behind the struct
This could even be extende, so that a whole array is allocated at once.
How would you assess such a technuique with regard to:
Portability
Maintainability / Extendability
Performance
Readability
Is this reasonable? If it is ok to use, are there any ideas on how to design a possible interface for that?
You could use a flexible array member instead of a pointer:
struct entry {
size_t buf_len;
int something;
char buf[];
};
// ...
struct entry *e = malloc(sizeof *e + buf_len);
e->buf_len = buf_len;
Portability and performance are fine. Readability: not perfect but good enough.
Extendability: you can't use this for more than one member at a time, you'd have to fall back to your explicit pointer version. Also, the explicit pointer version means that you have to muck around to ensure correct alignment if you use it with a type that doesn't have an alignment of 1.
If you are seriously thinking about this I'd consider revisiting your entire data structure's design to see if there is another way of doing it. (Maybe this way is actually the best way, but have a good think about it first).
As to portability, I am unaware of any issues, as long as the sizes are found via suitable calls to sizeof(), as in your code.
Regarding maintainability, extendability and readability, you should certainly wrap allocation and de-allocation in a well-commented function. Calls to...
entry *allocate_entry_with_buffer();
void deallocate_entry_with_buffer(entry **entry_with_buffer);
...do not need to know implementation details of how the memory actually gets handled. People use stranger things like custom allocators and memory pools quite frequently.
As for speed, this is certainly faster than making lots of small allocations. I used to allocate whole 2D matrices with a similar strategy...
It should work, but in fact you are using a pointer for a useless indirection. Windows API (for example) uses another method for variable size structs : the variable size buffer is last in struct and is declared to be char buf[1].
Your struct would become :
struct entry {
size_t buf_len;
int something;
char buf[1];
};
The allocation is (still no error checking) :
size_t buf_len = 4; // size of the buffer
struct entry *e;
e = malloc( sizeof(*e) + buf_len - 1); // struct already has room for 1 char
e->buf_len = buf_len; // set buffer size
That's all e.buf is guaranteed to be a char array of size buf_len.
That way ensures that even if the variable part was not a character array but a int, long, or anything array, the alignement would be given by the last element being a array of proper type and size 1.
For starters, the line:
e->buf = e + sizeof(*e); // the buffer lies behind the struct
Should be:
e->buf = e + 1; // the buffer lies behind the struct
This is because e + 1 will be equal to the address at the end of the structure. As you have it, it will only be the number of bytes into the structure equal to the number of bytes in a pointer.
And, yes, it's reasonable. However, I prefer this approach:
struct entry {
size_t buf_len;
int something;
char buf[1];
};
This way, you don't mess with the pointers. Just append as many bytes as needed, and they will grow the size of your buf array.
Note: I wrote a text editor using an approach similar to this but used a Microsoft c++ extension that allowed me to declare the last member as char buf[]. So it was an empty array that was exactly as long as the number of extra bytes I allocated.
seems fine to me - put comments in though
Or you could do this - which is quite common
struct entry {
size_t buf_len;
int something;
char buf;
};
ie make the struct itself variable length. and do
size_t buf_len = 4; // size of the buffer
struct entry *e = NULL;
// check that it packs right
e = malloc(sizeof(size_t) + sizeof(int) + buf_len ); // allocate struct and buffer
e->buf_len = buf_len; // set buffer size
...... later
printf(&e.buf);
I have a small program that creates a semver struct with some variables in it:
typedef struct {
unsigned major;
unsigned minor;
unsigned patch;
char * note;
char * tag;
} semver;
Then, I would like to create a function which creates a semver struct and returns it to the caller. Basically, a Factory.
That factory would call an initialize function to set the default values of the semver struct:
void init_semver(semver * s) {
s->major = 0;
s->minor = 0;
s->patch = 0;
s->note = "alpha";
generate_semver(s->tag, s);
}
And on top of that, I would like a function to generate a string of the complete semver tag.
void generate_semver(char * tag, semver * s) {
sprintf( tag, "v%d.%d.%d-%s",
s->major, s->minor, s->patch, s->note);
}
My problem appears to lie in this function. I have tried returning a string, but have heard that mallocing some space is bad unless you explicitly free it later ;) In order to avoid this problem, I decided to try to pass a string to the function to have it be changed within the function with no return value. I'm trying to loosely follow something like DI practices, even though I'd really like to separate the concerns of these functions and have the generate_semver function return a string that I can use like so:
char * generate_semver(semver * s) {
char * full_semver;
sprintf( full_semver, "v%d.%d.%d-%s",
s->major, s->minor, s->patch, s->note);
return full_semver; // I know this won't work because it is defined in the local stack and not outside.
}
semver->tag = generate_semver(semver);
How can I do this?
My problem appears to lie in this function. I have tried returning a string, but have heard that mallocing some space is bad unless you explicitly free it later.
Explicitly freeing dynamically allocated memory is required to avoid memory leaks. However, it is not necessarily a task that the end users need to perform directly: an API often provides a function to deal with this.
In your case, you should provide a deinit_semver function that does the clean up of memory that init_semver has allocated dynamically. These two functions behave in a way that is similar to constructor and destructor; init_semver is not a factory function, because it expects the semver struct to be allocated, rather than allocating it internally.
Here is one way of doing it:
void init_semver(semver * s, int major, int minor, int pathc, const char * note) {
s->major = major;
s->minor = minor;
s->patch = pathc;
size_t len = strlen(note);
s->note = malloc(len+1);
strcpy(s->note, note);
s->tag = malloc(40 + len);
sprintf(s->tag, "v%d.%d.%d-%s", major, minor, patch, note);
}
void deinit_semver(semver *s) {
free(s->note);
free(s->tag);
}
Note the changes above: rather than using fixed values for the components of struct semver, this code takes the values as parameters. In addition, the code copies the note into a dynamically allocated buffer, rather than pointing to it directly.
The deinit function does the clean-up by free-ing both fields that were allocated dynamically.
A char * on its own is just a pointer to memory. To accomplish what you want you will either need to instead use a fixed size field, i.e. char[33], or you can dynamically allocate the memory as needed.
As it is, your generate_semver function is attempting to print to an unknown address. Let's look at one solution.
typedef struct {
unsigned major;
unsigned minor;
unsigned patch;
char note[32];
char tag[32];
} semver;
Now, in your init_semver function, the line previously s->note = "alpha"; will become a string copy, as arrays are not a valid lvalue.
strncpy(s->note, "alpha", 31);
s->note[31] = '\0';
strncpy will copy a string from the second parameter to the first up to the number of bytes in the third parameter. The second line ensures that a trailing null terminator is in place.
Similarly, in the generate_semver function, it would directly work in the buffer:
void generate_semver(semver * s) {
snprintf( s->tag, 32, "v%d.%d.%d-%s",
s->major, s->minor, s->patch, s->note);
}
This will directly print to the array in the structure, with a maximum character limit. snprintf does append a trailing null terminator (unlike strncpy), so we don't need to worry about adding it ourselves.
You mention having to free allocated memory, and then say: "In order to avoid this problem". Well, it's not so much a problem, but rather a necessity of the C language. It's common to have functions that allocate memory, and require the caller to free it again.
The idiomatic way is to have a pair of "create" and "destroy" functions. So I'd suggest doing it like this:
// Your factory function
semver* create_semver() {
semver* instance = malloc(sizeof(*instance));
init_semver(instance); // will also allocate instance->tag and ->note
return instance;
}
// Your destruction function
void free_semver(semver* s) {
free(semver->tag);
free(semver->note);
free(semver);
}
I'm pretty new at C programming, and this type of thing keeps popping up. As a simple example, suppose I have a struct http_header with some char pointers:
struct http_header {
char* name;
char* value;
};
I want to fill an http_header where value is the string representation of an int. I "feel" like, semantically, I should be able to write a function that takes in an empty header pointer, a name string, and an int and fills out the header appropriately.
void fill_header(struct http_header *h, char* name, int value)
{
h->name = name;
char *value_str = malloc(100);
sprintf(value_str, "%d", value);
h->value = value_str;
}
int main(int argc, const char * argv[])
{
struct http_header h;
char *name = "Header Name";
int val = 42;
fill_header(&h, name, val);
...
free(h.value);
}
Here, the calling code reads exactly as my intent, but in this case I'm creating the value string dynamically, which means I'd have to free it later. That doesn't smell right to me; it seems like the caller then knows too much about the implementation of fill_header. And in actual implementations it may not be so easy to know what to free: consider filling an array of http_headers where only one of them needed to have its value malloced.
To get around this, I'd have to create the string beforehand:
void fill_header2(struct http_header *h, char* name, char *value_str)
{
h->name = name;
h->value = value_str;
}
int main(int argc, const char * argv[])
{
struct http_header h;
char *name = "Header Name";
int value = 42;
char value_str[100];
sprintf(value_str, "%d", value);
fill_header2(&h, name, value_str);
}
As this pattern continues down the chain of structures with pointers to other structures, I end up doing so much work in top level functions the lower level ones seem hardly worth it. Furthermore, I've essentially sacrificed the "fill a header with an int" idea which I set out to write in the first place. I'm I missing something here? Is there some pattern or design choice that will make my life easier and keep my function calls expressing my intent?
P.S. Thanks to all at Stackoverfow for being the best professor I've ever had.
Well, I would go with the first approach (with a twist), and also provide a destroy function:
struct http_header *make_header(char *name, int value)
{
struct http_header *h = malloc(sizeof *h);
/* ... */
return h;
}
void destroy_header(struct http_header *h)
{
free(h->name);
free(h);
}
This way the caller doesn't have to know anything about http_header.
You might also get away with a version that leaves the main allocation (the struct itself) to the caller and does it's own internal allocation. Then you would have to provide a clear_header which only frees that fill allocated. But this clear_header leaves you with a partially-valid object.
I think your problem is simply that you are programming asymmetrically. You should once and for all decide who is responsible for the string inside your structure. Then you should have two functions, not only one, that should be called something like header_init and header_destroy.
For the init function I'd be a bit more careful. Check for a 0 argument of your pointer, and initialize your DS completely, something like *h = (http_header){ .name = name }. You never know if you or somebody will end up in adding another field to your structure. So by that at least all other fields are initialized with 0.
If you are new at C programming, you might perhaps want to use the Boehm's conservative garbage collector. Boehm's GC works very well in practice, and by using it systematically in your own code you could use GC_malloc instead of malloc and never bother about calling free or GC_free.
Hunting memory leaks in C (or even C++) code is often a headache. There are tools (like valgrind) which can help you, but you could decide to not bother by using Boehm's GC.
Garbage collection (and memory management) is a global property of a program, so if you use Boehm's GC you should decide that early.
The general solution to your problem is that of object ownership, as others have suggested. The simplest solution to your particular problem is, however, to use a char array for value, i.e., char value[12]. 2^32 has 10 decimal digits, +1 for the sign, +1 for the null-terminator.
You should ensure that 1) int is not larger than 32-bits at compile-time, 2) ensure that the value is within some acceptable range (HTTP codes have only 3 digits) before calling sprintf, 3) use snprintf.
So by using a static array you get rid of the ownership problem, AND you use less memory.
I am writing a light weight serialization function and need to include two variable sized arrays within this.
How should I track the size of each?
How should I define the struct?
Am I going about this all wrong?
EDIT: the result must be a contiguous block of memory
This resolves to something like
typedef struct
{
size_t arr_size_1, arr_size_2;
char arr_1[0/*arr_size_1 + arr_size_2*/];
} ...;
The size(s) should be in the front of the dynamic sized data, so that it doesn't move when expanding your array.
You cannot have 2 unknown sized arrays in your struct, so you must collapse them into one and then access the data relative from the first pointer.
typedef struct MyStruct_s
{
int variable_one_size;
void* variable_one_buf;
int variable_two_size;
void* variable_two_buf;
} MyStruct;
MyStruct* CreateMyStruct (int size_one, int size_two)
{
MyStruct* s = (MyStruct*)malloc (sizeof (MyStruct));
s->variable_one_size = size_one;
s->variable_one_buf = malloc (size_one);
s->variable_two_size = size_two;
s->variable_two_buf = malloc (size_two);
}
void FreeMyStruct (MyStruct* s)
{
free (s->variable_one_buf);
free (s->variable_two_buf);
free (s);
}
Since the data should be continuous in memory it is necessary to malloc a chunk of memory of the right size and manage it's contents more or less manually. You probably best create a struct that contains the "static" information and related management functions that do the memory management and give access to the "dynamic" members of the struct:
typedef struct _serial {
size_t sz_a;
size_t sz_b;
char data[1]; // "dummy" array as pointer to space at end of the struct
} serial;
serial* malloc_serial(size_t a, size_t b) {
serial *result;
// malloc more memory than just sizeof(serial), so that there
// is enough space "in" the data member for both of the variable arrays
result = malloc(sizeof(serial) - 1 + a + b);
if (result) {
result->sz_a = a;
result->sz_b = b;
}
return result;
}
// access the "arrays" in the struct:
char* access_a(serial *s) {
return &s->data[0];
}
char* access_b(serial *s) {
return &s->data[s->sz_a];
}
Then you could do things like this:
serial *s = ...;
memcpy(access_a(s), "hallo", 6);
access_a(s)[1] = 'e';
Also note that you can't just assign one serial to another one, you need to make sure that the sizes are compatible and copy the data manually.
In order to serialize variably-sized data, you have to have a boundary tag of some sort. The boundary tag can be either a size written right before the data, or it can be a special value that is not allowed to appear in the data stream and is written right after the data.
Which you choose depends on how much data you are storing, and if you are optimizing for size in the output stream. It is often easier to store a size before-hand, because you know how big to make the receiving buffer. If you don't then you have to gradually resize your buffer on load.
In some ways, I'd do things like Dan Olson. However:
1) I'd create the final struct by having two instances of a simpler struct that has just one variable array.
2) I'd declare the array with byte* and use size_t for its length.
Having said this, I'm still not entirely clear on what you're trying to do.
edit
If you want it contiguous in memory, just define a struct with two lengths. Then allocate a block big enough for both blocks that you want to pass, plus the struct itself. Set the two lengths and copy the two blocks immediately after. I think it should be clear how the lengths suffice to make the struct self-describing.