I am working with sending data to and from a client/server application. I have created my own header scheme. This is the flow I am in question about:
char *data = returnmydata();
int one = 1;
int two = 2;
int three = 3;
So what happens is I have a char buffer that contains all my data. But I need to append to the front of this data my one, two, and three integers which is used to let the other side know how to handle the data.
Instead of allocating enough at the start for the 3 header items, is it possible to reallocate an addition 12 bytes and move the start of the data to data+12?
I currently just memcpy the 3 integers at the start then offset the data +12. However I think my code would be better if I could just pass a struct that contains the 3 ints and a pointer to the data. Then I can use this function to create the full data buffer with the headers in it.
Another option is to alloc another buffer memcpy the 3 integers to the front, and then memcpy the data to the new buffer. However I would like to keep memory usage low.
Use functions realloc and memmove.
If the size of the initially allocated buffer is equal to some value N then you can write
char *tmp = realloc( data, N + 12 );
if ( tmp != NULL )
{
data = tmp;
memmove( data + 12, data, N );
// then you can insert some values in the first 12 bytes of the allocated buffer
}
Related
I am rewriting this post because I managed to figure out the problem. The problem with my extremely broken output was due to an improper dynamic memory allocation.
Basically I needed to allocate memory for an array of pointers that pointed to structs, but the array itself was nested inside of another struct and the nesting confused me slightly and I ended up over complicating it.
So I had a struct named Catalog, that my array was in and that array pointed to another struct named Books.
When I originally allocated memory for it I was only allocated memory for an array, not an array of pointers:
catalog->arrayP = malloc(INITIAL_CAPACITY * sizeof( Books );
// But I should have done this:
catalog->arrayP = (Books *) malloc(INITIAL_CAPACITY * sizeof( Books );
// That first (Books *) was extremely important
The second issue I was having was that when I was trying to update the memory to allow for more books I was actually decreasing it:
catalog->arrayP = realloc(catalog->arrayP, 2 * sizeof( catalog->arrayP));
// I did this thinking it would just increase the memory to twice that of what it currently was, but it didn't
cataloc->capacity = catalog->capacity * 2;
catalog->arrayP = realloc(catalog->arrayP, catalog->capacity * sizeof( catalog->arrayP));
So whenever I needed to grow my array of pointers I ended up just allocating enough memory for 2 books rather than double the current.
Frankenstein; Or, The Modern Prometh..Shelley, Mary Woll.
Your printing results kind of give away the answer. You forgot the null terminator on your strings and printf invaded the next field until reached the null terminator.
In the following fields it couldn't find and invaded even more stuff.
Here's a minimal example
#include <stdio.h>
#include <string.h>
struct test{
char test[37]; // space for 36 chars + null
char test2[16]; // space for 15 chars + null
};
int main(void) {
struct test Test;
strcpy(Test.test, "randomrandomrandomrandomrandomrandom"); // Copy the 37 bytes
strcpy(Test.test2, "notnotnotnotnot"); // Copy the 16 bytes
//Replace null terminator with trash for demonstration purposes
Test.test[36] = '1'; // replaces 37th byte containing the terminator (\0) with trash
printf("%38s", Test.test); // should print randomrandomrandomrandomrandomrandom1notnotnotnotnot
return 0;
}
I am developing a c program which connects to a mysql database reads data from a a table of 4 million of data and writes its data to another 100 tables in another database after some calculations. To make it efficient I tried to use 100 threads to write data for 100 tables and 1 thread to read data from the database and write those in to a buffer. So that the 100 threads would read from the buffers.
But the problem is when I'm making the buffers. I used malloc to make the buffers. char *** queue; is declared in the header file so that it is global
int i = 0, j = 0;
queue = (char ***) malloc(100);
int threadRet;
for (i; i < 100; i++) {
queue[i] = (char **) malloc(2000);
for (j; j < 2000; j++) {
queue[i][j] = (char *) malloc(180);
}
}
and My buffer writing thread as I mentioned before exicutes the function void * thrededChunkPicker(void * parr)
sprintf(queue[tableNo][marker[tableNo]], "INSERT INTO usage_summary%s(mobile,`0`,ddate) VALUES('%s',%d,'%s') ON DUPLICATE KEY UPDATE `0` = VALUES(`0`), ddate=VALUES(ddate)", row[0] + 7, row[0], sumOfUsage, row[2]);
(marker[tableNo])++;
This is how I write to the buffer . As I've found out the segmentation fault occurs here.
I needed 100 buffers in each of which contains 2000 string arrays of 180. this code compiled successfully.but when runs it gives a segmentation fault.
As far as I can see, all your malloc() calls are problematic.
We pass the size in bytes to malloc(). malloc() does not have any idea of the size of the object in which you're going to store the returned pointer, whatsoever. So, it cannot automatically calculate the total size for the memory required.
We need to calculate the total size required and pass that size to malloc().
First to start with, please see this discussion on why not to cast the return value of malloc() and family in C..
That said, assuming queue is defined as char *** queue;, we need to
queue = malloc(100 * sizeof*queue );
queue[i] = malloc(2000 * sizeof(*queue[i]) );
and so on, if required.
Finally, always check for the success of malloc() through a NULL check on the returned pointer before using the same.
The biggest problem is that malloc allocates a number of bytes, not elements in an array. So you allocate 100 bytes for your three-star variable, which is not enough space for 100 elements. The same with the next call to malloc, it allocates 2000 bytes not 2000 elements.
I have problem with push to front new elements to char array. I have in my program a function which takes two arguments: a pointer to a char array, and a pointer to an int. This function fills an array with some data, and returns the count of elements in this array.
int function1( char* buffer , int* outSize);
buffer[1000]
int size;
result = function1(buffer , &size);
// after this a get some data in buffer and count how many data is in buffer. let's say 456 elements
// now a try push to front two new bytes as size this array
finallBuffer *char = new char[size+2];
memcpy(finallyBuffer, (char*)&size, 2); // here a get size in two bytes and put to finnalBuffer
// next I try just copy to finnalyBuffer received data from Buffer
strcat(finallyBuffer , buffer);
doSomething(finallyBuffer);
delete []finallyBuffer;
After this in finnalyBuffer I have only saved two bytes with size. Why don't I see data from buffer?
In finally I want to achieve new table with two new bytes in front. This two bytes is size old table. Let's say received data heve only 5 characters
char table[5] = {'a','b','b','c','t'};
So size is 5. 5 in two bytes is char size = {'5','0'};
Result should be looks like.
char table[5] = {'5','0','a','b','b','c','t'};
I am not sure to understand exactly what you want to do but i have tried to figure it. See comments in code bellow i hope it will help you.
int function1( char* buffer , int* outSize);
buffer[1000]
int size;
result = function1(buffer , &size);
// after this a get some data in buffer and count how many data is in buffer. let's say 456 elements
// now a try push to front two new bytes as size this array
char *finallBuffer = new char[size*sizeof(char)+2];
short ssize = (short)size;
memcpy(finallyBuffer, &ssize, 2); // here a get size in two bytes and put to finnalBuffer
// next I try just copy to finnalyBuffer received data from Buffer
/* NOTE: strcat is for null terminated string. If i understand what you want
to do you want store int in 4 bytes ?
I assume size is the actual size of the buffer
*/
//strcat(finallyBuffer , buffer);
memcpy(&finallyBuffer[2], buffer, sizeof(char)*size);
doSomething(finallyBuffer);
delete []finallyBuffer;
Those lines are wrong:
finallBuffer *char = new char[size+2];
memcpy(finallyBuffer, (char*)&size, 2);
They should be:
char * finalBuffer = new char[size+2];
memcopy(finallyBuffer, buffer, size);
But I really understand few things of what You are trying to do.
Edit:
Now I see what you are doing and Olivier gave you the answer. If you don't want (or can't) use std::vector for your buffer you can save the cost of reallocating the array, anyway.
Just start allocating for buffer 2 more bytes: char buffer[1002];.
Then to store values you can pass the pointer to the third element: char* ptr = buffer + 2;.
When you know the real size of the data, just put the bytes in buffer[0] and buffer[1].
A struct should be advised.
So I have this struct
#define MAX 128
typedef struct this_struct {
Type items[MAX];
} *SVar;
Lets say we create something like this
SVar first = malloc(sizeof(struct this_struct));
Now when I push values into the array and it fills to the MAX which is 128, I need to dynamically create a new array but I don't know how since the array is inside.
Here are my current thoughts on how I want to do it:
Create a new SVar names "second" with second->items[MAX *2]
free(first)
How can I go about doing this?
The typical way to do that is make your struct contain three values: first, a pointer to an array of variables, and second a count of the currently allocated array size, and in practice, you will need a third item to track the number of array slots you're actually using.
So, with your struct, it would be something like this:
Type *items;
int item_length; /* Number allocated */
int item_count; /* Number in use */
you initially allocate a "batch" of entries, say 100:
first = malloc(sizeof(this_struct));
first->items = malloc(sizeof(Type) * 100);
first->item_length = 100;
first->item_count = 0;
Then you add items one at a time. Simplistically, it's this:
first->items[first->item_count] = new_item;
first->item_count += 1;
But really you need to make sure each time you're not going to overflow the currently-allocated space, so it's really like this:
if (first->item_count == first->item_length) {
first->item_length += 100;
first->items = realloc(first->items, sizeof(Type) * first->item_length);
}
first->items[first->item_count] = new_item;
first->item_count += 1;
You're basically just using slots one at a time as long as your currently allocated space is large enough. Once you've used all the space you've allocated, realloc will either extend the array in place if there is room in the address space, or it will find and allocate a new larger space and move all the existing data to the new spot (and freeing up the old space).
In practice, you should check the return valueon the malloc and realloc calls.
A usual trick is to do something like this:
typedef struct {
int max_items; /* enough memory allocated for so many items */
...
Whatever_type items[1]; /* must be the last member */
} Dyn_array;
...
int n = 127;
Dyn_array *p = malloc(sizeof(Dyn_array) + n*sizeof(p.items[0]);
p->max_items = n + 1;
...
n = 1023;
p = realloc(p, sizeof(Dyn_array) + n*sizeof(p.items[0]);
p->max_items = n + 1;
and so on. The code using the structure performs out-of-bound reads and writes to the items array, which is declared to store one item only. This is OK, however, since C does not do any bounds checking, and the memory allocation policy must guarantee that there is always enough space available for num_items items.
I more or less have an idea, but I'm not sure if I've even got the right idea and I was hoping maybe I was just missing something obvious. Basically, I have and array of strings (C strings, so basically an array of pointers to character arrays) like so:
char **words;
Which I don't know how many words I'll have in the end. As I parse the string, I want to be able to resize the array, add a pointer to the word, and move on to the next word then repeat.
The only way I can think of is to maybe start with a reasonable number and realloc every time I hit the end of the array, but I'm not entirely sure that works. Like I want to be able to access words[0], words[1], etc. If I had char **words[10] and called
realloc(words, n+4) //assuming this is correct since pointers are 4 bytes
once I hit the end of the array, if I did words[11] = new word, is that even valid?
Keep track of your array size:
size_t arr_size = 10;
And give it an initial chunk of memory:
char **words = malloc( arr_size * sizeof(char*) );
Once you have filled all positions, you may want to double the array size:
size_t tailIdx = 0;
while( ... ) {
if( tailIdx >= arr_size ) {
char **newWords;
arr_size *= 2;
newWords = realloc(words, arr_size * sizeof(char*) );
if( newWords == NULL ) { some_error() };
words = newWords;
}
words[tailIdx++] = get_next_word();
}
...
free(words);
That approach is fine ,although you may want to do realloc(words, n * 2) instead. calling realloc and malloc is expensive so you want to have to reallocate as little as possible and this means you can go for longer without reallocating (and possibly copying data). This is how most buffers are implemented to amortize allocation and copy costs. So just double the size of your buffer every time you run out of space.
You are probably going to want to allocate multiple blocks of memory. One for words, which will contain the array of pointers. And then another block for each word, which will be pointed to by elements in the words array.
Adding elements then involves realloc()ing the words array and then allocating new memory blocks for each new word.
Be careful how you write your clean up code. You'll need to be sure to free up all those blocks.