A buffer that holds binary chunks of data is supposed to be copied to an array of structs (each struct represents a chunk), each chunk in the buffer is 20 bytes, first 4 bytes hold the hash value, then 8 bytes for an offset info, then 8 for a size:
thats the struct definition:
typedef struct{
khint_t hash; // 4
long int offset_start; // 8
size_t size; // 8
} header_offset, *offset_p;
and below is the code that's supposed to do whats mentioned:
offset_p *offsets;
size_t s= HEADER_OFFSET_SIZE;
header_offset t_offsets[n_files];
for (i=0; i< n_files; i++){
memcpy(&t_offsets[i].hash, buff, sizeof(khint_t));
buff+= sizeof(khint_t);
memcpy(&t_offsets[i].offset_start, buff, sizeof(long int));
buff+= sizeof(long int);
memcpy(&t_offsets[i].size, buff, sizeof(size_t));
buff+= sizeof(size_t);
printf("hash read: %p\n", t_offsets[i].hash);
printf("offset start read: %p\n", t_offsets[i].offset_start);
printf("offset size read: %p\n", t_offsets[i].size);
}
memmove(offsets, t_offsets, sizeof(header_offset)*n_files);
buff-= s*n_files;
free(buff);
return offsets;
I was struggling copying the chunks directly into a header_p* so i decided to have a temporary struct array thats copied from the buffer, then gets copied to a header_p*, I'd appreciate it even more if you could provide me with a way to do it without using a temporary struct array.
The printfs print the right data, although when calling this function, the array of the pointers returned does not hold the right data, or the same data that was printed within the loop.
I'd like to know, without further code, whether its the way im using pointers that causes the array of offset_p's not hold the right values.
No memory has been allocated for offsets, memmove() does not allocate the memory for you:
header_offset* offsets = malloc(sizeof(header_offset)*n_files);
Given that your are allocating memory to be returned the use of t_offsets is unrequired: just populate offsets directly.
EDIT:
To return a header_offset*[] as commented by alk:
header_offset** offsets = malloc(sizeof(header_offset*) * n_files);
for (i=0; i< n_files; i++){
*(offsets + i) = malloc(sizeof(header_offset));
...
}
offsets is supposedly an array of pointers, not structs.
The memcopy will probably overwrite a large chunk of memory past the end of the memory allocated for offsets (allthough we can't see how much memory was allocated for it).
I'd suggest:
offset_p *offsets = (offset_p*)malloc(sizeof(offset_p)*n_files);
And then a loop
for (i=0; i< n_files; i++){
offsets[i] = malloc(sizeof(header_offset));
memcpy(&(*offsets[i]).hash, buff, sizeof(khint_t));
buff+= sizeof(khint_t);
memcpy(&(*offsets[i]).offset_start, buff, sizeof(long int));
buff+= sizeof(long int);
memcpy(&(*offsets[i]).size, buff, sizeof(size_t));
buff+= sizeof(size_t);
}
To return an array of pointers to header_offset the array itself needs to be allocated (then referenced by offset_p *offsets;
) as well as the memory holding the data currently held by the temporary structures. The latter then will be references by the pointers held by offsets.
Related
I try something like below but all the time I have a segmentation fault.
I don't really want to use (e.g.) #define N 1000 and then declare int buffer[N].
Just in case..I'm not allowed to use any headers except stdio.h as well as dynamic memory.
void input (int *buffer, int *length);
int main()
{
int length, *buffer = NULL, *numbers = NULL;
input(buffer, &length);
}
void input(int *buffer, int *length) {
scanf("%d", length);
if (*length < 0) {
error = 1;
return;
}
for (int i = 0; i < *length; i++) {
scanf("%d", *buffer[i]);
}
}
How to pass an array with unknown 1-d dimension into function
In C, arrays cannot exist until their size is known.
There are other approaches though.
In C, code cannot pass an array to a function. some_function(some_array) converts the array some_array to the address of the first element of the array: &some_array[0]. That is what the function receives, a pointer, not an array. The original size information of the array is not passed, thus also pass the length to the function.
Sample:
Read desired length.
{
int length = 0;
scanf("%d", &length);
Form a variable length array, length >= 1.
if (length <= 0) {
return NULL;
}
int buffer[length];
Now call a function, passing the length and the address of the first element of the array.
// Do stuff with length and buf, like read data
foo1(length, buffer);
// foo1() receives the length & address of the first element of the array as an int *
// Do more stuff with length and buf, like write data
foo2(length, buffer);
}
At the end of the block }, buffer no longer available.
In C, you can't create an array if you can't know its size at compile time (or at least not in certain implementations and standards), so doing something like buffer[length] won't work (again at least not in certain implementations/standards).
What you need to do to make sure this works everywhere is to use a pointer (as I see you're trying to use here). However, what you're doing wrong here that causes your segfault with the pointers is you assign them the value of NULL. This also won't work due to how when you assign a pointer an arbitrary value, there is no memory allocated for the pointer (This applies for everything other than addresses of "regular" variables using the & operator and assigning other pointers that are checked to be OK). Your pointers are just pointing to address 0 and can't be used for anything.
What you need to do here to fix the pointers is to use dynamic memory allocation, so you can have a truly variable-sized array. Specifically, you need to use a function like malloc or calloc to allocate memory for the pointers so they are usable. In your case, using calloc and reading its documentation, we see that it takes 2 parameters: The number of elements it should allocate memory for and the size of each element. We also know that it returns a pointer to the starting address of the allocated memory and that in case of failure (which can only happen if you're out of memory), it returns NULL. Using this, we understand that in your case the call to calloc would be like this:
int *buffer = (int *) calloc(length, sizeof(int));
The sizeof() function returns the size of a data type in bytes. Here you allocated enough memory for the pointer to hold length integers (since you'll use it as an array you need enough memory for all the integers, you're not just pointing to 1 integer but storing all of them), and calloc is also noted to initialize every allocated element to 0, so you have an array of integers that are all initialized to 0 (Also note that type casting has been used to make sure the allocated memory block is appropriate for use with an integer array, you can read more about type casting in this small article from Tutorialspoint if you'd like). Then, after this has been allocated, you can start reading your integers into the array. The complete code looks like this:
void input (int *buffer, int *length);
int main() {
// NOTE: I don't see the numbers pointer used here, maybe remove it?
int length, *buffer, *numbers;
input(buffer, &length);
}
void input(int *buffer, int *length) {
scanf("%d", length);
if (*length < 0) {
// Consider printing the exact error here
error = 1;
return;
}
buffer = (int *) calloc(length, sizeof(int));
if (buffer == NULL) {
printf("Couldn't allocate memory for buffer\n");
error = 1;
return;
}
// Accessing the elements of an array doesn't need * and in fact * here can (and probably will) cause terrible things
for (int i = 0; i < *length; i++) {
scanf("%d", buffer[i]);
}
}
Also don't forget to call free() on the pointer after you're done using it to avoid memory leaks (in your case that'd be after the call to input()).
Hope this helped, good luck!
You cannot use arrays because their memory size must be known to the compiler at compile time. Also you can't use Variable Length Arrays because they are allocated at the point of declaration and deallocated when the block scope containing the declaration exits.
The solution to your problem might be to use malloc
So, to start off I've already looked at a few questions including this one and none of them seem to help.
I'm simply trying to write a function that extends the size of an array using realloc().
My code currently looks like this:
unsigned char *xtnd = malloc(4);
xtndc(&xtnd, 4);
// sizeof(*xtnd) should now be 8
void xtndc ( unsigned char ** bytesRef , uint8_t count ) {
*bytesRef = realloc(*bytesRef, (sizeof(**bytesRef)) + count);
}
But no matter what I do it seems that the size of xtnd is always 4. After running xtndc() on it it should now be 8 bytes long.
Any suggestions?
The type of **bytesRef is unsigned char, so sizeof(**bytesRef) is 1. sizeof doesn't keep track of dynamic allocations, it's a compile time tool that gives you the size of a type, in this case unsigned char.
You have to keep track of the array size manually to calculate the new required size.
Your program does in fact change the size of the memory block. It changes the size of your original memory block from 4 bytes to 5 bytes. It changes to 5 bytes because you are essentially doing sizeof(unsigned char) + 4 which 1 + 4 = 5. If you want to double the size instead, do count*sizeof(unsigned char) + count. There are two points to be noted here:
The sizeof function returns the size of the data type, not the size of the allocated bytes. There is no way to know the size of the dynamically allocated memory.
The function realloc (and malloc and calloc as well) is not always guaranteed to return the requested reallocation. It may or may not succeed all the time.
I fixed the problem with the following code.
typedef struct CArrPtr {
unsigned char* ptr;
size_t size;
} CArrPtr;
void xtndc ( CArrPtr *bytesRef, uint8_t count );
. . .
CArrPtr xtnd = { .ptr = malloc(4), .size = 4 };
xtndc( &xtnd, 4 );
// xtnd.size is now 8 bytes
. . .
void xtndc ( CArrPtr *bytesRef, uint8_t count ) {
unsigned char *nptr;
if((nptr = realloc(bytesRef->ptr, bytesRef->size + count)) != 0)
{
bytesRef->ptr = nptr;
bytesRef->size = bytesRef->size + count;
}
}
As I am somewhat new to C, what I learned from this is that malloc specifically creates a pointer to a memory block, but you have no direct access to information about the memory block. Instead, you must store the size of the array that you created with malloc somewhere as well.
Since in the past I'd been initializing arrays with unsigned char arr[size]; and then using sizeof on it, I was under the impression that sizeof returned the size of the array, which is of course wrong as it gives you the size of a type.
Glad I could learn something from this.
sizeof is used to calculate size of data type or array. Pointer and array are very similar, but they are different things. For int *ap, sizeof(ap) will return 4 on x86, sizeof(*ap) will return 4; for int a[10], sizeof(a) will return 40.
sizeof expression is processed at compile time, so it will be a constant written into the executable file before you run the program.
malloc and realloc don't maintain size.
If realloc succeeds, it will reallocate the requested size. So you don't need to check the size after realloc returns, but you should check the return value of realloc to ensure that realloc succeeds.
How can i extract bytes from offset offset of tvb with length length length? type of tvb is :
uint8_t *tvb;
uint8_t *extractBytes(uint8_t *tvb, guint8 offset, guint8 length)
{
// do ...
// extract bytes and return
}
I don't know how can I do this ?
Thanks in advance.
You will need to allocate memory for the extracted bytes. Then it's a simple matter of copying the correct bytes:
uint8_t *extractBytes(uint8_t *tvb, guint8 offset, guint8 length)
{
uint8_t *new = malloc (length);
if (new) {
mempcy (new, tvb+offset, length);
}
return new; /* Returns NULL on allocation failure */
}
Don't forget to free() the allocated memory once you are done using it.
Note that the malloc() call above allocates length number of bytes. If you want to allocate memory for elements that are larger than uint8_t, you will have to multiply the amount by the size of the element. To be on the safe side, and to guard against future changes, you can allocate the memory as follows:
new = malloc (length * sizeof *new);
Now, sizeof *new will always be the correct element size.
If you don't need to change the extracted bytes, and the original buffer (tvb) doesn't change while you're using the bytes, you can access them directly using something like:
int i;
for (i = 0; i < length; i++) {
do_something (tvb[offset+i]);
}
I'm trying to allocate an array 64 bytes in size and then loop over the array indexes to read a byte each from the inputfile. but when I don't malloc() the array indexes, the loop stays in index0 (so each time it loops, it replaces the content in index0 with the next byte, instead of putting each byte in the next array index and keeping them all chronological).
When I use malloc() it uses the array indexes properly, but it's an infinite loop and uses gigs of ram.
Here's my code:
struct BitIO {
FILE *FilePointer;
uint8_t *Buffer[64];
uint64_t BitsAvailable;
uint64_t BitsUnavailable;
} BitIO;
void Init_BitIO(const char *FileName, const char *Mode) {
BitIO.FilePointer = fopen(FileName, Mode);
malloc(sizeof(BitIO));
while (!feof(BitIO.FilePointer)) {
size_t BytesRead = 0;
for (int i = 0; i < 64; i++) {
BitIO.Buffer[i] = (uint8_t*)malloc(1);
BytesRead = fread(BitIO.Buffer[i], 1, 1, BitIO.FilePointer);
}
}
}
If you're "trying to allocate an array 64 bytes in size", you may consider
uint8_t Buffer[64];
instead of
uint8_t *Buffer[64];
(the latter is an array of 64 pointers to byte)
After doing this, you will have no need in malloc as your structure with a 64 bytes array inside is allocated statically.
The 'main' loop will look like
for (int i = 0; i < 64; i++) {
BytesRead += fread(&BitIO.Buffer[i], 1, 1,BitIO.FilePointer);
}
But, of course, I would advise a more efficient form:
BytesRead = fread(BitIO.Buffer, 1, 64, BitIO.FilePointer);
Point 1
You need to collect the return value of malloc() into some variable (and check for malloc() success before you use the returned pointer) to make use of the allocated memory. Then, seeing your usage, I believe you're confused with the struct member variable types. You don't need a uint8_t *Buffer[64]; as the struct member based on your usage.
1.1. If you want to use dynamic memory, then, change the structure member as
uint8_t *Buffer;
and inside for loop you do
BitIO.Buffer[i] = malloc(sizeof(uint8_t)); //allocate memory
BytesRead = fread(BitIO.Buffer[i], 1, 1,BitIO.FilePointer);
or, better, as you're looping for fixed number of time, you can get the memory allocated at one time outside the for loop
BitIO.Buffer = malloc( 64 * sizeof(uint8_t));
and then loop to read one element at a time.
1.2. Otherwise, change the structure member as
uint8_t Buffer[64];
and get rid of the malloc() completely.
Point 2:
Read Why is “while ( !feof (file) )” always wrong?
Point 3:
Please see why not to cast the return value of malloc() and family in C.
I would like to allocate memory for a buffer that will contain, via memcpy in the future, a struct that contains a pointer that has been previously dynamically allocated memory.
That is, I have a struct
struct test_struct {
int num;
char *values;
};
Where test_struct.values contains num amount of strings of length LENGTH. I know I can't get the size of memory a pointer has been allocated, so I just keep track of it via num. What is the easiest/cleanest way of getting the size of this struct?
The only solution I can come up with is something like
buf = malloc(sizeof(test_struct) + (num * LENGTH));
But I'm new to this low-level memory management stuff, so there might be something better.
If you would like to memcpy two structs then the memory in both of them must be continuous. But you would have to determine num beforehand.
struct test_struct {
int num;
char ** values;
} * TestStruct;
int _num = 0;
// find _num
TestStruct = malloc (sizeof (struct test_struct) + (sizeof(char*) * _num) + (LENGTH * _num));
TestStruct->num = _num;
TestStruct->values = &TestStruct + sizeof (struct test_struct);
for (int i = 0; i < _num; i++){
TestStruct->values[i] = &TestStruct + sizeof (struct test_struct) + (i * LENGTH);
}
The reason I changed char * to char ** is because using char * it becomes harder to access the strings after the first (I'm assuming they're null terminated). Also, after calling memcpy, you must update all the string pointers in the new struct.
To memcpy you would do this:
memcpy (buf, TestStruct->values[0], LENGTH * TestStruct->num);
But in buf, however, you would only see the first string (unless your strings are not null-terminated). You would have to increment the pointer after every null terminated character until you know, with num, that you've reached the end of the buffer.
Now that I understand more of the context of your request, consider the following.
If you're using UDP packets, you should send the data in one packet so that it arrives in the order you expect. When more than one packet is sent, it may arrive out of order. Because of this, you need to make sure the size of the data is <= 512 bytes - which is the maximum size of a UDP packet. Also, you need to make sure all the data is in contiguous memory. I'm going to assume you have your data already in the struct you've provided in this example:
// this function puts the struct in contiguous memory
int PrepareBuffer (struct test_struct TestStruct, char ** buffer){
char * cast = (char *) &TestStruct->num;
* buffer = malloc ((TestStruct->num * LENGTH) + sizeof (int));
for (int i = 0; i < sizeof (int); i++) *buffer[i] = cast[i];
for (int i = 0; i < (TestStruct->num * LENGTH); i++) *buffer[i + sizeof (int)] = TestStruct->values[i];
return 0;
}
You will have to implement another function on the receiving end that maps the buffer to struct test_struct. Also, I have omitted error checking for clarity. You should check for how big the packet is going to be before to allocate memory (it has to be <= 512). You should also check to make sure malloc returns a none-null pointer.
You should only need to allocate 4 bytes (for the integer on 32 bit linux) and 4 bytes for the char * (in 32 bit. 64 is 8).
What you're really asking though, is how do I know how much memory I need to allocate to the region pointed to by char *value. You figure this out in the wya you're doing. Then set value to the location of buf. There's a comment blow me that is the correct way if you have multiple string, and you don't want to just jam them all together in that region and have to figure out which is which yourself.
I'm assuming that you want to allocate memory for both the structure and the buffer that values points to. If so, this is correct. To point at the extra space, do buf->values = buf + 1; (this is assuming you declare buf as struct test_struct buf;