I'm trying to allocate an array 64 bytes in size and then loop over the array indexes to read a byte each from the inputfile. but when I don't malloc() the array indexes, the loop stays in index0 (so each time it loops, it replaces the content in index0 with the next byte, instead of putting each byte in the next array index and keeping them all chronological).
When I use malloc() it uses the array indexes properly, but it's an infinite loop and uses gigs of ram.
Here's my code:
struct BitIO {
FILE *FilePointer;
uint8_t *Buffer[64];
uint64_t BitsAvailable;
uint64_t BitsUnavailable;
} BitIO;
void Init_BitIO(const char *FileName, const char *Mode) {
BitIO.FilePointer = fopen(FileName, Mode);
malloc(sizeof(BitIO));
while (!feof(BitIO.FilePointer)) {
size_t BytesRead = 0;
for (int i = 0; i < 64; i++) {
BitIO.Buffer[i] = (uint8_t*)malloc(1);
BytesRead = fread(BitIO.Buffer[i], 1, 1, BitIO.FilePointer);
}
}
}
If you're "trying to allocate an array 64 bytes in size", you may consider
uint8_t Buffer[64];
instead of
uint8_t *Buffer[64];
(the latter is an array of 64 pointers to byte)
After doing this, you will have no need in malloc as your structure with a 64 bytes array inside is allocated statically.
The 'main' loop will look like
for (int i = 0; i < 64; i++) {
BytesRead += fread(&BitIO.Buffer[i], 1, 1,BitIO.FilePointer);
}
But, of course, I would advise a more efficient form:
BytesRead = fread(BitIO.Buffer, 1, 64, BitIO.FilePointer);
Point 1
You need to collect the return value of malloc() into some variable (and check for malloc() success before you use the returned pointer) to make use of the allocated memory. Then, seeing your usage, I believe you're confused with the struct member variable types. You don't need a uint8_t *Buffer[64]; as the struct member based on your usage.
1.1. If you want to use dynamic memory, then, change the structure member as
uint8_t *Buffer;
and inside for loop you do
BitIO.Buffer[i] = malloc(sizeof(uint8_t)); //allocate memory
BytesRead = fread(BitIO.Buffer[i], 1, 1,BitIO.FilePointer);
or, better, as you're looping for fixed number of time, you can get the memory allocated at one time outside the for loop
BitIO.Buffer = malloc( 64 * sizeof(uint8_t));
and then loop to read one element at a time.
1.2. Otherwise, change the structure member as
uint8_t Buffer[64];
and get rid of the malloc() completely.
Point 2:
Read Why is “while ( !feof (file) )” always wrong?
Point 3:
Please see why not to cast the return value of malloc() and family in C.
Related
I try something like below but all the time I have a segmentation fault.
I don't really want to use (e.g.) #define N 1000 and then declare int buffer[N].
Just in case..I'm not allowed to use any headers except stdio.h as well as dynamic memory.
void input (int *buffer, int *length);
int main()
{
int length, *buffer = NULL, *numbers = NULL;
input(buffer, &length);
}
void input(int *buffer, int *length) {
scanf("%d", length);
if (*length < 0) {
error = 1;
return;
}
for (int i = 0; i < *length; i++) {
scanf("%d", *buffer[i]);
}
}
How to pass an array with unknown 1-d dimension into function
In C, arrays cannot exist until their size is known.
There are other approaches though.
In C, code cannot pass an array to a function. some_function(some_array) converts the array some_array to the address of the first element of the array: &some_array[0]. That is what the function receives, a pointer, not an array. The original size information of the array is not passed, thus also pass the length to the function.
Sample:
Read desired length.
{
int length = 0;
scanf("%d", &length);
Form a variable length array, length >= 1.
if (length <= 0) {
return NULL;
}
int buffer[length];
Now call a function, passing the length and the address of the first element of the array.
// Do stuff with length and buf, like read data
foo1(length, buffer);
// foo1() receives the length & address of the first element of the array as an int *
// Do more stuff with length and buf, like write data
foo2(length, buffer);
}
At the end of the block }, buffer no longer available.
In C, you can't create an array if you can't know its size at compile time (or at least not in certain implementations and standards), so doing something like buffer[length] won't work (again at least not in certain implementations/standards).
What you need to do to make sure this works everywhere is to use a pointer (as I see you're trying to use here). However, what you're doing wrong here that causes your segfault with the pointers is you assign them the value of NULL. This also won't work due to how when you assign a pointer an arbitrary value, there is no memory allocated for the pointer (This applies for everything other than addresses of "regular" variables using the & operator and assigning other pointers that are checked to be OK). Your pointers are just pointing to address 0 and can't be used for anything.
What you need to do here to fix the pointers is to use dynamic memory allocation, so you can have a truly variable-sized array. Specifically, you need to use a function like malloc or calloc to allocate memory for the pointers so they are usable. In your case, using calloc and reading its documentation, we see that it takes 2 parameters: The number of elements it should allocate memory for and the size of each element. We also know that it returns a pointer to the starting address of the allocated memory and that in case of failure (which can only happen if you're out of memory), it returns NULL. Using this, we understand that in your case the call to calloc would be like this:
int *buffer = (int *) calloc(length, sizeof(int));
The sizeof() function returns the size of a data type in bytes. Here you allocated enough memory for the pointer to hold length integers (since you'll use it as an array you need enough memory for all the integers, you're not just pointing to 1 integer but storing all of them), and calloc is also noted to initialize every allocated element to 0, so you have an array of integers that are all initialized to 0 (Also note that type casting has been used to make sure the allocated memory block is appropriate for use with an integer array, you can read more about type casting in this small article from Tutorialspoint if you'd like). Then, after this has been allocated, you can start reading your integers into the array. The complete code looks like this:
void input (int *buffer, int *length);
int main() {
// NOTE: I don't see the numbers pointer used here, maybe remove it?
int length, *buffer, *numbers;
input(buffer, &length);
}
void input(int *buffer, int *length) {
scanf("%d", length);
if (*length < 0) {
// Consider printing the exact error here
error = 1;
return;
}
buffer = (int *) calloc(length, sizeof(int));
if (buffer == NULL) {
printf("Couldn't allocate memory for buffer\n");
error = 1;
return;
}
// Accessing the elements of an array doesn't need * and in fact * here can (and probably will) cause terrible things
for (int i = 0; i < *length; i++) {
scanf("%d", buffer[i]);
}
}
Also don't forget to call free() on the pointer after you're done using it to avoid memory leaks (in your case that'd be after the call to input()).
Hope this helped, good luck!
You cannot use arrays because their memory size must be known to the compiler at compile time. Also you can't use Variable Length Arrays because they are allocated at the point of declaration and deallocated when the block scope containing the declaration exits.
The solution to your problem might be to use malloc
My current concat function:
char* concat(char* a, int a_size,
char* b, int b_size) {
char* c = malloc(a_size + b_size);
memcpy(c, a, a_size);
memcpy(c + a_size, b, b_size);
free(a);
free(b);
return c;
}
But this used extra memory. Is it possible to append two byte arrays using realloc without making extra memory space?
Like:
void append(char* a, int a_size, char* b, int b_size)
...
char* a = malloc(2);
char* b = malloc(2);
void append(a, 2, b, 2);
//The size of a will be 4.
While Jean-François Fabre answered the stated question, I'd like to point out that you can manage such byte arrays better by using a structure:
typedef struct {
size_t max; /* Number of chars allocated for */
size_t len; /* Number of chars in use */
unsigned char *data;
} bytearray;
#define BYTEARRAY_INIT { 0, 0, NULL }
void bytearray_init(bytearray *barray)
{
barray->max = 0;
barray->len = 0;
barray->data = NULL;
}
void bytearray_free(bytearray *barray)
{
free(barray->data);
barray->max = 0;
barray->len = 0;
barray->data = NULL;
}
To declare an empty byte array, you can use either bytearray myba = BYTEARRAY_INIT; or bytearray myba; bytearray_init(&myba);. The two are equivalent.
When you no longer need the array, call bytearray_free(&myba);. Note that free(NULL) is safe and does nothing, so it is perfectly safe to free a bytearray that you have initialized, but not used.
To append to a bytearray:
int bytearray_append(bytearray *barray, const void *from, const size_t size)
{
if (barray->len + size > barray->max) {
const size_t len = barray->len + size;
size_t max;
void *data;
/* Example policy: */
if (len < 8)
max = 8; /* At least 8 chars, */
else
if (len < 4194304)
max = (3*len) / 2; /* grow by 50% up to 4,194,304 bytes, */
else
max = (len | 2097151) + 2097153 - 24; /* then pad to next multiple of 2,097,152 sans 24 bytes. */
data = realloc(barray->data, max);
if (!data) {
/* Not enough memory available. Old data is still valid. */
return -1;
}
barray->max = max;
barray->data = data;
}
/* Copy appended data; we know there is room now. */
memmove(barray->data + barray->len, from, size);
barray->len += size;
return 0;
}
Since this function can at least theoretically fail to reallocate memory, it will return 0 if successful, and nonzero if it cannot reallocate enough memory.
There is no need for a malloc() call, because realloc(NULL, size) is exactly equivalent to malloc(size).
The "growth policy" is a very debatable issue. You can just make max = barray->len + size, and be done with it. However, dynamic memory management functions are relatively slow, so in practice, we don't want to call realloc() for every small little addition.
The above policy tries to do something better, but not too aggressive: it always allocates at least 8 characters, even if less is needed. Up to 4,194,304 characters, it allocates 50% extra. Above that, it rounds the allocation size to the next multiple of 2,097,152 and substracts 24. The reasoning behid this is complex, but it is more for illustration and understanding than anything else; it is definitely NOT "this is best, and this is what you should do too". This policy ensures that each byte array allocates at most 4,194,304 = 222 unused characters. However, 2,097,152 = 221 is the size of a huge page on AMD64 (x86-64), and is a power-of-two multiple of a native page size on basically all architectures. It is also large enough to switch from so-called sbrk() allocation to memory mapping on basically all architectures that do that. It means that such huge allocations use a separate part of the heap for each, and the unused part is usually just virtual memory, not necessarily backed by any RAM, until accessed. As a result, this policy tends to work quite well for both very short byte arrays, and very long byte arrays, on most architectures.
Of course, if you know (or measure!) the typical size of the byte arrays in typical workloads, you can optimize the growth policy for that, and get even better results.
Finally, it uses memmove() instead of memcpy(), just in case someone wishes to repeat a part of the same byte array: memcpy() only works if the source and target areas do not overlap; memmove() works even in that case.
When using more advanced data structures, like hash tables, a variant of the above structure is often useful. (That is, this is much better in cases where you have lots of empty byte arrays.)
Instead of having a pointer to the data, the data is part of the structure itself, as a C99 flexible array member:
typedef struct {
size_t max;
size_t len;
unsigned char data[];
} bytearray;
You cannot declare a byte array itself (i.e. bytearray myba; will not work); you always declare a pointer to a such byte arrays: bytearray *myba = NULL;. The pointer being NULL is just treated the same as an empty byte array.
In particular, to see how many data items such an array has, you use an accessor function (also defined in the same header file as the data structure), rather than myba.len:
static inline size_t bytearray_len(bytearray *const barray)
{
return (barray) ? barray->len : 0;
}
static inline size_t bytearray_max(bytearray *const barray)
{
return (barray) ? barray->max : 0;
}
The (expression) ? (if-true) : (if-false) is a ternary operator. In this case, the first function is exactly equivalent to
static inline size_t bytearray_len(bytearray *const barray)
{
if (barray)
return barray->len;
else
return 0;
}
If you wonder about the bytearray *const barray, remember that pointer declarations are read from right to left, with * as "a pointer to". So, it just means that barray is constant, a pointer to a byte array. That is, we may change the data it points to, but we won't change the pointer itself. Compilers can usually detect such stuff themselves, but it may help; the main point is however to remind us human programmers that the pointer itself is not to be changed. (Such changes would only be visible within the function itself.)
Since such arrays often need to be resized, the resizing is often put into a separate helper function:
bytearray *bytearray_resize(bytearray *const barray, const size_t len)
{
bytearray *temp;
if (!len) {
free(barray);
errno = 0;
return NULL;
}
if (!barray) {
temp = malloc(sizeof (bytearray) + len * sizeof barray->data[0]);
if (!temp) {
errno = ENOMEM;
return NULL;
}
temp->max = len;
temp->len = 0;
return temp;
}
if (barray->len > len)
barray->len = len;
if (barray->max == len)
return barray;
temp = realloc(barray, sizeof (bytearray) + len * sizeof barray->data[0]);
if (!temp) {
free(barray);
errno = ENOMEM;
return NULL;
}
temp->max = len;
return temp;
}
What does that errno = 0 do in there? The idea is that because resizing/reallocating a byte array may change the pointer, we return the new one. If the allocation fails, we return NULL with errno == ENOMEM, just like malloc()/realloc() do. However, since the desired new length was zero, this saves memory by freeing the old byte array if any, and returns NULL. But since that is not an error, we set errno to zero, so that it is easier for callers to check if an error occurred or not. (If the function returns NULL, check errno. If errno is nonzero, an error occurred; you can use strerror(errno) to get a descriptive error message.)
You probably also noted the sizeof barray->data[0], used even when barray is NULL. This is okay, because sizeof is not a function, but an operator: it does not access the right side at all, it only evaluates to the size of the thing the right side refers to. (You only need to use parentheses when the right size is a type.) This form is nice, because it lets a programmer change the type of the data member, without changing any other code.
To append data to such a byte array, we probably want to be able to specify whether we anticipate further appends to the same array, or whether this is probably the final append, so that only the exact needed amount of memory is needed. For simplicity, I'll only implement the exact size version here. Note that this function returns a pointer to the (modified) byte array:
bytearray *bytearray_append(bytearray *barray,
const void *from, const size_t size,
int exact)
{
size_t len = bytearray_len(barray) + size;
if (exact) {
barray = bytearray_resize(barray, len);
if (!barray)
return NULL; /* errno already set by bytearray_resize(). */
} else
if (bytearray_max(barray) < len) {
if (!exact) {
/* Apply growth policy */
if (len < 8)
len = 8;
else
if (len < 4194304)
len = (3 * len) / 2;
else
len = (len | 2097151) + 2097153 - 24;
}
barray = bytearray_resize(barray, len);
if (!barray)
return NULL; /* errno already set by the bytearray_resize() call */
}
if (size) {
memmove(barray->data + barray->len, from, size);
barray->len += size;
}
return barray;
}
This time, we declared bytearray *barray, because we change where barray points to in the function. If the fourth parameter, final, is nonzero, then the resulting byte array is exactly the size needed; otherwise the growth policy is applied.
yes, since realloc will preserve the start of your buffer if the new size is bigger:
char* concat(char* a, size_t a_size,
char* b, size_t b_size) {
char* c = realloc(a, a_size + b_size);
memcpy(c + a_size, b, b_size); // dest is after "a" data, source is b with b_size
free(b);
return c;
}
c may be different from a (if the original memory block cannot be resized in-place contiguously to the new size by the system) but if that's the case, the location pointed by a will be freed (you must not free it), and the original data will be "moved".
My advice is to warn the users of your function that the input buffers must be allocated using malloc, else it will crash badly.
So, to start off I've already looked at a few questions including this one and none of them seem to help.
I'm simply trying to write a function that extends the size of an array using realloc().
My code currently looks like this:
unsigned char *xtnd = malloc(4);
xtndc(&xtnd, 4);
// sizeof(*xtnd) should now be 8
void xtndc ( unsigned char ** bytesRef , uint8_t count ) {
*bytesRef = realloc(*bytesRef, (sizeof(**bytesRef)) + count);
}
But no matter what I do it seems that the size of xtnd is always 4. After running xtndc() on it it should now be 8 bytes long.
Any suggestions?
The type of **bytesRef is unsigned char, so sizeof(**bytesRef) is 1. sizeof doesn't keep track of dynamic allocations, it's a compile time tool that gives you the size of a type, in this case unsigned char.
You have to keep track of the array size manually to calculate the new required size.
Your program does in fact change the size of the memory block. It changes the size of your original memory block from 4 bytes to 5 bytes. It changes to 5 bytes because you are essentially doing sizeof(unsigned char) + 4 which 1 + 4 = 5. If you want to double the size instead, do count*sizeof(unsigned char) + count. There are two points to be noted here:
The sizeof function returns the size of the data type, not the size of the allocated bytes. There is no way to know the size of the dynamically allocated memory.
The function realloc (and malloc and calloc as well) is not always guaranteed to return the requested reallocation. It may or may not succeed all the time.
I fixed the problem with the following code.
typedef struct CArrPtr {
unsigned char* ptr;
size_t size;
} CArrPtr;
void xtndc ( CArrPtr *bytesRef, uint8_t count );
. . .
CArrPtr xtnd = { .ptr = malloc(4), .size = 4 };
xtndc( &xtnd, 4 );
// xtnd.size is now 8 bytes
. . .
void xtndc ( CArrPtr *bytesRef, uint8_t count ) {
unsigned char *nptr;
if((nptr = realloc(bytesRef->ptr, bytesRef->size + count)) != 0)
{
bytesRef->ptr = nptr;
bytesRef->size = bytesRef->size + count;
}
}
As I am somewhat new to C, what I learned from this is that malloc specifically creates a pointer to a memory block, but you have no direct access to information about the memory block. Instead, you must store the size of the array that you created with malloc somewhere as well.
Since in the past I'd been initializing arrays with unsigned char arr[size]; and then using sizeof on it, I was under the impression that sizeof returned the size of the array, which is of course wrong as it gives you the size of a type.
Glad I could learn something from this.
sizeof is used to calculate size of data type or array. Pointer and array are very similar, but they are different things. For int *ap, sizeof(ap) will return 4 on x86, sizeof(*ap) will return 4; for int a[10], sizeof(a) will return 40.
sizeof expression is processed at compile time, so it will be a constant written into the executable file before you run the program.
malloc and realloc don't maintain size.
If realloc succeeds, it will reallocate the requested size. So you don't need to check the size after realloc returns, but you should check the return value of realloc to ensure that realloc succeeds.
How can i extract bytes from offset offset of tvb with length length length? type of tvb is :
uint8_t *tvb;
uint8_t *extractBytes(uint8_t *tvb, guint8 offset, guint8 length)
{
// do ...
// extract bytes and return
}
I don't know how can I do this ?
Thanks in advance.
You will need to allocate memory for the extracted bytes. Then it's a simple matter of copying the correct bytes:
uint8_t *extractBytes(uint8_t *tvb, guint8 offset, guint8 length)
{
uint8_t *new = malloc (length);
if (new) {
mempcy (new, tvb+offset, length);
}
return new; /* Returns NULL on allocation failure */
}
Don't forget to free() the allocated memory once you are done using it.
Note that the malloc() call above allocates length number of bytes. If you want to allocate memory for elements that are larger than uint8_t, you will have to multiply the amount by the size of the element. To be on the safe side, and to guard against future changes, you can allocate the memory as follows:
new = malloc (length * sizeof *new);
Now, sizeof *new will always be the correct element size.
If you don't need to change the extracted bytes, and the original buffer (tvb) doesn't change while you're using the bytes, you can access them directly using something like:
int i;
for (i = 0; i < length; i++) {
do_something (tvb[offset+i]);
}
After using malloc() to initialize 5000 bytes of memory, how would I reference the bytes in this memory space? For example, if I need to point to a starting location of data within the memory, how would I go about that?
EDIT: Does it matter what I use to point to it? I mean I am seeing people use bytes/int/char? Is it relevant?
Error I get:
You can use the subscript array[n] operator to access the index you are interested in reading/writing, like so:
uint8_t* const bytes = (uint8_t*)malloc(5000);
bytes[0] = UINT8_MAX; // << write UINT8_MAX to the first element
uint8_t valueAtIndexZero = bytes[0]; // << read the first element (will be UINT8_MAX)
...
free(bytes), bytes = 0;
char * buffer = malloc(5000);
buffer[idx] = whatever;
char * p = buffer + idx;
*p = whatever;
Malloc doesn't initialize the bits allocated by it. Use calloc() rather.
int *p = malloc (5000); // p points to the start of the dynamically allocated area.
As has been mentioned by others, you could do something like this:
int nbytes = 23; // number of bytes of space to allocate
byte *stuff = malloc(nbytes * sizeof stuff[0]);
stuff[0] = 0; // set the first byte to 0
byte x = stuff[0]; // get the first byte
int n = 3;
stuff[n] = 0; // set the nth byte to 0
x = stuff[n]; // nth byte, or in the case of some other type, nth whatever - just make sure it's a safe value, from 0 (inclusive) to the number (nbytes here) of things you allocated (exclusive)
However, a couple of things to note:
malloc will not initialise the memory, but calloc will (as mentioned by Prasoon Saurav)
You should always check to see if the memory allocation failed (see below for an example)
int nbytes = 23; // or however many you want
byte *stuff = malloc(nbytes * sizeof stuff[0]);
if (NULL == stuff) // memory allocation failed!
{
//handle it here, e.g. by exiting the program and displaying an appropriate error message
}
stuff[0] = 0; // set the first byte to 0
byte x = stuff[0]; // get the first byte
int n = 3;
stuff[n] = 0; // set the nth byte to 0
x = stuff[n]; // nth byte, or in the case of some other type, nth whatever
malloc() returns a pointer to the allocated memory:
typedef unsigned char byte;
byte * mem = malloc( 5000 );
byte val = mem[1000]; /* gets the 1000th byte */
After using malloc() to initialize 5000 bytes of memory, how would I
reference the bytes in this memory space? For example, if I need to
point to a starting location of data within the memory, how would I go
about that?
Does it matter what I use to point to it? I mean I am seeing people
use bytes/int/char? Is it relevant?
as you have seen malloc allocates a block of memory counted in bytes, you can assign a pointer to that block and depending on the pointer type the compiler knows how to reference individual elements:
unsigned char *memblob = malloc( 1024 );
short* pshort = (short*)memblob;
now if you reference the second short value i.e. *(pshort + 1) or pshort[1] the compiler knows that it needs to add 2 bytes (sizeof(short)) in order get the next element.
float* pfloat = (float*)memblob;
now if you reference the second float value i.e. *(pfloat + 1) or pfloat[1] the compiler knows that it needs to add 4 bytes (sizeof(float)) in order get the next element.
same with own defined data types:
typedef struct s
{
short a;
long b;
} mystruct_t;
mystruct_t* pstruct = (mystruct_t*)memblob;
pstruct + 1 accesses the struct at offset sizeof(mystruct_t)
so it is really up to you how you want to use the allocated memory