I'm trying to code a buffer for an input file. The Buffer should always contain a defined amount of data. If a few bytes of the data were used, the buffer should read data from the file until it has the defined size again.
const int bufsize = 10;
int *field = malloc(bufsize*sizeof(int)); //allocate the amount of memory the buffer should contain
for(i=0;i<bufsize;++i) //initialize memory with something
*(field+i) = i*2;
field += 4; //Move pointer 4 units because the first 4 units were used and are no longer needed
field= realloc(field,bufsize*sizeof(int)); //resize the now smaller buffer to its original size
//...some more code were the new memory (field[6]-field[9]) are filled again...
Here is a short example of how I'm trying to do it at the moment (without files, because this is the part thats not working), but the realloc() always returns NULL. In this example, the first 4 units were used, so the pointer should move forward and the missing data at the end of the memory (so that it will again contain 10 elements) should be allocated. What am I doing wrong?
I would be very thankful if someone could help me
You need memmove() instead
memmove(field, field + 4, (bufsize - 4) * sizeof(*field));
you don't need to realloc() because you are not changing the size of the buffer, just think about it.
If you do this
field += 4;
now you lost the reference to the begining of field so you can't even call free on it, nor realloc() of course. Read WhozCraig comment for instance.
Doing realloc() for the same size doesn't make that much sense.
Using realloc() the way you do causes some other problems, for instance when it fails you also run into the same problem, you loose reference to the original pointer.
So the recommended method is
void *pointer;
pointer = realloc(oldPointer, oldSize + nonZeroNewSize);
if (pointer == NULL)
handleFailure_PerhapsFree_oldPointer();
oldPointer = pointer;
So the title of your question contains the answer to it, what you need is to move the data from offset 4 * sizeof(int) bytes to the begining of the pointer, for which memmove() is the perfect tool, notice that you could also think of using memcpy() but memcpy() cannot handle the case of overlapping data, which is your case.
Your problem should be named as Cyclic Buffer.
You should call malloc() just once when opening file and once free() when you close it.
You don't need to call realloc() at all. All is necessary is advaning pointer by amount of read data, wrapping its value around size of buffer and replacing old data with new data from file.
Your problem with realloc(): you must pass same pointer to it which was previously returned from malloc() or realloc() without offsetting it!
Related
I have a question about dynamic memory allocation.
Context: I'm writing a program that reads a text file of words and counts the frequency with which each word occurs (one word per line).
This particular function reads the file, counts the lines and characters, then dynamically allocates memory to the array of string pointers, an array storing the count of characters for each line and the strings themselves. (The other parts are less directly relevant to my question).
Question: How often should I reallocate memory if I run out of space? I set a constant ("memstart") for setting the initial memory allocation value. In the below code snippet I realloc for every line over the value of "memstart". Would the program process faster if a reallocated a larger block of memory instead of increasing the memory space by 1 "variable type" each time?
What would be best practice for something like this?
Code Snip:
int read_alloc(FILE* fin, FILE *tmp, char **wdp, int *sz){
int line_cnt= 0, chr, let=1;
do{
chr=getc(fin);
let++;
//count characters
if(chr!=EOF){
chr=tolower(chr);
fputc(chr, tmp);
}
//convert to lcase and write to temp file
if ('\n' == chr || chr==EOF){
sz[(line_cnt)]=((let)*sizeof(char)); //save size needed to store string in array
*(wdp+(line_cnt))=malloc((let)*sizeof(char)); //allocate space for the string
if ((line_cnt-1) >= memstart){
realloc(wdp, (sizeof(wdp)*(memstart+line_cnt))); //if more space needed increase size
realloc(sz, (sizeof(sz)*(memstart+line_cnt)));
}
line_cnt++;
let=1;
}
} while (EOF != chr);
return (line_cnt);
}
While the question is about how often realloc should be called, looking at OP's code, I think it's better to start with how safely it should be done.
The C11 standard states (n1570 draft, ยง 7.22.3.5, The realloc function, emphasis mine):
Synopsis
#include <stdlib.h>
void *realloc(void *ptr, size_t size);
Description
The realloc function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
If ptr is a null pointer, the realloc function behaves like the malloc function for the specified size. (...). If memory for the new object cannot be allocated, the old object is not deallocated and its value is unchanged.
Returns
The realloc function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Now let's consider this snippet from the question, where sz is declared as int* sz;
realloc(sz, (sizeof(sz)*(memstart+line_cnt)));
The return value is lost, so we can't know if the call succeeded and if it did, sz is invalidated. Moreover, sizeof(sz) is the size of the pointer, not of the pointed type (int).
A more safe (and correct) pattern would be:
size_t new_size = /* Whatever, let's say */ size + SOME_COSTANT + size / 2;
void *tmp = realloc(ptr, new_size * sizeof *ptr);
if ( tmp == NULL ) {
/* Deal with the error, e.g. log a message with perror, return NULL
(if this is in a function) or just give up, but remeber that
realloc doesn't invalidate nor free 'ptr' on failure */
exit(EXIT_FAILURE);
}
ptr = tmp; // <- on success, realloc invalidated ptr
size = new_size;
Now, to answer the question, realloc should be called only when needed, because it involves potentially expansive system calls. So either allocate a big chunk ahead or choose a growing stratey like doubling (or 1.5 times) the size every time.
It's worth noting that if possible, the OS could perform the reallocation without copying any element of the original array.
The classic answer is to double each time, but a factor of 1.5 might be better. The important bit is that you multiply your array size by some factor each time, rather than adding additional space each time.
Each re-allocation might need to copy the previous array into a new one. We'd like to minimize these copies. If we will be adding n items, and we start with an array of size a, increase by a factor of r each re-allocation, to end with a value of n, the sequence of (re-)allocations will be a, ar, ar^2, ar^3, ..., n. The sum of that sequence is (nr-a)/(r-1). Thus the total space is of order O(n).
Suppose instead we start with a, and this time add r each time. The sequence is a, a+r, a+2r, a+3r, ..., n. The sum of that sequence will be 0.5*((n^2-a^2)/r + a + n). In this case the total space of order O(n^2). Much worse!
With a constant factor of 2, the array will be in the worse case 1/2 empty. That's probably ok. You can always shrink the allocation when you're done and know the final size.
As pointed out in another answer, there are several bugs in the manner in which you call realloc(), but that wasn't the question.
Please look at the following code
char *line = (char *) malloc(100);
char *newline,*source = line;
int size=100;
newline = realloc ( line , size +=size);
// assuming that newline has been successfully assigned the demanded memory and line is freed
Now my question here is that can i write in future expression like
source = newline +( line - source );
I am having doubt in mind because i am using the line pointer which is freed after the successful operation of the realloc() but My program ( this only a snippet from it ) is
still working?so is it safe to use line pointer after realloc() has done?
No, it's not safe to use the line pointer after realloc is done. realloc changes the size of the block pointed to by line. If the size increases, the old location may not have enough contiguous space to accommodate the new larger block. So the location of the block in the memory changes. Use the pointer returned by realloc.
If the old location does not have enough contiguous space for the larger block requested by the user, realloc tries to find a new block of the required size (like malloc), copies the elements from the old block and frees the old block.
If realloc fails, the old pointer is valid and must be freed. If realloc succeeds, the old pointer is considered invalid and should not be freed.
Also, the fact that your program is working is not always a good way to check if something is correct. For example, if you declare int a[10] and you access a[10] or a[11] it may not fail most of the time but it's still undefined behaviour.
so is it safe to use 'linepointer afterrealloc()` has done?
No, because realloc() may free your input buffer while making the new one(if new one is larger than the previous size). And realloc would copy the previous user data from the old buffer into new buffer. So user should use the new buffer as it contains all older information not the old one.
This is about realloc()
void *realloc(void *p, size_t size) realloc changes the size of the
object pointed to by p to size. The contents will be unchanged up to
the minimum of the old and new sizes. If the new size is larger, the
new space is uninitialized. realloc returns a pointer to the new
space, or NULL if the request cannot be satisfied, in which case *p is
unchanged.
gcc (GCC) 4.7.2
c89
Hello,
All error checking removed from snippet - to keep the code short.
I have a problem freeing some memory that I have allocated and copied a string to.
My program will check for digits and increment the pointer until it gets to an non-digit.
When I go to free the memory I get a stack dump with invalid free.
I think this is because I have incremented the pointer and now it is pointing to halfway down the string, as that is when the non-digits start.
If I don't increment its ok to free. However, if I do increment it and then try and free I get the stack dump.
int parse_input(const char *input)
{
char *cpy_input = calloc(strlen(input) + 1, sizeof(char));
size_t i = 0;
apr_cpystrn(cpy_input, input, strlen(input) + 1);
/* Are we looking for a range of channels */
for(i = 0; i < strlen(cpy_input); i++) {
if(isdigit(*cpy_input)) {
/* Do something here */
cpy_input++;
}
}
/* Where finished free the memory */
free(cpy_input); /* Crash here */
return 0;
}
I resolved the issue by declaring another pointer and assigning the address, so it points to the first character, then I free that. It works ok i.e.
char *mem_input = cpy_input;
free(mem_input);
My question is why do I need to declare another pointer to be able to free the memory? Is there another way of doing this?
Many thanks in advance,
You need to save the original pointer. Only the original pointer can be used when freeing the memory. You can just create another variable to hold the original pointer.
Or put the loop in a separate function. As variables by default is passed by value, i.e. copied, when you change the pointer in the function you only change the copy of the pointer.
Besides that, your loop seems a little weird. You loop using an index from zero to the length of the string, so you can easily use that index instead of modifying the pointer. Either that, or change the loop to something like while (*cpy_input != '\0'). I have never seen the two variants mixed.
By the way, you have a bug in that code. You only increase the pointer if the current character is a digit. But if the first character is not a digit, the loop will just loop until it reaches the end of the string, but the pointer will not be increased and you will check the first character over and over again. If you just want to get leading digits from the string (if any), you could use a loop such as
for (; isdigit(*cpy_input); cpy_input++)
{
/* do something, using `*cpy_input` */
}
Or of course
for (int i = 0; i < strlen(cpy_input); i++)
{
/* do something, using `cpy_input[i]` */
}
char *cpy_input = calloc(strlen(input) + 1, sizeof(char));
let's say cpu_input is 0x1000. Point is same pointer should be freed in free().
As per your logic if input length is 5, then after for loop cpy_input points to 0x1005 location. And if you calls free(cpy_input) it's free(0x1005), which is invalid pointer for free and it's getting crashed.
Well, of course there is another way of doing this: just decrement the cpy_input pointer exactly as many times as you incremented it. Or subtract the length of the string (assuming you saved it) from the final cpy_input value. That way you will restore the original cpy_input value and properly free the memory.
The bottom line here is simple: you have to pass to free the same pointer value that you received from calloc. There no way around it. So, in one way or another you have to be able to obtain the original pointer value. Saving it in another pointer is actually the best solution in your situation. But if you know how to do it in any other way - go ahead and use whatever you like most.
calloc returns the pointer to memory block requested from the memory.So you can free the same pointer location only which is returned from calloc.
Either free the original pointer or free backup copy of this.
you can change your loop for
for(i = 0; i < strlen(cpy_input); i++) {
if(isdigit(cpy_input[i])) {
/* Do something here */
}
}
or do Pointer Arithmetic to get the initial value later
It is important to understand, that a pointer just is a memory address.
The resource management system behind free and calloc will keep some book keeping data associated with the chunk of memory, in particular how big the chunk is, you requested by calling calloc. This might be in some lookup container, which stores it related to the pointer returned by calloc (i.e. the initial value of cpu_input), or this information is stored in memory right in front of the chunk, which is as far as I know more common.
If you now pass the changed value in cpu_input to free, it will either not find the book keeping data in its lookup container or it will look for the book keepin data in front of the pointer, where it will find the data of you string, which probably makes no sense at all.
So your solution of keeping a copy of the original pointer is an appropriate one.
My program is written in C and it is a disk emulator. I finished writing it and it runs when I comment out certain lines of my test program, but I get a memory error where I un-comment them. I suspect it is with my char* 's.
The line I comment out (and where the program crashes) is
free(buffer);
where buffer is the char* that the string of bytes that was read into from the disk. It was initially allocated 30 bytes using malloc.
char* buffer = (char *) malloc(sizeof(char) * 30);
There is too much to just post it all here, so I am going to put the parts where I am writing/copying to char* 's in the hopes that someone will see what I am doing wrong.
I don't think it is anything too complicated, I am just not familiar enough with C the recognize obvious memory mistakes.
// In the event of a cache miss:
// block_buffer to pass to add_cache_entry
char cMissBuffer[BLOCK_SIZE];
// read content of block from disk
fread(cMissBuffer,sizeof(char),BLOCK_SIZE,diskEntity.pBlockStore);
// add to cache
if(1==add_cache_entry(i,cMissBuffer)) return 1;
.
.
.
// some of what is in add_cache_entry
int add_cache_entry(int v, char *block_buffer)
{
// ...
// construct a new queue element
QueueElement *block_to_cache = (QueueElement*)malloc(sizeof(QueueElement));
block_to_cache->blkidx = v;
block_to_cache->content=(char*)malloc(BLOCK_SIZE);
strcpy(block_to_cache->content,block_buffer);
// ...
}
In the test, BLOCK_SIZE is 5, QueueElement is a struct, content is a char* with BLOCK_BYTES of info.
Here is an excerpt from running the executable (dumping the queue)...I think that the lack of a '\0' could have something to do with the issue...
after adding cache entry (5):
DUMP:
BLOCK 5 FLAG:0 CONTENT:222220000000
BLOCK 4 FLAG:0 CONTENT:222220000000
BLOCK 3 FLAG:0 CONTENT:000000000000
BLOCK 2 FLAG:0 CONTENT:000000000000
BLOCK 1 FLAG:1 CONTENT:11100
I think I get extra space because malloc allocates more space than I require, but I read that is normal.
Any thoughts?
A probable cause for the behaviour is that strcpy() requires the source string to be null terminated, which is not the case here as fread() does not append a null terminator for you (nor could it in this case as fread() is reading the exact buffer size). strcpy() also appends a null terminator which means the strcpy() call will definitely be writing beyond the block_to_cache->content buffer.
If the data is not be used as a C style string use memcpy() to copy the data instead:
memcpy(block_to_cache->content, block_buffer, BLOCK_SIZE);
Other points:
Check the return value of fread(), to ensure it successfully populated the buffer before attempting to use it.
it is unnecessary to cast the return value of malloc() (see Do I cast the result of malloc?).
check return value of malloc() to ensure memory was successful allocated.
sizeof(char) is guaranteed to be one so can be removed from argument to malloc().
Could someone explain what second parameter in realloc really is as I cant find a way to test it.
So suppose that we have something like this
int *p = malloc(sizeof(int)); //we have just enough space to store single int value
now if I want to store 2 int values in p do I need to send to realloc as second parameter 2 * sizeof(int) new size of the block or sizeof(int) as it needs to extend memory for size of int
In case I should send to realloc total value of new block in this case 2 * sizeof(int), what will it do in case I send it just sizeof(int), just return original pointer and do nothing within memory or something else?
The second parameter is the new size of the memory block (total new size) in bytes. If you want room for 2 ints, you'd use 2 * sizeof(int).
just return original pointer and do nothing within memory or something else?
This is not guaranteed by the specifications. Many implementations will just return the original pointer unchanged, but they could just as easily move it. You will not get a new memory allocation large enough for 2 ints, however, merely the original data values in a memory location large enough for sizeof(int).
You want to tell realloc the new total size of the allocation. So from your example, 2 * sizeof(int).
Your second question isn't entirely clear, so I'll try to wrap all those pieces into one sentence. If you call realloc with the same size value as the original call to malloc, it is up to the implementation whether to return the original pointer or to move the data to a (implementation-defined) more convenient place; if you then try to store two ints in the space for which you only requested one int then you've triggered undefined behavior. It could clobber other allocated memory (causing wrong results in calculations) or (more likely) it could clobber malloc's own bookkeeping data (likely your program will abort with a Segfault somewhat later on when malloc actually takes a look at the data.