So I'm working on a small memory allocation package, and I've wanted the initialization of pointers to save the size of the space allocated as well as an indicator that it was allocated via one of my functions (which is the character 'q' before the size in memory). So, I've tried to do the following:
int qmem_alloc(unsigned num_bytes, void ** rslt){
*rslt = malloc(num_bytes+sizeof(int)+sizeof(char));
*((int*)rslt) = num_bytes;
*(char*)(rslt+sizeof(int)) = 'q';
rslt = rslt+sizeof(int) + sizeof(char);
if(*rslt == NULL)
return -1;
else if(errno != 0){
//Catch the rest of the errors
return -2;
}
return 0;
}
However, it seems in my main function that the memory directly before the address of rslt does not contain what it should after being passed back. Am I doing something bad here by changing the pointer address?
You're missing a level of indirection in a few places. Anyplace you use rslt before dereferencing you should be using *rslt:
int qmem_alloc(unsigned num_bytes, void ** rslt){
*rslt = malloc(num_bytes+sizeof(int)+sizeof(char));
if(*rslt == NULL)
return -1;
*((int*)*rslt) = num_bytes;
*(char*)(*rslt+sizeof(int)) = 'q';
*rslt = *rslt+sizeof(int) + sizeof(char);
if(errno != 0){
//Catch the rest of the errors
return -2;
}
return 0;
}
Also, the memory returned by malloc is properly aligned for any use. Because you return sizeof(int)+sizeof(char) == 5 bytes past that (assuming a 4 byte int) which means the pointer you return is probably not. You'll want to add at least 3 more bytes to put the returned buffer on an 8 byte boundary.
Within your function rslt is the address of the pointer. You should not be modifying or even accessing that. If you're trying to change/read the address where the pointer is pointing to, than you need to use *rslt. If you're trying the modify/read the value of what the pointer is pointing to, you need to use **rslt.
#dbush describes the result of the above in code.
Related
This question already has answers here:
Initializing a pointer in a separate function in C
(2 answers)
Closed 3 years ago.
My goal is to pass a pointer to double to a function, dynamically allocate memory inside of the function, fill resulted array with double values and return filled array. After lurking attentively everywhere in StackOverflow, I have found two related topics, namely Initializing a pointer in a separate function in C and C dynamically growing array. Accordingly, I have tried to write my own code. However, the result was not the same as it was described in aforementioned topics. This program was run using both gcc and Visual Studio.
First trial.
int main()
{
double *p;
int count = getArray(&p);
<...print content of p...>
return 0;
}
int getArray(double *p)
{
int count = 1;
while(1)
{
if(count == 1)
p = (double*)malloc(sizeof(double));
else
p = (double*)realloc(p, count*sizeof(double));
scanf("%lf", &p[count-1]);
<...some condition to break...>
count++;
{
<... print the content of p ...>
return count;
}
(Here comes the warning from compiler about incompatible argument type. Ignore it).
Input:
1.11
2.22
3.33
Output:
1.11
2.22
3.33
0.00
0.00
0.00
Second trial.
int main()
{
double *p;
int count = getArray(&p);
<...print content of p...>
return 0;
}
int getArray(double **p)
{
int count = 1;
while(1)
{
if(count == 1)
*p = (double*)malloc(sizeof(double));
else
{
double ** temp = (double*)realloc(*p, count*sizeof(double));
p = temp;
}
scanf("%lf", &(*p)[count-1]);
<...some condition to break...>
count++;
{
<... print the content of p ...>
return count;
}
Input:
1.11
2.22
Segmentation error.
I tried this method on several different *nix machines, it fails when the loop uses realloc. SURPRISINGLY, this code works perfect using Visual Studio.
My questions are: first code allows to allocate and reallocate the memory and even passes all this allocated memory to main(), however, all the values are zeroed. What is the problem? As for the second program, what is the reason of the segmentation error?
The right way of doing it is like this:
int getArray(double **p)
{
int count = 0;
while(1)
{
if(count == 0)
*p = malloc(sizeof(**p));
else
*p = realloc(*p, (count+1)*sizeof(**p));
scanf("%lf", &((*p)[count]));
<...some condition to break...>
count++;
{
<...print content of p...>
return count;
}
If you pass a pointer to a function and you want to change not only the value it is pointing at, but change the address it is pointing to you HAVE to use a double pointer. It is simply not possible otherwise.
And save yourself some trouble by using sizeof(var) instead of sizeof(type). If you write int *p; p = malloc(sizeof(int));, then you are writing the same thing (int) twice, which means that you can mess things up if they don't match, which is exactly what happened to you. This also makes it harder to change the code afterwards, because you need to change at multiple places. If you instead write int *p; p = malloc(sizeof(*p)); that risk is gone.
Plus, don't cast malloc. It's completely unnecessary.
One more thing you always should do when allocating (and reallocating) is to check if the allocation was successful. Like this:
if(count == 0)
*p = malloc(sizeof(**p));
else
*p = realloc(*p, (count+1)*sizeof(**p));
if(!p) { /* Handle error */ }
Also note that it is possible to reallocate a NULL pointer, so in this case the malloc is not necessary. Just use the realloc call only without the if statement. One thing worth mentioning is that if you want to be able to continue execution if the realloc fails, you should NOT assign p to the return value. If realloc fails, you will lose whatever you had before. Do like this instead:
int getArray(double **p)
{
int count = 0;
// If *p is not pointing to allocated memory or NULL, the behavior
// of realloc will be undefined.
*p = NULL;
while(1)
{
void *tmp = realloc(*p, (count+1)*sizeof(**p));
if(!tmp) {
fprintf(stderr, "Fail allocating");
exit(EXIT_FAILURE);
}
*p = tmp;
// I prefer fgets and sscanf. Makes it easier to avoid problems
// with remaining characters in stdin and it makes debugging easier
const size_t str_size = 100;
char str[str_size];
if(! fgets(str, str_size, stdin)) {
fprintf(stderr, "Fail reading");
exit(EXIT_FAILURE);
}
if(sscanf(str, "%lf", &((*p)[count])) != 1) {
fprintf(stderr, "Fail converting");
exit(EXIT_FAILURE);
}
count++;
// Just an arbitrary exit condition
if ((*p)[count-1] < 1) {
printf("%lf\n", (*p)[count]);
break;
}
}
return count;
}
You mentioned in comments below that you're having troubles with pointers in general. That's not unusual. It can be a bit tricky, and it takes some practice to get used to it. My best advice is to learn what * and & really means and really think things through. * is the dereference operator, so *p is the value that exists at address p. **p is the value that exists at address *p. The address operator & is kind of an inverse to *, so *&x is the same as x. Also remember that the [] operator used for indexing is just syntactic sugar. It works like this: p[5] translates to *(p+5), which has the funny effect that p[5] is the same as 5[p].
In my first version of above code, I used p = tmp instead of *p = tmp and when I constructed a complete example to find that bug, I also used *p[count] instead of (*p)[count]. Sorry about that, but it does emphasize my point. When dealing with pointers, and especially pointers to pointers, REALLY think about what you're writing. *p[count] is equivalent to *(*(p+count)) while (*p)[count] is equivalent to *((*p) + count) which is something completely different, and unfortunately, none of these mistakes was caught even though I compiled with -Wall -Wextra -std=c18 -pedantic-errors.
You mentioned in comments below that you need to cast the result of realloc. That probably means that you're using a C++ compiler, and in that case you need to cast, and it should be (double *). In that case, change to this:
double *tmp = (double*)realloc(*p, (count+1)*sizeof(**p));
if(!tmp) {
fprintf(stderr, "Fail allocating");
exit(EXIT_FAILURE);
}
*p = tmp;
Note that I also changed the type of the pointer. In C, it does not matter what type of pointer tmp is, but in C++ it either has to be a double* or you would need to do another cast: *p = (double*)tmp
I have an array that I want to increase the size of during runtime. I assign values to the elements of the array by using a loop, and when the index of the loop hits the number of elements of the array, I want to increase the size of the array.
What I did, actually works; I can assign values to elements of the array that I would normally not be able to assign any values to without increasing the array's size. The bad side is, it gives me a crash after the program runs and finishes smoothly. What is wrong here? Is it that maybe the memory that I try to allocate for the array is already filled?
int main()
{
int arr[3];
int num_of_elements = sizeof(arr)/sizeof(arr[0]); // This gives '3', I checked
for(i = 0; i < 10; i++)
{
if(i == num_of_elements)
{
num_of_elements = num_of_elements + 10;
realloc(arr, num_of_elements);
}
arr[i] = i+10;
printf("%d\n", arr[i]);
}
return 0;
}
Well you are invoking undefined behavior. From standard ยง7.22.3.5
void *realloc(void *ptr, size_t size);
If ptr is a null pointer, the realloc function behaves like the malloc
function for the specified size. Otherwise, if ptr does not match a
pointer earlier returned by a memory management function, or if the
space has been deallocated by a call to the free or realloc function,
the behavior is undefined. If memory for the new object cannot be
allocated, the old object is not deallocated and its value is
unchanged.
By memory management function - it means malloc etc. arr is not dynamically allocated memory. So passing this to realloc is undefined behavior - in your case that behavior leads you to crash in program.
It would work if you do this
int *arr = malloc(sizeof(int)*3);
if( arr == NULL){
perror("Malloc failed");
exit(EXIT_FAILURE);
}
...
int *p = realloc(arr,num_of_elements*sizeof(int));
^^^^^
if(p == NULL ){
perror("realloc failed");
exit(EXIT_FAILURE);
}
arr = p;
Check how realloc is used.
The takeaways will be:-
Check the return value of realloc, malloc.
You were trying to reallocate extra 10 elements for which you need 10*sizeof(int) amount of memory.
Don't do arr = realloc(arr,SIZE) in case realloc fails you will have memory leak.
Why realloc to p after all you do arr=p annyway?
Two reasons so far
The answer to this is when realloc fails then it returns NULL now if you assign arr to NULL then you may have a situation where you lose the only reference to the previously allocated memory - leading to a memory leak. That's why we do it like this.
Note this from standard
The realloc function returns a pointer to the new object (which may
have the same value as a pointer to the old object), or a null pointer
if the new object could not be allocated.
Notice that may part - it might be the same address as before as pointed by arr or it might be different one. That explains why we should store it in some temporary pointer and then we assign it later.
You should have done this:
arr = realloc(arr, num_of_elements);
^^^^^
realloc() does not necessarily extend or shrink the allocated memory in-place, it invalidates the dynamic memory that its first argument points to and allocates new memory, while preserving the content of the previous memory.
One possible implementation is :
void* realloc(void* ptr, size_t size) {
void* ret = malloc(size);
if (ret == NULL) return ret;
if (ptr != NULL) {
memcpy(ret, ptr, /* previous size from system */);
free(ptr);
}
return ret;
}
I am a newbie in C and I am trying to program a simple text editor, I have already written a 100 lines of stupid messy code, but it just worked. Until this SEGFAULT started showing up. I am going with the approach of switching terminal to canonical mode, and getting letter by letter from the user and do the necessary with each of 'em. The letters are added to a buffer, which is realloced extra 512 byte when the buffer is half filled, which I know is a stupid thing to do. But the cause of the SEGFAULT cant be determined. Help would be appreciated. Here's my code:
char* t_buf
int t_buf_len = 0;
int cur_buf_sz = 0;
void mem_mgr(char *buffer, unsigned long bytes){ //Function to allocate requested memory
if(buffer == NULL){
if((buffer = malloc(sizeof(char) * bytes)) == NULL){
printf("ERROR:Cannot get memory resources(ABORT)\n\r");
exit(1);
}
}
else{
realloc(buffer, bytes);
}
}
void getCharacter(){
if(t_buf_len >= (cur_buf_sz/2))
mem_mgr(t_buf, cur_buf_sz+=512);
strcpy(t_buf, "Yay! It works!");
printf("%s %d", t_buf, cur_buf_sz);
}
There are things you need to understand first,
The buffer pointer is a local variable inside the mem_mgr() function, it points to the same memory t_buf points initially, but once you modify it, it's no longer related to t_buf in any way.
So, when you return from mem_mgr() you lose the reference to the allocated memory and.
To fix this, you can pass a poitner to the pointer, and alter the actual pointer by dereferencing it.
The realloc() function, behaves exactly like malloc() if the first argument is NULL, if you read the documentation you would know that.
Memory allocation functions MUST be checked to ensure they returned a valid legal pointer, that's why you need a temporary poitner to store the return value of realloc(), because if it returns NULL, meaning that there was no memory to fulfill the request, you would lose reference to the original block of memory and you can't free it anymore.
You need to pass a pointer to your pointer to mem_mgr(), like this
int
mem_mgr(char **buffer, unsigned long bytes)
{
void *tmp = realloc(*buffer, bytes);
if (tmp != NULL) {
*buffer = tmp;
return 0;
}
return -1;
}
And then, to allocate memory
void
getCharacter()
{
if (t_buf_len >= (cur_buf_sz / 2)) {
if (mem_mgr(&t_buf, cur_buf_sz += 512) != -1) {
strcpy(t_buf, "Yay! It works!");
printf("%s %d", t_buf, cur_buf_sz);
}
}
}
The call to
mem_mgr(t_buf, cur_buf_sz+=512);
cannot change the actual parameter t_buf. You will either have to return the buffer from mem_mgr
t_buf = mem_mgr(t_buf, cur_buf_sz+=512);
or pass a pointer to t_buf
mem_mgr(&t_buf, cur_buf_sz+=512);
Furthermore, a call to realloc may change the address of the memory buffer, so you will have to use
char *tmpbuf = realloc(buffer, bytes);
if (!tmpbuf)
// Error handling
else
buffer = tmpbuf;
realloc(NULL, bytes); will behave like a malloc, so you don't need a separate branch here. This makes in total:
char *mem_mgr(char *buffer, unsigned long bytes){ //Function to allocate requested memory
char *tmpbuf = realloc(buffer, bytes);
if (!tmpbuf) {
// Error handling
}
return tmpbuf;
}
which somehow questions the reason of existence of the function mem_mgr.
On RHEL6, I'm facing a strange problem with realloc(). At some point in the program, realloc() returns NULL (the old pointer has an address and there's plently of memory available). What's being allocated is 200 structure elements (structure below). For some reason, when I do a realloc() instead, it works, but I then have to assign the old pointer to the new one. Below is a simplified version of my code.
This is perhaps a server tuning issue more than a programming one. What is your opinion?
Thanks.
//hearder file
typedef struct { /* Variable Node Detail Record */
long next;
long mask;
char *value;
// more stuff...
} NODETEST;
extern NODETEST *oldNodes;
extern NODETEST *newNodes;
//program
#define MAXSIZE 200
// do some stuff with oldNodes....
int alloc_nodes (void)
{
// Allocate or grow the table
oldNodes = (NODETEST *) malloc(MAXSIZE * sizeof(NODETEST));
if( oldNodes == NULL ) {
//handle exception...
exit(1);
}
//oldNodes = (NODETEST *) realloc(oldNodes,MAXSIZE * sizeof(NODETEST)); // *** FAILS
newNodes = (NODETEST *) realloc(oldNodes,MAXSIZE * sizeof(NODETEST)); // *** WORKS
if( newNodes == NULL ){
printf("errno=%d\n", errno );
}else{
oldNodes = newNodes; }
}
Your first call malloc with a size S and then realloc with the same size S. This is wrong: you have to pass to realloc the new wanted size (independently of the current size - it is not an increment). Here, there is a big chance realloc returns exactly the same pointer it received. BTW it is not clear why you want to do with a malloc immediately followed by a realloc. Gives us more detail.
If you want a dynamic table whose size auto-adjusts, you need to allocate an initial size storing its size in a variable (e.g. alloc_size) and keep the current number of occupied elements in another variable (e.g. n_elem) . When you add an element you increment this number. When the table is full reallocate it. Here is a sketch
NODETEST *newNodes = NULL;
int allocated_elem = 0;
int n_elem = 0;
#define ALLOC_INCR 200
then at each addition:
if (n_elem >= alloc_size) { // the first time realloc is as malloc since nodes == NULL
alloc_size += ALLOC_INCR;
nodes = (NODETEST *) realloc(nodes, alloc_size * sizeof(NODETEST));
if (nodes == NULL) {
//handle exception...
exit(1);
}
}
// add your element at nodes[n_elem]
n_elem++;
Recall that realloc acts like malloc when the received pointer is NULL (case of the first call). Thus it allocates the initial table. Subsequent calls reallocate it by adjusting the size with a constant increment (here 200). Some other schemes are possible for the enlargement of the table, for instance you can multiply the size by a factor (e.g. 2) starting from 32:
if (n_elem >= alloc_size) { // the first time realloc is as malloc since nodes == NULL
alloc_size = (alloc_size == 0) ? 32 : alloc_size * 2;
Regarind the FAIL and WORKS comments: it is clear that if you assign oldNodes (in the FAIL code) then newNodes is not assigned and keeps its initial value which is zero (NULL) since it is declared as a global variable and not initialized (well I suppose, it is extern here). Thus the test if (newNodes == NULL) will probably fail.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I'm trying to implement a solution to copy a large string in memory in C.
Can you give me any advice about implementation or any reference?
I'm thinking to copy byte by byte since I don't know the length (probably I can't calculate it with strlen() since the string is very large).
Another concern is that I will have to reallocate memory on every step and I don't know how is the best way to do that. Is there any way that I can reallocate using only the reference to the last position of the memory already alocated and filled? Thus if the memory allocation fails, it will not affect the rest of the memory already filled.
What is the best value to return from this function? Should I return the number of bytes that were succesfully copied?
If there is a memory allocation fail, does realloc() set any global variable that I can check in the main function after I call the copying function? As I don't want to just return NULL from it if at some point realloc() fails, but I want to return a value more useful.
strlen() won't fail, as it uses size_t to descirbe the string's size, and size_t is large enough to hold the size of any object on the machine the program runs on.
So simply do
#define _XOPEN_SOURCE 500 /* for strdup */
#include <string.h>
int duplicate_string(const char * src, char ** pdst)
{
int result = 0;
if (NULL == ((*pdst) = strdup(src)))
{
result = -1;
}
return result;
}
If this fails try using an more clever structure to hold the data, for example by chopping it into slices:
#define _XOPEN_SOURCE 700 /* for strndup */
#include <string.h>
int slice_string(const char * src, char *** ppdst, size_t s)
{
int result = 0;
size_t s_internal = s + 1; /* Add one for the 0-terminator. */
size_t len = strlen(src) + 1;
size_t n =len/s_internal + (len%s_internal ?1 :0);
*ppdst = calloc(n + 1, sizeof(**ppdst)); /* +1 to have a stopper element. */
if (NULL == (*ppdst))
{
result = -1;
goto lblExit;
}
for (size_t i = 0; i < n; ++i)
{
(*ppdst)[i] = strndup(src, s);
if (NULL == (*ppdst)[i])
{
result = -1;
while (--i > 0)
{
free((*ppdst)[i]);
}
free(*ppdst);
*ppdst = NULL;
goto lblExit;
}
src += s;
}
lblExit:
return result;
}
Use such functions by trying dump copy first and if this fails by slicing the string.
int main(void)
{
char * s = NULL;
read_big_string(&s);
int result = 0;
char * d = NULL;
char ** pd = NULL;
/* 1st try dump copy. */
result = duplicate_string(s, &d);
if (0 != result)
{
/*2ndly try to slice it. */
{
size_t len = strlen(s);
do
{
len = len/2 + (len%2 ?1 :0);
result = slice_string(s, &pd, len);
} while ((0 != result) || (1 == len));
}
}
if (0 != result)
{
fprintf(stderr, "Duplicating the string failed.\n");
}
/* Use copies. */
if (NULL != d)
{
/* USe result from simple duplication. */
}
if (NULL != pd)
{
/* Use result from sliced duplication. */
}
/* Free the copies. */
if (NULL != pd)
{
for (size_t i = 0; pd[i]; ++i)
{
free(pd[i]);
}
}
free(pd);
free(d);
return 0;
}
realloc() failing
If there is a memory allocation fail, does realloc() set any global variable that I can check in the main function after I call the copying function? As I don't want to just return NULL from it if at some point realloc() fails, but I want to return a value more useful.
There's no problem with realloc() returning null if you use realloc() correctly. If you use realloc() incorrectly, you get what you deserve.
Incorrect use of realloc()
char *space = malloc(large_number);
space = realloc(space, even_larger_number);
If the realloc() fails, this code has overwritten the only reference to the previously allocated space with NULL, so not only have you failed to allocate new space but you also cannot release the old space because you've lost the pointer to it.
(For the fastidious: the fact that the original malloc() might have failed is not critical; space will be NULL, but that's a valid first argument to realloc(). The only difference is that there would be no previous allocation that was lost.)
Correct use of realloc()
char *space = malloc(large_number);
char *new_space = realloc(space, even_larger_number);
if (new_space != 0)
space = new_space;
This saves and tests the result of realloc() before overwriting the value in space.
Continually growing memory
Another concern is that I will have to reallocate memory on every step and I don't know how is the best way to do that. Is there any way that I can reallocate using only the reference to the last position of the memory already allocated and filled? Thus if the memory allocation fails, it will not affect the rest of the memory already filled.
The standard technique for avoiding quadratic behaviour (which really does matter when you're dealing with megabytes of data) is to double the space allocated for your working string when you need to grow it. You do that by keeping three values:
Pointer to the data.
Size of the data area that is allocated.
Size of the data area that is in use.
When the incoming data won't fit in the space that is unused, you reallocate the space, doubling the amount that is allocated unless you need more than that for the new space. If you think you're going to be adding more data later, then you might add double the new amount. This amortizes the cost of the memory allocations, and saves copying the unchanging data as often.
struct String
{
char *data;
size_t length;
size_t allocated;
};
int add_data_to_string(struct String *str, char const *data, size_t datalen)
{
if (str->length + datalen >= str->allocated)
{
size_t newlen = 2 * (str->allocated + datalen + 1);
char *newdata = realloc(str->data, newlen);
if (newdata == 0)
return -1;
str->data = newdata;
str->allocated = newlen;
}
memcpy(str->data + str->length, data, datalen + 1);
str->length += datalen;
return 0;
}
When you've finished adding to the string, you can release the unused space if you wish:
void release_unused(struct String *str)
{
char *data = realloc(str->data, str->length + 1);
str->data = data;
str->allocated = str->length + 1;
}
It is very unlikely that shrinking a memory block will move it, but the standard says:
The realloc function deallocates the old object pointed to by ptr and returns a
pointer to a new object that has the size specified by size. The contents of the new
object shall be the same as that of the old object prior to deallocation, up to the lesser of
the new and old sizes.
The realloc function returns a pointer to the new object (which may have the same
value as a pointer to the old object), or a null pointer if the new object could not be
allocated.
Note that 'may have the same value as a pointer to the old object' also means 'may have a different value from a pointer to the old object'.
The code assumes that it is dealing with null terminated strings; the memcpy() code copies the length plus one byte to collect the terminal null, for example, and the release_unused() code keeps a byte for the terminal null. The length element is the value that would be returned by strlen(), but it is crucial that you don't keep doing strlen() on megabytes of data. If you are dealing with binary data, you handle things subtly differently.
use a smart pointer and avoid copying in the first place
OK, let's use Cunningham's Question to help figure out what to do. Cunningham's Question (or Query - your choice :-) is:
What's the simplest thing that could possibly work?
-- Ward Cunningham
IMO the simplest thing that could possibly work would be to allocate a large buffer, suck the string into the buffer, reallocate the buffer down to the actual size of the string, and return a pointer to that buffer. It's the caller's responsibility to free the buffer they get when they're done with it. Something on the order of:
#define BIG_BUFFER_SIZE 100000000
char *read_big_string(FILE *f) /* read a big string from a file */
{
char *buf = malloc(BIG_BUFFER_SIZE);
fgets(buf, BIG_BUFFER_SIZE, f);
realloc(buf, strlen(buf)+1);
return buf;
}
This is example code only. There are #includes which are not included, and there's a fair number of possible errors which are not handled in the above, the implementation of which are left as an exercise for the reader. Your mileage may vary. Dealer contribution may affect cost. Check with your dealer for price and options available in your area. Caveat codor.
Share and enjoy.