I have a function that recieves a pointer to dynamic array of 100 ints. But instead of 100 I have just 50 allocated by malloc or calloc before that.
Is there a way that I could check if any ellement (like 79th for example) is allocated rather than wonder what this SIGSEGV actually means ?
My question is purely theoretic and I have no actual code to show.
No, the pointer does not store its size. You may be better off storing the size and the pointer in a struct and passing it instead:
typedef struct
{
size_t size;
int *ptr;
} my_data;
void myFunc(my_data *data)
{
size_t i;
for(i = 0; i < data->size; i++)
{
// data->ptr[i];
}
}
void myFunc2(my_data *data, size_t index)
{
if(index < data->size)
{
// memory location exists
}
}
Well, you could do such a thing according to your description, given an array and looking for an index (which is slightly different from "any raw pointer"). And with some more work, it is even possible to do such a thing for any pointer.
The malloc function necessarily stores information about how much was allocated. Unluckily, there is no standard how this must be done. Some compilers over-allocate and store the size immediately preceding the allocated data. Others may store addresses in a map, yet others may do something else, you don't know.
However, most (all?) C libraries and at least one linker that I know of have explicit support for overloading/hooking/replacing allocation functions.
For example in the GNU C library, you can set __malloc_hook. and GNU ld lets you do such a thing at linker level with __wrap_malloc.
You could thus overload/hook malloc and free with a function that simply calls the real malloc function and stores the information how much was allocated yourself somewhere (e.g. by over-allocating and using the first word, or whatever you like).
Then write a function which takes a base pointer and an index. That function looks at the allocation info (now you know where to find it!), and can trivially check whether the index is in range. This does not work for "just any pointer".
An alternative solution which works for "just any pointer" would be to write an allocator that satisfies allocations from separate arenas rather than simply wrapping the real malloc. All allocations coming from the same arena have the same allocation size. Given any pointer, you would then only need to iterate over all your arenas and look whether the address is within the arena's start and end address.
However, one should normally be quite sure how much one has allocated, this should not be guesswork, or random luck, or something to figure out at runtime.
Also, given the presence of ready-to-use memory debuggers, I doubt it is really worth investing time in doing such a thing application-side. Just use something like valgrind, no need to write any code at all.
No, there's no portable and reliable way to check this from within the code.
There exist tools -- such as valgrind -- that may help diagnose certain types of memory bugs.
No, there isn't.
This is when you break out your dynamic analysis tool (e.g. valgrind), or use a real container that keeps information about its size.
Some years ago i used one library, i forget its name. Using it, you can create try-catch block and try to access to unknown data e.g. x[79] in try-block, and, if memory is not allocated in it, exception was generated.
Related
Suppose the following function:
float *dosomething(const float *src, const int N)
{
float *dst = (float *)malloc(sizeof(float) * N);
if(!dst)
{
printf("Cannot allocate memory\n");
exit(EXIT_FAILURE);
}
for(int i = 0; i < N; i++)
dst[i] = src[i] * 2;
return dst;
}
In this case we don't need allocate memory previously if we want to use it right?
Now, just another case:
void dosomething(float *dst, const float *src, const int N)
{
for(int i = 0; i < N; i++)
dst[i] = src[i] * 2;
}
In the last case we need to allocate memory previously. So I share it and I'm wondering which is the best method for returning an array. Which of them provide more security to an user of the library or class? which method is most recommended? why?
What's better practice or a better idea depends on what you're actually trying to do.
A function like char *strdup(const char *s) (POSIX) is implemented like the first case, it takes a string as an argument, allocates memory for another of the same length and then copies the source to the new piece of memory. It's convenient and saves you from manually doing the common action of allocating a buffer for the copy of the string. You could assume this is simply like a call to malloc and then strcpy/memcpy.
Then you've got a function like char *strcpy(char *dest, const char *src), which is like the second case, where you have control of where the string is going to be copied to. This way you're not forced into having the string copied into a dynamically allocated, not of your choice, piece of memory.
The first way might come in handy if you needed to create and initialise some sort of dynamic structure (list, tree, etc), but then again the second way also suffices and gives you control of what piece of memory is being used; you can use dynamically allocated memory on the heap, or local variables on the stack, etc.
Personally, I would usually go the second way, because I have more control of what variable's being initialised, and I'm not forced into having to use a newly malloc'd piece of memory (what if I wanted my local variable to be initialised?). You could always then write a wrapper function that makes a call to malloc and then to your function using the newly allocated memory as the destination.
It's really up to you and your design and what you're trying to achieve, there are no right and wrong ways and as long as you remember the allocated memory you shouldn't have any problems. I wouldn't say either of the two is more "secure."
There is no RIGHT answer.
C language is inherently insecure, i.e. you can only make data secure if you make a copy and return the copy. Thus hiding the real location of the original from the caller.
What is more important is how to handle the memory de-allocation of shared data that usually dictates the approach is more correct.
In the example you cite the only data being accessed is the data the caller has already passed (and already owns). So the fact you allocate memory, do something with the data and return the allocated memory to the caller is just fine. Just document that is how the function works (like strdup() works on C strings, the caller is responsible for using free() on any returned non-NULL pointer).
FWIW you don't "share" the data. The caller invokes the function to do work on the data on its behalf, once the function returns no more access occurs. If there was a retained (by the function) memory pointer (or other data) it would be correct to describe the situation as sharing data. Since at some point in the future that retained memory pointer (or other data) maybe utilized in some way.
There is no definite "this is better than the other". I never actually think about these things, and just do whatever comes to mind. Which is likely to be the more "natural" solution for the problem at hand. And if it turns out to be "bad" along the way... well, luckily we are not programming by engraving on stone tablets.
In your case, without knowing anything about the software at all, nothing "feels" better. That's actually quite common; almost everything you do in programming can be done in different ways, and often there's no actual difference other than personal preference or just random "that's what I came up with first".
For example, your second solution lets the caller copy to existing memory, which might be part of a larger object. On the other hand, he has to provide the destination memory every time. Although this could also mean saving allocations by using just one memory block for multiple calls. The first solution seems slightly more convenient for the simple case, but 'locks' the user in that case: there's always a fresh memory block allocated.
As an assignment in operating systems we have to write our own code for malloc and free in C programming language, I know if i asked the code for it there is no point of me to study. i'm facing the problem of not knowing where to include initializing char array with 50000 bytes and making two lists free and used. in my function i can't trigger malloc or free to happen automatically. and a 3rd party main program will be used to test my functions.....
if my file is mymalloc.c or what ever
void* myalloc(size_t size)
{
//code for allocating memory
}
void myfree(void *ptr)
{
//code for free the memory
}
where do the code for initiating memory space and lists will go..
I will provide you with the basic concept which you can use to write your own code for malloc() and free() functions using C.
Assume that we have a contiguous block of memory of a certain size. It will be our abstract sense of memory which will carry all the requested memory allocations plus the data structures that are used to hold data about those allocated blocks.
We use a simple linked list to carry the data related to the allocated as well as free blocks of memory.
Its structure is as follows.
struct block{
size_t size; /*Specifies the size of the block to which it refers*/
int free; /*This is the flag used to identify whether a block is free
or not*/
struct block *next; /*This points to the next metadata block*/
};
You will need 2 source files for this purpose. One is mymalloc.h which is the header file which contains the initialization parts and the function prototypes of the rest of the functions that we are going to implement. The other is the mymalloc.c source file which contains all the necessary function implementations.
There needs to be a function to initialize the first free memory block.
And another function to split a block of memory which has more than enough space to give to the requested size. And another method to scan through the linked list and merge any consecutive blocks that are free, so that it prevents external fragmentation.
Note: We use the First-fit-algorithm to find a free block to allocate memory.
I think this will help anyone who is in search of a simple way to write their own malloc and free functions using C. Please follow the following link for a detailed explanation.
http://tharikasblogs.blogspot.com/p/how-to-write-your-own-malloc-and-free.html
I think you only have to implement a memory manager. So you don't have to use brk, sbrk, ...
Just put used memory in a simple array and fragment it somehow. Since it's homework you want to make it as simple as possible or else you run into problems due to complexity/time constraints of your assignment.
You only have to decide which tactic you want to use. I'd suggest to use the buddy system. Though it's a bit more complicated than the most simple ones.. maybe fixed sized fragmentation is simpler..
Maybe this is also a good read.
Don't do something low-level as suggested in the other answers..
The implementation greatly depends upon operating system and architecture, anyhow you may take a look at this: http://www.raspberryginger.com/jbailey/minix/html/lib_2ansi_2malloc_8c-source.html
(and study how it works!).
If you are on a unix system you can look the manual of brk and sbrk. Those system calls "push/set" the limit of the heap.
Using those you can manage your memory pages, allocating them as you need.
I would advise a chained-list to manage your different allocated spaces and building functions to split them or to merge them if they are free.
If you need to try your code with high-level applications, you can name your functions malloc/free, compile them to a shared-object (.so) and then use LD_PRELOAD and LD_LIBRARY_PATH environment variables to load your .so and replace system's malloc.
Every command you call then will use your shared object and thus your malloc, telling you if your malloc is stable or if it fails to comply with reality.
If you need a clear example of this i'd be happy to put some code here, but I do not want to make my answer too hard to read.
First, you could make a fake malloc which always fail
/* fake malloc */
void* myalloc(size_t sz)
{ return NULL; }
but that is "cheating". You want to make a malloc which is useful.
You probably want to make a system call which asks the kernel for memory. Of course, you'll need the symetrical syscall to release memory. On Linux and many Posix systems you'll often use mmap and munmap syscalls.
(You could also use sbrk, but using mmap with munmap is easier and more general)
The idea is that you get big chunks of memory (with mmap) and then you manage smaller memory zones inside. The interesting detail is how to manage these smaller zones. You may want to deal with large malloc differently than "small" allocations.
You really want to read wikipedia page on memory allocation
You could have a global static variable that is initialized to zero. Then check that variable at the start of your malloc and free function. In your malloc function, if the variable is zero then initialize whatever you need, and then set the variable to non-zero. In your free function, just return if the variable is zero.
More like that, is a simple malloc :
void* my_malloc(size_t size)
{
return (sbrk(size));
}
man sbrk will help you.
The problem now is to create a free and to create a efficient malloc :-)
if you want to test your malloc you can do like this :
$> LD_PRELOAD=/mypath/my_malloc.so /bin/ls
but you need to create a dynamic library before because malloc is a .so
Is there any way in C to know if a memory block has previously been freed with free()? Can i do something like...
if(isFree(pointer))
{
//code here
}
Ok if you need to check whether a pointer has already been freed you may want to check your design. You should never have to either track reference count on a pointer or if it's freed. Also some pointers are not dynamically allocated memory so I hope you mean ones called with malloc(). This is my opinion but again if you have a solid design you should know when the things your pointers point to are done being used.
The only place I have seen this not work is in monolithic kernels because pages in memory need a usage count because of shared mappings among other things.
In your case simply set unused pointers to NULL and check that. This gives you a guaranteed way of knowing in the case that you have unused fields in structures that were malloced. A simple rule is wherever you free a pointer that needs to be checked in the above way just set it to NULL and replace isFree() with if pointer == NULL. This way no reference count needs to be tracked and you know for sure if your pointer is valid and not pointing to garbage.
No, there is no way.
You can, however, use a little code discipline as follows:
Always always always guard allocations with malloc:
void * vp;
if((vp = malloc(SIZE))==NULL){
/* do something dreadful here to respond to the out of mem */
exit(-1);
}
After freeing a pointer, set it to 0
free(vp); vp = (void*)0;
/* I like to put them on one line and think of them as one peration */
Anywhere you'd be tempted to use your "is freed" function, just say
if(vp == NULL)[
/* it's been freed already */
}
Update
#Jesus in comments says:
I can't really recommend this because as soon as you're done with that
memory the pointer should go out of scope immediately (or at least at
the end of the function that releases it) these dangling pointers
existence just doesn't sit right with me.
That's generally good practice when possible; the problem is that in real life in C it's often not possible. Consider as an example a text editor that contains a doubly-linked list of lines. The list is really simple:
struct line {
struct line * prev;
struct line * next;
char * contents;
}
I define a guarded_malloc function that allocates memory
void * guarded_malloc(size_t sz){
return (malloc(sz)) ? : exit(-1); /* cute, eh? */
}
and create list nodes with newLine()
struct line * newLine(){
struct line * lp;
lp = (struct line *) guarded_malloc(sizeof(struct line));
lp->prev = lp->next = lp-contents = NULL ;
return lp;
}
I add text in string s to my line
lp->contents = guarded_malloc(strlen(s)+1);
strcpy(lp->contents,s);
and don't quibble that I should be using the bounded-length forms, this is just an example.
Now, how can I implement deleting the contents of a line I created with the char * contents going out of scope after freeing?
I see nobody has addressed the reason why what you want is fundamentally impossible. To free a resource (in this case memory, but the same applies to basically any resource) means to return it to a resource pool where it's available for reuse. The only way the system could provide a reasonable answer to "Has the memory block at address X already been freed?" is to prevent this address from ever being reused, and store with it a status flag indicating whether it was "freed". But in this case, it has not actually been freed, since it is not available for reuse.
As others have said, the fact that you're trying to answer this question means you have fundamental design errors you need to address.
In general the only way to do this portably is to replace the memory allocation functions. But if you're only concerned about your own code, a fairly common technique is to set pointers to NULL after you free() them, so any subsequent use will throw an exception or segfault:
free(pointer);
pointer = NULL;
For a platform-specific solution, you may be interested in the Win32 function IsBadReadPtr (and others like it). This function will be able to (almost) predict whether you will get a segmentation fault when reading from a particular chunk of memory.
Note: IsBadReadPtr has been deprecated by Microsoft.
However, this does not protect you in the general case, because the operating system knows nothing of the C runtime heap manager, and if a caller passes in a buffer that isn't as large as you expect, then the rest of the heap block will continue to be readable from an OS perspective.
Pointers have no information with them other than where they point. The best you can do is say "I know how this particular compiler version allocates memory, so I'll dereference memory, move the pointer back 4 bytes, check the size, makes sure it matches..." and so on. You cannot do it in a standard fashion, since memory allocation is implementation defined. Not to mention they might have not dynamically allocated it at all.
On a side note, I recommend reading 'Writing Solid Code' by Steve McGuire. Excellent sections on memory management.
I think all malloc(sizeof(structure)) can be replaced this way:
char[sizeof(structure)]
Then when is malloc necessary?
When you don't know how many object of some kind you need (e.g. linked list elements);
when you need to have data structures of size known only at runtime (e.g. strings based on unknown input); this is somewhat mitigated by the introduction of VLAs in C99, but see the next point:
when you know at compile time their size (or you can use VLAs), but it's just too big for the stack (typically a few MBs at most) and it would make no sense to make such thing global (e.g. big vectors to manipulate);
when you need to have an object whose lifetime is different than what automatic variables, which are scope-bound (=>are destroyed when the execution exits from the scope in which they are declared), can have (e.g. data that must be shared between different objects with different lifetimes and deleted when no one uses it anymore).
Notice that it isn't completely impossible to do without dynamic memory allocation (e.g. the whole rockbox project works almost without it), but there are cases in which you actually need to emulate it by using a big static buffer and writing your own allocator.
By the way, in C++ you will never use malloc()/free(), but the operators new and delete.
Related: a case in which trying to work without malloc has proven to be a big mess.
You will use malloc to dynamically allocate memory, either because:
you don't know at compile-time how much memory will be required,
you want to be able to reallocate memory later on (for instance using realloc),
you want to be able to discard the allocated memory earlier than by waiting for its release based on the scope of your variable.
I can see your point. You could think you could always using a declarative syntax for all of these, even using variables to declare the size of your memory spaces, but that would:
be non-standard,
give you less control,
possibly use more memory as you will need to do copies instead of re-allocating.
You will probably get to understand this in time, don't worry.
Also, you should try to learn more about the memory model. You don't use the same memory spaces when using a dynamic allocation and when using a static allocation.
For first pointers, visit:
Dynamic Memory Allocation
Static Memory Allocation
Stack vs Heap
Stack vs Heap?
How C Programming Works - Dynamic Data Structures
Friendly advice: I don't know if you develop C on *NIX or Windows, but in any case if you use gcc, I recommend using the following compilation flags when you teach yourself:
-Wall -ansi -pedantic -Wstrict-prototypes
You should read about dynamic memory allocation. You obviously don't know what it is.
The main difference between the two is that memory allocated with malloc() exists until you say so. Static memory such as char buff[10]; only exists in the function scope.
malloc is a dynamic memory allocator which helps u up to assign memory to ur variables according to ur need and therefore reduces the loss of memory.It is also supported by realloc() function through which u can edit the memory required which u have defined earlier through malloc() or calloc(). So in short we can say that malloc() can be used for managing the memory space and making use of the necessary memory without wasting it.
You never should do this the way you are proposing. Others already told you about the difference of allocating storage on the heap versus allocation on the function stack. But if and when you are allocating on the stack you should just declare your variable:
structure A = { /* initialize correctly */ };
There is no sense or point in doing that as an (basically) untyped char array. If you also need the address of that beast, well, take the address of with &A.
When you don't know how much memory to allocate at compile time. Take a very simple program, where you need to store the numbers entered by the user in linked list. Here you dont know how many numbers will be entered by the user. So as user enters a number you will create a node for it using malloc and store it in the linked list.
If you use char[sizeof(structure)] instead of malloc, then I think no dynamic memory allocation is done.
Besides the fact that your char[] method cannot resize or determine the size at runtime, your array might not be properly aligned for the type of structure you want to use it for. This can result in undefined behaviour.
Is there a one-liner that will free the memory that is being taken by all pointers you created using mallocs? Or can this only be done manually by freeing every pointer separately?
you could do that by creating some kind of "wrapper" around malloc.
(warning that's only pseudo code showing the idea, there is no checking at all)
void* your_malloc(size_t size)
{
void* ptr = malloc(size);
// add ptr to a list of allocated ptrs here
return ptr;
}
void your_free(void *pointer)
{
for each pointer in your list
{
free( ptr_in_your_list );
}
}
But it doesn't sound like a good idea and I would certainly not do that, at least for general purpose allocation / deallocation. You'd better allocate and free memory responsibly when it is no longer needed.
You might want to look into memory pools. These are data structures built to do exactly this.
One common implementation is in the Apache Portable Runtime, which is used in the Apache web server, as well as other projects, such as Subversion.
malloc on it's own has implementation-defined behavior. So there isn't a necessity for it to keep track of all the pointers it has, which obviously puts a damper on the idea.
You'd need to make your own memory manager that tracks the pointers, and then provides a function called free_all or something that goes through the list of pointers it has and calls free on them.
Note, this sounds like a somewhat bad idea. It's better to be a bit more strict/responsible about your memory usage, and free things when you're done; not leave them hanging about.
Perhaps with a bit more background on where you want to apply your idea, we might find easier solutions.
Check out dlmalloc
ftp://g.oswego.edu/pub/misc/malloc.h
look at the following functions
/*
mspace is an opaque type representing an independent
region of space that supports mspace_malloc, etc.
*/
typedef void* mspace;
/*
create_mspace creates and returns a new independent space with the
given initial capacity, or, if 0, the default granularity size. It
returns null if there is no system memory available to create the
space. If argument locked is non-zero, the space uses a separate
lock to control access. The capacity of the space will grow
dynamically as needed to service mspace_malloc requests. You can
control the sizes of incremental increases of this space by
compiling with a different DEFAULT_GRANULARITY or dynamically
setting with mallopt(M_GRANULARITY, value).
*/
mspace create_mspace(size_t capacity, int locked);
/*
destroy_mspace destroys the given space, and attempts to return all
of its memory back to the system, returning the total number of
bytes freed. After destruction, the results of access to all memory
used by the space become undefined.
*/
size_t destroy_mspace(mspace msp);
...
/*
The following operate identically to their malloc counterparts
but operate only for the given mspace argument
*/
void* mspace_malloc(mspace msp, size_t bytes);
void mspace_free(mspace msp, void* mem);
void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size);
void* mspace_realloc(mspace msp, void* mem, size_t newsize);
You might want to do something called "arena allocation", where you allocate certain requests from a common "arena" which can be freed all at once when you're done.
If you're on Windows, you can use HeapCreate to create an arena, HeapAlloc to get memory from the heap/arena you just created, and HeapDestroy to free it all at once.
Note that when your program exit()s, all the memory you allocated with malloc() is freed.
Yes, you can do that unless you write your own defintion of malloc() and free(). You should probably call myCustomMalloc() instead of regular malloc() and you should be keeping track of all the pointers in some memory location and when you call the myCustomFree() method, you should be able to clear all the pointers that was created using your myCustomMalloc(). Note: both your custom methods will be calling malloc() and free() internally
By this way you can achieve your goal. I am a java person but I use to work a lot in C in my early days. I assume that you're trying to achieve a common solution where memory is being handled by the compiler. That has a cost of performance as it is seen in Java. You dont have to worry about allocation and freeing the memory. But that has a severe effect on performance. Its a tradeoff that you have to live with.