I am breaking my head in understanding the BoehmGC allocation scheme - GC_malloc. I am not getting how it allocates memory, not seen any malloc or mmap which GC_malloc internally calls.
Can someone kindly help me? Any links or code snippet will be of big help.
Huge thanks in advance.
Boehm GC source code
enter code here
254 /* Allocate lb bytes of composite (pointerful) data */
255 #ifdef THREAD_LOCAL_ALLOC
256 void * GC_core_malloc(size_t lb)
257 #else
258 void * GC_malloc(size_t lb)
259 #endif
260 {
261 void *op;
262 void **opp;
263 size_t lg;
264 DCL_LOCK_STATE;
265
266 if(SMALL_OBJ(lb)) {
267 lg = GC_size_map[lb];
268 opp = (void **)&(GC_objfreelist[lg]);
269 LOCK();
270 if( EXPECT((op = *opp) == 0, 0) ) {
271 UNLOCK();
272 return(GENERAL_MALLOC((word)lb, NORMAL));
273 }
274 /* See above comment on signals. */
275 GC_ASSERT(0 == obj_link(op)
276 || (word)obj_link(op)
277 <= (word)GC_greatest_plausible_heap_addr
278 && (word)obj_link(op)
279 >= (word)GC_least_plausible_heap_addr);
280 *opp = obj_link(op);
281 obj_link(op) = 0;
282 GC_bytes_allocd += GRANULES_TO_BYTES(lg);
283 UNLOCK();
284 return op;
285 } else {
286 return(GENERAL_MALLOC(lb, NORMAL));
287 }
288 }
There are two possibilities:
It returns a pointer given by GENERAL_MALLOC (two returns)
it sets op = *opp (the line with the EXPECT) and then it returns op. I'll say that the second is to reuse freed blocks.
For the second case: look at the value of opp before the if:
opp = (void **)&(GC_objfreelist[lg]);
In opp there is a pointer to the "free" list of objects.
The if probably checks if there is any block in that list. If there isn't (== 0) then it uses GENERAL_MALLOC.
Look at the os_deps.c file where (most) of the OS-dependent functions are implemented.
mmap can be used by Boehm-GC if it's configured to use that. (See the various GC_unix_get_mem(bytes) functions.)
If mmap isn't used, the other (bare) allocator used sbrk.
Related
This question already has answers here:
What's the need of array with zero elements?
(5 answers)
Closed 6 years ago.
In linux kernel (version 4.8),
"struct pid" is defined as following (from file: http://lxr.free-electrons.com/source/include/linux/pid.h). Here "numbers[1]" (at line 64) is a static array which can have only one element (because of array size is mentioned as 1).
57 struct pid
58 {
59 atomic_t count;
60 unsigned int level;
61 /* lists of tasks that use this pid */
62 struct hlist_head tasks[PIDTYPE_MAX];
63 struct rcu_head rcu;
64 struct upid numbers[1];
65 };
But then, in the following code at line 319 and 320 (from file: http://lxr.free-electrons.com/source/kernel/pid.c), array "numbers" is inside a for loop as 'numbers[i]'. How is it even correct because variable 'i' cannot have any value other than zero without causing segmentation fault? I have checked the value of 'i' during the loops to see if it ever goes more than 1. Yes it goes but still i don't see any segmentation fault. Am i missing something here?
297 struct pid *alloc_pid(struct pid_namespace *ns)
298 {
299 struct pid *pid;
300 enum pid_type type;
301 int i, nr;
302 struct pid_namespace *tmp;
303 struct upid *upid;
304 int retval = -ENOMEM;
305
306 pid = kmem_cache_alloc(ns->pid_cachep, GFP_KERNEL);
307 if (!pid)
308 return ERR_PTR(retval);
309
310 tmp = ns;
311 pid->level = ns->level;
312 for (i = ns->level; i >= 0; i--) {
313 nr = alloc_pidmap(tmp);
314 if (nr < 0) {
315 retval = nr;
316 goto out_free;
317 }
318
319 pid->numbers[i].nr = nr;
320 pid->numbers[i].ns = tmp;
321 tmp = tmp->parent;
322 }
Is it possible to have number of elements in an array more than array's size which is defined at compile time?
Yes. It is call undefined behavior and code should not be written to allow that.
How is it even correct because variable 'i' cannot have any value other than zero without causing segmentation fault?
It is possible; because code broke the contract. Writing outside an array's bounds may work. It may crash the program. It is undefined behavior.
C is not specified to prevent array access outside its bounds nor cause a seg fault. Such an access may be caught or not. Code itself needs to be responsible for insuring access is within bounds.
There are no training wheels and few safety nets specified in C
This code is from the file /fs/proc/array.c in the Linux headers. What does the int whole parameter mean? I want to know why sometimes you need to accumulate min_flt and maj_flts from the sig_struct and other times it's ok to just read their values straight out of the task_struct
346 static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
347 struct pid *pid, struct task_struct *task, int whole)
...
406 /* add up live thread stats at the group level */
407 if (whole) {
408 struct task_struct *t = task;
409 do {
410 min_flt += t->min_flt;
411 maj_flt += t->maj_flt;
412 gtime = cputime_add(gtime, t->gtime);
413 t = next_thread(t);
414 } while (t != task);
415
416 min_flt += sig->min_flt;
417 maj_flt += sig->maj_flt;
418 thread_group_times(task, &utime, &stime);
419 gtime = cputime_add(gtime, sig->gtime);
420 }
...
431 if (!whole) {
432 min_flt = task->min_flt;
433 maj_flt = task->maj_flt;
434 task_times(task, &utime, &stime);
435 gtime = task->gtime;
436 }
"whole" is just an argument name and in this context it seems to indicate "do_task_stat for the whole thread group".
do_task_stat is a static function only used within /fs/proc/array.c It is used in two places: proc_tid_stat (TID is "thread ID") and proc_tgid_stat (TGID is "thread group ID").
See Linux - Threads and Process for good explaination of thread groups.
I'm using zdelta library (http://cis.poly.edu/zdelta/) to compress a bunch of binary files and have been running into the issue where decompression almost always segfaults, even with the command line interface. Just wondering if anyone ran into this before?
I did some error isolation: compression output with my code is same as what I got from CLI (command is ./zdc reference.bin fileToCompress.bin > compressedFile.bin.del) so I assume compression works fine. The confusing part is say I use A.bin as reference and compress against itself, then everything works perfectly. As soon as I try a different file it segfaults (compress B.bin with A.bin being the reference, for example). Same with the decompression CLI.
Code for compression, bufferIn is the uncompressed data and bufferOut is an output buffer area which is large enough (ten times as input buffer, so even if the compression grows the file things should still work):
int rv = zd_compress(reference, refSize,
bufferIn, inputSize,
bufferOut, &outputSize);
Documentation for compress:
433 /* computes zdelta difference between target data and reference data
434 *
435 * INPUT:
436 * ref pointer to reference data set
437 * rsize size of reference data set
438 * tar pointer to targeted data set
439 * tsize size of targeted data set
440 * delta pointer to delta buffer
441 * the delta buffer IS allocated by the user
442 * *dsize size of delta buffer
443 *
444 *
445 * OUTPUT parameters:
446 * delta pointer to zdelta difference
447 * *dsize size of zdelta difference
448 *
449 * zd_compress returns ZD_OK on success,
450 * ZD_MEM_ERROR if there was not enough memory,
451 * ZD_BUF_ERROR if there was not enough room in the output
452 * buffer.
453 */
454 ZEXTERN int ZEXPORT zd_compress OF ((const Bytef *ref, uLong rsize,
455 const Bytef *tar, uLong tsize,
456 Bytef *delta, uLongf* dsize));
==============================
Code for decompression, bufferIn is the compressed data and bufferOut is an output buffer area which is 1000 times than the input (bad practice yes, but I'd like to figure out the segfault first..):
int rv = zd_uncompress(reference, refSize,
bufferOut, &outputSize,
bufferIn, inputSize);
Documentation for uncompress:
518 /* rebuilds target data from reference data and zdelta difference
519 *
520 * INPUT:
521 * ref pointer to reference data set
522 * rsize size of reference data set
523 * tar pointer to target buffer
524 * this buffer IS allocated by the user
525 * tsize size of target buffer
526 * delta pointer to zdelta difference
527 * dsize size of zdelta difference
528 *
529 *
530 * OUTPUT parameters:
531 * tar pointer to recomputed target data
532 * *tsize size of recomputed target data
533 *
534 * zd_uncompress returns ZD_OK on success,
535 * ZD_MEM_ERROR if there was not enough memory,
536 * ZD_BUF_ERROR if there was not enough room in the output
537 * buffer.
538 */
539 ZEXTERN int ZEXPORT zd_uncompress OF ((const Bytef *ref, uLong rsize,
540 Bytef *tar, uLongf *tsize,
541 const Bytef *delta, uLong dsize));
The size variables are all properly initialized. Whenever I run decompression it segfaults deep inside zdelta library at a memcpy in zdelta/inffast.c, seems like a bad destination (only except the case I mentioned above). Anyone had this issue before? Thanks!
I figured this problem was caused by a negation of an unsigned variable, in file inffast.c at line 138:
ptr = rwptr[best_ptr] + (sign == ZD_PLUS ? d : -d);
d is declared of type uInt, so the negation in the false part will (most likely) overflow, which was the cause of the bad destination address of memcpy().
SImply changing this into:
if(ZD_PLUS == sign)
{
ptr = rwptr[best_ptr] + d;
}
else
{
ptr = rwptr[best_ptr] - d;
}
Resolves the issue.
Same story for line 257 in infcodes.c:
c->bp = rwptr[best_ptr] + (c->sign == ZD_PLUS ? c->dist : -c->dist);
I want to read to and write from process' memory through /dev/mem.
First, I get process' memory map through a linux kernel module coded by myself, output is like this:
start_code_segment 4000000000000000
end_code_segment 4000000000019c38
start_data_segment 6000000000009c38
end_data_segment 600000000000b21d
start_brk 6000000000010000
brk 6000000000034000
start_stack 60000fffffde7b00
Second, I can convert virtual address(VA) to PA thorough the linux kernel module, for example, I can convert VA:0x4000000000000008 to PA:0x100100c49f8008
Third, function read_phy_mem can get memory data in PA:0x100100c49f8008，code at the final.
Problem: My problem is when I read text segment PA memory, everything is OK, but if I read data segment PA memory, *((long *)mapAddr) in line 243 will cause system to go down. Also, I tried
memcpy( &data, (void *)mapAddr, sizeof(long) )
but it still make the system go down.
other info: my computer is IA64, OS is Linux 2.6.18, when system is down, I can get output Info from console like this, then system will restart.
Entered OS MCA handler. PSP=20010000fff21320 cpu=0 monarch=1
cpu 0, MCA occurred in user space, original stack not modified
All OS MCA slaves have reached rendezvous
MCA: global MCA
mlogbuf_finish: printing switched to urgent mode, MCA/INIT might be dodgy or fail.
Delaying for 5 seconds...
code of function read_phy_mem
/*
* pa: physical address
* data: memory data in pa
*
* return int: success or failed
*/
188 int read_phy_mem(unsigned long pa,long *data)
189 {
190 int memfd;
191 int pageSize;
192 int shift;
193 int do_mlock;
194 void volatile *mapStart;
195 void volatile *mapAddr;
196 unsigned long pa_base;
197 unsigned long pa_offset;
198
199 memfd = open("/dev/mem", O_RDWR | O_SYNC);
200 if(memfd == -1)
201 {
202 perror("Failed to open /dev/mem");
203 return FAIL;
204 }
205
206 shift = 0;
207 pageSize = PAGE_SIZE; //#define PAGE_SIZE 16384
208 while(pageSize > 0)
209 {
210 pageSize = pageSize >> 1;
211 shift ++;
212 }
213 shift --;
214 pa_base = (pa >> shift) << shift;
215 pa_offset = pa - pa_base;
224 mapStart = (void volatile *)mmap(0, PAGE_SIZE, PROT_READ | PROT_WRITE,MAP_SHARED | MAP_LOCKED, memfd, pa_base);
226 if(mapStart == MAP_FAILED)
227 {
228 perror("Failed to mmap /dev/mem");
229 close(memfd);
230 return FAIL;
231 }
232 if(mlock((void *)mapStart, PAGE_SIZE) == -1)
233 {
234 perror("Failed to mlock mmaped space");
235 do_mlock = 0;
236 }
237 do_mlock = 1;
238
239 mapAddr = (void volatile *)((unsigned long)mapStart + pa_offset);
243 printf("mapAddr %p %d\n", mapAddr, *((long *)mapAddr));
256 if(munmap((void *)mapStart, PAGE_SIZE) != 0)
257 {
258 perror("Failed to munmap /dev/mem");
259 }
260 close(memfd);
269 return OK;
270 }
Can anyone understand why text segment works well but data segment does not?
I guess, its happening because code-section remain in memory while process executes(if not a DLL code), Whereas data section leave in & out continuously.
Try with stack-Segment. And check if its working?
Write your own test program and allocate memory dynamically in KBs and keep that memory in use within a loop. Than try it with your code to read memory segments of test program. I think it will work.
I have done similar work in windows to replace BIOS address from IVT.
Should be root user.
/*
180 * Create a regular file
181 */
182 static int
183 ufs_create(ap)
184 struct vop_create_args /* {
185 struct vnode *a_dvp;
186 struct vnode **a_vpp;
187 struct componentname *a_cnp;
188 struct vattr *a_vap;
189 } */ *ap;
190 {
191 int error;
192
193 error =
194 ufs_makeinode(MAKEIMODE(ap->a_vap->va_type, ap->a_vap->va_mode),
195 ap->a_dvp, ap->a_vpp, ap->a_cnp);
196 if (error)
197 return (error);
198 return (0);
199 }
Please help me to get information from line 182-189...this is strange for me.. What is this function title means? (I mean, what is return value, what is input parameter?)
Thank you all.
The return type is int and it takes one argument, a struct vop_create_args* named ap. This is K&R notation.
It's an old-style (pre-prototype) function declaration. The function is local to the current translation unit, returns an int, ap is the parameter it accepts and it's of the type:
struct vop_create_args *
All that other stuff is just comments, presumably echoing the actual definition of the structure so that the information is held locally as well (so a lazy coder doesn't have to go looking for it, a somewhat dangerous practice if the actual and local definitions get out of step).
It's equivalent to:
static int ufs_create (struct vop_create_args *ap) { ...