I would like to dispatch some tasks out via fork, and collect some information about the results of those tasks in an array.
My thought is to use mmap to share a data structure between the two, and have the child process update the array of structs with the result of the activity, but i'm having some issues.
#include <time.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <sys/mman.h>
#include <stdlib.h>
typedef struct message_s {
int status;
char message[256];
} message_t;
void child(message_t **result) {
result[0]->status = 2;
sprintf((char *) &result[0]->message, "Hello World!");
usleep(1);
}
void parent() {
printf("Parent\n");
usleep(1);
return;
}
int main() {
size_t result_size = 1000 * sizeof(message_t);
message_t **result_ar = mmap(NULL, result_size,
PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANON, -1, 0);
pid_t child_pid = fork();
switch (child_pid) {
case 0:
child(result_ar);
exit(0);
case -1:
exit(-1);
break;
default:
parent();
}
int child_status;
waitpid(child_pid, &child_status, 0);
printf("\nRESULT: %i: %s\n\n", result_ar[0]->status, result_ar[0]->message);
msync(result_ar, result_size, MS_SYNC);
munmap(result_ar, result_size);
return 0;
}
I'm really sort of lost here, and while i can find "similar" questions on SO, they either do not have answers or are plagued with other issues, or both.
When the code above runs, i get the following concatenated result.
Process: concurrent_test [2537]
Path: /Users/USER/Library/Caches/*/concurrent_test
Identifier: concurrent_test
Version: 0
Code Type: X86-64 (Native)
OS Version: Mac OS X 10.10.5 (14F1509)
Report Version: 11
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000
VM Regions Near 0:
-->
__TEXT 000000010144a000-000000010144c000 [ 8K] r-x/rwx SM=COW /Users/USER/Library/Caches/*
Application Specific Information:
crashed on child side of fork pre-exec
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 concurrent_test 0x000000010144bb33 child + 35
1 concurrent_test 0x000000010144bc1f main + 127
2 concurrent_test 0x000000010144b5a9 _start + 247
3 concurrent_test 0x000000010144b4b1 start + 33
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x00007fff5e7b5908 rcx: 0x000000010144beb3 rdx: 0xffffffffffffffff
rdi: 0x0000000000000000 rsi: 0x0000000000000000 rbp: 0x00007fff5e7b5730 rsp: 0x00007fff5e7b5720
r8: 0x0000000000000303 r9: 0x0000000000000000 r10: 0x0000000000000030 r11: 0x0000000000000206
r12: 0x00007fff5e7b57e0 r13: 0x0000000000000000 r14: 0x00007fff5e7b57f0 r15: 0x0000000000000001
rip: 0x000000010144bb33 rfl: 0x0000000000010246 cr2: 0x0000000000000000
Logical CPU: 6
Error Code: 0x00000006
Trap Number: 14
And I'm using:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin14.5.0
Thread model: posix
You have one indirection too much in your code. Try message_t *result_ar = mmap(...); and result[0].status = 2; to fix this issue. Also, don't forget the error checking! mmap() can fail.
Related
I have the following simple program:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include <assert.h>
#include <errno.h>
#define handle_error_en(en, msg) \
{do { errno = en; perror(msg); exit(EXIT_FAILURE); } while (0);}
#define status_check(stat) ((stat != 0) ? (handle_error_en(stat, "status error")) : ((void)0))
static void* thread_start(void *arg)
{
printf("%s\n", "thread working..");
sleep(1);
return NULL;
}
int main(int argc, char const *argv[])
{
int status;
pthread_t thread;
status = pthread_create(&thread,
NULL,
thread_start,
NULL);
status_check(status);
status = pthread_join(thread, NULL);
status_check(status);
printf("%s\n", "exit program..");
return 0;
}
When I run Valgind with
Valgrind --tool=helgrind ./threads_simple
I get 54 errors, all being data race errors. Just picking one of them:
Possible data race during write of size 4 at 0x1003FD2D8 by thread #1
==17334== Locks held: none
==17334== at 0x1003E2FDF: spin_unlock (in /usr/lib/system/libsystem_platform.dylib)
==17334== by 0x1003F7EAF: pthread_join (in /usr/lib/system/libsystem_pthread.dylib)
==17334== by 0x100000E37: main (threads_simple.c:29)
==17334==
==17334== This conflicts with a previous write of size 4 by thread #2
==17334== Locks held: none
==17334== at 0x1003E2FDF: spin_unlock (in /usr/lib/system/libsystem_platform.dylib)
==17334== by 0x1003F6FD6: _pthread_start (in /usr/lib/system/libsystem_pthread.dylib)
==17334== by 0x1003F43EC: thread_start (in /usr/lib/system/libsystem_pthread.dylib)
==17334== Address 0x1003fd2d8 is in the Data segment of /usr/lib/system/libsystem_pthread.dylib
I have written many more small thread-programms, all give errors like this. Is my Valgrind installation corrupt? I installed it on mac via brew.
As RHEL6 introduced a new arena allocator design because of the same number of arena’s are increased for a single thread which is resulting into more virtual memory usage.
RHEL6 is allocating separate chunks of memory for each thread. Number of Arena per thread are calculated as below as per my understanding on 32/64 bit system:
On 32 Bit system :
Number of Arena = 2 * Number of cores .
On 64 Bit system :
Number of Arena = 8 * Number of cores .
Please validate if my understanding have some gap.
My machine has 4 cores and 64 bit system,
$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 69
Stepping: 1
CPU MHz: 759.000
BogoMIPS: 4589.41
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 3072K
NUMA node0 CPU(s): 0-3
$ uname -a
Linux admin 3.13.0-71-generic #114-Ubuntu SMP Tue Dec 1 02:34:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
So it has 4*8=32 Arena's per thread. Each arena has a maximum size of 64MB; pet thread virtual memory can be reached to 32*64=2GB(per thread).
Please let me know if my understanding is correct.
I have created a sample program. It is allocated 1.5 GB of heap memory.
#include <pthread.h>
#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <malloc.h>
#include <unistd.h>
static void *thread(void *arg)
{
int i=0;
char *x[1024];
unsigned int pid;
char str[15];
char cmd[30]="cat /proc/";
pid = getpid();
sprintf(str, "%d", pid);
strcat(cmd,str);
strcat(cmd,"/status");
system(cmd);
for(i=0;i<1536;i++)
{
x[i] = malloc(1*1024*1024);
printf("Memory Allocated %d:\n ",i);
system(cmd);
}
for(i=0;i<1536;i++)
{
*x[i] = 5;
free(x[i]);
}
return NULL;
}
int main(void)
{
unsigned i;
pthread_t t;
pthread_create(&t, NULL, thread, NULL);
pthread_join(t, NULL);
return 0;
}
thread t is heap memory allocation is lesser than 2GB. but it generates the coredump after allocating 1GB approx. For more detail please refer the below detail:
Name: arena
State: S (sleeping)
Tgid: 12511
Ngid: 0
Pid: 12511
PPid: 4417
TracerPid: 0
Uid: 1000 1000 1000 1000
Gid: 1000 1000 1000 1000
FDSize: 256
Groups: 4 24 27 30 46 108 124 129 1000
VmPeak: 1133924 kB
VmSize: 1133924 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 4568 kB
VmRSS: 4568 kB
VmData: 1127636 kB
VmStk: 136 kB
VmExe: 4 kB
VmLib: 2012 kB
VmPTE: 2092 kB
VmSwap: 0 kB
Threads: 2
SigQ: 0/46560
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000006
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000001fffffffff
Seccomp: 0
Cpus_allowed: ff
Cpus_allowed_list: 0-7
Mems_allowed: 00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 1027
nonvoluntary_ctxt_switches: 4
Why the coredump generated? is my above understanding is incorrect to calculate the thread maximum virtual memory?
Also what are the consequences happened if virtual memory maximum limit reached
Please note I have set ulimit to unlimited
$ ulimit
unlimited
Why the coredump generated?
You have
char *x[1024];
…
for(i=0;i<1536;i++)
{
x[i] = malloc(1*1024*1024);
- a plain case of array overflow.
Memory allocation generally isn't specific to any distribution. Typically, the allocator is the glibc implementation. However, the amount of memory a thread can allocate should be a function of the kernel, not the allocator. Some of the reasons you wouldn't be able to allocate memory might include exhausting physical memory, exhausting swap memory, ulimit, cgroups, etc...
If you still suspect the allocator though, you can try using mmap to directly map the memory into the process.
To check whether I can change code at run time or not, I wrote a small piece of code(below) in linux.
int add(int a, int b)
{
printf("reached inside the function");
return a+b;
}
int main()
{
int x=10;
int y = 20;
int * p;
int z;
int (*fp) (int , int);
fp = add;
p = (int *)fp;
*(p+0) = 0;
z = add(x,y);
}
As there is no issue from c coding point of view, compiler compiles is perfectly and link also happens. But at run time it fails with below error:
Segmentation fault (core dumped)
Above error is perfect, because code segment is not supposed to be changed at run time, But I want to know how it is controlled at run time.
To know more about the code area restrictions, I ran readelf on the output file and result shows below in section headers:
[13] .text PROGBITS 08048330 000330 0001cc 00 AX 0 0 16
where section header flag shows as "AX" , means this section is just allocatable and executable. It does not support writing ("W").
and with a small change in the elf file I was able to modify the flag of this section as "WAX" , as below:
[13] .text PROGBITS 08048330 000330 0001cc 00 WAX 0 0 16
But still I get the same "segmentation fault" error.
I want to know - how is it achieved by the system?
The system is ignoring the W flag here:
$ gcc -Wall file.c
$ readelf -S a.out | grep .text
[14] .text PROGBITS 08048330 000330 0001cc 00 AX 0 0 16
$ objcopy a.out --set-section-flags .text=alloc,code,data a.out
$ readelf -S a.out | grep .text
[14] .text PROGBITS 08048330 000330 0001cc 00 WAX 0 0 16
$ gdb -q a.out
Reading symbols from a.out...(no debugging symbols found)...done.
(gdb) r
Starting program: a.out
Program received signal SIGSEGV, Segmentation fault.
0x0804842f in main ()
(gdb) x/i 0x0804842f
0x804842f <main+45>: movl $0x0,(%eax)
(gdb)
You still cannot write to p. You can change the memory page protection at runtime using mprotect:
#include <stdio.h>
#include <unistd.h>
#include <stdint.h>
#include <sys/mman.h>
int add(int a, int b)
{
printf("reached inside the function");
return a+b;
}
int main()
{
int x=10;
int y = 20;
int * p;
int z;
int (*fp) (int , int);
long pagesize;
fp = add;
p = (int *)fp;
pagesize = sysconf(_SC_PAGESIZE);
if(mprotect((void *)((uintptr_t)p & ~((uintptr_t)pagesize - 1)), pagesize, PROT_READ | PROT_WRITE | PROT_EXEC) == -1)
perror("Error mprotect()");
*(p+0) = 0;
z = add(x,y);
return 0;
}
this will leave you with the bad instruction to fix:
$ gcc -Wall file.c
$ ./a.out
Segmentation fault
$ gdb -q a.out
Reading symbols from a.out...(no debugging symbols found)...done.
(gdb) r
Starting program: a.out
Program received signal SIGSEGV, Segmentation fault.
0x08048484 in add ()
(gdb) x/i 0x08048484
0x8048484 <add>: add %al,(%eax)
(gdb)
Does the segmentation fault happen at the same place?
It could be that the OS ignores the W flag, but I don't think that's the case here. Assuming the OS honours the flag, the following is relevant.
You are overwriting the first instruction of the add function with 0, which in x86 assembly is (assuming 4 bytes int here)
00000000 0000 add [bx+si],al
00000002 0000 add [bx+si],al
This most likely ends up accessing invalid memory, at bx+si.
I am trying to find a regular expression in a large memory mapped file
by using regexec() function. I discovered that the program crashes when the file size
is the multiple of the page size.
Is there a regexec() function that has the length of the string
as additional argument?
Or:
How to find a regex in a memory mapped file?
Here is the minimal example that ALWAYS crashes
(if I run less that 3 threads program doesn't crash):
ls -la ttt.txt
-rwx------ 1 bob bob 409600 Jun 14 18:16 ttt.txt
gcc -Wall mal.c -o mal -lpthread -g && ./mal
[1] 11364 segmentation fault (core dumped) ./mal
And the program is:
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <stdio.h>
#include <assert.h>
#include <pthread.h>
#include <regex.h>
void* f(void*arg) {
int size = 409600;
int fd = open("ttt.txt", O_RDONLY);
char* text = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);
fd = open("/dev/zero", O_RDONLY);
char* end = mmap(text + size, 4096, PROT_READ, MAP_PRIVATE | MAP_FIXED, fd, 0);
close(fd);
assert(text+size == end);
regex_t myre;
regcomp(&myre, "XXXXX", REG_EXTENDED);
regexec(&myre, text, 0, NULL, 0);
regfree(&myre);
return NULL;
}
int main(int argc, char* argv[]) {
int n = 10;
int i;
pthread_t t[n];
for (i = 0; i < n; ++i) {
pthread_create(&t[n], NULL, f, NULL);
}
for (i = 0; i < n; ++i) {
pthread_join(t[n], NULL);
}
return 0;
}
P.S.
This is the output from gdb:
gdb ./mal
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/bob/prog/c/mal...done.
(gdb) r
Starting program: /home/srdjan/prog/c/mal
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff77ff700 (LWP 11817)]
[New Thread 0x7ffff6ffe700 (LWP 11818)]
[New Thread 0x7ffff6799700 (LWP 11819)]
[New Thread 0x7fffeffff700 (LWP 11820)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6799700 (LWP 11819)]
__strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:72
72 ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory.
(gdb) bt
#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:72
#1 0x00007ffff78df254 in __regexec (preg=0x7ffff6798e80, string=0x7fffef79b000 'a' <repeats 200 times>..., nmatch=<optimized out>,
pmatch=0x0, eflags=<optimized out>) at regexec.c:245
#2 0x00000000004008e6 in f (arg=0x0) at mal.c:24
#3 0x00007ffff7bc4e9a in start_thread (arg=0x7ffff6799700) at pthread_create.c:308
#4 0x00007ffff78f24bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5 0x0000000000000000 in ?? ()
(gdb)
Celada correctly identifies the problem - the file data does not necessarily include a null terminator.
You could fix the problem by mapping a page of zeroes immediately after the file:
int fd;
char *text;
fd = open("ttt.txt", O_RDONLY);
text = mmap(NULL, 409600, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);
fd = open("/dev/zero", O_RDONLY);
mmap(text + 409600, 4096, PROT_READ, MAP_PRIVATE | MAP_FIXED, fd, 0);
close(fd);
(Note that you can close fd immediately after the mmap(), because mmap() adds a reference to the open file description).
You should of course add error-checking to the above. Also, many UNIX systems support a MAP_ANONYMOUS flag which you can use instead of opening /dev/zero (but this is not in POSIX).
The problem is that regexec() is used to match a null-terminated string against the precompiled pattern buffer, but an mmaped file is not necessarily (indeed not usually) null-terminated. Thus, it is looking beyond the end of the file to find a NUL character (0 byte).
You would need a version of regexec() that takes a buffer and a size argument instead of a null-terminated string, but there doesn't appear to be one.
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/types.h>
#define CODESIZE 7
static unsigned char original_code[CODESIZE];
static unsigned char jump_code[CODESIZE] =
"\xb8\x00\x00\x00\x00" /* movq $0, %rax */
"\xff\xe0" /* jump *%rax */
;
void (*sync_readahead)( struct address_space *mapping, struct file_ra_state *ra, struct file *filp, pgoff_t offset, unsigned long req_size ) = (void (*)(struct address_space *, struct file_ra_state *, struct file *, pgoff_t , unsigned long ) )0xc0197100;
int hijack_start(void);
void hijack_stop(void);
void intercept_init(void);
void intercept_start(void);
void intercept_stop(void);
void fake_printk(struct address_space *mapping, struct file_ra_state *ra, struct file *filp, pgoff_t offset, unsigned long req_size);
int hijack_start()
{
printk(KERN_INFO "I can haz hijack?\n" );
intercept_init();
return 0;
}
void hijack_stop()
{
intercept_stop();
return;
}
void intercept_init()
{
printk(KERN_INFO "in the intercept_init\n" );
memcpy( original_code, sync_readahead, 7 );
*(long *)&jump_code[1] = (long)fake_printk;
memcpy( sync_readahead, jump_code, 7 );
printk(KERN_INFO "in the hijack?\n" );
//real_printk=NULL;
printk(KERN_INFO "begin the hijack?\n" );
memcpy( sync_readahead, jump_code, CODESIZE );
printk(KERN_INFO "begin the hijack?\n" );
return;
}
void intercept_stop()
{
memcpy( sync_readahead, original_code, CODESIZE );
}
void fake_printk(struct address_space *map, struct file_ra_state *a, struct file *fil, pgoff_t offse, unsigned long req_siz)
{
printk(KERN_INFO "in the fake printk\n");
// return ret;
}
MODULE_LICENSE("GPL");
module_init( hijack_start );
module_exit( hijack_stop );
I want to replace Linux kernel function by address (/proc/kallsyms), but when I memcpy the new function to the address (Linux kernel):
memcpy( sync_readahead, jump_code, CODESIZE );
there are errors (segmentation fault). I have seen some examples to replace Linux kernel function in the same way. Would you please help me to solve the problem? Thank you very much.
Information as follows:
ubuntu kernel: [ 574.826458] *pde = 0087d067 *pte = 00197161
ubuntu kernel: [ 574.826468] Modules linked in: hijack(+) test(+) binfmt_misc bridge stp bnep input_polldev video output vmblock vsock vmmemctl vmhgfs pvscsi acpiphp lp ppdev pcspkr psmouse serio_raw snd_ens1371 gameport snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore snd_page_alloc vmci i2c_piix4 parport_pc parport intel_agp agpgart shpchp mptspi mptscsih mptbase scsi_transport_spi floppy fbcon tileblit font bitblit softcursor vmxnet
ubuntu kernel: [ 574.826491]
ubuntu kernel: [ 574.826493] Pid: 4694, comm: insmod Tainted: G D (2.6.28-11-generic #42-Ubuntu) VMware Virtual Platform
ubuntu kernel: [ 574.826496] EIP: 0060:[<f7c92101>] EFLAGS: 00010246 CPU: 0
ubuntu kernel: [ 574.826498] EIP is at intercept_init+0x41/0x70 [hijack]
ubuntu kernel: [ 574.826499] EAX: f5ec4b60 EBX: 00000000 ECX: ffffffff EDX: 00004c4c
ubuntu kernel: [ 574.826501] ESI: f7c9252c EDI: c0197100 EBP: f5edbe18 ESP: f5edbe0c
ubuntu kernel: [ 574.826502] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
ubuntu kernel: [ 574.826506] f7c921a6 f7c92130 00000000 f5edbe24 f7c92147 f7c921d5 f5edbf8c c010111e
ubuntu kernel: [ 574.826618] ---[ end trace ccc07e4b4d814976 ]---
Kernel function hijacking is very tricky business, and it needs to be exactly right in order to not run into all kinds of issues.
I am currently working on a module that does this, and it (at the time of this writing) works for 2.6.18+ kernels:
https://github.com/cormander/tpe-lkm
You'll be most interested in the hijacks.c file.
Many portions of this process are architecture, kernel version dependent, and CPU feature dependent as well.
UPDATE
The module now uses the 0XE9 jump opcode and should work for you. The nitty gritty details are in hijacks.c, and the "high level" logic you'll be most interested in is in the hijack_syscalls() function in security.c