Implementing Paranoid Array in C - c

I have the following assignment for an online class and was wondering if anyone was familiar with paranoid arrays, as it's very difficult to get help for this specific class.
Your paranoid array will expose a very simple interface provided in parray.h. You can
assume the parray will not be free’d. The parray new call creates an array of a set-number
of entries (count argument) of a fixed size (size argument). Internally, the parray will
not arrange the elements consecutively in memory, so the parray entry function returns a
pointer to a given entry (specified by argument index).
In order to trigger a segfault upon overflow within an element, you should use guard
pages. A guard page’s main purpose is to trigger segfaults when it is accessed. Thus, pagetable read, write, and execute permissions on a guard table are disabled, so any access to
the page will trigger the fault. When a guard page is placed immediately after a buffer or
data structure, any buffer overflow bugs affecting that piece of memory will hit the guard
page, triggering an instant segfault
Every entry in your array should be bounded by guard pages on each side. For example,
an array with 10 entries should use 11 guard pages: one before the first entry, one after the
last entry, and nine in between consecutive entries.
My parray_new and parray_entry call is as follows:
typedef char byte;
parray_t* parray_new(int size, int count)
{
struct parray* p = NULL;
// TODO: Allocate and return parray
// Add guard pages first at this time
int pagesize = getpagesize();
p->size = (size * count) + (pagesize * count) + pagesize;
p->array = malloc(p->size + pagesize - 1);
if(posix_memalign(&p->array, p->size, count))
{
exit(0);
}
return p;
}
void* parray_entry(struct parray* p, int index)
{
//int pagesize = getpagesize();
byte* entry = NULL;
// TODO: compute correct entry
if (mprotect(&p->array, p->size-1, PROT_READ))
{
exit(0);
}
if (mprotect(&p->array, p->size, PROT_WRITE))
{
exit(0);
}
entry = (void*)(p->array + index);
return entry;
}
I also have the following handler:
static void handler(int sig, siginfo_t *si, void* unused)
{
// TODO: Use fprintf or perror to print
// a message indicating a segmentation fault
// happened and provide the memory address
// where the fault happened
fprintf(stderr, "Segmentation Fault\n k = %d, %p\n", sig, si >si_addr);
}
Finally, the main method:
int main(void)
{
struct sigaction sa;
/*
* TODO: Overwrite the signal handler for
* SIGSEGV
*/
memset(&sa, '\0', sizeof(sa));
sa.sa_flags = SA_SIGINFO;
sa.sa_sigaction = handler;
if (sigaction(SIGSEGV, &sa, NULL) == -1)
{
perror("sigaction");
exit(EXIT_FAILURE);
}
}
There's also several tests to run in the main method, but I've left those out because I encounter an error before I even reach them. What happens is, the handler prints forever (Segmentation fault, k = 11, 0x8). I do not know the significance of 11 or 0x8, but it does not stop printing that sequence until I force it.
Any help would be greatly appreciated, and I apologize for the length of this post. Thanks
Edit: from what I can see, the handler continues to print. It's not so much that I'm getting a seg fault (I might be), but whatever I put in the handler it continues to print. Also if I change it to perror it does the same. What can I do to allow the program to continue after the handler?

Related

How to implement a dynamic–grow stack with pthread?

From APUE, I learned about guardsize and stackaddr of a stack. If a rsp pointer is going to be lower than stackaddr, the rsp enters the guard stack area, and a signal emits to notify the program.
I am wondering if it is possible to implement dynamic–grow(which grows dynamically) stack using this feature. Can you show
how?
The stack can be grown "dynamically"
Set up an alternate signal stack.
Set up signal handler for SIGSEGV with SA_ONSTACK to use the alternate stack.
When the program runs out of stack, it will be issued a SIGSEGV signal.
The signal handler can use getrlimit/setrlimit with RLIMIT_STACK to change the stack size.
getrlimit and setrlimit are not explicitly mentioned in man signal-safety but I don't see why they would be a problem in a signal handler.
Here is a sample test program. It is annotated. Invoke with -d to dynamically increase the stack size from within the signal handler:
// stacktest.c -- test dynamically growing stack
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <signal.h>
#include <assert.h>
#include <setjmp.h>
#include <sys/time.h>
#include <sys/resource.h>
volatile int opt_t = 0;
volatile int opt_d = 0;
volatile int opt_i = 0;
void *top; // initial top of stack
// signal handler
volatile int may_grow = 1; // grow stack within signal handler
volatile int err1 = 0; // error on get
volatile int err2 = 0; // error on set
volatile int hitno; // number of signals
jmp_buf jbuf;
// alternate signal stack
unsigned char altstk[64 * 4096] __attribute__((aligned(4096)));
// xprtstr -- print string in signal handler
void
xprtstr(const char *str)
{
size_t len = strlen(str);
write(1,str,len);
}
// xprtstr -- print number in signal handler
void
xprtnum(unsigned long val,const char *sym)
{
static const char *hex = "0123456789ABCDEF";
xprtstr(" ");
xprtstr(sym);
xprtstr("=");
char buf[100];
char *bp = &buf[50];
*bp-- = 0;
for (int idx = 0; idx < 16; ++idx, --bp) {
*bp = hex[val & 0x0F];
val >>= 4;
}
++bp;
xprtstr(bp);
}
// sigfault -- SIGSEGV handler
void
sigfault(int signo,siginfo_t *info,void *vp)
{
++hitno;
xprtstr("ISR hit");
xprtnum(signo,"signo");
xprtnum(hitno,"hitno");
xprtstr("\n");
struct rlimit rlim;
do {
if (opt_t) {
xprtstr("ISR test\n");
siglongjmp(jbuf,1);
break;
}
// dynamically grow the stack
if (may_grow) {
xprtstr("ISR grow\n");
may_grow = 0;
err1 = getrlimit(RLIMIT_STACK,&rlim);
xprtnum(rlim.rlim_cur,"rlim_cur");
xprtnum(rlim.rlim_cur / 1024,"rlim_cur");
xprtstr("\n");
rlim.rlim_cur += 8 * 1024 * 1024;
err2 = setrlimit(RLIMIT_STACK,&rlim);
getrlimit(RLIMIT_STACK,&rlim);
xprtnum(rlim.rlim_cur,"rlim_cur");
xprtnum(rlim.rlim_cur / 1024,"rlim_cur");
xprtstr("\n");
break;
}
// stop the program
xprtstr("ISR stop\n");
siglongjmp(jbuf,2);
} while (0);
}
// loop -- recursive function to overflow stack
void
loop(unsigned char *old)
{
// get some space on the stack
unsigned char cur[4096];
// get current stack frame address
void *frame = __builtin_frame_address(0);
// get amount of space used on stack
size_t dif = top - frame;
// show where we are
printf("loop: top=%p frame=%p dif=%8.8zX/%zu may_grow=%d\n",
top,frame,dif,dif / 1024,may_grow);
// keep consuming more stack
// NOTE: we don't actually use cur/old but ensure that the compiler won't
// optimize it away
loop(cur);
}
#define SHOWFLAGS(_msk) \
if (flags & (_msk)) \
printf(" " #_msk)
void
showsa(const struct sigaction *sa,const char *who)
{
int flags = sa->sa_flags;
// show the signal flags
printf("showflags: %s flags=%8.8X",who,flags);
SHOWFLAGS(SA_NOCLDSTOP);
SHOWFLAGS(SA_NOCLDWAIT);
SHOWFLAGS(SA_NODEFER);
SHOWFLAGS(SA_ONSTACK);
SHOWFLAGS(SA_RESETHAND);
SHOWFLAGS(SA_RESTART);
SHOWFLAGS(SA_SIGINFO);
// show the signal mask
for (int signo = 1; signo < 32; ++signo) {
if (sigismember(&sa->sa_mask,signo))
printf(" S%d",signo);
}
printf("\n");
}
// setup_signal -- set up signal handler
void
setup_signal(int signo)
{
struct sigaction sa;
assert(sigaction(signo,NULL,&sa) == 0);
sa.sa_sigaction = sigfault;
showsa(&sa,"BEF");
sa.sa_flags |= SA_SIGINFO;
sa.sa_flags |= SA_ONSTACK;
sa.sa_flags &= ~SA_RESETHAND;
assert(sigaction(signo,&sa,NULL) == 0);
assert(sigaction(signo,NULL,&sa) == 0);
showsa(&sa,"AFT");
}
// setup_altstack -- set up alternate signal stack
void
setup_altstack(void)
{
stack_t ss;
sigaltstack(NULL,&ss);
printf("setup_altstack: ss_sp=%p ss_flags=%8.8X ss_size=%zu\n",
ss.ss_sp,ss.ss_flags,ss.ss_size);
void *sp = altstk;
sp += 16 * 4096;
printf("altstk=%p sp=%p\n",altstk,sp);
ss.ss_sp = sp;
ss.ss_size = sizeof(altstk) / 2;
ss.ss_flags = SS_ONSTACK;
assert(sigaltstack(&ss,NULL) == 0);
sigaltstack(NULL,&ss);
printf("setup_altstack: ss_sp=%p ss_flags=%8.8X ss_size=%zu\n",
ss.ss_sp,ss.ss_flags,ss.ss_size);
if (ss.ss_sp != sp) {
printf("setup_altstack: failed\n");
exit(99);
}
}
#define SHOWOPT(_opt,_reason) \
if (_opt) \
printf(#_opt " -- " _reason)
int
main(int argc,char **argv)
{
setlinebuf(stdout);
--argc;
++argv;
for (; argc > 0; --argc, ++argv) {
char *cp = *argv;
if (*cp != '-')
break;
cp += 2;
switch (cp[-1]) {
case 'd':
opt_d = ! opt_d;
break;
case 'i':
opt_i = ! opt_i;
break;
case 't':
opt_t = ! opt_t;
break;
}
}
// show the options
SHOWOPT(opt_d,"dynamically grow stack within signal handler");
SHOWOPT(opt_i,"grow stack initially");
SHOWOPT(opt_t,"test sigsetjmp/siglongjmp");
// set up alternate signal stack and signal handler
if (opt_d || opt_t) {
setup_altstack();
setup_signal(SIGSEGV);
}
// test our sigsetjmp/siglongjmp
if (opt_t) {
for (int try = 1; try <= 2; ++try) {
if (! sigsetjmp(jbuf,1)) {
printf("main: ptr try=%d\n",try);
unsigned int *ptr = NULL;
*ptr = 23;
}
else
printf("main: resume try=%d\n",try);
}
opt_t = 0;
}
// set up large stack outside of signal handler
if (opt_i) {
struct rlimit rlim;
getrlimit(RLIMIT_STACK,&rlim);
printf("rlim_cur=%lu/%lu\n",rlim.rlim_cur,rlim.rlim_cur / 1024);
rlim.rlim_cur = 32 * 1024 * 1024;
setrlimit(RLIMIT_STACK,&rlim);
getrlimit(RLIMIT_STACK,&rlim);
printf("rlim_cur=%lu/%lu\n",rlim.rlim_cur,rlim.rlim_cur / 1024);
}
printf("main: loop\n");
// dynamically grow the stack
may_grow = opt_d;
// top of stack
top = __builtin_frame_address(0);
if (! sigsetjmp(jbuf,1))
loop(NULL);
else
printf("main: stop\n");
return 0;
}
UPDATE:
This describes how to make "the" stack grow dynamically (very interesting!), but I take the question to be about the stacks of threads other than the initial one, or at least inclusive of those stacks. I don't think this answer addresses them. – 
John Bollinger
Although the question was tagged with pthreads, I'm not sure if OP actually was talking about subthreads doing this.
I'm not sure that this can be done for subthreads (created via pthread_create). At least not by using a default pthread_create call.
Without special pthread_attr_t values, the default for pthread_create is to malloc a stack [of a default size]. With attributes, the caller might set a larger stack size. And/or the caller will [usually] do an explicit malloc and pass the address along (with the size).
AFAICT, from reading glibc source, pthread_create will not set up a guard area if the user provides a stack pointer. Setting up the guard area [if there is to be one] is the responsibility of the caller.
But, in either case, if the stack overflows (with a guard area), a signal will be generated (SIGSEGV ?).
But, what can one do at that point???
The [pthread internal] function that calls the user's start_routine, the start_routine itself, and any functions that the start routine has called, already have pointers to things on that "old" stack [we must assume this].
So, the per-thread stack can not be moved (i.e. no realloc).
The only way to [possibly] do this is for the caller to provide an explicit stack pointer [and size]. As mentioned, the caller must set up the guard pages (via mprotect, I assume).
Although userfaultfd et. al. might be usable/preferable, I'm going to assume that the caller must use an explicit mmap call (vs. using the heap malloc/realloc).
The main thread's stack [generally] grows downward from the top of the virtual memory space. It can grow until all physical memory is used up and the paging disk is full.
However, for a per-thread (subthread) stack, the caller must decide on the maximum per-thread stack size before creating the thread. More on this below.
It can set up a stack (via mmap) of a smaller initial size. Once thread is created, the base/top stack address must remain constant.
Edit: The following is modified by additional thoughts below.
If a fault occurs, the signal handler could try an extension of the stack. There may be a few ways to do this. My best guess for this is:
It may have to copy/save the existing stack data [somewhere ;-)] if the remap below does not copy the existing data the way realloc does.
Temporarily undo any guard pages.
unmap/remap the stack at the same address but with a larger size (via mremap and/or mmap using MREMAP_FIXED/MAP_FIXED).
If necessary, copy back the stack data onto the "new" stack.
Set up new guard pages.
return from the signal handler [and hope ;-)].
As I said, subthread stacks can't grow "infinitely" as the main thread stack can (via setrlimit).
The sizes/addresses in the example below are not "to scale" ...
Consider two threads (e.g. tA and tB) that start with a small size:
tA's stack at xxxx1000 with size 1000
tB's stack at xxxx2000 with size 1000
If tB hits its guard page and faults, what happens? It has no room the extend its stack downward without colliding with the top of tA's stack.
So, we must map the stacks with sufficient space to grow to the "maximum" and we must know that beforehand.
We need to "space out" the stack addresses so they have room to grow [even if not all pages are mapped initially].
Let's assume that the "maximum" size is 10000. A mapping that would work is:
tA's stack at xxxx1000 with size 1000
tB's stack at xxx11000 with size 1000
Now, if tB hits its guard page, it can extend its stack up to the maximum of 10000
Additional thoughts ...
We probably must initially map the entire maximum region. Otherwise, other unrelated mmap calls may grab space in the proposed extension area. Either mmap from the heap manager, shared memory mappings, or even mmap calls done when setting up other threads.
So, we don't need to use mremap to ever increase the size of the region. Rather, we should use madvise with MADV_DONTNEED on the area that we're not currently using.
If we do this, we don't need a signal handler or guard pages to invoke it to extend the area. Just doing MADV_DONTNEED will keep resource usage low(er).
The thread function can (after popping the stack a bit) release the "popped" area with MADV_DONTNEED.
Other MADV_* options might be better.
All of the above strikes me as craziness! The only use would be a [hugely] multithreaded app that is doing deeply recursive functions.
But, if that's the case, it would be better to convert the recursive functions into ones that don't use actual program stack based recursion. Rather, the function manages an array of structs as a software controlled [pseudo] "stack". Each struct has all the variables that were function/automatic scope in the program stack frame.
The size of this array can be controlled with realloc. Or, we can implement the "stack" as a linked list of these structs with a "slab" allocation scheme. With the slab scheme, all threads could share the same slab allocator. This has the advantage that the amount of memory required can be smaller than if each thread had its own/private slab allocator.
So, IMO, don't do this on the real, per-thread program stack!
Okay, so that's what I've come up with. I've not written code for it or tested it. That's an exercise I think I'll leave to the reader ;-)

Questions on scull_follow function in linux device drivers 3rd edition

I never found the definition for scull_follow in the book, so I'm trying to understand it based off a github repo(https://github.com/martinezjavier/ldd3).
Here is the code I am trying to understand:
struct scull_qset *scull_follow(struct scull_dev *dev, int n) {
struct scull_qset *qs = dev->data;
/* Allocate first qset explicitly if need be */
if (!qs) { // if NULL
qs = dev->data = kmalloc(sizeof(struct scull_qset), GFP_KERNEL);
if (qs == NULL)
return NULL; /* Never mind */
memset(qs, 0, sizeof(struct scull_qset));
}
/* Then follow the list */
while (n--) {
if (!qs->next) {
qs->next = kmalloc(sizeof(struct scull_qset), GFP_KERNEL);
if (qs->next == NULL)
return NULL; /* Never mind */
memset(qs->next, 0, sizeof(struct scull_qset));
}
qs = qs->next;
continue;
}
return qs;
}
Here is struct scull_qset:
struct scull_qset {
void **data;
struct scull_qset *next;
};
Conceptually, I understand that all scull_follow does is that it follows the list up to the right position so that you know where to start reading/writing.
I'm mainly confused on this part of the code.
/* Allocate first qset explicitly if need be */
if (!qs) { // if NULL
qs = dev->data = kmalloc(sizeof(struct scull_qset), GFP_KERNEL);
if (qs == NULL)
return NULL; /* Never mind */
memset(qs, 0, sizeof(struct scull_qset));
}
Let's say the user opens this driver and attempts to read from it first without writing. that should mean that it should go into the if statement and allocate some memory. Then, why is there a second check for if qs is NULL or not? Won't it always be not NULL because it was allocated some memory?
After that what is the memset function for? I understand that it copies 0 into qs, but what is the point of that other than initializing this memory region? Is it so that when you call the copy_to_user function in your read function it'll know that because it is filled with 0s, nothing of 'value' has been written to it so you will just get a blank output when reading, assuming the first operation you do is reading?
Thanks for answering my questions.
This:
qs = dev->data = kmalloc(sizeof(struct scull_qset), GFP_KERNEL);
if (qs == NULL)
return NULL;
Is standard C programming good practice: whenever a function can fail, you always need to check the return value. This is true for any kind of function that can fail, not only malloc() and friends. In this case, kmalloc() can fail to allocate memory, returning NULL, so the code is checking for that error. If that happens, the function safely aborts execution by doing a return NULL;, and the caller will then handle that as needed.
This:
memset(qs, 0, sizeof(struct scull_qset));
Is standard kernel programming good practice: whenever you allocate uninitialized memory (like kmalloc() does), it could contain sensitive kernel data. You never want uninitialized data to reach userspace through a copy_to_user() or similar calls. In order to avoid this, you need to make sure to initialize it before making it available to userspace. Filling it with zeroes using memset() is one of the simplest way to do this.
In the case an user program does a read as the first syscall on the scull driver, it would just read a bunch of 0 bytes.

Making a Device Driver in Minix

I'm trying to create a character device driver on Minix. I would like it to be able to accept read() and write() calls. My understanding is that I would need to use sys_safecopyfrom() for the function which runs the read() function and sys_safecopyto() for the function which runs the write() function. The issue is that I keep getting a similar error (although not exactly the same, but I think that the differences are memory locations) when I run it like this. The error is:
verify_grant: grant verify failed: access invalid: want 0x..., have 0x...
grant 2 verify to copy ... -> ... by ... failed err -1
read: Operation not permitted
The "..." are memory locations and the error is similar for write except for the memory locations and it says "write" instead of "read" on the last line.
I think that the relevant code is the following:
#include <minix/drivers.h>
#include <minix/chardriver.h>
#include <stdio.h>
#include <stdlib.h>
#include <minix/ds.h>
...
static struct chardriver hello_tab =
{
.cdr_open = hello_open,
.cdr_close = hello_close,
.cdr_read = hello_read,
.cdr_write = hello_write,
};
...
static ssize_t hello_read(devminor_t UNUSED(minor), u64_t position,
endpoint_t endpt, cp_grant_id_t grant, size_t size, int UNUSED(flags),
cdev_id_t UNUSED(id))
{
u64_t dev_size;
char *ptr;
int ret;
char *buf = HELLO_MESSAGE;
printf("hello_read()\n");
/* This is the total size of our device. */
dev_size = (u64_t) strlen(buf);
/* Check for EOF, and possibly limit the read size. */
if (position >= dev_size) return 0; /* EOF */
if (position + size > dev_size)
size = (size_t)(dev_size - position); /* limit size */
/* Copy the requested part to the caller. */
ptr = buf + (size_t)position;
if ((ret = sys_safecopyfrom(endpt, grant, 0, (vir_bytes) ptr, size)) != OK)
return ret;
/* Return the number of bytes read. */
printf("Message is :%s", ptr);
return size;
}
static ssize_t hello_write(devminor_t UNUSED(minor), u64_t position,
endpoint_t endpt, cp_grant_id_t grant, size_t size, int UNUSED(flags),
cdev_id_t UNUSED(id))
{
u64_t dev_size;
char *ptr;
int ret;
char *buf = HELLO_MESSAGE;
printf("hello_write()\n");
/* This is the total size of our device. */
dev_size = (u64_t) strlen(buf);
/* Check for EOF, and possibly limit the read size. */
if (position >= dev_size) return 0; /* EOF */
if (position + size > dev_size)
size = (size_t)(dev_size - position); /* limit size */
/* Copy the requested part to the caller. */
ptr = buf + (size_t)position;
if ((ret = sys_safecopyto(endpt, grant, 0, (vir_bytes) ptr, size)) != OK)
return ret;
/* Return the number of bytes read. */
return size;
}
The hello_read function is based off of the hello_write functions but I think that it should still work and should read the information into ptr.
Also, I'm a bit hazy on how I would go about getting the second argument in the write() function (the buffer) in my hello_write() function. Is it contained in one of hello_read()'s arguments?
Thanks for your help!
So, I know it's been a long time and there's no activity here but I thought I would answer the question.
I am going to start by saying the the error occurs when passing the wrong arguments into sys_safecopyto/from.
Now to really debug this I would want to see the rest of the code you had. But for anyone else who comes across this problem I'm going to give some tips
look at how many bytes you are passing the the sys_safecopy funcitons
make sure you are putting the correct offset with the buffer when writing. For
the case I used it in that was (buffer_ptr + current_size)
make sure if you are using an earlier version of minix that you are putting in the correct amount of parameters into the sys_safecopy funcitons (could be 5 args or 6 args, the last one on older versions of minix for the hello driver would just be "D" ;) )

Why a child process fails to write to a shared memory?

I have a simple program that writes to a shared memory and reads from a shared memory, but while reading I am getting segmentation fault.
When I am debugging, the child process is not writing information to the shared memory, and after that, the parent process is trying to read from the shared memory which has no data and is throwing segmentation fault at 1st printf, in parent printf("%d\n",ptr->nread);
Why the child process is not able to write data to the shared memory? (it is failing at ptr->nread=20; line)
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
#define SIZE 5*1024
struct databuf{
int nread;
char *buf;
int xyz;
};
struct databuf* ptr;
main()
{
int shmid,pid;
shmid = shmget((key_t)1,SIZE,IPC_CREAT|0777);
pid = fork();
if(pid==0)
{
ptr = (struct databuf *)shmat(shmid,(char*)0,0);
ptr->nread=20;
ptr->buf=ptr+sizeof(ptr->nread);
strcpy(ptr->buf, "abc");
ptr->xyz=20;
}
else
{
wait(0);
ptr = (struct databuf *)shmat(shmid,(char*)0,0);
printf("%d\n",ptr->nread);
printf("%s\n",ptr->buf);
printf("%d\n",ptr->xyz);
}
return 0;
}
If ptr->nread is failing then you should put a error checking code something like this, before acessing the ptr.
ptr = (struct databuf *)shmat(shmid,(char*)0,0);
if (data == (struct databuf *)(-1)) {
perror("shmat failed");
exit(1);
}
ptr->nread=20;
Ref: http://linux.die.net/man/2/shmat
ptr->buf=ptr+sizeof(ptr->nread);
could be written as:
ptr->buf=(char*)ptr+sizeof(struct databuf)+ptr->nread;
or
ptr->buf=(char*)ptr+ptr->nread;
The string can now be accessed in the parent process.
Brief Explanation:
If you're using shared memory, you have to make sure all the data you want to access in other processes is in the shared memory segment. Leave the data at a specified offset(in your case ptr+ptr->nread) in the memory segment. And be careful not to overwrite the existing data in the shared memory. sizeof(ptr->nread) will yield the sizeof(int).
Leaving to one side all the other issues with the code, I think:
shmid = shmget((key_t)1, SIZE, IPC_CREAT|0777) ;
is probably a mistake, unless you can (somehow) guarantee that (key_t)1 is not in use elsewhere. For parent-child communication, as in this case, the simpler approach is:
shmid = shmget((IPC_PRIVATE, SIZE, IPC_CREAT|0777) ;
In general, when something apparently inexplicable happens, I find it helps to make sure I have checked for error returns. In this case: shmget() returning -1 or shmat() returning -1... and I suspect that both have happened.
Worked solution added by #WhozCraig:
The following example works, and likely does what you're trying to accomplish. Pay note to how we calculate the address we store in ptr->buf after we home ptr to our shared memory base address. We have to leave room for the structure, so we calculate the address to start the first byte past the structure back-side.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
struct databuf
{
int nread;
char *buf;
int xyz;
};
#define SIZE (5*1024)
int main()
{
// acquire shared memory first (read/write access)
int shmid = shmget(IPC_PRIVATE, SIZE, IPC_CREAT|0666);
if (shmid == -1)
{
perror("Failed to acquire shared emory.");
exit(EXIT_FAILURE);
}
// fork child process
pid_t pid = fork();
// both parent and child need this. may as well do both before
// special casing child vs. parent logic.
struct databuf *ptr = shmat(shmid,(char*)0,0);
if (ptr == (void*)(-1))
{
perror("Failed to map shared memory to our process");
exit(EXIT_FAILURE);
}
// child process
if (pid==0)
{
ptr->nread = 20;
ptr->buf = ((char*)ptr) + sizeof(*ptr);
strcpy(ptr->buf, "abc");
ptr->xyz = 30;
}
// parent process
else
{
wait(NULL);
printf("ptr = %p, ptr->buf = %p\n", ptr, ptr->buf);
printf("%d\n",ptr->nread);
printf("%s\n",ptr->buf);
printf("%d\n",ptr->xyz);
}
return 0;
}
Sample Output (varies by system obviously)
ptr = 0x80000, ptr->buf = 0x8000c
20
abc
30
it is worth noting that this:
ptr->buf = ((char*)ptr) + sizeof(*ptr);
could be written as the following, using typed-pointer math:
ptr->buf = (char*)(ptr + 1);
The location where the cast is applied is important. The first applies it before we do any math, so we need to account for octet count. The second applies it after the pointer math, so simply adding one to to the typed pointer ptr will move us to the memory immediately past our ptr structure base.
Best of luck.
If you have a pointer from type T, an addition by one to the pointer will increment it by sizeof(T).
So you have to replace:
ptr->buf=ptr+sizeof(ptr->nread);
to
ptr->buf= ((char*)ptr )+sizeof(ptr->nread);
If you don't do that, your pointer will be incremented by sizeof(T)^2.

LightWeight IP: Buffer not freeing

I'm using an TCP/IP stack called lwip. I have implemented a function below to send data packets, inspired from a similar callback function that receives data packets.
Each time a packet is received, I create a buffer using the pbuf_alloc function. Then, I send the packet using udp_sendto. Finally, I free the buffer using pbuf_free. (See the code below.)
For some reason, pbuf_free is not freeing the buffer. (I get a buffer overflow after n packets, where n is the pool size.) The lwip wiki warns that:
The network driver may also not assume that the pbuf memory is
actually freed when it calls pbuf_free.
How can I force pbuf_free to free my buffer? How is the buffer overflow avoided?
(My implementation below.)
static err_t IAP_tftp_send_data_packet(struct udp_pcb *upcb, struct ip_addr *to, int to_port, int block)
{
err_t err;
struct pbuf *pkt_buf;
char packet[TFTP_DATA_PKT_LEN_MAX];
int bytesRead;
int bytesToSend;
/* Specify that we are sending data. */
IAP_tftp_set_opcode(packet, TFTP_DATA);
/* Specify the block number that we are sending. */
IAP_tftp_set_block(packet, block);
bytesRead = IAP_tftp_set_data(packet, block);
if(bytesRead != 0) {
bytesToSend = TFTP_DATA_PKT_LEN_MAX - (512 - bytesRead + 1);
} else {
bytesToSend = TFTP_DATA_PKT_LEN_MAX - 512;
}
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);
if (!pkt_buf)
{
print("(TFTP) Buffer overflow!\r\n");
}
/* Copy the file data onto pkt_buf. */
memcpy(pkt_buf->payload, packet, bytesToSend);
err = udp_sendto(upcb, pkt_buf, to, to_port);
/* free the buffer pbuf */
printf("%d\n\r", pbuf_free(pkt_buf));
return err;
}
What version of lwIP are you using?
Depending on different versions the answers vary a lot.
The memp_malloc() allocation function called inside the pbuf_alloc() has failed or the pbufs chaining has failed.So, it returns NULL.
pbuf_alloc() will also return NULL, if the passed arguments also contains NULL.(due to NULL arguments check).
In newer versions, could you show what value the MEMP_OVERFLOW_CHECK macro contains? The lwIP shows a diferent behavior when the macro value >= 2.
And another cause might be if you are using multi-threading, the locking mechanisms inside the pbuf_alloc() fail, might cause it to return NULL.
Some versions require that you call pbuf_init(), before calling pbuf_alloc().
You can try this:
pkt_buf = NULL;//Use NULL, just incase the NULL is not 0 as per your compiler.
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_REF);
if(pkt_buf == NULL)
{
printf("pbuf_alloc failed.\n");
}
else
{
/* Do something with the allocated pbufs and free it. */
}
PBUF_REF will allocate no buffer memory for pbuf. The pbuf should be used in a single thread only and if the pbuf gets queued, then pbuf_take should be called to copy the buffer.
You can also try PBUF_RAM which will allocate buffer in RAM.
For more informtaion, you can also browse the source files of the version of lwIP, that you are using.
The easiest solution seems to be to make the buffer static, i.e. re-use the same buffer for each call:
static struct pbuf *pkt_buf = NULL;
if( pkt_buf == NULL )
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);
if( pkt_buf == NULL )
{
print("(TFTP) Buffer overflow!\r\n");
}
If your scenario involves unloading/reloading the driver, it will leak memory. To fix that, make the buffer static outside the IAP_tftp_send_data_packet() function, and call pbuf_free() when the driver unloads (assuming lwip tells you).
Just a passing thought, possibly completely nonsensical. In this code:
if(bytesRead != 0) {
bytesToSend = TFTP_DATA_PKT_LEN_MAX - (512 - bytesRead + 1);
} else {
bytesToSend = TFTP_DATA_PKT_LEN_MAX - 512;
}
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);
...is it possible for bytesRead to assume the value 513 - TFTP_DATA_PKT_LEN_MAX ?
If it happened, wouldn't the request to allocate zero bytes fail? (this could be tested by printing the value of bytesToSend upon buffer overflow, and checking if it is nonzero).
struct pbuf does not represent a continuous region of memory. It is rather a chain of memory locations. Thus this will not work in general case:
memcpy(pkt_buf->payload, packet, bytesToSend);
You need to scatter-copy your data. The memcpy() from the code snippet may overflow the payload buffer and cause all kinds of side effects including inability to free the pbuf chain cleanly.

Resources