Fail to allocate semaphore/netconn Tiva C TM4C1294 - c

I am using tm4c1294+lwip1.4.1+FreeRTOS.
As netconn_alloc() is called for socket communication, it allocates an unused semaphore. the number of semaphore is defined as SYS_SEM_MAX, so it can not be over SYS_SEM_MAX.
However, as semaphores are allocated continuously it reaches SYS_SEM_MAX and stop working since I guess sys_sem_free() does not deallocate it properly
Here is function that creates a semaphore implemented in sys_arch.c
err_t
sys_sem_new(sys_sem_t *sem, u8_t count)
{
void *temp;
u32_t i;
/* Find a semaphore that is not in use. */
for(i = 0; i < SYS_SEM_MAX; i++) {
if(sems[i].queue == 0) {
break;
}
}
if(i == SYS_SEM_MAX) {
#if SYS_STATS
STATS_INC(sys.sem.err);
#endif /* SYS_STATS */
return ERR_MEM;
}
/* Create a single-entry queue to act as a semaphore. */
#if RTOS_FREERTOS
sem->queue = xQueueCreate(1, sizeof(void *));
if(sem->queue == NULL) {
#endif /* RTOS_FREERTOS */
#if SYS_STATS
STATS_INC(sys.sem.err);
#endif /* SYS_STATS */
return ERR_MEM;
}
/* Acquired the semaphore if necessary. */
if(count == 0) {
temp = 0;
xQueueSend(sem->queue, &temp, 0);
}
/* Update the semaphore statistics. */
#if SYS_STATS
STATS_INC(sys.sem.used);
#if LWIP_STATS
if(lwip_stats.sys.sem.max < lwip_stats.sys.sem.used) {
lwip_stats.sys.sem.max = lwip_stats.sys.sem.used;
}
#endif
#endif /* SYS_STATS */
/* Save the queue handle. */
sems[i].queue = sem->queue;
/* Return this semaphore. */
return (ERR_OK);
}
Here is another function that frees semaphore implemented in sys_arch.c
void
sys_sem_free(sys_sem_t *sem)
{
/* Delete Sem , By Jin */
vQueueDelete(sem->queue);
/* Clear the queue handle. */
sem->queue = 0;
/* Update the semaphore statistics. */
#if SYS_STATS
STATS_DEC(sys.sem.used);
#endif /* SYS_STATS */
}
Whenever netconn_free() is called sys_sem_free() deallocates the semaphore, but it does not free the semaphore assigned in sem[] array.
I added vQueueDelete(sem->queue); that was suggested by someone, but still all same.
Not only functions creates/frees semaphore but also functions handling mbox are same as functions above, so functions handling mbox could be wrong as well.
Someone already reported this issue to TI, but it seems they have not solved the problems yet.
Therefore, I may need to implement my own functions handling semaphore/mbox in sys_arch.c, but I don't have any clues so far.
Can anyone give me any ideas? or anything?
Thanks,
Jin

The sys_arch.txt file in /doc is somewhat helpful. Apparently, looking at that document, and at what lwip 1.3.2 used to do, it looks like the ports/tiva-tm4c129/sys_arch.c is incorrect and incomplete.
sys_sem_free() should indeed be doing the vQueueDelete() as you discovered. It should not be doing "sem->queue = 0". If you look at netconn_free() over in src/api/api_msg.c, you can see it calls sys_sem_free() and then sys_sem_set_invalid(). The queue handle will be needed in the second function, and should not be clobbered in the first.
sys_sem_set_invalid() should make a sweep over the sems[] array and if it finds a match to the sem->queue, it should zero out that copy in sems[]. Once that is done, it should set sem->queue to 0.
This, I think, best matches what's in Dunkel's sys_arch.txt document, and fixed the resource leak on my system.
I concur that the mailboxes are in the same shape. Fwiw, I went ahead and modified those in similar fashion to that just described for the sems.
btw, the lwip files I was working with were from the TivaWare_C_Series 2.1.0.12573 third_party folder.

Related

How can I Execute/Call a user-space defined function from Linux kernel space module?

I am developing a Linux module which I want to use to run my C program from kernel mode.
My problem here, in function read() of the module, I need to use a function named eval_keycode(), which is defined in my user space program.
When I try to compile my module, this error occurs :
error: implicit declaration of function ‘eval_keycode’
which is confirming my problem described above.
This is the read() function of my module :
ssize_t exer_read(struct file *pfile, char __user *buffer, size_t length, loff_t *offset) {
struct file *f = pfile->private_data;
enum { MAX_BUF_SIZE = 4096 };
size_t buf_size = 0;
char *buf = NULL;
ssize_t total = 0;
ssize_t rc = 0;
struct input_event *ev;
int yalv;
/* Allocate temporary buffer. */
if (length) {
buf_size = min_t(size_t, MAX_BUF_SIZE, length);
ev = kmalloc(buf_size, GFP_KERNEL);
if (ev == NULL) {
return -ENOMEM;
}
}
/* Read file to buffer in chunks. */
do {
size_t amount = min_t(size_t, length, buf_size);
rc = kernel_read(f, ev, amount, offset);
if (rc > 0) {
/* Have read some data from file. */
if (copy_to_user(buffer, ev, rc) != 0) {
/* Bad user memory! */
rc = -EFAULT;
} else {
/* Update totals. */
total += rc;
buffer += rc;
*offset += rc;
length -= rc;
for (yalv = 0; yalv < (int) (rc / sizeof(struct input_event)); yalv++) {
if (ev[yalv].type == EV_KEY) {
if (ev[yalv].value == 0)
eval_keycode(ev[yalv].code);
}
}
if (rc < amount) {
/* Didn't read the full amount, so terminate early. */
rc = 0;
}
}
}
}
while (rc > 0 && length > 0);
/* Free temporary buffer. */
kfree(buf);
if (total > 0) {
return total;
}
return rc;
}
This is my user space eval_keycode() defined function :
void eval_keycode(int code)
{
static int red_state = 0;
static int green_state = 0;
switch (code) {
case 260:
printf("BTN left pressed\n");
/* figure out red state */
red_state = red_state ? 0 : 1;
change_led_state(LED_PATH "/" red "/brightness", red_state);
break;
case BTN_RIGHT:
printf("BTN right pressed\n");
/* figure out green state */
green_state = green_state ? 0 : 1;
change_led_state(LED_PATH "/" green "/brightness", green_state);
break;
}
}
How can call the eval_keycode function from user space in order to solve this problem ?
Thank you.
You can, but it is a really bad idea. You need to establish a pointer to your user mode function, arrange for the process containing that function to be running (in the kernel) when you invoke it. That is a lot of work, and is fundamentally malware due to the security holes it creates. Additionally, in the mad dash to lock the door to the now empty barn in the wake of spectre et al, new layers of hackery are being deployed in newer CPUs to make this even harder.
A different approach:
In your original query, you are running this driver as a "tee"; that is, you take the input you receive from the device, give a copy to the caller, and call eval_keycode with each input. Eval_keycode doesn't modify the data, and the kernel module discards it afterwards. So Eval_keycode doesn't really need to be a function; or rather, there could be a user function:
void ProcessEvents(int fd) {
struct input_event ev;
while (read(fd, &ev, sizeof ev) == sizeof ev) {
eval_keycode(&ev);
}
}
if you could arrange for all the events to be fed into that fd. With this setup, your problem becomes more plumbing than kernel renovation. The user creates a pipe/socket/fifo/... and passes the write end to your kernel module (yay more ioctl()s). Your kernel module can then carefully use kernel_write() ( or vfs_write if you are stuck in the past ) to make these events available to the user handler. It wants to be careful about where its blocking points are.
You could extend this to work as a transform; that is where your driver transforms the events via a user mode handler; but at that point, you might really consider FUSE a better solution.
There is no traditional (in the way a library works) way to "call" a user space "function".
Your user space code should be running in its' own process (or another user space process), in which you would implement communications (through shared memory, interprocess calls [IPC], device files, interrupts..) where you handle the exchange of data, and act on the data (e.g. calling your eval_keycode function).
You basically want an upcall. You can find some explanation about that here, but it doesn't seem like Linux has an official upcall API.
However, as others have already mentioned, this isn't very good design. Upcalls are useful to servers implemented in the kernel.
If your exer_read() is only called for your own code (on your files for which you're implementing the driver), then perhaps inotify would be a better design.
If your exer_read() can be called for any file (e.g. you want any file write on the machine to change the LED state), then you want your userspace process containing eval_keycode() to poll some character device, and you want your module to write the code to this character device instead of calling eval_keycode().
If, however, change_led_state() is synchronous, and you actually need the read to block until it returns, then you are advised to reconsider your design... but that's a valid use case for upcalls.

Linux Kernel Where to lock and unlock semaphores?

Within the Linux Kernel (specifically for device drivers), how would I know what variables to lock and when they need locking? In particular, why does the locking in the following code only happen after dev has been set, even though dev points to a global variable scull_devices?
struct scull_qset {
void **data; /* pointer to an array of pointers which each point to a quantum buffer */
struct scull_qset *next;
};
struct scull_dev {
struct scull_qset *data; /* Pointer to first quantum set */
int quantum; /* the current quantum size */
int qset; /* the current array size */
unsigned long size; /* amount of data stored here */
unsigned int access_key; /* used by sculluid and scullpriv */
struct semaphore sem; /* mutual exclusion semaphore */
struct cdev cdev; /* Char device structure initialized in scull_init_module */
};
struct scull_dev *scull_devices; /* allocated dynamically in scull_init_module */
int scull_open(struct inode *inode, struct file *filp)
{
struct scull_dev *dev; /* device information */
dev = container_of(inode->i_cdev, struct scull_dev, cdev);
filp->private_data = dev; /* for other methods */
/* now trim to 0 the length of the device if open was write-only */
if ( (filp->f_flags & O_ACCMODE) == O_WRONLY) {
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
scull_trim(dev); /* empty out the scull device */
up(&dev->sem);
}
return 0; /* success */
}
If the code for scull_init_module is needed for a more complete picture, here it is:
int scull_major = SCULL_MAJOR;
int scull_minor = 0;
int scull_quantum = SCULL_QUANTUM;
int scull_qset = SCULL_QSET;
int scull_nr_devs = SCULL_NR_DEVS;
int scull_init_module(void)
{
int result, i;
dev_t dev = 0;
/* assigns major and minor numbers (left out for brevity sake) */
/*
* allocate the devices -- we can't have them static, as the number
* can be specified at load time
*/
scull_devices = kmalloc(scull_nr_devs * sizeof(struct scull_dev), GFP_KERNEL);
if (!scull_devices) {
result = -ENOMEM;
goto fail;
}
memset(scull_devices, 0, scull_nr_devs * sizeof(struct scull_dev));
/* Initialize each device. */
for (i = 0; i < scull_nr_devs; i++) {
scull_devices[i].quantum = scull_quantum;
scull_devices[i].qset = scull_qset;
init_MUTEX(&scull_devices[i].sem);
scull_setup_cdev(&scull_devices[i], i);
}
/* some other stuff left out for brevity sake */
return 0; /* succeed */
fail: /* isn't this a little redundant? */
scull_cleanup_module();
return result;
}
/*
* Set up the char_dev structure for this device.
*/
static void scull_setup_cdev(struct scull_dev *dev, int index)
{
int err, devno = MKDEV(scull_major, scull_minor + index);
cdev_init(&dev->cdev, &scull_fops);
dev->cdev.owner = THIS_MODULE;
dev->cdev.ops = &scull_fops;
err = cdev_add (&dev->cdev, devno, 1);
/* Fail gracefully if need be */
if (err)
printk(KERN_NOTICE "Error %d adding scull%d", err, index);
}
Locking in the example has nothing to do with the global scull_devices variable, but the locking is used to protect attributes of one scull_dev.
E.g. assume there exists a read() operation which copies size bytes from data while the mentioned scroll_trim() operation frees data.
So, when process #1 calls open() and process #2 tries to read() from an already opened device at the same time, the read() operation can access freed data and oopses.
That is why you need to protect data against races. Semaphores are one way; mutexes another one which is often more appropriate. Spinlocks and atomic variables might work too.
lock - it is way to protect critical section
critical section - in your driver code, if multiple instances are accessing same area, that is critical section.
multiple instances - it could be thread, regular ioctl cmd(from user space), and softirq and irq. It depends on your driver implementation.
Based on "context", you should use different lock too.
thread context which can sleep -> semaphore/mutex
non-sleeping context -> spinlock
softirq, tasklet -> spin_lock_bh
irq -> spin_lock_irq, spin_lock_irqsave
It is completely based on your requirements.
Let's take an example. If you are working on network driver, your netdev has stats and packet buffer and those needs to be protected by lock since it can be updated by multiple instances like net_rx_softirq, net_tx_softirq, ioctl/netlink from userspace request an so on.
In this case, based on your resource's context, you need to use different lock/mutex and sometimes you need more than 1 lock.

Multiply defined error when using fprintf(), while modifying the kernel of minix 3.1.6

I am modifying kernel of minix 3.1.6 as part of my undergraduate OS project, and I intend to first analyze the scheduling algorithm previously implemented with it by
storing scheduling related info for each process in some file, which can be analyzed later.
While modifying proc.c, I am getting Multiply defined error when using fprintf() to store data in some file, at the same time fopen() is working fine.
I am very new to all this, so any help will be appreciated.
Thanks in advance...........
here is the piece of code ..
PRIVATE void sched(rp, queue, front)
register struct proc *rp; /* process to be scheduled */
int *queue; /* return: queue to use */
int *front; /* return: front or back */
{
/* This function determines the scheduling policy. It is called whenever a
* process must be added to one of the scheduling queues to decide where to
* insert it. As a side-effect the process' priority may be updated.
*/
int time_left = (rp->p_ticks_left > 0); /* quantum fully consumed */
/* Check whether the process has time left. Otherwise give a new quantum
* and lower the process' priority, unless the process already is in the
* lowest queue.
*/
if (! time_left) { /* quantum consumed ? */
rp->p_ticks_left = rp->p_quantum_size; /* give new quantum */
if (rp->p_priority < (NR_SCHED_QUEUES-1)) {
rp->p_priority += 1; /* lower priority */
}
}
FILE *fp = fopen("/usr/newlog.txt","a");
/* If there is time left, the process is added to the front of its queue,
* so that it can immediately run. The queue to use simply is always the
* process' current priority.*/
fprintf(fp,"{\nprocess name: %s\nprocess quantum: %d\ntime left: %d\nprocess queue: %d\n}\n\n",
rp->p_name,rp->p_quantum_size,rp->p_ticks_left,rp->p_priority);
*queue = rp->p_priority;
*front = time_left;
fclose(fp);
}

Thread synchronization issues of APUE

I'm reading APUE and I am confused with thread synchronization of chapter 11. Below is a code snippet.
#define NHASH 29
#define HASH(fp) (((unsigned long)fp)%NHASH)
struct foo *fh[NHASH];
pthread_mutex_t hashlock = PTHREAD_MUTEX_INITIALIZER;
struct foo {
int f_count;
pthread_mutex_t f_lock;
struct foo *f_next; /* protected by hashlock */
int f_id;
/* ... more stuff here ... */
};
struct foo *
foo_alloc(void) /* allocate the object */
{
struct foo *fp;
int idx;
if ((fp = malloc(sizeof(struct foo))) != NULL) {
fp->f_count = 1;
if (pthread_mutex_init(&fp->f_lock, NULL) != 0) {
free(fp);
return(NULL);
}
idx = HASH(fp);
pthread_mutex_lock(&hashlock);
fp->f_next = fh[idx];
fh[idx] = fp;
pthread_mutex_lock(&fp->f_lock);
pthread_mutex_unlock(&hashlock);
/* ... continue initialization ... */
pthread_mutex_unlock(&fp->f_lock);
}
return(fp);
}
My doubts are:
Why place pthread_mutex_lock(&fp->f_lock) before the pthread_mutex_unlock(&hashlock)? Could I place it afterward instead?
Since fp is local variable, could pthread_mutex_lock(&fp->f_lock) and pthread_mutex_unlock(&fp->f_lock) be removed all together?
No, because the actions after the pthread_mutex_lock(&hashlock) expose the newly created structure to other threads by adding it to the the fh list. While the hashlock is held, no-one can access the variable; as soon as the hashlock is released, it becomes accessible to other threads via the hash, but locking the fp_>f_lock mutex prevents anyone messing with fp.
Not with the code as written. If the whole structure was initialized except for the hashing, then you could do without locking the fp->f_lock mutex; just before completing, you'd lock the hashlock, hook the newly allocated item into the hash tables, and then release the hashlock, and you would be safe. If you need any exclusive access after the structure is added into the hash table, you have to acquire its mutex. The way it is written, that's a non-waiting acquisition of the mutex; there is no other process that could have access to the variable.
if ((fp = malloc(sizeof(struct foo))) != NULL) {
fp->f_count = 1;
if (pthread_mutex_init(&fp->f_lock, NULL) != 0) {
free(fp);
return(NULL);
}
idx = HASH(fp);
/* ... complete initialization except for adding to hash table ... */
pthread_mutex_lock(&hashlock);
fp->f_next = fh[idx];
fh[idx] = fp;
pthread_mutex_unlock(&hashlock);
}
So, there is logic behind what is done; it is correct.
I guess that there's a second thread that's looping through the created objects, doing something with them. In this case:
No, because the loop thread might access the newly created object before being initialized at all.
No, because the loop thread might access the newly created object being half initialized.

OS X: Generate core dump without bringing down the process?

I know how to generate a core dump on OS X when a process crashes, but what I really need to do is attach to a process, generate a core dump, then resume that process (without killing it).
A long time ago (maybe a year and a half ago) I had C code that would do this... It used the OS X kernel libraries to connect to a process, read all of its thread states and memory, and write that into a Mach-O file on disk. This worked great (and it's exactly what I'm looking for), but now I can't seem to find that code for the life of me. I seem to recall that code was related somewhat to the OS X system internals book, but that's just a vague recollection.
Does anyone know the code I'm talking about and could point me at it? If not does anyone know a good way of doing this preferably with some example code?
Edit: Here is the answer.
Information: http://osxbook.com/book/bonus/chapter8/core/
Program that will do it for you: http://osxbook.com/book/bonus/chapter8/core/download/gcore-1.3.tar.gz
I believe you are looking for this information
Specifically:
/* UNIX Third Edition, circa early 1973 */
/* ken/sig.c */
core()
{
int s, *ip;
extern schar;
/* u is the user area */
u.u_error = 0; /* reset error code to "no error" */
u.u_dirp = "core"; /* file name to search for */
ip = namei(&schar, 1); /* do search; schar means it's a kernel string */
if (ip == NULL) { /* failed to find */
if (u.u_error) /* because of some error */
return(0); /* so bail out */
ip = maknode(0666); /* didn't exist; so create it */
}
if (!access(ip, IWRITE)) { /* check "write" permission; 0 means OK */
itrunc(ip); /* truncate the core file */
/* first we write the user area */
u.u_offset[0] = 0; /* offset for I/O */
u.u_offset[1] = 0; /* offset for I/O */
u.u_base = &u; /* base address for I/O (user area itself) */
u.u_count = USIZE*64; /* bytes remaining for I/O; USIZE=8 */
u.u_segflg = 1; /* specify kernel address space */
writei(ip); /* do the write */
/*
* u_procp points to the process structure
* p_size is the size of the process's swappable image (x 64 bytes) */
*/
s = u.u_procp->p_size - USIZE; /* compute size left to write */
/*
* This sets up software prototype segmentation registers to implement
* text(=0 here), data(=s here), and stack(=0 here) sizes specified.
*/
estabur(0, s, 0);
u.u_base = 0; /* base address for I/O (start of space) */
u.u_count = s*64; /* s is in units of 64 bytes, so adjust */
u.u_segflg = 0; /* specify user address space */
writei(ip); /* do the write */
}
iput(ip); /* decrement inode reference count */
return(u.u_error==0); /* done */
}

Resources