What should I be using if (on a multicore system) I wanted to make a kernel module function be ran only by one core at a time? Said in other words, avoid two cores running the same function at the same time; aka, one of the cores should wait for the other one to finish running the function.
Mutex? Spinlock? Something else?
You need to use variants of spinlock() i.e raw_spin_lock_irqsave(), raw_spin_lock_irqrestore() (https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/linux/spinlock.h#n188), not mutex() cause they're sleep-able, so they might wake up on some other CPUs. And spinlock will make sure your code won't be executed by other cores. It's been very well documented in Linux kernel tree at Documentation/spinlock.txt (https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/spinlocks.txt).
Since rakib has already suggested using spinlocks, I'm going to explain a bit more about how to use them.
After setting up the spinlock, e.g.
static DEFINE_SPINLOCK(your_lock);
You can simply wrap the contents of your function with spin_lock_irqsave/spin_lock_irqrestore, e.g.:
static void function_to_protect()
{
unsigned int flags;
spin_lock_irqsave(&your_lock, flags);
/* function body here */
spin_lock_ireqrestore(&your_lock, flags);
}
If you know for sure that your lock won't be touched by an interrupt handler, you can elect to use the lighter weight functions spin_lock and spin_unlock and omit the flags variable.
Reference: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/spinlocks.txt
Related
We are using Colibri iMX7D for our product development. Our application is running on embedded Linux with GTK+3.
In our application, there will be a thread running always to get data from ADC and the data will not be processed until I get a interrupt from GPIO. So, to implement this, I have a global boolean variable. In the interrupt thread (libsoc), I will be setting the boolean to be true. In the ADC thread, I will be processing the data if that variable is true. When another GPIO interrupt occurs, I will set the variable to be false in the interrupt thread.
After lot of googling, I found it is not good to use a shared variable inside multiple threads and found concepts like mutex and semaphores. It is a little bit confusing as both looks similiar and I am not sure about what to use for my application.Suggestions or directions to implement this are welcome.
In addition to the problem of data consistency during simultaneous read and write access, your architecture as presented has a major disadvantage in terms of performance. Indeed, the fact of using a boolean variable, this forces you to probably loop in your ADC thread in something that looks like this:
while ( bDataPresent == false ) ...
You should know that in this case, your ADC thread is in permanent execution, therefore, it consumes resources: CPU, memory, ...
This is not the case when you use semaphore (or other synchronization mechanism) because when the thread is blocked on a semaphore call, it does not consume as much resource.
I'm looking to create a state of uninterruptible sleep for a program I'm writing. Any tips or ideas about how to create this state would be helpful.
So far I've looked into the wait_event() function defined in wait.h, but was having little luck implementing it. When trying to initialize my wait queue the compiler complained
warning: parameter names (without types) in function declaration
static DECLARE_WAIT_QUEUE_HEAD(wq);
Has anyone had any experience with the wait_event() function or creating an uninterruptible sleep?
The functions that you're looking at in include/linux/wait.h are internal to the Linux kernel. They are not available to userspace.
Generally speaking, uninterruptible sleep states are considered undesirable. Under normal circumstances, they cannot be triggered by user applications except by accident (e.g, by attempting to read from a storage device that is not responding correctly, or by causing the system to swap).
You can make sleep 'signal-aware`.
sleep can be interrupted by signal. In which case the pause would be stopped and sleep would return with amount of time still left. The application can choose to handle the signal notified and if needed resume sleep for the time left.
Actually, you should use synchronization objects provided by the operating system you're working on or simply check the return value of sleep function. If it returns to a value bigger than zero, it means your procedure was interrupted. According to this return value, call sleep function again by passing the delta (T-returnVal) as argument (probably in a loop, in case of possible interrupts that might occur again in that time interval)
On the other hand, if you really want a real-uninterruptible custom sleep function, I may suggest something like the following:
void uninterruptible_sleep(long time, long factor)
{
long i, j;
__asm__("cli"); // close interrupts
for(i=0; i<time; ++i)
for(j=0; j<factor; ++j)
; // custom timer loop
__asm__("sti"); // open interrupts
}
cli and sti are x86 assembly instructions which allow us to set IF (interrupt flag) of the cpu. In this way, it is possible to clear (cli) or set (sti) all the interrupts. However, if you're working on a multi-processor system, there needs to be taken another synchronization precautions too, due to the fact that these instructions will only be valid for single microprocessor. Moreover, this type of function as I suggested above, will be very system (cpu) dependant. Because, the inner loop requires a clock-cycle count to measure an exact time interval (execution number of instructions per second) depending on the cpu frequency. Thus, if you really want to get rid of every possible interrupt, you may use a function as I suggested above. But be careful, if your program gets a deadlock situation while it's in cli state, you will need to restart your system.
(The inline assembly syntax I have written is for gcc compiler)
I would like to be able to 'capture' an hrtimer interrupt with a linux kernel module and replay the interrupt at a later period in time. Any thoughts on how to go about doing this?
Use case: A program that calls sleep(1). My module will grab the hrtimer interrupt when it fires after 1 second, wait for 'x' amount of time, then re-fire the interrupt, waking the process.
Note: I do not want to hook the sleep system call.
Thanks!
Quite honestly, writing a Linux kernel module just to modify the behavior of sleep() for a single application sounds like overkill.
For most cases you should be able to use a preloadable shared object to intercept/override the sleep() function family with your own implementations. The application will call your implementation and your code may then call the real function with a modified parameter list.
This method is much simpler and less intrusive than anything involving kernel programming, although it will not work if your application is statically linked or if it uses direct system calls instead of library functions.
I am trying to implement my own new schedule(). I want to debug my code.
Can I use printk function in sched.c?
I used printk but it doesn't work. What did I miss?
Do you know how often schedule() is called? It's probably called faster than your computer can flush the print buffer to the log. I would suggest using another method of debugging. For instance running your kernel in QEMU and using remote GDB by loading the kernel.syms file as a symbol table and setting a breakpoint. Other virtualization software offers similar features. Or do it the manual way and walk through your code. Using printk in interrupt handlers is typically a bad idea (unless you're about to panic or stall).
If the error you are seeing doesn't happen often think of using BUG() or BUG_ON(cond) instead. These do conditional error messages and shouldn't happen as often as a non-conditional printk
Editing the schedule() function itself is typically a bad idea (unless you want to support multiple run queue's etc...). It's much better and easier to instead modify a scheduler class. Look at the code of the CFS scheduler to do this. If you want to accomplish something else I can give better advice.
It's not safe to call printk while holding the runqueue lock. A special function printk_sched was introduced in order to have a mechanism to use printk when holding the runqueue lock (https://lkml.org/lkml/2012/3/13/13). Unfortunatly it can just print one message within a tick (and there cannot be more than one tick when holding the run queue lock because interrupts are disabled). This is because an internal buffer is used to save the message.
You can either use lttng2 for logging to user space or patch the implementation of printk_sched to use a statically allocated pool of buffers that can be used within a tick.
Try trace_printk().
printk() has too much of an overhead and schedule() gets called again before previous printk() calls finish. This creates a live lock.
Here is a good article about it: https://lwn.net/Articles/365835/
It depends, basically it should be work fine.
try to use dmesg in shell to trace your printk if it is not there you apparently didn't invoked it.
2396 if (p->mm && printk_ratelimit()) {
2397 printk(KERN_INFO "process %d (%s) no longer affine to cpu%d\n",
2398 task_pid_nr(p), p->comm, cpu);
2399 }
2400
2401 return dest_cpu;
2402 }
there is a sections in sched.c that printk doesn't work e.g.
1660 static int double_lock_balance(struct rq *this_rq, struct rq *busiest)
1661 {
1662 if (unlikely(!irqs_disabled())) {
1663 /* printk() doesn't work good under rq->lock */
1664 raw_spin_unlock(&this_rq->lock);
1665 BUG_ON(1);
1666 }
1667
1668 return _double_lock_balance(this_rq, busiest);
1669 }
EDIT
you may try to printk once in 1000 times instead of each time.
I'm looking for advice on how to best implement thread-safe IO (e.g. for printf going to a debug serial port) in an environment in which the operating system scheduler may yet to have started, be running, suspended or may have stopped or crashed. I'm using Newlib and FreeRTOS.
At the moment, I'm doing the (seemingly-standard) FreeRTOS approach of having the _write system call (Newlib) queue characters into a FreeRTOS queue which is then emptied from an interrupt service routine (filling the serial port hardware FIFO then waiting for the FIFO empty interrupt).
This has the disadvantage that (at least on FreeRTOS) the queue can only be used safely when the scheduler is running, and debug output can not be printed when interrupts are disabled (as they are during boot until the scheduler starts, or after a fatal error condition (exactly where debug printf output would be most useful :-).
Would it be best to have the _write system call query the scheduler and/or interrupt status, and use queues if the scheduler is running and use blocking/polling serial IO when interrupts are disabled? Is there a more elegant idea I haven't thought of yet?
Thanks
Perhaps slightly more elegant would be to use indirection in the _write system call. Instead of checking the status, use a function pointer to achieve the desired functionality based on whether the scheduler is running, interrupts enabled, etc. Then set the appropriate callback function when the program state changes.
int (*writeCallback)(int, const void *, unsigned int);
int
_write(int fd, const void *buffer, unsigned int count)
{
return writeCallback(fd, buffer, count);
}
...
writeCallback = polling_write;
...
writeCallback = rtos_write;
...
Alternatively, you define a different debug function that bypasses the standard library stream system (i.e. printf) to use in places that you know will not have interrupt/RTOS facilities (e.g. exception handlers).