This question is related to linux scheduling and processor activity with different phases in a thread which may be doing something actively, or idle/waiting or may be sleeping. When a thread is actively doing something, the processor will be executing instruction (and processor throughput i.e. no of instruction it is executing per sec) will be high. My questing is how does the processor behave (may be roughly) when a thread is waiting vs when it is sleeping? Can a waiting thread be sleeping? When a thread is sleeping, does it mean the processor is idle? When a processor is idle, does it mean all the thread are sleeping? When a processor is idle, does Linux literally put the processor in a mode such that it does not execute any instructions (i.e. clock gated)?
When a thread is sleeping it is actually waiting for the OS to put it into the execution queue. Sleeping occurs when a thread asks the OS for interrupting the thread itself for some amount of time. It is normally achieved by using functions as sleep(). The OS removes the thread from the execution queue for the requested time and when it's over, it continues running the thread. Other I/O functions act in a similar way: if a thread calls read(), the OS will remove the thread from the execution queue until there's data in the resource being read.
On the other hand, a processor is idle when it has no code to run nor any hardware events to attend. The idle state is a hardware state: the CPU is not running code, just waiting for some event to occur (i.e.: scheduling timer or I/O interrupt) that signals the processor to run code. A 8086 processor can achieve the idle state by executing the HLT (halt) instruction. After running HLT the processor stops running code until a hardware interrupt (like the ones listed above) is received.
Related
im wondering whether I understand the concept of a RTOS, and more specifically the scheduling process, correctly.
So, I think I understand the process of a timer interrupt (i omitted the interrupt enable/disable commands for better readability here)
1. program runs...
2. A timer tick occurs that triggers a Timer Interrupt
3. The Timer ISR is called
The timer ISR looks like this:
3.1. Kernel saves context (registers etc.)
3.2. Kernel checks if there is a higher priority task
3.3. If so, the Kernel performs the context switch
3.4. Return from Interrupt
4. Program runs with another task executing
But how does the process looks like, when an Interrupt occurs from lets say a I/O Pin?
1. program runs
2. an interrupt is triggered because data is available
3. a general ISR is called?
3.1. Kernel saves context
3.2. Kernel have to call the User defined ISR, because the Kernel doesn't know what to do now
3.1.1 User ISR runs and does whatever it should do (maybe change priority of a task, that should run now, because the data is now available)
3.1.2 return from User ISR
3.3. Kernel checks if there is a higher priority task available
3.4. If so the Kernel performs a context switch
3.5. Return from Interrupt
4. program runs with the different task
In this case the kernel must implement a general ISR, so that all interrupts are mapped to this ISR. For example (as far as i know) the ATmega168p microcontroller has 26 interrupt vectors. So there should be a processor specific file, that maps all the Interrupts to a general ISR. The Kernel-ISR determines what caused the interrupt and calls the specific User-ISR (that handles the actual interrupt).
Did I misunderstood something?
Thank you for your help
There is a clear distinction between the OS tick interrupt and the OS scheduler - you have however conflated the two. When the OS tick ISR occurs, the tick count is incremented, if that increment causes a timer or delay expiry, that is a scheduling event, and scheduling events causes the scheduler to run on exit from the interrupt context.
Different RTOS may have subtle differences, but in general in any ISR, if a scheduling event occurred, the scheduler runs immediately before exiting the interrupt context, setting up the threading context for whatever thread is due to run by the scheduling policy (normally highest priority ready thread).
Scheduling events include:
OS timer expiry
Task delay expiry
Timeslice expiry (for round-robin scheduling).
Semaphore give
Message queue post
Task event flag set
These last three can occur in any ISR (so long as they are "try semantics" non-blocking/zero timeout), the first three as a result of the tick ISR. So the scheduler will run on exit from the interrupt context when any interrupt has caused at least one scheduling event (there may have been nested or multiple simultaneous interrupts).
Scheduling events may occur in the task context also including on any potentially blocking action such as:
Semaphore give
Semaphore take
Message queue receive
Message queue post
Task event flag set
Task event flag wait
Task delay start
Timer wait
Explicit "yield"
The scheduler runs also when a thread triggers a scheduling event, so context switches do not only occur as the result of an interrupt.
To summarise and with respect to your question specifically; the tick or any other interrupt does not directly cause the scheduler to run. An interrupt, any interrupt can perform an action that makes the scheduler due to run. Unlike the thread context where such an action causes the scheduler to run immediately, in the interrupt context, the scheduler is deferred until all pending interrupts have been serviced and runs on exit from the interrupt context.
For details of a specific RTOS implementation of context switching see ยงยง3.05, 3.06 and 3.10 of MicroC/OS-II: The Real Time Kernel (the kernel and the book were specifically developed to teach such principles, so it is a useful resource and the principles apply to other RTOS kernels). In particular Listings 3.18 to 3.20 and Figure 3.10 and the associated explanation.
I saw this piece of code on disk read in Linux 0.11 kernel:
static inline void lock_buffer(struct buffer_head * bh)
{
cli();
while (bh->b_lock)
sleep_on(&bh->b_wait);
bh->b_lock=1;
sti();
}
IIUC, cli() will block the interrupt (not blocking all as explained here: https://c9x.me/x86/html/file_module_x86_id_31.html, but still, block some interrupts which means it changes the default behavior).
And sleep_on will call schedule, which will pass the control flow to another process.
However, what makes me confused is that here we will switch to another process with some of the interrupts blocked, which seems error-prone because the other process should expect the default behavior. So is this a correctly written piece of code (if so, why?) or it is just a wrongly written one which will cause unexpected behaviors?
I presume that the interrupt handler of the disk drive will be the one to wakeup(&bh->b_wait), which could lead to a missed wakeup if interrupts were not disabled in the process waiting for this block.
Remember that condition variables (sleep_on, wakeup) have no memory: sleep_on will suspend until wakeup is called; it doesn't matter if wakeup is called just before sleep_on.
From the point in time of testing bh->b_lock, the caller is racing with the interrupt handler; thus cli (or, more typical unix splbio()) blocks the interrupt handler, preventing the race.
Since the kernel saves the interrupt state (mask, priority, ...) with the process state, when sleep_on cause a reschedule, it is most likely that interrupts will be re-enabled; or at least eventually will be. The disk interrupt will eventually run, waking-up this process.
When this process is rescheduled, its saved interrupt state (disabled) will be restored, so that the test & assignment of b_lock will also prevent interference from the disk interrupt handler.
Thought about this again. I think this is the intended behavior. It means that before the disk read finishes (unlock_buffer being called), all the following executions will be in uninterruptible mode (interrupt blocked). When the buffer is unlocked and the head of queue is woken up,
while (bh->b_lock)
sleep_on(&bh->b_wait);
bh->b_lock=1;
sti();
will be executed and because we are in uninterruptible mode, it will execute to sti() without switching to other process. So other processes waiting on the same signal will sleep again (bh->b_lock is 1) when scheduled and only 1 process continues its execution.
If I have a process whose all threads are running on different cores, what will sched_yield do in that case? Get relinquished and immediately issued again? Because what I see is that sched_yield does not cause the thread calling it, wait any cycles at all, it seems that the call is not even going to the kernel.
As far as the linux kernel is concern, one core is one CPU, so sched_yield would causes the calling thread to relinquish the CPU but if the thread is the only one in the queue, it won't wait any cycle.
So yes in your case sched_yield would immediatly return immediatly if there are no thread waiting on the current core.
Note that sched_yield is not designed to control how threads are run on multiple CPU.
The man page says:
If the calling thread is the only thread in the highest priority
list at that time, it will continue to run after a call to sched_yield().
So if there's nothing more important to run, the function will basically do nothing.
I have some confusion when looking at how interrupt handler(ISR) is run. In Wiki http://en.wikipedia.org/wiki/Context_switch, it describes interrupt handling with 2 steps:
1) context switching
When an interrupt occurs, the hardware automatically switches a part of the
context (at least enough to allow the handler to return to the interrupted code).
The handler may save additional context, depending on details of the particular
hardware and software designs.
2) running the handler
The kernel does not spawn or schedule a special process to handle interrupts,
but instead the handler executes in the (often partial) context established at
the beginning of interrupt handling. Once interrupt servicing is complete, the
context in effect before the interrupt occurred is restored so that the
interrupted process can resume execution in its proper state.
Let's say the interrupt handler is the upper half, is for a kernel space device driver (i assume user space device driver interrupt follow same logic).
when interrupt occurs:
1) current kernel process is suspended. But what is the context situation here? Based on Wiki's description, kernel does not spawn a new process to run ISR, and the context established at the beginning of interrupt handling, sounds so much like another function call within the interrupted process. so is interrupt handler using the interrupted process's stack(context) to run? Or kernel would allocate some other memory space/resource to run it?
2) since here ISR is not a 'process' type that can be put to sleep by scheduler. It has to be finished no matter what? Not even limited by any time-slice bound? What if ISR hang, how does the system deal with it?
Sorry if the question is fundamental. I have not delved into the subject long enough.
Thanks,
so is interrupt handler using the interrupted process's stack(context) to run? Or kernel would allocate some other memory space/resource to run it?
It depends on the CPU and on the kernel. Some CPUs execute ISRs using the current stack. Others automatically switch to a special ISR stack or to a kernel stack. The kernel may switch the stack as well, if needed.
since here ISR is not a 'process' type that can be put to sleep by scheduler. It has to be finished no matter what?
Yep, or you're risking to hang your computer. You see, interrupts interrupt processes and threads. In fact, most CPUs have no concept of a thread or a process and to them it doesn't matter what gets interrupted/preempted (it can even be another ISR!), it's just not going to execute again until the ISR finishes.
Not even limited by any time-slice bound? What if ISR hang, how does the system deal with it?
It hangs, especially if it's a single-CPU system. It may report an error and then hang/reboot. In fact, in Windows (since Vista?) hung or too slowly executing deferred procedures (DPCs), which aren't ISRs but are somewhat like them (they execute between ISRs and threads in terms of priority/preemption) can cause a "bugcheck". The OS monitors execution of DPCs and it can do that concurrently on multiple CPUs.
Anyway, it's not a normal situation and typically there's no way out of it other than a system reset. Look up watchdog timers. They help to discover such bad hangs and perform a reset. Many electronic devices have them.
Think about interrupt handler as a function running in its own thread with high priority. When interrupt is set by device, any other activity with lowest priority is suspended, and ISR is executed. This is like thread context switch.
When ISR hangs (for example, in endless loop), the whole computer hangs - assuming that we are talking about ISR in PC driver. Any activity with lower that ISR priority is not allowed, so computer looks dead. However, it still reacts on the hardware remote debugger commands, if one is attached.
I have to develop an application that tries to emulate the executing flow of an embedded target. This target has 2 levels of priority : the highest one being preemptive on the lowest one. The low priority level is managed with a round-robin scheduler which gives 1ms of execution to each thread in turn.
My goal is to write a library that provide the thread_create, thread_start, and all the system calls that are available on my target and use POSIX functions to reproduce the behavior natively on a standard PC.
Thus, when an high priority thread executes, low priority threads should be suspended whatever they are doing at that very moment. It is to the responsibility of the low priority thread's implementation to ensure that it won't be perturbed.
I now it is usually unsafe to suspend a thread, which explains why I didn't find any "suspend(pid)" function.
I basically imagine two solutions to the problem :
-find a way to suspend the low priority threads when a high priority thread starts (and resume them when there is no more high priority activity)
-periodically call a very small "suspend_if_necessary" function everywhere in my low-priority code, and whenever an high priority must start, wait for all low-priority process to call that function and be suspended, execute as single high priority thread, then resume them all.
Even if it is not-so-clean, I quite like the second solution, but still have one problem : how to call the function everywhere without changing all my code?
I wonder if there is an easy way to doing that, somewhat like debugging code does : add a hook call at every line executed that checks for a flag and run some specific code when that flag changes?
I'd be very happy if there is an easy solution to that problem, since I really need to be representative with the behavior of the target execution flow...
Thanks in advance,
Goulou.
Unfortunately, it's not really possible to implement what you want with true threads - even if the high prio thread is restarted, it can take arbitrarily long before the high prio thread is scheduled back in and goes to suspend all the low priority threads. Moreover, there is no reliable way to determine whether the high priority thread is blocked or not using only POSIX threads; you could try tracking things manually, but this runs the risk of both false positives (the thread's blocked on something, but the low prio threads think it's running and suspend itself) and false negatives (you miss a resumed annotation, or there's lag between when the thread's actually resumed and when it marks itself as running).
If you want to implement a thread priority system with pure POSIX, one option is to not use threads, but rather use setcontext for cooperative multitasking. This would allow you to swap between threads at a user level. However you must explicitly yield the CPU in this case. It also doesn't help with blocking syscalls, which would then block all threads in your app; but since you're writing an emulator this might not be an issue.
You may also be able to swap threads using setcontext within a signal handler; I've not tested this case myself, but it could be worth a try scheduling using setcontext in a SIGALRM handler.
To suspend a thread, you sleep it. If you want to be able to wake it on command, sleep it using sigwait, which puts the thread to sleep until it gets a signal. You can send a specific thread a signal with pthread_kill (crazy name, but it actually just sends signals to a thread). This is a very fast way to sleep and wake up threads. 40x Faster than condition variables and very easy.