I'm new to kernel programming and was thinking on how one can make single process running which means that other kernel processes should die or get starved. What can be optimal approach to this situation. I'm working on kernel version 2.6.38.
while (1); should do a good job of starving other processes - especially if you first disable interrupts.
Disable CPU quotas. Raise your process priority to the highest possible. Create as many for(;;) threads as there are cores. Raise their priority to the highest available. Set them running.
Not so sure about Linux, TBH. Such an activity on Windows needs the power switch for recovery.
Related
I was recently doing some experiments with the fork function and I was concerned by a "simple" (short) question:
Does fork uses concurrency or parallelism (if more than one core) mechanisms?
Or, is it the OS which makes the best choice?
Thanks for your answer.
nb: Damn I fork bombed myself again!
Edit:
Concurrency: Each operation run on one core. Interrupts are received in order to switch from one process to another (Sequential computation).
Parallelism: Each operation run on two cores (or more).
fork() duplicates the current process, creating another independent process. End of story.
How the kernel chooses to schedule these processes is a different, very broad question. In general the kernel will try to use all available resources (cores) to run as many tasks as possible. When there are more runnable tasks than cores, it has to start making decisions about who gets to run, and for how long.
The fork function creates a separate process. It is up to the operating system how it handles different processes.
Of course, if only once core is available, the OS has no other choice but running all processes interleaved.
If more cores are available, every sane OS will distribute the processes to the different cores, so every core runs at least one process.
However, even then, more processes can be active than there are cores. So even then, it is up to the OS to decide which processes can be run parallel (by distributing to cores) and which have to be run interleaved (on a single core).
In fact, fork() is a system call (aka. system service) which creates a new process from the current process (read the return code to see who you are, the parent or the child).
In the UNIX work, processes shares the CPU computing time. This works like that :
a process is running
the clock generates an interrupt, calling the kernel and pausing the process
the kernel takes the list of available processes, and decide to resume one (this is called scheduling)
go to point 1)
When there is multiples processor cores, kernels are able to dispatch processes on them.
Well, you can do something. Write a complex program say, O(n^3), so that it takes a good amount of time to compute. fork() four times (if you have quad-core). Now open any graphical CPU monitor. Anything cool?
I would like to run a long term task on a dedicated core and would like that task to be minimally interrupted / preempted. I can see 2 solutions. Which one is better or any other solution?
1) Set affinity and isolate core using isolcpus
2) Make the thread real time using SCHED_FIFO and set the priority high
- if this is the better choice how high the priority should be? Can I set it to 99?
What I am concerned about is being preempted by kernel threads, IPIs ...
Regarding the first solution you mentioned, by adding parameter isolcpus = [CPU no.] during boot will instruct Linux scheduler to not run any task on that CPU unless requested by user using CPU Affinity. But this CPU may receive interrupts and that can also be avoided by setting IRQ Affinity, so that the isolated CPU doesn’t receive any interrupt. Finally in your code of the task you set the Affinity to the isolated CPU and you are good to go.
But Even if you follow these steps, kernel tasks are executed on the isolated CPU core if you are not using a real-time kernel from RP_PREEMPT, hence it might not be possible to completely isolate a CPU core unless you are using RT kernel.
Refer - http://elinux.org/CPU_Shielding_capability
The second solution about using SCHED_FIFO scheduling policy and using a high priority value will still not prevent the kernel threads, Timer tick interrupts, IPIs etc., from pre-empting your task. Because the scheduling policies and priority is for kernel to schedule all other User-space processes and threads and does not apply to kernel threads or processes.
So by setting high priority to your task does not mean you will get 100% CPU dedicated to your task. Also the alternative, manually setting the CPU mask of your task to a CPUSET in the system, can cause problems and suboptimal load balancer performance. Your task will still get interrupted from time to time by Linux code, including other tasks - such as the timer tick interrupt and the scheduler code, IPIs from other CPUs and stuff like work queue kernel threads, although the interruption should be quite minimal if you have don’t have much activity going on in your other cores.
But the cleanest way to achieve this should come from Kernel tweak which I found from this link http://www.linuxjournal.com/article/6799?page=0,2. Though I haven’t tried this personally, I think it’s worth giving a look at this article as well before you decide upon the method you will use.
what happens when we set different processor affinity to process and its thread in linux.
I am trying to start a process affined to a core (say 1) which have two threads one of which need to run on other core (say 0)
When i tried to set affinity to thread different to process the program got executed. but I want to know the hidden impacts of this approach.
Threads and processes are vastly the same thing. Whether you call pthread_setaffinity... or use the sched_setaffinity syscall, they both affect the current thread's affinity mask. This may be your "process" thread, or a thread you created.
However, note that a new thread created by pthread_create inherits a copy of its creator's CPU affinity mask [1].
That means that setting the affinity and creating a thread is not the same as creating a thread and setting the affinity. In the first case, both threads will compete over the same processor (which is most probably not what you want) and in the second case they will be bound to different processors.
Also note that while binding threads to a dedicated processor (core) may have advantages in some situations, it may just as well be a very stupid thing to do. Playing with the affinity mask means you limit the scheduler in what it can do to make your program run. If the core you bound your thread to isn't available, your thread will not run, end of story.
This is a very similar reasoning/strategy as disabling swap to make the system "faster" (some users still do that!). By doing so they usually gain nothing, all they do is limit what the memory manager can do by removing one option of providing a free page once it runs out of unused physical RAM. Usually this means something more or less valuable from the buffer cache is purged when instead some private page that wasn't used in hours could have been swapped out.
Usually people use affinity because they have this idea that the scheduler will make threads bounce between processor cores all the time and this is bad. Processor migration indeed is not cheap, but the scheduler has a mechanism which makes sure it does not happen before a certain minimum amount of time (there is a /proc thingie for that too). After a longer amount of time, all advantages of staying at the old core (TLB, cache) are usually gone anyway, so running on a different core which is readily available is actually better than waiting for a particular core to maybe, eventually become available.
NUMA architectures may be a different topic, but I'd assume (though I don't know for sure) that the scheduler is smart enough not to silently migrate a thread to a different node. In general, however, I'd recommend not to play with affinity at all.
Affinity is a common first line approach to limiting jitter in HPC. Typically LINUX processes and threads and such are constrained to a small but sufficient set of CPUs and the application is constrained to the remainder of the CPUs.
Affinity is very useful with device drivers. Consider for example an Infiniband adapter being used by an application. The adapter will perform best if the driver thread(s) are constrained to CPUs on the same (or closest if none) NUMA node as the adapter. LINUX doesn't know the application thread so can't even consider any affinity for performance.
Can setting the cpu affinity in linux for a multithreaded program where each thread runs on each core effectively block any other process from being scheduled by the os on that core. Effectively I want to guarantee the use of a core to my process and have all other non critical programs bound to a minimal number of cores.
Or am I missing something with the linux scheduler, or maybe I need my own.
Can setting the cpu affinity in linux for a multithreaded program
where each thread runs on each core effectively block any other
process from being scheduled by the os on that core
No, setting the cpu affinity prevents the scheduler from using some cores for your threads. That is, it will only schedule your threads on certain cores - it doesn't do anything to other threads.
You can probably achieve what you want using setpriority. If your requirements are that stringent, you might look into sched_setscheduler and choose SCHED_RR or SCHED_FIFO.
When the scheduler is actively involved, taskset and nice will only give a hint to the scheduler about your preferences. The scheduler is free to reschedule any of the threads on any of the available cores based on the workload. You can use perf to monitor context switches and cpu migrations.
You have two options:
You can force the scheduler to follow your orders trough sched_setscheduler as user417896 sugggested.
You can use cgroups/cpuset to define two cpusets,say system and isolated, and isolate the target cores by moving all the system threads to system cpuset and run your program using cgexec on the isolated cpuset. You can assign cores and memory to a cpuset and in order to isolate it set cpu_exclusive bit and you are all set. You an also use cset (http://code.google.com/p/cpuset/) if you are using older kernels to automate this process for you.
I hope it helps.
I'm writing a a very small daemon that must remain responsive even when a system is under severe stress. I'm looking at the differences between SCHED_FIFO and SCHED_RR in regards to scheduling, as well as trying to determine a sensible priority.
Which scheduler would be appropriate for a small but critical monitoring daemon, what priority would be reasonably safe? I'm still coming up a little fuzzy when trying to understand the differences between the two.
My program is allocating under 3k (and uses mlockall()), it writes about 600 bytes to xenbus then sleeps, but its impossible for me to tell how much time (in ms) it will take to actually write the data.. since what is written depends on a configuration file.
Thanks in advance for any suggestions / explanations.
The infamous pchdtvr program, which captures digital TV signals, uses SCHED_FIFO to make sure that the TV packets are written to disk no matter what. It can capture 4 shows at once while playing Doom on an old computer.
The program is infamous because it was released under GPL and the author tried to revoke the GPL retroactively. This act provoked a minor firestorm. Anyway, you can find a recent version to study at http://frequal.com/pmn/pchdtvr.html.
SCHED_FIFO can't be preempted (context switched to another process) unless another process of higher priority shows up in the execution queue.
SCHED_RR can be preempted by a time quantum (delay given to a process to execute).
They are both "real-time" priorities of linux based schedulers.
I'm not an expert for scheduling schemes, but have a look at
man sched_setscheduler
it details what the difference between the different scheduling algorithms are, and provide links to other scheduling functions. SCHED_FIFO actually sounds pretty dangerous, but is described as the most aggressive scheduling:
A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted by a higher priority process, or it calls sched_yield(2).
Beware not to lock up your system. I would personally do some empirical tests to see what priority fits the best and how they behave exactly.
If all your other tasks use the standard scheduler, it makes no difference; SCHED_FIFO and SCHED_RR only affect the scheduling of these tasks with each other.
So on a normal system it makes no difference. FIFO is easiest to understand, so use that I guess.
If you have several tasks of different priorities, only the higher one will be run if they are all ready to be run (and there is only one CPU core)