I have implemented a new scheduler for the xv6 OS, but I don't know how to measure the performance of this new one. I found one benchmark to measure the time, but will the benchmarks for Unix work well on xv6? any ideas...
Related
I'm doing some investigation in profiling scalability tests on openMPI, with Linux Ubuntu 18.04.
I could profile the benchmark within some useful MPI profiling tools like mpiP, Scalasca, etc. However, still, there is an open question for me,
What is the kernel usage (time, memory, I/O, etc) to do an MPI job? I need to see kernel movements to profile the MPI tasks (processes) over different ranks, How can I profile the kernel usage? I think all the (mentioned above) profilers provide for me are from the user point of view, right?
The operating system is able to determine how to arrange difference processes/threads onto different cpu cores, the os scheduler does the work well. So when do we really need to call functions like sched_setafficity() for a process, or pthread_setaffinity_np() for a pthread?
It doesn't seem to be able to raise any performance dramatically, if it can, then I suppose we need to re-write linux process scheduler right?
Just wish to know when do we need to call these functions, in my applications?
Thanks.
It's very helpful in some computationally intensive real time processes related to DSP(Digital Signal Processing).
Let's say One real time DSP related process PROCESS0 is running on core CPU0. Because of some scheduling algorithms CPU0 pre-emption need to happen such that process0 has to run on another CPU. This switching of realtime process is a overhead. Hence affinity. We direct to kernel that the process0 should run on CPU0.
I want to know which part of the code is the scheduler of project OPTEEhttps://github.com/OP-TEE.
More specifically, the code decides that on which CPU secure world and normal word processes are running for multicore architecture.
Any advice is genuinely appreciated.
I'm one of the Linaro developers working directly with OP-TEE and OP-TEE as such has no scheduler, instead it's being entirely scheduled by Linux kernel. It's not tied to a single core either, after being in Linux kernel and you're about to re-enter secure world it could be any of the cores where you continue running your ongoing job. So in short, there is no scheduler in OP-TEE.
I use time command to measure execution time of a c program and I see that it spends so much time in kernel mode (although I expect it to run mostly in user mode). I don't know why, and I don't have any clue where to search for the problem.
this is an example:
real 0m44.548s
user 0m19.956s
sys 1m19.944s
These are information about the test program. it is streamcluster from parsec benchmark tools.
Application domain: data mining
Data sharing: low
Data exchange: medium
Parallelization model: data-parallel
Contains many pthread_mutexes and pthread_conditions
CPU bound
Little memory allocations or writing to files
I run this program on a virtual machine.
Today my boss and I were having a discussion about some code I had written. My code downloads 3 files from a given HTTP/HTTPS link. I had multi-threaded the download so that all 3 files are downloading simultaneously in 3 separate threads. During this discussion, my boss tells me that the code is going to be shipped to people who will most likely be running old hardware and software (I'm talking Windows 2000).
Until this time, I had never considered how a threaded application would scale on older hardware. I realize that if the CPU has only 1 core, threads are useless and may even worsen performance. I have been wondering if this download task is an I/O operation. Meaning, if an API is blocked waiting for information from the HTTP/HTTPS server, will another thread that wants to do some calculation be scheduled meanwhile? Do older OSes do such scheduling?
Another thing he said: Since the code is going to be run on old machines, my application should not eat the CPU. He said use Sleep() calls after CPU intensive tasks to allow other programs some breathing space. Now I was always under the impression that using Sleep() is terrible in any program. Am I wrong? When is using Sleep() justified?
Thanks for looking!
I have been wondering if this download task is an I/O operation.
Meaning, if an API is blocked waiting for information from the
HTTP/HTTPS server, will another thread that wants to do some
calculation be scheduled meanwhile? Do older OSes do such scheduling?
Yes they do. That's the joke of having blocked IO. The thread is suspended and other calculations (threads) take place until an event wakes up the blocked thread. That's why it makes completely sense to split it up into threads even for single core machines instead of doing some poor man scheduling between the downloads yourself in a single thread.
Of course your downloads affect each other regarding bandwith, so threading won't help to speedup the download :-)
Another thing he said: Since the code is going to be run on old
machines, my application should not eat the CPU. He said use Sleep()
calls after CPU intensive tasks to allow other programs some breathing
space.
Actually using sleep AFTER the task finished won't help here. Doing Sleep after a certain time of calculation (doing sort of time slicing) before going on with the calculation could help. But this is only true for cooperative systems (e.g. like Windows 3.11). This does not play a role for preemptive systems where the scheduler uses time slicing to allocate calculation time to threads. Here it would be more important to think about lowering the priority for CPU intensive tasks in order to give other tasks precedence...
Now I was always under the impression that using Sleep() is terrible
in any program. Am I wrong? When is using Sleep() justified?
This really depends on what you are doing. If you implement sort of busy waiting for a certain flag being set which is set maybe after few seconds it's better to recheck if it's set after going to sleep for a while in order to give up your scheduled time slice instead of just buring CPU power with checking for a flag never being set.
In modern systems there is no sense in introducing Sleep in a calculation as it will only slow down your calculation.
Scheduling is subject to the OS's scheduler. He's the one with the "big picture". In my opinion every approach to "do it better" is only valid inside the scope of a specific application where you have the overview over certain relationships that are not obvious to the scheduler.
Addendum:
I did some research and found that Windows supports preemptive multitasking from Windows 95. The Windows NT-line (where Windows 2000 belongs to) always supported preemptive multitasking.