I am trying to understand how many processors are supported by Linux Kernel.
grep NR_CPUS /boot/config-`uname -r`
Will give me the maximum number of processors supported by kernel, which I can override using kernel command line parameter nr_cpus.
To find number of online cpus, i can use num_online_cpus() function
Then what is nr_cpu_ids?
#include <linux/kernel.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
static int __init test_hello_init(void)
{
pr_info("%s: In init NR_CPUs=%d, nr_cpu_ids=%d\n", __func__, NR_CPUS, nr_cpu_ids);
pr_info("Number of cpus available:%d\n", num_online_cpus());
return -1;
}
static void __exit test_hello_exit(void)
{
pr_info("%s: In exit\n", __func__);
}
module_init(test_hello_init);
module_exit(test_hello_exit);
[11548.627338] test_hello_init: In init NR_CPUs=8192, nr_cpu_ids=128
[11548.627340] Number of cpus available:6
What is the difference between NR_CPUs and ncr_cpu_ids. Are they not same?
nr_cpu_ids is the total number of CPUs or processors in the machine while NR_CPUS is the total number of CPUs the Linux O/S can handle.
Related
I am developing a linux kernel module, which looks like this:
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Me");
MODULE_DESCRIPTION("Something Something");
int checkSomething(void) {
int someCpuFeature = 0;
__asm__("mov $1, %eax");
__asm__("cpuid");
__asm__("mov %%ecx, %0" : "=r" (someCpuFeature));
if (someCpuFeature & 32) {
return 1;
}
return 0;
}
int __init init_module(void) {
if (!checkSomething()) {
printk(KERN_INFO "Exiting\n");
return 0;
} else {
printk(KERN_INFO "Continuing\n");
}
return 0;
}
static void __exit exit_module(void) {
printk(KERN_INFO "Unloading Module\n");
}
And when i loaded it, i tried to see it's output from dmesg.
but instead of only printing Exiting/Continuing, it also printed a call trace,
and said BUG: scheduling while atomic: insmod/24641/0x06100800.
I searched this bug and found that it has some connection to the scheduler and sleeping at places you shouldn't sleep in, but this is the only functionality of the code,
So i think it has something to do with the cpuid instruction, but i don't know exactly what it is.
any ideas?
Hm, considering the interrupts raised on my machine now, it would look like your assembler is incorrect.
Instead of writing this yourself, I'd recommend relying on the cpuid kernel module (and thus, reading /dev/cpu/NUMBER/cpuid after seeking to your level of interest), as that's future-proof external API. You can also look at /arch/x86/include/asm/processor.h, and use kernel functions like cpu_has to detect your feature. Don't reinvent the wheel – querying details of a CPU on an SMP machine is bound to be painful, and the poor kernel developers had to go through the pain to make this work themselves.
I wrote a simple Linux kernel module to issue hlt instruction
#include <linux/kernel.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
static int __init test_hello_init(void)
{
asm("hlt");
return 0;
}
static void __exit test_hello_exit(void)
{
}
module_init(test_hello_init);
module_exit(test_hello_exit);
Loading this module on my Virtual Machine, I don't see my VM is halted.
Am I missing something?
HLT doesn't stop your machine, only make that core sleep (in C1 idle) until the next interrupt.
You can try adding cli instruction before hlt, so only an NMI can wake that CPU up and make the function return.
static int __init test_hello_init(void) {
asm("cli");
asm("hlt");
return 0;
}
I am testing a small code which involves creating a thread on my ARMv8 device.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
void *fun(void *arg)
{
sleep(2);
printf("%s: Exiting now\n", __func__);
pthread_exit(0);
//return 0;
}
int main()
{
pthread_t th;
pthread_create(&th, NULL, fun, NULL);
sleep(3);
printf("%s: Wait over\n", __func__);
return 0;
}
When I compile the code into a 32-bit executable and run it on the device, it seems to run fine.
fun: Exiting now
main: Wait over
But when I build it into 64-bit executable, I get the following output:
fun: Exiting now
Aborted (core dumped)
As far as I know, ARMv8 is completely backward-compatible. But here, the 32-bit binary runs while 64-bit binary hangs up.
Apart from that, while trying out things, I found when I change the fun() to:
void *fun(void *arg)
{
sleep(2);
printf("%s: Exiting now\n", __func__);
//pthread_exit(0);
return 0;
}
Both 32-bit and 64-bit binaries run fine.
I looked into this article:
Pthread: Why people bother using pthread_exit? and didn't get much information as to where the problem might be.
Here is some more information about my device:
Linux Version: Linux version 4.4.67 (lnxbuild#ecbld-bd213-lnx) (gcc version 4.9.3 (GCC) ) #1 SMP PREEMPT
Total Memory : 3721056
So here are my questions:
Can there be any reason why the 32-bit binary runs while the 64-bit crashes on an ARMv8 device?
How does this error fix when I replace pthread_exit(0) with return 0?
Is there anything else that I might be missing here?
Minor Update:
I have tried changing the wait times and making various combinations of pthread_join(), pthread_detach() and pthread_exit() at all possible places, but to no avail. This is the simplest version I could provide.
My code is as follows:
#include <sys/ptrace.h>
#include <stdio.h>
int
main()
{
printf("PTRACE_CONT: %d\n", PTRACE_CONT);
printf("PTRACE_SYSCALL: %d\n", PTRACE_SYSCALL);
printf("PTRACE_SINGLESTEP: %d\n", PTRACE_SINGLESTEP);
printf("PTRACE_SYSEMU: %d\n", PTRACE_SYSEMU);
printf("PTRACE_SYSEMU_SINGLESTEP: %d\n", PTRACE_SYSEMU_SINGLESTEP);
printf("PTRACE_LISTEN: %d\n", PTRACE_LISTEN);
return 0;
}
I'm compiling with the default flags on Ubuntu16.04 (Linux x86_64 4.40-38), with gcc v5.4.0.
This throws an error that PTRACE_SYSEMU is undeclared. While the man ptrace page states it exists. This is repeated for PTRACE_SYSEMU_SINGLESTEP if the line containing PTRACE_SYSEMU is commented out. Which the man page states PTRACE_SYSEMU_SINGLESTEP is only available for x86, except a patch was merged to unify the x86 and x64 handling of PTRACE_SYSEMU_SINGLESTEP in 2008.
This produces the same error on 32bit (well i686), or 64bit (AMD64). Is this distro specific? What is going on?
I can confirm neither of these values are defined are in my /usr/include/x86_64/linux/sys/ptrace.h. But they are defined in kernel sources?!?
On Ubuntu 16.04 (and also 14.04), these are defined in <asm/ptrace-abi.h>, which is included by <asm/ptrace.h>, which in turn is included by <linux/ptrace.h>, but not by <sys/ptrace.h>
Since these request codes are linux specific (not part of any standard), if you want them, you need to #include <linux/ptrace.h>
Sysemu is used in user-mode linux as optimization and described at http://sysemu.sourceforge.net/ site. It is feature for UML (when special kernel runs as ordinary process) and not for typical users of ptrace.
Its implementation in x86 linux can be checked by TIF_SYSCALL_EMU flag in lxr of linux kernel (ptrace_resume)
http://lxr.free-electrons.com/source/kernel/ptrace.c?v=4.10#L767
767 static int ptrace_resume(struct task_struct *child, long request,
768 unsigned long data)
...
780 #ifdef TIF_SYSCALL_EMU
781 if (request == PTRACE_SYSEMU || request == PTRACE_SYSEMU_SINGLESTEP)
782 set_tsk_thread_flag(child, TIF_SYSCALL_EMU);
783 else
784 clear_tsk_thread_flag(child, TIF_SYSCALL_EMU);
785 #endif
http://lxr.free-electrons.com/ident?i=TIF_SYSCALL_EMU
The only definition is for x86:
http://lxr.free-electrons.com/source/arch/x86/include/asm/thread_info.h?v=4.10#L85
85 #define TIF_SYSCALL_EMU 6 /* syscall emulation active */
I have written kernel module to measure the correctness of ndelay() kernel function.
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/time.h>
#include <linux/delay.h>
static int __init initialize(void)
{
ktime_t start, end;
s64 actual_time;
int i;
for(i=0;i<10;i++)
{
start = ktime_get();
ndelay(100);
end = ktime_get();
actual_time = ktime_to_ns(ktime_sub(end, start));
printk("%lld\n",(long long)actual_time);
}
return 0;
}
static void __exit final(void)
{
printk(KERN_INFO "Unload module\n");
}
module_init(initialize);
module_exit(final);
MODULE_AUTHOR("Bhaskar");
MODULE_DESCRIPTION("delay of 100ns");
MODULE_LICENSE("GPL");
the dmesg output is like this:
[16603.805783] 514
[16603.805787] 350
[16603.805789] 373
[16603.805791] 323
[16603.805793] 362
[16603.805794] 320
[16603.805796] 331
[16603.805797] 312
[16603.805799] 304
[16603.805801] 350
I have gone through one of the posts in stackoverflow:
Why udelay and ndelay is not accurate in linux kernel?
But I want a fine tuned nanosecond delay (probably in the range of 100-250ns) in kernel space. Can anyone please suggest me any alternative for doing this?
You can use
High resolution timers (or hrtimers)
hrtimer_init
hrtimer_start
hrtimer_cancel
functions. An example is available here
If you are targeting x86 only system, you can use rdtsc() call to get the CPU clock counts. The rdtsc() api has very little overhead. But you do need to convert from CPU clock to the ns, it is dependent on how fast your CPU clock is running.
static unsigned long long rdtsc(void)
{
unsigned int low, high;
asm volatile("rdtsc" : "=a" (low), "=d" (high));
return low | ((unsigned long long)high) << 32;
}
Otherwise you can use the kernel high resolution timers API.
The high-resolution timer API
Linux Kernel ktime - high resolution timer type, APIs, RDTSC usages