I know that a process switches between user mode and kernel mode for running. I am confused that for every line of code, we should possibly need the kernel. Below is the example, could I get explanation of the kernels role in execution of the following coding lines. Does the following actually require kernel mode.
if(a < 0)
a++
I am confused that for every line of code, we should possibly need the kernel.
Most code in user-space is executed without the kernel being involved. The kernel becomes involved (and the CPU switches from user-space to kernel) when:
a) The user-space code explicitly asks the kernel to do something (calls a system call).
b) There's an IRQ (from a device) that interrupts user-space code.
c) The kernel is providing some functionality that user-space code is unaware of. The most common reason is virtual memory management; but debugging and profiling are other reasons.
d) Asynchronous notifications (e.g. something causing a switch to kernel so that kernel can redirect the program to a suitable signal handler).
e) The user-space code does something illegal (crashes).
Does the following actually require kernel mode.
That code (if(a < 0) a++;) probably won't require kernel's assistance; but it possibly might. For example, if the variable a is in memory that was previously sent to swap space, then any attempt to access a is a request for the kernel to fetch that data from swap space. In a similar way, if the executable file was memory mapped but not loaded yet (a common optimization to improve program startup time), then attempting to execute any instruction (regardless of what the instruction is) could ask the kernel to fetch the code from the executable file on disk.
Short answer:
It depends on what you are trying to do, following code depending on which enviroment and how its compiled it shouldn't need to use the kernel. The CPU executes machine code directly, only trapping to the kernel on instructions like syscall, or on faults like page-fault or an interrupt.
The ISA is designed so that a kernel can set up the page tables in a way that stops user-space from taking over the machine, even though the CPU is fetching bytes of its machine code directly. This is how user-space code can run just as efficiently when it's just operating on its own data, doing pure computation not hardware access.
Long answer:
Comparing something and increasing value of something shouldn't require use of a kernel, On x86 (64 bit) architecture following could be represented like this (in NASM syntax):
; a is in RAX, perhaps a return value from some earlier function
cmp rax, 0 ; if (a<0) implemented as
jnl no_increase ; a jump over the inc if a is Not Less-than 0
inc rax
no_increase:
Actual compilers do it branchlessly, with various tricks as you can see on the Godbolt compiler explorer.
Clearly there aren't any syscalls so this piece of code can be ran on any x86 device but it wouldn't be meaningful
What requires kernels are the system calls now sys calls aren't required to have a device that can output something in theory you can output something by finding a memory location that let's say corresponds to video memory and you can manipulate pixels to output something in the screen but for userland this isn't possible due virtual memory.
A userspace application needs a kernel to exist if a kernel did not exist then userspace wouldn't exist :) and please note not every kernel let's a userspace.
So only doing something like:
write(open(stdout, _O_RDWR), "windows sucks linux rocks", 24);
would obviously require a kernel.
Writing / reading to arbitary memory location for example: 0xB8000 to manipulate video memory doesn't need a kernel.
TL:DR; For example code you provided it needs a kernel to be in userspace but can be written in a system where userspace and kernel doesn't exist at all and work perfectly fine (eg: microcontrollers)
In simpler words: It doesn't require a kernel to be work since it doesn't use any system calls, but for meaningful operation in a modern operating system it would atleast require a exit syscall to exit with a code otherwise you will see Segmentation fault even though there isn't dynamic allocation done by you.
Whatever the code we write is but obvious in the realm of user mode.. Kernel mode is only going to be in picture when you write any code that performs any system call..
and since the if() is not calling any system function it's not going to be in kernel mode.
Related
Imagine we have a usual instruction such as this one
mov [eax], ebx
and eax contains some address that we would like to write to.
The idea is to write a c program that tells you which address contains the instruction, if we already know the address that it's going to be writing to.
The real question:
write a c program using the free sony pspsdk that would accomplish the same thing.
The psp uses MIPS III / IV and the instruction would look something like
sw a0 $00(t0)
##which literally spells out store register a0 at offset t0 + 0 bytes. where t0 would
## contain something like 0x08800000
disclaimer: it is still useful to know how to do this on windows, so if somebody only knows how to do this on windows or even osx, That would still be appreciated as it could provide relevant information on similar programming practices to accomplish this particular task.
Intercepting an instruction that writes to a particular address is not a normal activity in programs.
It is a feature provided by some debuggers. There are at least three ways debuggers may be able to do this:
A debugger can examine the program code and find where a particular instruction writes to a particular address. This is actually a hugely complicated activity that requires interpreting the instructions. Often, a debugger cannot do it completely; as doing so in general is equivalent to completely interpreting and executing the program the same way the computer processor does, and it is very slow to do in software. Instead, the debugger may plan part of program execution and put in a breakpoint at a spot where it is unable to easily continue, such as at a branch instruction that depends on a value the debugger is not prepared to compute. A breakpoint is a special instruction that interrupts program execution and, in this case, results in the operating system transferring control to the debugger. At that time, the debugger removes the breakpoint, requests that the instruction be single-stepped (that the processor execute the single instruction and then interrupt program execution immediately), examines the result, and continues.
A debugger can mark the page of memory containing the desired address as no-access. Then, whenever the program accesses that memory, the hardware will interrupt program execution, and the operating system will transfer control to the debugger. The debugger examines the instruction that caused the interruption. If the instruction is accessing the target address, the debugger acts on that. If it is not, the debugger changes the memory protection to allow access, requests that the instruction be single-stepped, changes the memory protection to disallow access, and resumes the program to wait for the next interruption. (Instead of single-stepping the instruction, the debugger might just emulate it, since that might avoid changing the memory protection twice, which can be expensive.)
Some computer processor models have features to support this sort of debugging feature. The debugger can request that a portion of memory be monitored, so that the hardware interrupts program execution when a particular address is accessed, instead of when any part of a whole memory page is accessed.
I cannot speak to the Sony platform you are using. You would have to check its documentation or ask others regarding the availability of such features. Since this is a feature most often used by debuggers, investigating the documentation regarding debugging could be a way to find out whether the system supports such a feature.
I have some hardware that i want to emulate; i wonder if i can do it at a low level like this. The hardware has many registers, which i arrange in a struct:
#include <stdint.h>
struct MyControlStruct
{
uint32_t data_reg_1;
uint32_t data_reg_2;
uint32_t dummy[2]; // to make the following registers have certain addresses
uint32_t control_reg_1;
uint32_t control_reg_2;
};
volatile struct MyControlStruct* MyDevice = (struct MyControlStruct*)0xDeadF00;
So, i want to support the following syntax for hardware access on Windows and linux:
MyDevice->data_reg_1 = 42;
MyDevice->data_reg_2 = 100;
MyDevice->control_reg_1 = 1;
When the last line of code is executed, i want the hardware emulator to "wake up" and do some stuff. Can i implement this on Windows and/or linux? I thought about somehow catching the "segmentation fault" signal, but not sure whether this can be done on Windows, or at all.
I looked at the manual page of mmap; it seems like it can help, but i couldn't understand how i can use it.
Of course, i could abstract the access to hardware by defining functions like WriteToMyDevice, and everything would be easy (maybe), but i want to understand if i can arrange access to my hardware in this exact way.
In principle, you could code (unportably) a handler for SIGSEGV which would trap and handle access to unwanted pages, and which could check that a specified address is accessed.
To do that under Linux, you'll need to use the sigaction system call with SA_SIGINFO and use the ucontext_t* third argument of your signal handler.
This is extremely unportable: you'll have to code differently for different Unixes (perhaps even the version number of your Linux kernel could matter) and when changing processors.
And I've heard that Linux kernels are not very quick on such handling.
Other better kernels (Hurd, Plan9) offer user-level pagination, which should help.
I initially misunderstand your question. You have a piece of memory mapped hardware and you want your emulation to be binary compatible. On Windows you could allocate the memory for the structure using VirtualAlloc and make it a guard page and catch any access to it using SEH.
In actuality your emulator is (rather crudely) possible on linux with pure user space code.
To build the emulator, simply have a second thread or process (using shared memory, or perhaps an mmap'd file and inotify) watching the memory which is emulating the memory mapped device
For the real hardware driver, you will need a tiny bit of kernel code, but that could simply be something that maps the actual hardware addresses into user space with appropriate permissions. In effect this regresses a modern multiuser operating environment down to acting like an old dos box or a simple micro-controller - not great practice, but workable at least where security is not a concern.
Another thing you could consider would be running the code in a virtual machine.
If the code you will be exercising is your own, it's probably better to write it in a portable manner to begin with, abstracting out the hardware access into functions that you can re-write for each platform (ie, OS, hardware version or physical/emulated). These techniques are more useful if it's someone else's existing code you need to create an environment for. Another thing you can consider (if the original isn't too tightly integrated) is using dynamic-library level interception of specific functions, for example with LD_PRELOAD on linux or a wrapper dll on windows. Or for that matter, patching the binary.
Let's say there is a simple program like:
#include<stdio.h>
void main()
{
int x;
printf("Cool");
fd = open("/tmp/cool.txt", O_READONLY)
}
The open is a system call here. I suppose when the shell runs it, it makes some hundred other system calls to implement it? How about a declaration like int x - at some point should it have some additional system calls in the backdrop to get the memory from the computer?
I am not sure what is the boundary between a system call and a normal stuff ... everything, in the end, needs the operating system's help right?!
Or is it like the C generates an executable (code) which can be run on the processor and need no OS assistance is needed until a system call is reached - at which point it has to do something to load the OS instructions etc ...
A bit vague :) Please clarify.
I'm not answering the questions in order, so I'm prefixing my answers with the questions. I've taken the liberty of editing them a bit. You didn't specify the processor architecture, but I'm assuming you want to know about x86, so the processor-level details will pertain to x86. Other architectures can behave differently (memory management, how system calls are made, etc.). I'm also using Linux for examples.
Does the c compiler generate executable code that can be run straight on the processor without need for OS assistance until a system call is reached, at which point it has to do something to load the OS instructions?
Yes, that is correct. The compiler generates native machine code that can be run straight on the processor. The executable files that you get from the compiler, however, contain both the code and other needed data, for example, instructions on where to load the code in the memory. On Linux the ELF format is typically used for executables.
If the process is completely loaded into memory and has sufficient stack space, it will not need further OS assistance before it wants to make a system call. When you make a system call, it is just an instruction in the machine code that calls the OS. The program itself does not need to "load the OS instructions" in any way. The processor handles transferring execution to the OS code.
With Linux on the x86 architecture, one way for the machine code to make a system call is to use the software interrupt vector 128 to transfer execution to the operating system. In x86 assembly (Intel syntax), that is expressed as int 0x80. Linux will then perform tasks based on the values that the calling program placed into processor registers before making the system call: the system call number is found in the eax processor register and the system call parameters are found in other processor registers. After the OS is done, it will return a result in the eax register, and has possibly modified buffers pointed to by the system call parameters etc. Note however, that this is not the only way to make a system call.
However, if the process is not entirely in memory, and execution moves to a part of the code that is not in memory at the moment, the processor causes a page fault, which moves execution to the operating system, which then loads the required part of the process into memory and transfers execution back to the process, which can then continue execution normally, without even noticing that anything happened.
I'm not entirely sure on the next point, so take it with a grain of salt. The Wikipedia article on stack overflow (the computer error, not this site :) seems to indicate that stacks are usually of fixed size, so int x; should not cause the OS to run, unless that part of the stack is not in the memory (see previous paragraph). If you had a system with dynamic stack size (if it is even possible, but as far as I can see, it is), int x; could also cause a page fault when the stack space is used up, prompting the operating system to allocate more stack space for the process.
Page faults cause the execution to move to the operating system, but are not system calls in the usual sense of the word. System calls are explicit calls to the OS when you want it to perform some work for you. Page faults and other such events are implicit. Hardware interrupts continuously transfer the execution from your process to the OS so that it can react to them. After that it transfers the execution back to your process, or some other process.
On a multitasking OS, you can run many programs at once even if you have only one processor/core. This is accomplished by running only one program at a time, but switching between programs quickly. The hardware timer interrupt makes sure that control is transferred back to the OS in a timely fashion, so that one process can't hog the CPU all for itself. When control is passed to the OS and it has done what it needs to, it may always start a different process from the one that was interrupted. The OS handles all this totally transparently, so you don't have to think about it, and your process won't notice it. From the viewpoint of your process, it is executing continuously.
In short: Your program executes system calls only when you explicitly ask it to. The operating system may also swap parts of your process in and out of the memory when it wants to, and generally does things related and unrelated to your process in the background, but you don't normally need to think about that at all. (You can reduce the amount of page faults, though, by keeping your program as small as possible, and things like that)
In this case open() is an explicit system call, but I suppose when the shell runs it, it makes some hundred other system calls to implement it.
No, the shell has got nothing to do with an open() call in your c program. Your program makes that one system call, and shell doesn't come into the picture at all.
The shell will only affect your program when it starts it. When you start your program with the shell, the shell does a fork system call to fork off a second process, which then does an execve system call to replace itself with your program. After that, your program is in control. Before the control gets to your main() function though, it executes some initialization code, that was put there by the compiler. If you want to see what system calls a process makes, on Linux you can use strace to view them. Just say strace ls, for example, to see what system calls ls makes during its execution. If you compile a c program with just a main() function that returns immediately, you can see with strace what system calls the initialization code makes.
How does the process get its memory from the computer etc.? It has to involve some system calls again right? I am not sure what is the boundary between a system call and normal stuff. Everything in the end needs the OS help, right?
Yep, system calls. When your program is loaded into memory with the execve system call, it takes care of getting enough memory for your process. When you need more memory and call malloc(), it will make a brk system call to grow the data segment of your process if it has run out of internally cached memory to give you.
Not everything needs explicit help from the OS. If you have enough memory, have all your input in memory, and you write your output data to memory, you won't need the OS at all. That is, as long as you only do calculations on data you already have in memory, don't need more memory, and don't need to communicate with the outside world, you don't need the OS. On the other hand, a program that does not communicate with the outside world at all is a pretty useless one, because it can't get any input, and cannot give any output. Even if you calculate the millionth decimal of pi, it doesn't matter at all if you don't output it to the user.
This answer got quite big, so in case I missed something or didn't explain something clearly enough, please leave me a comment and I'll try to elaborate. If anyone spots any mistakes, be sure to point them out also.
I want to make a simple just-in-time compiler with c on Linux.
How can I allocate memory such that I can write out raw x86 code to it and execute it as any other function?
See mprotect(). Once you have filled a (n-)page-sized memory region (allocated with mmap()) with code, change its permissions to disallow writes and allow execution.
In addition to using mprotect correctly to provide first write and then execute permission, on some OS/hardware operations you may need to flush the I-cache. At this moment (mid-2010), all recent x86 processors have separate level 1 caches for instructions and data, and somebody has to make sure that if you write new instructions into memory (which will update the D-cache), you don't then try to execute stale bits from the I-cache. Exactly how to flush the I-cache from userspace will depend on both your hardware and the OS. My advice would be to read Intel's documentation on "self-modifying code" for their IA-32 multiprocessors. This should be enough to get you through.
I am trying to understand how the linux syscall sched_setaffinity() works. This is a follow-on from my question here.
I have this guide, which explains how to use the syscall and has a pretty neat (working!) example.
So I downloaded the Linux 2.6.27.19 kernel sources.
I did a 'grep' for lines containing that syscall, and I got 91 results. Not promising.
Ultimately, I'm trying to understand how the kernel is able to set the instruction pointer for a specific core (or processor.)
I am familiar with how single-core-single-thread programs work. One might issue a 'jmp foo' instruction, and this basically sets the IP to the memory address of the 'foo' label. But when one has multiple cores, one has to say "fetch the next instruction at memory address foo, and set the instruction pointer for core number 2 to begin execution there."
Where, in the assembly code, are we specifying which core performs that operation?
Back to the kernel code: what is important here? The file 'kernel/sched.c' has a function called sched_setaffinity(), but returns type "long" - which is inconsistent with its manual page. So what is important here? Which of these modules shows the assembly instructions issued? What module is reading the 'task_struct', looking at the 'cpus_allowed' member, and then translating that into an instruction? (I've also thumbed through the glibc source - but I think it just makes a call to the kernel code to accomplish this task.)
sched_setaffinity() simply tells the scheduler which CPUs is that process/thread allowed to run on, then calls for a re-schedule.
The scheduler actually runs on each one of the CPUs, so it gets a chance to decide what task to execute next on that particular CPU.
If you're interested in how you can actually call some code on other CPUs, I suggest you take a look at smp_call_function_single(). In case we want to call something on another CPU, this calls generic_exec_single(). The latter simply adds the function to the target CPU's call queue and forces a reschedule through some IPI stuff (if the queue was empty).
Bottom line is: there no actual SMP variant of the _jmp_ instruction. Instead, code running on other CPUs cooperates in order to accomplish the task.
I think the thing you are not understanding is that the kernel is running on all the CPU cores. At every timer interrupt (~1000 per second), the scheduler runs on each CPU and chooses a process to run. There is no one CPU that somehow tells the others to start running a process. sched_setaffinity() works by just setting flags on the process. The scheduler reads these flags and will not run that process on its CPU if it is set not to.
Where, in the assembly code, are we specifying which core performs that operation?
There is no assembly involved here. Every task (thread) is assigned to a single CPU (or core in your terms) at a time. To stop running on a given CPU and resume on another, the task has to "migrate" (also this). When a task migrates from one CPU to another, the scheduler picks the CPU which is more idle among the CPUs allowed by sched_setaffinity().
There is no magic assembly instructions issued. The kernel has a more low-level view of the hardware, each CPU is a separate object, very different than how it looks like for user-space processes (in user-space, CPUs are almost invisible).
Check this out: B Operating System Programming Guidelines