How can one print some arbitrary text on-screen without the call to any function ?
Another way of asking this question is how are i/o functions implemented ?
I tried searching in google, but unluckily, no result was found similarly to as if it was some sort of a top secret thing.
For study purposes.
I personally don't feel even close to a real programmer without knowing this.
Well, in the end, loosely speaking, the low level software in the computer sets a special memory location or uses a special instruction that changes the voltage on some pins on the CPU, and the hardware responds to these changes.
But user-level processes don't have access to those instructions or memory locations. Messing with stuff that drives the hardware is the responsibility of "device drivers" that execute in the kernel. They use these special memory locations or instructions, and each one is given responsibility over a particular hardware device.
User-level processes communicate with device drivers via system calls as mentioned in the comments. A system call is not quite like a normal function call -- you don't just call the code. After setting up a "request" for what it wants to do, the user-level process pokes the kernel, usually by using a software interrupt instruction. The kernel wakes up, looks at what you request, and then decides itself what code to execute. The kernel code runs at a higher privilege level and will call directly into the device drivers that access the hardware.
This is how the kernel keeps processes safe from each other.
To actually get from stdout to the screen is a lengthy process:
the standard library ends up making a system call that writes to a "pipe" that is attached to stdout. This is where it leaves your process.
The other end of the pipe is being read by the console. The console is a user-level process, so it has to do a system call to do the reading.
The console decides what to display, and how to make it visible to you. There will be a bunch more layers, but eventually there will be system calls into drivers that control the graphics hardware. They will mess with the bits that turn into pixels on the screen, making your text visible.
Related
I have been reading about system calls and how they work in Linux. I still have more reading to do but one thing that nothing I have read has answered is, WHY do we need system calls?
I understand that system calls are requests from user space program for the kernel to do something, but my question is basically: Why can't the user space program do the thing itself? Why doesn't Glibc do the actual operation instead of just being a wrapper for a system call?
For example, if I call fopen() in my program, why does glibc call the open system call? Why doesn't glibc just do the operation itself?
I understand that it would mean that glibc developers would have a lot more work and that they would have to have an intimate knowledge of Linux, but isn't glibc already very closely related to Linux kernel?
Also, I understand the system call functions are run in ring 0 in the CPU...but what's really the point of that? If I execute a program, I am giving it express permission to run, so what security is added by separating what code can be run in different contexts since you are giving it all permission anyway?
Why doesn't glibc just do the operation itself?
Well that is more less the ways things went in good old MS/DOS systems: no separation between kernel code and user code, and user code could happily directly access the hardware.
This just has 2 major problems:
It works (rather) fine on a single user and not multi tasking system, but as soon as multiple programs can simultaneously run in a system, you have to synchronize hardware accesses and memory usage => those are the parts dedicated to the kernel
There is no protection of the system from a poorly coded program. In a modern OS, an erroneous program can crash, but the system itself should survive. In MS/DOS a program crash usually ended in a system reboot.
For those reasons, all modern OS (except maybe some lightweight embedded ones) use isolation between different user processes and the kernel. And that just mean that you need a way to allow a user mode process to require a privileged action (reading or writing a physical disk is) from the kernel: that is exactly what system calls are made for.
Why doesn't glibc just do the operation itself?
Short answer: Because it can't.
Long answer:
A program running in Linux can run in two modes : UserLand or KernelLand.
The Kernel Land has every rights and can do everything, including talking with hardware, or providing userspace callbacks. For instance, when you call fopen(), the kernel does all the dirty talking with your filesystem (ext4 for instance), the caching, everything down to talking with the SATA Controller to access data on the hard-drive.
GLibc could do that using the device exposed by the kernel in /dev, but that would mean recoding from scratch all the filesystems layers, the sockets, the firewalling...
The kernel just provides easy usable API for programmers to have elevated rights and communicate with the devices. That's how Linux (and most modern OS) is made.
What security is added by separating what code can be run in different contexts since you are giving it all permission anyway?
The permissions are managed by the kernel. If you don't have syscall, you don't have permissions. Or should the program you run check their own permission? Once again, it would be reinventing the wheel every time.
If the code generated by a C implementation were the only thing that were going to be running on the target system (as it would be for many freestanding implementations, and for a very small number of hosted implementations) and if implementation knew precisely what hardware it would be running upon (true of some freestanding implementations, but seldom true for hosted ones), its runtime library might be able to perform operations like "fopen" by directly communicating with the storage hardware. It is rare, however, for either condition to apply, much less both of them.
If multiple programs will be using storage device, it will generally be necessary that they either coordinate their actions somehow or else that sequences of operations performed by different programs do not overlap, and that every program "forget" anything it thinks it knows about the state of storage any time another program might have written to it.
Otherwise, suppose a disk contains a single file and program #1 uses "fopen" to open it for reading. Each directory sector holds 8 entries, so the program would read the first directory sector and observe that slot #0 identifies the file of interest while #1-#7 are blank.
Now suppose program #2 uses "fopen" to create a file for writing. It would read the directory sector, observe that slots #1-#7 are blank, and rewrite the directory sector with information about the new file in slot #1.
Finally, suppose program #1 wants to write a file. If it doesn't know about program #2, it might reasonably believe it knows what the directory contains (it had read it earlier, and has no reason to believe it's changed), place information about the new file in slot #1, and replace the directory sector on disk with its new version, obliterating the entry written by program #2.
Having both programs route their operations through an operating system ensures that when program #2 wants to create its file, it can exploit the fact that it had just read the directory for program #1 (and thus doesn't need to reread it). More importantly, when program #1 goes to write a file, the operating system will know that the directory contains the file written by program #2, and will thus ensure that the new file gets placed in slot #2.
Contrary to what other answers say, even microcomputer C implementations running on platforms like MS-DOS essentially always relied upon the OS for file I/O. Some would include their own console I/O routines because the ones in MS-DOS were about four times as slow as they should have been, but the need for coordination when using file I/O meant that very few programs would try to do it themselves.
1- You don't wanna deal with low-level hardware communications. At least most people don't. Each of them has hundreds of commands.
2- Make a simple mistake and your CPU/RAM or I/O device might be useless forever.
3- When you are part of a network, you can share resources. The system calls and kernel keeps your co-worker from damaging your hard disk.
Another consideration is that the OS kernel needs to provide an abstraction for the myriad different types of hardware via a uniform API - without which you'd invariably be making device specific calls in your program.
While the previously-idle disk spins up for two seconds, or the networked disk gets connected for thirty seconds, what is the library going to do?
The full answer to your question is very broad but let me take a simple example based upon your question about fopen.
Let us say that we have a large system that has hundred or thousands of users. One of those users is, say the HR department with files containing confidential information about employees.
If that disk could be accessed at will in user mode, then any person on the system could open any file on the system, including those with confidential information.
In other words operating systems managed SHARED resources. These include disk, CPU, and memory. If these could be controlled in user mode, there would be no way to ensure that these were shared equitably.
I want to make a program that will gather information about the keystrokes of a user (keycode, press and release times) and will use them as a biometric for authenticating the user continuously. My approach is to gather the keystrokes using a kernel module (because you can't just kill a kernel module), than the kernel module will send the information to another process that will analyze the data gathered by the kernel module, it will save it to a database and will return an answer to the kernel (the user is authenticated or not) and the kernel will lock the computer if the user is not authenticated. the whole module will not be distributed.
my questions are:
1. How can I call a process from the kernel and also send him the data?
2. how can I return a message to the kernel from the process?
#basile-starynkevitch 's answer and his arguments notwithstanding there is an approach you can take that is perfectly correct and technically allowed by the linux kernel.
Register a keyboard notifier call back function using the kernel call register_keyboard_notifier() in your kernel module. As a matter of fact it's designed for exactly this!
Your notifier call back function will look something like:
int keysniffer_callback(struct notifier_block *notifier_block,
unsigned long scancode,
void *param)
{
// do something with the scancode
return NOTIFY_OK;
}
See https://www.kernel.org/doc/Documentation/input/notifier.txt for starters.
I want to make a program that will gather information about the keystrokes of a user
That should go in practice into your display server, which you did not mention (Xorg, Wayland, MIR, ...?). Details matter a big lot!
My approach is to gather the keystrokes using a kernel module
I strongly believe this is a wrong approach, you don't need any kernel module.
I want to make a program that gathers data about the user keystrokes
Then use ordinary Unix machinery. The keyboard is some character device (and you could have several keyboards, or none, or some virtual one...) and you could read(2) from it. If you want to code a keylogger, please tell that explicitly.
(be aware that a keylogger or any other cyberspying activity can be illegal when used without consent and without permission; in most countries, that could send you to jail: in France, Article 323-1 du Code Pénal punishes that by at least 2 years of jail; and most other countries have similar laws)
the kernel module will send the information to another process [....] it will save it to a database
This is practically very difficult to get (and you look confused). Databases are in user-land (e.g. some RDBMS like PostGresSQL, or some library accessing files like sqlite). Notice that a kernel driver cannot (easily and reliably) even access to files.
All application programs (and most daemons & servers) on Linux are started with execve(2) (e.g. by some unix shell process, or by some daemon, etc...), and I see no reason for you to make an exception. However, some programs (mostly init, but also a few others, e.g. /sbin/hotplug) are started by the kernel, but this is exceptional (and should be avoided, and you don't need that).
How can I call a process from the kernel
You should not do that. I see no reason for your program to avoid being started by execve from some other process (perhaps your init, e.g. systemd).
and also send him the data?
Your process, as all other processes, is interacting with the kernel thru system calls (listed in syscalls(2)). So your application program could use read(2), write(2), poll(2) etc.. Be aware of netlink(7).
how can I return a message to the kernel from the process?
You don't. Use system calls, initiated by application code.
the kernel will lock the computer if the user is not authenticated.
This does not have any sense. Screen locking is a GUI artefact (so is not done by kernel code, but by ad-hoc daemon processes). Of course some processes do continue to run when locking is enabled. And many processes are daemons or servers which don't belong to "the" user (and continue to run when "the computer is locked"). At heart, Linux & POSIX is a multi-user and multi-tasking operating system. Even on a desktop Linux system used by a single physical person, you have dozens of users (i.e. uid-s many of them specialized to a particular feature, look into your /etc/passwd file, see passwd(5)) and more than a hundred processes (each having its pid), use top(1) or ps(1) as ps auxw to list them.
I believe you have the wrong approach. Take first several days or weeks to understand more about Linux from the application point of view. So read some book about Linux programming, e.g. ALP or something newer. Read also something like: Operating Systems: Three Easy Pieces
Be aware that in practice, most Linux systems having a desktop environment are using some display server. So the (physical) keyboard is handled by the X11 or Wayland server. You need to read more about your display server (with X11, things like EWMH).
Hence, you need to be much more specific. You are likely to need to interact with the display server, not the kernel directly.
At last, a rule of thumb is to avoid bloating your kernel with extra and useless driver code. You very probably can do your thing in userland entirely.
So, spend a week or more reading about OSes & Linux before coding a single line of code. Avoid kernel modules, they will bite you, and you probably don't need them (but you might need to hack your display server or simply your window manager; of course details are different with X11 and with Wayland). Read also about multiseat Linux systems.
At last, most Linux distributions are made of free software, whose source code you can study. So take time to look into the source code of relevant software for your (ill-defined) goals. Use also strace(1) to understand the system calls dynamically done by commands and processes.
As I understood, the UNIX function read() will cause an interrupt(TRAP) and invoke the system call read. I also remembered that it has to switch to "Kernel Mode" before invoking the system call read and the switching is expensive..
I was wondering that why the read operation has to be delegated to system call in "Kernel Mode", instead of being done in "User Mode" completely.
For example, if there could be a service in "User Mode" which manages the access permissions of files, the read operation can just request this service, not disturbing the Kernel..
And for the disk driver, it is said in this link that
Device drivers can run in either user or kernel mode
Does anyone have ideas about this? Why does read have to be in Kernel Mode?
Is not the way Operating Systems are designed. The definition of OS is to handle the computers' hardware and to bring resources to their users. Operating Sysmtes also have the concept of user mode and kernel mode (as you said).
By having these concepts, OS define an specific line to what a user might do and what not. Letting them manage hardware is definitely something OS don't want users to do.
read usually involves a hardware access. Accessing hardware is cumbersome and error prone and can leave the computer in an unusable state. Operating System uses drivers to control the computer's hardware.
Issuing a read (assuming a hard disk IO) generally makes a driver to send a set of commands to the disk controller, read it's output, pass it to main memory, etc. This are dangerous operations that shouldn't be trusted to User Mode.
If there would be a service in user mode to handle this. Context switch still would be needed to be done, because the service would be running as another process.
Sure thing it can be done an Operating System that allows this. But modern operating systems aren't design to fulfill this behavior.
There are other approaches to building operating systems that relies on microkernels. A microkernel just do the minimum to get the pc started and leave everything else to other modules. Meaning that if a module crashes, the system still up. That's the case of specific drivers, filesystems, etc. I don't know if microkernels let these run in user space though.
Hope this helps!
First of all: it's no longer true that calling kernel is very expensive. It used to be when causing an exception/trap/fault/interrupt were the only way to switch from user mode to kernel mode in x86 systems, but that all changed with the addition of the systenter/sysexit machine code instructions, which perform a more lightweight transition.
Even if it is/were expensive, in terms of time consumed, system calls that deal with character and block device drivers should run in kernel mode because dealing with hardware devices involves reading and writting to hardware registers, which could be memory mapped or accessed thru I/O ports.
These registers must be protected from any access from userspace process. Not doing so may lead to any process to not to use the established API for reading a file, and directly use the hardware registers to read and write to the device. In the case of a disk with file, this would allow the userprocess to bypass the filesystem entirely, and hence, all the security and permission system.
So, if we need to protect these hardware registers so no user process can use them, code that does use them cannot run at the same priviledge level as any other user process. Hence, they run in another (more priviledged) mode, which is what is called "kernel mode".
Think on what would happen if you configure a Linux system so /dev/sda (usually the main harddisk in which the root filesystem lives) is read/write to anybody and everybody:
# chmod 666 /dev/sda
Having done this is more or less the equivalent of exposing the hard disk device to any user process. You can effectively write a program that could open, read, and write files stored within this device, but at the same time, you can write a program that open, read and write ANY files within the partition, no matter which permissions files have.
That said, there are cases in which a system runs only trusted applications. This kind of system doesn't need the level of protection that is present in a general purpose system, and hence it can benefit from the increased speed that comes by not depending on layers of APIs to isolate the process from the hardware. The most widely known example would be a videoconsole system. I recall that Windows CE used to run all its programs and device drivers at the same privilege too.
I was going through this tutorial about how to write a minimalist kernel. I read this in between :
The Run-Time Library
A major part of writing code for your OS is rewriting the run-time library, also known as libc. This is because
the RTL is the most OS-dependent part of the compiler package: the C
RTL provides enough functionality to allow you to write portable
programs, but the inner workings of the RTL are dependent on the OS in
use. In fact, compiler vendors often use different RTLs for the same
OS: Microsoft Visual C++ provides different libraries for the various
combinations of debug/multi-threaded/DLL, and the older MS-DOS
compilers offered run-time libraries for up to 6 different memory
models.
I am kind of confused with this part. Suppose I write my kernel in C code and against the advice use the inbuilt printf() function to print something. Finally my code will be translated to the machine code. When it will be executed, processer will directly run it. Why does the author says :
inner workings of the RTL are dependent on the OS in use ?
There are two separate issues:
What will printf() do when run inside of your kernel? Most likely it will crash or do nothing, since the RTL of the C compiler you use to develop your kernel is probably assuming some runtime environment with console, operating system, etc. Even if you're using a freestanding implementation of C/C++, the runtime will likely take over serial ports or whatnot to perform the output. You don't want that, probably, since your kernel's drivers will control the I/O. So you need to reimplement the underlying file I/O from the RTL.
What will printf() do when run in a user process that runs on top of your kernel? If the kernel protects access to hardware resources, it can't do anything. The underlying file I/O code from the RTL has to be aware of how to communicate with the kernel to open whatever passes for standard input/output "files" and to perform data exchange.
You need to be aware of whether you're using a free-standing or hosted implementation of the C/C++ compiler + RTL, and all of the implications. For kernel development, you'll be using a free-standing implementation. For userspace development, you'll want a hosted implementation, perhaps a cross-compiler, but the runtime library must be written as for a hosted implementation. Note that in both cases you can use the same compiler, you just need to point it to appropriate header files and libraries. On Linux, for example, kernel and userspace development can be done using the very same gcc compiler, with different headers and libraries.
The processor has no clue what a console is, or what a kernel is. Some code has to actually access the hardware. When you take printf() from a hosted C/C++ implementation, that implementation, somewhere deep in its guts, will invoke a system call for the particular platform it was meant to run on. That system call is meant to write to some abstraction that wraps the "console". On the other side of this system call is kernel code that will push this data to some hardware. It may not even be hardware directly, it may well be userspace of another process.
For example, whenever you run things in a GUI-based terminal on a Unix machine (KDE's Konsole, X11 xterm, OS X Terminal, etc.), the userland process invoking printf() has very, very far to go before anything hits hardware. Namely (even this is simplified!):
printf() writes data to a buffer
The buffer is flushed to (written to) a file handle. The write() library function is called.
The write() library function invokes a system call that transfers the control over to the kernel.
The kernel code copies the data from the userspace pages, since those can vanish at any time, to a kernel-side non-paged buffer.
The kernel code invokes the write handler for a given file handle - a file handle, in many kernels, is implemented as class with virtual methods.
The file handle happens to be a pseudo-terminal (pty) slave. The write method passes the data to the pty master.
The pty master fills up the read buffer of given pseudo-terminal, and wakes up the process waiting on the related file handle.
The process implementing the GUI terminal wakes up and read()s the file handle. This goes through the library to a syscall.
The kernel invokes the read handler for the pty master file handle.
The read handler copies its buffered data to the userspace.
The syscall returns.
The terminal process takes the data, parses it for control codes, and updates its internal data structure representing the emulated screen. It also queues an update event in the event queue. Control returns to the event loop of the GUI library/framework. This is done through an event since those events are usually coalesced. If there's a lot of data available, it will be all processed to update the screen data structure before anything gets repainted.
The event dispatcher dispatches the update/repaint event to the "screen" widget/window.
The event handler code in the widget/window uses the internal data structure to "paint" somewhere. Usually it'd be on a bitmap backing store.
The GUI library/framework code signals the operating system's graphics driver that new data is available on the backing store.
Again, through a syscall, the control is passed over to the kernel. The graphics driver running in the kernel will do the necessary magic on the graphics hardware to pass the backing bitmap to the screen. It may be an explicit memory copy, or a simple queuing of a texture copy with the graphics hardware.
Printf() is a high-level function that can be independent of the OS. It is however just part of the puzzle, it has dependencies itself. It needs to be able to write to stdout. Which will result in low-level OS dependent system calls, like create() to open the stdout stream and write() to send printf output there. Different OSes have different system calls so there's always an adaption layer, there will be in yours.
So sure, you can make printf() work in your kernel. Actually seeing the output of calls to printf() is going to be the real problem to solve. Nothing like a terminal window in kernel mode.
Im writing a custom device driver in linux that has to be able to respond very rapidly on interrupts. Code to handle this already exists in a user-space implementation but that is too slow as it relies on software constantly checking the state of the interrupt line. After doing some research, I found that you can register these interrupt lines from a kernel module, and execute a function given by a function pointer. However the code we want to execute is in the user-space, is there a way to call a function in the user-space from a kernel space module?
You are out of luck with invoking user-space functions from the kernel since the kernel doesn't and isn't supposed to know about individual user-space application functions and logic, not to mention that each user-space application has its own memory layout, that no other process nor the kernel is allowed to invade in that way (shared objects are the exception here, but still you can't tap into that from the kernel space). What about the security model, you aren't supposed to run user-space code (which is automatically considered unsafe code in the kernel context) in the kernel context in the first place since that will break the security model of a kernel right there in that instant. Now considering all of the above mentioned, plus many other motives you might want to reconsider your approach and focus on Kernel <-> User-space IPC and Interfaces, the file system or the user-mode helper API(read bellow).
You can invoke user space apps from the kernel though, that using the usermode-helper API. The following IBM DeveloperWorks article should get you started on using the usermode-helper Linux kernel API:
Kernel APIs, Part 1: Invoking user-space applications from the kernel
I think the easiest way is to register a character device which becomes ready when the device has some data.
Any process which tries to read from this device, then gets put to sleep until the device is ready, then woken up, at which point it can do the appropriate thing.
If you just want to signal readyness, a reader could just read a single null byte.
The userspace program would then just need to execute a blocking read() call, and would be blocked appropriately, until you wake it up.
You will need to understand the kernel scheduler's wait queue mechanism to use this.
Sounds like your interrupt line is already available to userspace via gpiolib? (/sys/class/gpio/...)
Have you benchmarked if gpio edge triggering and poll() is fast enough for you? That way you don't have to poll the status from the userspace application but edge triggering will report it via poll(). See Documentation/gpio.txt in kernel source.
If the edge triggering via sysfs is not good enough, then the proper way is to develop a kernel driver that takes care of the time critical part and exports the results to userspace via a API (sysfs, device node, etc).
I am also facing the same problem, I read this document http://people.ee.ethz.ch/~arkeller/linux/multi/kernel_user_space_howto-6.html, so planning to use signals. In my case there is no chance of losing signals, because
1. the system is closed loop, after signals executed then only I will get another signal.
2. And I am using POSIX real-time signals.