Should a function to change the priority of the calling process (eg: nice()) be implemented as a library call or as a system call? I was reading about it online and from what I understood, it used to be a system call but it's a library call now. Why so?
All functions are library calls. The question you're looking at is a sloppy shorthand for the question of whether or not there's a syscall that corresponds directly to the semantics, so that the library function can do something trivial along the lines of return syscall(SYS_foo, ...);.
Often it happens that at some point in history there was a syscall that corresponded fully to the function's operations at that point in history, but either:
new requirements on the function rendered the existing syscall incapable of meeting the function's needs, or
a bug was discovered in the syscall that's fundamental to its interface
In either case, if there's a reasonable way to implement the operation entirely with other syscalls, that usually makes more sense than implementing a "v2" of the syscall. Moreover, even if a new syscall is added, it's often added with further generality than the old interface needed, for extensibility or to be useful for satisfying other existing needs. Therefore it might not correspond directly to the function being implemented, just provide a means to obtain the functionality.
Related
I've heard filldir()/filldir64() lets the kernel specify the "dirent" (directory entry) layout depending on the binary type. Why do differing binary types matter?
I am also confused about dir_context, I assume filldir64() can be defined by other filesystems and implemented differently? So is that why there is an actor member?
filldir64(): https://elixir.bootlin.com/linux/latest/source/fs/readdir.c#L307
dir_context: https://elixir.bootlin.com/linux/latest/source/include/linux/fs.h#L1776
The reason for this is to run old binaries. The ABI of the readdir() system call has changed a few times. The way this works is each system call number calls a different function. There's a common implementation of the filldir loop that dispatches to the filesystem-specific calls to read directory entires, and again dispatches to the appropriate filldir() implementation.
dir_context is defined by the specific implementation of filldir() to retain some state between calls. Desipite being written in C, the filesystem is object-oriented code. When you see stuff like this, start thinking virtual method dispatch, becasue effectively that's what it is. But since it's done in C, the this pointer has to be passed manually as the first argument.
I was surprised to discover the man pages having entries for two conflicting variants of readdir.
in READDIR(2), it specifically states you do not want to use it:
This is not the function you are interested in. Look at readdir(3) for the POSIX conforming C library interface. This page documents the bare kernel system call interface, which is superseded by getdents(2).
I understand a function may become deprecated when another function comes along and does its job better, but I am not familiar with other cases of a userspace function coming in and replacing a kernel function of the same name. Is there a known reason it was chosen to go this route rather than coming up with a new function name (as the man page mentions getdents did when superseding readdir).
The programming interface, POSIX, is stable. You don't just go replacing functions in it unnecessarily because you want to implement the backend more efficiently. The Linux syscall readdir never implemented the readdir function because it has the wrong signature; it was an old, inefficient backend for implementing the readdir function. When a better backend came along, it was obsolete.
You have it completely backwards: it's the library function readdir(3) which predates Linux and its readdir(2) system call, and not the reverse.
Naming the syscall that way was certainly a poor decision, and probably has a story behind it, but it's pretty much irrelevant now, as nobody is using it.
On Unix, directories used to be simple files formatted in a special way, and the system call interface through which they were read was just read(2) [1]. Later systems introduced system calls like getdirentries (44BSD) and getdents (SVR3), but they weren't willing or capable to standardize on an interface, so we're still stuck with the high level and broken [2] readdir(3) library function as the only standard interface for reading a directory.
[1] On some systems like BSD you can still cat a directory, at least when using the default filesystem (FFS).
[2] it's broken because it's not signal safe, and it returns NULL for both error and EOF, which means that the only way it could be safely used is by first setting errno to 0, and checking both its return value and errno afterwards. Yuck.
How does one call Go code in C from threads that weren't created by Go?
What do I assign to a C function pointer such that threads not created by Go can call that pointer and enter into Go code?
Update0
I don't want to use SWIG.
The callbacks will be coming from threads Go hasn't seen before. Neither cgo/life nor anything in pkg/runtime demonstrates this behaviour AFAICT.
You can do this, but the solution is relatively slow (about 22µs per call on my machine).
The answer is for the C code to use C thread primitives to communicate with another goroutine that will actually run the callback.
I have created a Go package that provides this functionality: rog-go.googlecode.com/hg/exp/callback.
There is an example package demonstrating its use here. The example demonstrates a call back to an arbitrary Go closure from a thread created outside of the Go runtime. Another example is here. This demonstrates a typical C callback interface and layers a Go callback on top of it.
To try out the first example:
goinstall rog-go.googlecode.com/hg/exp/example/looper
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/looper
gotest
To try out the second example:
goinstall rog-go.googlecode.com/hg/exp/example/event
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/event
gotest
Both examples assume that pthreads are available. Of course, this is just a stop-gap measure until cgo is fixed, but the technique for calling arbitrary Go closures in a C callback will be applicable even then.
Here is the documentation for the callback package:
PACKAGE
package callback
import "rog-go.googlecode.com/hg/exp/callback"
VARIABLES
var Func = callbackFunc
Func holds a pointer to the C callback function.
When called, it calls the provided function f in a
a Go context with the given argument.
It can be used by first converting it to a function pointer
and then calling from C.
Here is an example that sets up the callback function:
//static void (*callback)(void (*f)(void*), void *arg);
//void setCallback(void *c){
// callback = c;
//}
import "C"
import "rog-go.googlecode.com/hg/exp/callback"
func init() {
C.setCallback(callback.Func)
}
I'll assume you mean from C code compiled with gcc?
IIRC, this either can't be done or can't easily be done using 6g+cgo and friends. Go uses a different calling convention (as well as the segmented stacks and such).
However, you can write C code for [685]c (or even [685]a) and call into go easily using package·function() (you can even call methods IIRC). See the Source of the runtime package for examples.
Update:
Coming back to this question after the update, and giving it some more thought. This can't be done in a standard fashion using 6c or cgo. Especially because the threads are not started by the go runtime, the current implementation would fail. The scheduler would suddenly have a thread under its control that it does not know about; additionally, that thread would be missing some thread-local variables the go runtime uses for managing stacks and some other things. Also, if the go function returns a value (or several) the C code can't access it on the currently supported platforms, as go returns values on the stack (you could access them with assembly though). With these things in mind, I do believe you could still do this using channels. It would require your C code to be a little too intimate with the inner workings of the go runtime, but it would work for a given implementation. While using channels may not be the solution you're looking for, it could possibly fit more nicely with the concepts of Go than callbacks. If your C code reimplemented at least the sending methods in The channel implementation (that code is written for 6c, so it would have to be adapted for gcc most likely, and it calls the go runtime, which we've determined can't be done from a non-go thread), you should be able to lock the channel and push a value to it. The go scheduler can continue to manage it's own threads, but now it can receive data from other threads started in C.
Admittedly, it's a hack; I haven't looked close enough, but it would probably take a few other hacks to get it working (I believe the channels themselves maintain a list of the goroutines that are waiting on them [EDIT: confirmed: runtime·ready(gp);], so you'd need something in your go code to wake up the receiving channel or to warranty the go code won't receive on the channel until you've already pushed a value). However, I can't see any reason this can't work, whereas there are definite reasons that running code generated by 6g on a thread created in C can't.
My original answer still holds though: barring an addition to the language or runtime, this can't yet be done the way you'd like (I'd love to be proven wrong here).
You can find a real-world application of rog's callback package in these bindings for the PortAudio audio I/O library: http://code.google.com/p/portaudio-go/. Might make it easier to understand..
(Thanks for implementing that, rog. It's just what I needed!)
In this SO thread, Brian Postow suggested a solution involving fake anonymous functions:
make a comp(L) function that returns the version of comp for arrays of length L... that way L becomes a parameter, not a global
How do I implement such a function?
See the answer I just posted to that question. You can use the callback(3) library to generate new functions at runtime. It's not standards compliant, since it involves lots of ugly platform-specific hacks, but it does work on a large number of systems.
The library takes care of allocating memory, making sure that memory is executable, and flushing the instruction cache if necessary, in order to ensure that code which is dynamically generated (i.e. the closure) is executable. It essentially generates stubs of code that might look like this on x86:
pop %ecx
push $THUNK
push %ecx
jmp $function
THUNK:
.long $parameter
And then returns the address of the first instruction. What this stub does is stores the the return address into ECX (a scratch register in the x86 calling convention), pushes an extra parameter onto the stack (a pointer to a thunk), and then re-pushes the return address. Then, it jumps to the actual function. This results in the function getting fooled into thinking it has an extra parameter, which is the hidden context of the closure.
It's actually more complicated than that (the actual function called at the end of the stub is __vacall_r, not the function itself, and __vacall_r() handles more implementation details), but that's the basic principle.
I don't believe you can do that with C99 -- there's no partial application or closure facility available unless you start manually generating machine code at runtime.
Apple's recently proposed blocks would work, though you need compiler support for that. Here's a brief overview of blocks. I have no idea when/if any vendor outside apple will support them.
It is not possible to generate ordinary functions during run-time in either C or C++. What Brian suggested is based on a big "if": "...if you can fake anonymous functions...". And the answer to that "if" is: no, you can't. (Although it is not clear what he meant by "fake".)
(In C++ it is possible to generate function-like objects at run time, but not ordinary functions.)
The above applies to standard C and C++ languages. Particular implementations can support various implementation-provided extensions and/or manually-implemented hacks, like "closures", "delegates" and similar stuff. Nothing of that, of course, have anything to do with standard C/C++ languages.
I have a function which is called explicitly by 4 other functions in my code base. Then in turn each of these functions is called by at least 10 other functions throughout my code. I know that I could, by hand, trace one of these function calls to the main function of my program (which has 30 function calls) but it seems like this would be a better job for the computer. I just want to know which of the functions in main() is calling this buried function.
Does anyone know of any software that could help?
Also, using a debugger is out of the question. That would have been too easy. The software only runs on a hand held device.
doxygen, correctly configured, is able to output an HTML document with navigable caller list and called-by list for every function in your code. You can generate call graphs as well.
Comment it out (or better, comment out its prototype) and try to compile your program. You should see, where it is referenced.
If your platform has an API to capture backtraces, I would just instrument up the function to use those and log them to a file for later analysis. There's no guarantee that this will find all callers (or callers-of-...-of-callers), but if you exercise all of the programs features while logging like this, you should find "most" of them. For relatively simple programs, it is possible to find all callers this way.
Alternatively, many sampling tools can get you this information.
However, I have a suspicion that you may be on a platform that doesn't have a lot of these features, so a static source-analysis tool (like mouviciel suggested) is likely your best option. Assuming that you can make it work for you, this has the added benefit that it should find all callers, not just most of them.
http://cscope.sourceforge.net/ I think this also can be useful.
I second mouviciel's suggestion of using doxygen for getting this info. The downside is that doxygen is working on the source code. You can only see what functions CAN POTENTIALLY call your function, not the ones that are ACTUALLY CALLING your function. If you are using Linux and you can change the source code of the function in question, you can obtain this info using the backtrace() and the backtrace_symbols() functions.