While trying to learn socket programming, I saw the following code:
int sock;
sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
I browsed through the man page and found that socket returns a file descriptor. I have tried searching the internet and other similar questions here but I couldn't understand what file descriptor really is. That would be great if someone could explain file descriptor in easy language.
There are two related objects: file descriptor and file description. People often confuse these two and think they are the same.
File descriptor is an integer in your application that refers to the file description in the kernel.
File description is the structure in the kernel that maintains the state of an open file (its current position, blocking/non-blocking, etc.). In Linux file descripion is struct file.
POSIX open():
The open() function shall establish the connection between a file and a file descriptor. It shall create an open file description that refers to a file and a file descriptor that refers to that open file description. The file descriptor is used by other I/O functions to refer to that file. The path argument points to a pathname naming the file.
The open() function shall return a file descriptor for the named file that is the lowest file descriptor not currently open for that process. The open file description is new, and therefore the file descriptor shall not share it with any other process in the system.
In Unix/ Linux operating systems, a file descriptor is an abstract indicator (handle) used to access a file or other IO(input/output) resource, such as a pipe or network socket.
Normally a file descriptors index into a per-process file descriptor table maintained by the kernel in Linux/Unix OS, that in turn indexes
into a system-wide table of files opened by all processes, called the file table.
This table records the "mode" with which the file or the other resource has been opened
for the following operations(There are more operations)
reading
writing
appending
writing
and possibly other modes.
It also indexes into a third table called the inode table that describes the actual underlying files.
File Descriptors are nothing but mappings to a file. You can also say these are pointers to a file that the process is using.
FDs are just integer values which act as pointers to process resources.
Whenever a process starts, an entry of the running process is added to the /proc/<pid> directory. This is the place where all of the data related to the process is kept. Also, on process start the kernel allocates 3 file-descriptors to the process for communication with the 3 data streams referred to as stdin, stdout and stderr.
the linux kernel uses an algorithm to always create a FD with the lowest possible integer value so these data-streams are mapped to the numbers 0, 1 and 2.
Let's say in you code you opened a file to read from or to write to. This means the process needs access to a resource and it has to create a mapping/pointer for this new resource.
To do this, the kernel automatically creates a FD as soon as the file is opened by your code.
If you run ls -l /proc/<pid>/fd/ you will se an additional FD created there with id 4 (can be some other number also if the program has used other resources)
I think of file descriptors as (indirect, higher-level) pointers to opaque file objects maintained by the kernel.
Normally, when you deal with objects maintained by a library, you pass to the library pointers to objects that you're not supposed to dereference and manipulate yourself.
For kernel objects, this it's not just that you're not supposed to manipulate them yourself -- you literally can't because they live in a different address space that's not at all accessible to you. And because they live in a different address space, pointers wouldn't be a meaningful way of referring to them.
You need a token or handle which the kernel would internally resolve to a pointer that's meaningful in the kernel address space. File descriptors are such tokens in integer form.
For the kernel:
your_process_id + your_file_descriptor => kernels_file_object_pointer
(or an EBADF error if a given filedescriptor may not be resolved to a file object pointer for the given process)
Related
Is it possible to check whether a file descriptor originated from a call to shm_open()? We already have isatty() that checks whether a file descriptor refers to a terminal. Is there something similar to know whether a file descriptor refers to a shared memory object?
Is it possible to check whether a file descriptor originated from a call to shm_open()? We already have isatty() that checks whether a file descriptor refers to a terminal. Is there something similar to know whether a file descriptor refers to a shared memory object?
Well, isatty() is not a system call, but just an ioctl(2) wrapper call that has to be made to a tty device (one device that supports it).
but you can use fstat(2) with the file descriptor to get the inode information. in the st_mode field of this structure there are bits that will tell you if it is a file, directory, socket, fifo, block device, char device, etc.
In you case, if you know the descriptor has been built with a call to shm_open, you have already answered your question. But try fstat(2) to see what does it return in those bits in your struct stat buffer.
Beware that what isatty(3) does is different than what you want to know. For isatty(3) to work you must pass not just a char device descriptor, but a char device that implements the ioctl call implemented in the tty driver, which for example will give you a false result when you use it with a magnetic tape descriptor.
In Solaris, I need to get IP address a specific process is using (sshd session), I have his ID.
How do they do it on linux ? After reading netstat.c source, this is the flow:
Iterate the process file descriptors, located at /proc/ProcessId/fd/,
If iterated file descriptor is a socket, they readlink, open and finally read the file descriptor.
So in solaris, I can detect the socket file descriptor of the process.
int fd=NULL;
struct dirent *dentp;
while ((dentp = readdir(dirp)) != NULL) { //iterate file descriptors
fd = atoi(dentp->d_name);
struct stat statb;
char temp_dir_path [100];
if (stat(temp_dir_path, &statb) != -1)
{
if (S_ISSOCK(statb.st_mode))
{
//What to do here ?? temp_dir_path is /proc/12345/fd/4
I saw there are methods like getpeername(..),getsockname(..) they receive as param the file descriptor of the current context process, I want to read file descriptor for another process.
Can I open the file descriptor and cast it to struct sockaddr_in ?
The socket file descriptor structure is different between linux and solaris.. I guess i need to do whatever they do in pfiles / lsof
I saw there are methods like getpeername(..),getsockname(..) they receive as param the file descriptor of the current context process, I want to read file descriptor for another process.
Can I open the file descriptor and cast it to struct sockaddr_in ?
No. You can open() it and use the file descriptor open() returns and try using getpeername() and getsockname() on the file descriptor you get. It might even work.
You'll probably be better served by using the method pfiles uses. Per the pfiles man page:
pfiles
Report fstat(2) and fcntl(2) information for all open files in
each process. For network endpoints, the local (and peer if
connected) address information is also provided. For sockets, the
socket type, socket options and send and receive buffer sizes are also
provided. In addition, a path to the file is reported if the
information is available from /proc/pid/path. This is not necessarily
the same name used to open the file. See proc(4) for more information.
The pfiles source code can be found at http://src.illumos.org/source/xref/illumos-gate/usr/src/cmd/ptools/pfiles/pfiles.c
Solaris provides a libproc interface library that does what you need. pfiles uses that - the library provides calls such as pr_getpeername() and pr_getsockname(). You can see the implementations in http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libproc/common/pr_getsockname.c
Note that there are actual system calls to get what you need directly from the kernel.
The OpenSolaris man pages for the libproc library can be found at http://illumos.org/man/3proc/all They are likely to be substantially similar to the Solaris 11 libproc implementation.
To use these tools, you have to be really careful. From the Pgrab man page for the function used to grab a process:
Grabbing a process is a destructive action. Stopping a process stops
execution of all its threads. The impact of stopping a process depends
on the purpose of that process. For example, if one stops a process
that's primarily doing computation, then its computation is delayed
the entire time that it is stopped. However, if instead this is an
active TCP server, then the accept backlog may fill causing connection
errors and potentially connection time out errors.
There are options to not stop the grabbed process, and to grab it read-only.
When using the open() function in C, I get a fd (file descriptor). I was wondering if it's the same thing as its process id, because as I know, fd is an integer.
No it is not.
PID is process identifier, and file descriptor is file handler identifier.
Specifically from Wikipedia about File Descriptors:
(...) file descriptor (FD) is an abstract indicator for accessing a file. The term is generally used in POSIX operating systems.
In POSIX, a file descriptor is an integer, specifically of the C type int. (...)
And for PID:
[PID] is a number used by most operating system kernels, — such as that of UNIX, Mac OS X or Microsoft Windows — to temporarily uniquely identify a process (...)
No, file descriptors are indices into the file table of your own process. They are always small integers (that is, up to the max-open-files limit for the process) because, among other things, the bitmap interface to select() wouldn't work if they were arbitrary numbers. On the other hand PIDs typically grow to at least 32767 before the wrap around.
An open file in general doesn't have a process ID of its own. And even in the case where one might arguably expect it to be connected to a particular process -- namely when the file handle comes from popen() -- there's no such direct connection and what goes on inside popen() is more complex than "treat this process as if it was a file".
No...
A file descriptor is an opaque handle that is used in the interface between user space and the kernel to identify file/socket resources. Therefore when you use open() or socket() (system calls to interface to the kernel) you are returned a file descriptor, which is an integer (it is actually an index into the processes u structure - but that is not important). Therefore if you want to interface directly with the kernel, using system calls to read(), write(), close() etc. the handle you use is a file descriptor.
A PID (i.e., process identification number) is an identification number that is automatically assigned to each process when it is created on a Unix-like operating system.A process is an executing (i.e., running) instance of a program. Each process is guaranteed a unique PID, which is always a non-negative integer.
One of the first things a UNIX programmer learns is that every running program starts with three files already opened:
Descriptive Name.............fd number...................... Description
Standard In 0 Input from the keyboard
Standard Out 1 Output to the console
Standard Error 2 Error output to the console
If you create any file descriptor, mostly you will get value as 3. Because 3 is least available +ve integer to allocate for fd. Because STDIN,STDOUT,STDERR are occupied with 0,1,2 respectively. That is why fd is called as smallest non negative integer.
My program does the following in chronological order
The program is started with root permissions.
Among other tasks, A file only readable with root permissions is open()ed.
Root privileges are dropped.
Child processes are spawned with clone() and the CLONE_FILES | CLONE_FS | CLONE_IO flags set, which means that while they use separate regions of virtual memory, they share the same file descriptor table (and other IO stuff).
All child processes execve() their own programs (the FD_CLOEXEC flag is not used).
The original program terminates.
Now I want every spawned program to read the contents of the aforementioned file, but after they all have read the file, I want it to be closed (for security reasons).
One possible solution I'm considering now is having a step 3a where the fd of the file is dup()licated once for every child process, and each child gets its own fd (as an argv). Then every child program would simply close() their fd, so that after all fds pointing to the file are close()d the "actual file" is closed.
But does it work that way? And is it safe to do this (i.e. is the file really closed)? If not, is there another/better method?
While using dup() as I suggested above is probably just fine, I've now --a day after asking this SO question-- realized that there is a nicer way to do this, at least from the point of view of thread safety.
All dup()licated file descriptors point to the same same file position indicator, which of course means you run into trouble when multiple threads/processes might simultaneously try to change the file position during read operations (even if your own code does so in a thread safe way, the same doesn't necessarily go for libraries you depend on).
So wait, why not just call open() multiple times (once for every child) on the needed file before dropping root? From the manual of open():
A call to open() creates a new open file description, an entry in the system-wide table of open files. This entry records the file offset and the file status flags (modifiable via the fcntl(2) F_SETFL operation). A file descriptor is a reference to one of these entries; this reference is unaffected if pathname is subsequently removed or modified to refer to a different file. The new open file description is initially not shared with any other process, but sharing may arise via fork(2).
Could be used like this:
int fds[CHILD_C];
for (int i = 0; i < CHILD_C; i++) {
fds[i] = open("/foo/bar", O_RDONLY);
// check for errors here
}
drop_privileges();
// etc
Then every child gets a reference to one of those fds through argv and does something like:
FILE *stream = fdopen(atoi(argv[FD_STRING_I]), "r")
read whatever needed from the stream
fclose(stream) (this also closes the underlying file descriptor)
Disclaimer: According to a bunch of tests I've run this is indeed safe and sound. I have however only tested open()ing with O_RDONLY. Using O_RDWR or O_WRONLY may or may not be safe.
I have two prgrams lets say prog1 and prog2. I am opening a file with prog1 and doing some operations
on it. Now without closing the file in prog1 i am sending its file descriptor to prog2 using unix
sockets which then does some operations in it.
Though i get the same descriptor i passed in prog1 but doing a fstat() on the fd recieved in prog2
throws an error saying Bad file descriptor. I have opened the file in prog1 with corerct permissions
that is read and write for all, still i get an error.
Why is it happening so. If my way of passing a file descriptor is wrong then please suggest a correct
one.
I believe this site has what you're looking for:
http://www.lst.de/~okir/blackhats/node121.html
There's also information in Linux's man 7 unix on using SCM_RIGHTS and other features of Unix sockets.
Fix for broken link: http://web.archive.org/web/20131016032959/http://www.lst.de/~okir/blackhats/node121.html
This is normal. Each program has its own file descriptors.
EDIT: Well, it seems that you can pass file descriptor using local socket.
You can see them in /proc/PID/fd, they are often symlinks to your files. What you can do with unix socket is allowing a write to a file from one program to another with sendmsg/recvmsg. See this question for more detail.
But there's way many better way to write concurrently to a file. You can use fifo, shm or even just pass your offset position between your 2 programs.
A file descriptor is a small int value that lets you access a file. It's an index into a file descriptor table, a data structure in the kernel that's associated with each individual process. A process cannot do anything meaningful with a file descriptor from another process, since it has no access to any other process's file descriptor table.
This is for basic security reasons. If one process were able to perform operations on an open file belonging to another process, chaos would ensue. Also, a file descriptor just doesn't contain enough information to do what you're trying to do; one process's file descriptor 0 (stdin) might refer to a completely different file than another process's file descriptor 0. And even if they happen to be the same file, each process needs to maintain its own information about the state of that open file (how much it's read/written, etc.).
If you'll describe what you're trying to accomplish, perhaps we can help.
EDIT :
You want to pass data from one program to another. The most straightforward way to do this is to create a pipe (man 2 pipe). Note that the second process will have to be a child of the first.
An alternative might be to create a file that the second process can open and read (not trying to share a file descriptor), or you might use sockets.