I understand file descriptors are kernel handle to identify the file , while inode number of a file is pointer to a structure which has other details about file(Correct me if I am wrong). But I am unable to get the difference between them.
An inode is an artifact of a particular file-system and how it manages indirection. A "traditional *ix" file-system uses this to link together files into directories, and even multiple parts of a file together. That is, an inode represents a physical manifestation of the file-system implementation.
On the other hand, a file descriptor is an opaque identifier to an open file by the Kernel. As long as the file remains open that identifier can be used to perform operations such as reading and writing. The usage of "file" here is not to be confused with a general "file on a disk" - rather a file in this context represents a stream and operations which can be performed upon it, regardless of the source.
A file descriptor is not related to an inode, except as such may be used internally by particular [file-system] driver.
The difference is not substantial, both are related to the abstract term called "file". An inode is a filesystem structure that represents files. Whereas, a file descriptor is an integer returned by open syscall. By definition:
Files are represented by inodes. The inode of a file is a structure kept by the filesystem which holds information about a file, like its type, owner, permissions, inode links count and so on.
On other the hand, a file descriptor
File Descriptors:
The value returned by an open call is termed a file descriptor and is essentially an index into an array of open files kept by the kernel.
The kernel doesn't represent open files by their names, instead it uses an array of entries for open files for every process, so a file descriptor in effect is an index into an array of open files. For example, let's assume you're doing the following operation in a process:
read(0, 10)
0 denotes the file descriptor number, and 10 to read 10 bytes. In this case, the process requests 10 bytes from the file/stream in index 0, this is stdin. The kernel automatically grants each process three open streams:
Descriptor No.
0 ---> stdin
1 ---> stdout
2 ---> stderr
These descriptors are given to you for free by the kernel.
Now, when you open a file, in the process via open("/home/myname/file.txt") syscall, you'll have index 3 for the newly opened file, you open another file, you get index 4 and so forth. These are the descriptors of the opened files in the process:
Descriptor No.
0 ---> stdin
1 ---> stdout
2 ---> stderr
3 ---> /home/user100/out.txt
4 ---> /home/user100/file.txt
See OPEN(2) it explains what goes underneath the surface when you call open.
The fundamental difference is that an inode represents a file while a file descriptor (fd) represents a ticket to access the file, with limited permission and time window. You can think an inode as kind of complex ID of the file. Each file object has a unique inode. On the other hand, a file descriptor is an "opened" file by a particular user. The user program is not aware of the file's inode. It uses the fd to access the file. Depending on the user's permissions and the mode the user program choses to open the file (read-only for example) a fd is allowed a certain set of operations on the file. Once the fd is "closed" the user program can't access the file unless it opens another fd. At any given time, there can be multiple fds accessing a file in the same or different user programs.
Related
How can I obtain the file descriptor flags 'fd flags' residing in the File Descriptor Table associated with any open file descriptor? I would like to know to write the code manually or at least what functions and structures to research further while not using the fcntl function or viewing the /proc entries.
The image below shows that the file descriptor table for a given process has a flags field and a file pointer field. How can I programatically access these fields?
You can access /proc/PID/fdinfo/FD file which contains flags in octal:
$ cat /proc/$$/fdinfo/0
pos: 0
flags: 0100002
mnt_id: 20
In this example, 0100002 = O_LARGFILE|O_RDWR
fcntl() is a system call, not "a library". It is the way you retrieve the flags associated with a file descriptor.
In it's main page it says:
fchdir() is identical to chdir(); the only difference is that the
directory is given as an open file descriptor.
And the prototype is given as following:
int chdir(const char *path);
int fchdir(int fd);
My question is how can a directory be passed as a file descriptor? Do directories also have a corresponding descriptor like files?
Do directories also have a corresponding descriptor like files?
Yes. The Unix philosophy(and Linux) is to treat everything as a stream of bytes. So yes, you can do open(2) on a directory and get its file descriptor.
Not only directories but sockets, pipes and devices can also be opened using open(2) system call and do operations on it as though it's a file.
As you can read on the Opengroup documentation for fchdir,
A conforming application can obtain a file descriptor for a file of type directory using open(), provided that the file status flags and access modes do not contain O_WRONLY or O_RDWR.
So by calling open on a directory, one obtains a file descriptor to a directory. So in a sense, yes, all directories have file descriptors.
The underlying OS takes care to map the file descriptors to the proper filesystem object, thus abstracting away whatever this object may be at the lowest level.
I have an application that creates multiple instances (processes) of itself and these processes have a shared data structure. In that struct there is a file descriptor used for logging data to file. There is a check in the logging function that checks to see if the file descriptor is -1 and if it is then it opens the file and sets the value of the shared file descriptor.
Other processes / threads do the same check but at this time the fd is != -1. So the file does not get opened. They then continue to writing to the file. The write fails most of the time and returns -1. When the write did not fail I checked the file path of the fd using readlink. The path was some other file than the log file.
I am assuming that this is because even though the file descriptor value was always 11, even in subsequent runs, that value refers to a different file for each process. So it is the eleventh file that process has open? So the log file is not even regarded as open for these processes and even if they do open the file the fd would be different.
So my question is this correct? My second question is how do I then re-implement this method given that multiple processes need to write to this log file. Would each process need to open that file.. or is there another way that is more efficient.. do I need to close the file so that other processes can open and write to it..?
EDIT:
The software is an open source software called filebench.
The file can be seen here.
Log method is filebench_log. Line 204 is the first check I mentioned where the file is opened. The write happens at line 293. The fd value is eleven among all processes and the value is the same: 11. It is actually shared through all processes and setup mostly here. The file is only opened once (verified via print statements).
The shared data struct that has the fd is called
filebench_shm
and the fd is
filebench_shm->shm_log_fd
EDIT 2:
The error message that I get is Bad file descriptor. Errno is 9.
EDIT 3:
So it seems that each process has a different index table for the fds. Wiki:
On Linux, the set of file descriptors open in a process can be accessed under the path /proc/PID/fd/, where PID is the process identifier.
So the issue that I am having is that for two processes with process IDs 101, 102 the file descriptor 11 is not the same for the two processes:
/proc/101/fd/11
/proc/102/fd/11
I have a shared data structure between these processes.. is there another way I can share an open file between them other than an fd since that doesn't work?
It seems that it would be simplest to open the file before spawning the new processes. This avoids all the coordination complexity regarding opening the file by centralizing it to one time and place.
I originally wrote this as a solution:
Create a shared memory segment.
Put the file descriptor variable in the segment.
Put a mutex semaphore in the segment
Each process accesses the file descriptor in the segment. If it is not open, lock the semaphore, check if it is open, and if not open the
file. Release the mutex.
That way all processes share the same file descriptor.
But this assumes that the the underlying file descriptor object is also in the shared memory, which I think it is not.
Instead, use the open then fork method mentioned in the other answer, or have each process open the file and use flock to serialize access when needed.
I am confused with the concept of file position as used in lseek. Is this file position maintained at inode level or a simple variable which could have different values for different process working on the same file?
Per the lseek docs, the file position is associated with the open file pointed to by a file descriptor, i.e. the thing that is handed to your by open. Because of functions like dup and fork, multiple descriptors can point to a single description, but it's the description that holds the location cursor.
Think about it: if it were associated with the inode, then you would not be able to have multiple processes accessing a file in a sensible manner, since all accesses to that file by one process would affect other processes.
Thus, a single process could have track many different file positions as it has file descriptors for a given file.
It's not an 'inode', but FILEHANDLE inside kernel.
Inode is a part of file description of the *nix specific file system on the disk. FAT32, for example, has no inodes, but supported by Linux.
For know the relation between file descriptors and open files, we need to examine three data structures.
the per-process file descriptor table
the system wide table of open file descriptors
the file system i-node table.
For each process kernel maintains a table of open file descriptors. Each entry in this table records information about a single file descriptor including
a set of flags controlling the operation of the file descriptor.
a reference to the open file description
The kernel maintains a system wide table of all open file descriptors. An open file description stores all information related to an open file including:
the current file offset(as updated by read() and write(),or explicitly modified using lseek())
status flags specified when opening the file.
the file access mode (read-only,write only,or read-write,as specified in open())
setting relating to signal-driven I/O and
a reference to i-node object for this file.
Reference-Page 94,The Linux Programming Interface by Michael Kerrisk
I have a process using C on Linux OS that writes data to a file. It uses open()/write() functions and I've been wondering if another process rm'd or mv'd the file. How can my process find out and recreate the file?
You can use fstat() to get the information about the open file. If the st_nlink field is zero, the file has been removed from the file system (possibly by being moved to a different file system, but there's no real way for you to determine that). There's a decent chance you have the only remaining reference to that file - though there might be other processes also holding it open. The disk space won't be released until the last process with an open file descriptor for the file finally closes the file.
If the st_nlink field is still positive, then your file still has a name somewhere out in the file system. You then need to use stat() to determine whether the st_dev and st_ino fields for the given file name match the same fields from the file descriptor. If the name still exists and has the same device and inode number, then it is 'the same' file (though the contents may have changed). If there's a difference, then the open file is different from the file specified by name.
Note that if you want to be sure that the given name is not a symbolic link to a moved copy of the file, then you would have to use lstat() on the file when you open it (to ensure it isn't a symlink at that point), and again when you check the file (instead of using stat()).
You can use the stat call to do this.
struct stat st;
if(stat("/tmp",&st) == 0)
printf(" /tmp is present\n");
else
/* Write code to create the file */