golang File struct is like this:
type File struct{
*file
}
and File struct functiona is also design to recive a pointer,why it design like this?
It is explained in the Go os package source code comments.
For example, this is safe:
package main
import "os"
func main() {
f, err := os.Create("/tmp/atestfile")
if err != nil {
*f = os.File{}
}
// finalizer runs
}
Package os
go/src/os/types.go:
// File represents an open file descriptor.
type File struct {
*file // os specific
}
go/src/os/file_plan9.go:
// file is the real representation of *File.
// The extra level of indirection ensures that no clients of os
// can overwrite this data, which could cause the finalizer
// to close the wrong file descriptor.
type file struct {
fd int
name string
dirinfo *dirInfo // nil unless directory being read
}
go/src/os/file_unix.go:
// +build darwin dragonfly freebsd linux nacl netbsd openbsd solaris
// file is the real representation of *File.
// The extra level of indirection ensures that no clients of os
// can overwrite this data, which could cause the finalizer
// to close the wrong file descriptor.
type file struct {
pfd poll.FD
name string
dirinfo *dirInfo // nil unless directory being read
nonblock bool // whether we set nonblocking mode
}
go/src/os/file_windows.go:
// file is the real representation of *File.
// The extra level of indirection ensures that no clients of os
// can overwrite this data, which could cause the finalizer
// to close the wrong file descriptor.
type file struct {
pfd poll.FD
name string
dirinfo *dirInfo // nil unless directory being read
}
Related
In my code, I create a file with a random name using mkstemp() function (Im on Linux). What this function returns is an int being a file descriptor.
int fd;
char temp[] = "tempXXXXXX";
fd = mkstemp(temp);
Later I can access the file using fdopen() through that int file descriptor.
FILE *file_ptr = NULL;
file_ptr = fdopen(fd);
But at the end of my program, I would like to see if the file still exists with the random name it was given when I created it (the program should change that file name if successful). I can set a flag if the rename() function run on that file is successful, but I still don't know how to delete it when I only have its file descriptor.
if rename files => remove the temp file
How can I do that? Or is there a way to get the files name if I have its file descriptor?
Neither C nor POSIX (since you are using POSIX library functions) defines a way to delete a file via an open file descriptor. And that makes sense, because the kind of deletion you're talking about is actually to remove a directory entry, not the file itself. The same file can be hard linked into the directory tree in multiple places, with multiple names. The OS takes care of removing its data from storage, or at least marking it as available for reuse, after the last hard link to it is removed from the directory tree and no process any longer has it open.
A file descriptor is associated directly with a file, not with any particular path, notwithstanding the fact that under many circumstances, you obtain one via a path. This has several consequences, among them that once a process opens a file, that file cannot be pulled out from under it by manipulating the directory tree. And that is the basis for one of the standard approaches to your problem: unlink (delete) it immediately after opening it, before losing its name. Example:
#include <stdlib.h>
#include <unistd.h>
int make_temp_file() {
char filename[] = "my_temp_file_XXXXXX";
int fd;
fd = mkstemp(filename);
if (fd == -1) {
// handle failure to open ...
} else {
// file successfully opened, now unlink it
int result = unlink(filename);
// ... check for and handle error conditions ...
}
return fd;
}
Not only does that (nearly) ensure that the temp file does not outlive the need for it, but it also prevents the contents from being accessible to users and processes to which the owning process does not explicitly grant access.
Even though this doesn't exactly answer the question you're asking about mkstemp, consider creating a temporary file that will automatically be deleted, unless you rename it.
Instead of mkstemp you could call open combined with the creation flag O_TMPFILE to create a temporary, unnamed file that is automatically deleted when file is closed.
See open(2):
O_TMPFILE (since Linux 3.11)
Create an unnamed temporary regular file. The pathname argu‐
ment specifies a directory; an unnamed inode will be created
in that directory's filesystem. Anything written to the
resulting file will be lost when the last file descriptor is
closed, unless the file is given a name.
Instead of a filename, you call open with the path where you prefer to place the temporary file, like:
temp_fd = open("/path/to/dir", O_TMPFILE | O_RDWR, S_IRUSR | S_IWUSR);
If you like to give the temporary file a permanent location/name, you can call linkat on it later:
linkat(temp_fd, NULL, AT_FDCWD, "/path/for/file", AT_EMPTY_PATH);
Note: Filesystem support is required for O_TMPFILE, but mainstream Linux filesystems do support it.
readlink provide you the name of your file depending of the file descriptor if you use the path /proc/self/fd/ adding you fd.
Then use remove for deleting the file passing the name readlink gave you
ssize_t readlink(const char *path, char *buf, size_t bufsiz); (also load ernno)
int remove(const char *filename); (returns zero is successful, otherwise nonzero)
I hope something like that could helped you ?
⚠ Don't copy/past this you must edit "filename"; _BUFFER, _BUFSIZE ⚠
#include<stdio.h>
#include <unistd.h>
#include <stdlib.h>
int delete_file(int fd) {
char *str_fd = itoa(fd, str_fd, 10);
char *path = strcat("/proc/self/fd/", str_fd);
if (read_link(path, buffer, bufsize) == -1)
return -1;
int del = remove(filename);
if (!del)
printf("The file is Deleted successfully");
else
printf("The file is not Deleted");
return 0;
}
(feel free to edit this, i didn't test the code and i let you handel the buffer and buffer size)
I need to concurrently read from a file in different offsets using C.
dup unforunately creates a file descriptor that shares offset and flags with the original.
Is there a function like dup that does not share the offset and flags?
EDIT I only have access to the file pointer FILE* fp; I do not have the file path
EDIT This program is compiled for windows in addition to mac and many flavors of linux
SOLUTION
We can use pread on posix systems, and I wrote a pread function for windows which solves this problem
https://github.com/Storj/libstorj/blob/master/src/utils.c#L227
On Linux, you can recover the filename from /proc/self/fd/N, where N is the integral value of the file descriptor:
sprintf( linkname, "/proc/self/fd/%d", fd );
Then use readlink() on the resulting link name.
If the file has been renamed or deleted, you may be out of luck.
But why do you need another file descriptor? You can use pread() and/or pwrite() on the original file descriptor to read/write from/to the file without affecting the current offset. (caveat: on Linux, pwrite() to a file opened in append mode is buggy - POSIX states that pwrite() to a file opened in append mode will write to the offset specified in the pwrite() call, but the Linux pwrite() implementation is broken and will ignore the offset and append the data to the end of the file - see the BUGS section of the Linux man page)
No, neither C nor POSIX (since you mention dup()) has a function for opening a new, independent file handle based on an existing file handle. As you observed, you can dup() a file descriptor, but the result refers to the same underlying open file description.
To get an independent handle, you need to open() or fopen() the same path (which is possible only if the FILE refers to an object accessible through the file system). If you don't know what path that is, or if there isn't any in the first place, then you'll need a different approach.
Some alternatives to consider:
buffer some or all of the file contents in memory, and read as needed from the buffer to serve your needs for independent file offsets;
build an internal equivalent of the tee command; this will probably require a second thread, and you'll probably not be able to read one file too far ahead of the other, or to seek in either one;
copy the file contents to a temp file with a known name, and open that as many times as you want;
if the FILE corresponds to a regular file, map it into memory and access its contents there. The POSIX function fmemopen() could be useful in this case to adapt the memory mapping to your existing stream-based usage.
On windows (assuming VisualStudio), you can get access to the OS file handle from the stdio FILE handle.
From there, reopen it and convert back to a new FILE handle.
This is windows only, but I think Andrews answer will work for Linux and probably the Mac as well - unfortunately there is no portable way to have it work on all systems.
#include <Windows.h>
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
FILE *jreopen(FILE* f)
{
int n = _fileno(f);
HANDLE h = (HANDLE)_get_osfhandle(n);
HANDLE h2 = ReOpenFile(h, GENERIC_READ, FILE_SHARE_READ, 0);
int n2 = _open_osfhandle((intptr_t)h2, _O_RDONLY);
FILE* g = _fdopen(n2, "r");
return g;
}
I was able to use pread and pwrite on POSIX systems, and I wrapped ReadFile/WriteFile on Windows Systems into pread and pwrite functions
#ifdef _WIN32
ssize_t pread(int fd, void *buf, size_t count, uint64_t offset)
{
long unsigned int read_bytes = 0;
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
overlapped.OffsetHigh = (uint32_t)((offset & 0xFFFFFFFF00000000LL) >> 32);
overlapped.Offset = (uint32_t)(offset & 0xFFFFFFFFLL);
HANDLE file = (HANDLE)_get_osfhandle(fd);
SetLastError(0);
bool RF = ReadFile(file, buf, count, &read_bytes, &overlapped);
// For some reason it errors when it hits end of file so we don't want to check that
if ((RF == 0) && GetLastError() != ERROR_HANDLE_EOF) {
errno = GetLastError();
// printf ("Error reading file : %d\n", GetLastError());
return -1;
}
return read_bytes;
}
ssize_t pwrite(int fd, const void *buf, size_t count, uint64_t offset)
{
long unsigned int written_bytes = 0;
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
overlapped.OffsetHigh = (uint32_t)((offset & 0xFFFFFFFF00000000LL) >> 32);
overlapped.Offset = (uint32_t)(offset & 0xFFFFFFFFLL);
HANDLE file = (HANDLE)_get_osfhandle(fd);
SetLastError(0);
bool RF = WriteFile(file, buf, count, &written_bytes, &overlapped);
if ((RF == 0)) {
errno = GetLastError();
// printf ("Error reading file :%d\n", GetLastError());
return -1;
}
return written_bytes;
}
#endif
I need to fill a buffer space with file descriptors of files from a defined source directory. So I have the startup code:
int main(int argc, char* argv[])
{
DIR *src=opendir(argv[1]);
struct dirent *DirEntry;
char* buffer[200];
do {
DirEntry = readdir(src);
if(DirEntry != NULL) {
//put file into buffer
}
}while(DirEntry!=NULL);
}
How do I complete this loop to place all file descriptors of a given directory into the array called 'buffer'? Should I use an object of DirEntry like DirEntry->d_name to return a file descriptor that I then put into the array?
If you need to move files from a source directory to a destination directory, you are going to need file names much more than you need file descriptors. With the names, you can open and close descriptors whenever needed; without the names, you can't create the files in the target directory sensibly. However, we can handle file descriptors too.
So, assuming you have strdup(), you might use:
typedef struct File
{
char *name;
int fd;
} File;
And in your loop:
if (DirEntry != NULL)
{
buffer[i].name = strdup(DirEntry->d_name);
if (buffer[i].name != 0)
buffer[i].fd = open(buffer[i].name, O_RDONLY);
i++;
}
where buffer is an array of File and i is a convenient integer:
enum { MAX_FILES = 4096 };
int i;
File buffer[MAX_FILES];
You should also add a condition to the main condition to ensure no overflow (or replace the fixed size buffer with a dynamically allocated one):
if (DirEntry != NULL && i < MAX_FILES)
You could sensibly break the loop if i reaches the limit. You could test whether the name represents a file (as opposed to FIFO, block device, character device, socket, symlink, directory, ...); you'd probably use stat() or lstat() for that. The file descriptor would be negative (-1) if the open() call failed. You might conserve entries by not incrementing i if the memory allocation fails, but it is probably not worth worrying about. If the memory allocation for a file name fails, there isn't going to be much else that works.
A file descriptor is "int" typed value that returned by open() system call. It's not in DirEntry structure. So, firstly, you should define the buffer array as int type, then you can open each file in the loop with open() system call, and save open() returned file descriptor in buffer array.
You question doesn't quite make sense. If you want file descriptors you have to open the file or directory with open() to get a file descriptor.
If you just want to store the names of the files in an array, then you can create a two dimensional array using malloc or calloc and copy the d_name member to the next available slot in the array. Alternatively you can use what you have and use a function such as strdup to copy the string to your array, but be careful because you'll need to free it later using free().
If you actually do want file descriptors, you will need to create an array of int rather than char *.
I have probably pointer problem with fp, cause I get (SIGSEGV) error.
But I'm not much experienced in C and I doesn't see it.
What I'm trying to do. I do Server application for simple game, I handle clients in new threads. I use function pthread_create, it has handle function called handle_client, I need to get in some argumets socketdescritpor(it works) and filedescriptor for writing logs(there is probably the problem).
In my main I open the log file, then I put the filedescriptor to my struct, I get the struct inside the handle_client function and in this function I want to get back the file decriptor (fp) of the log file, to be able write to file. I use fflush(fp) to save the data after each fprintf, I open the file once and then each client should be able write to file throuhgt this descripor, but I've probably done something bad with pointers (I mean geting fp in and out from the struct, there is piece of my code where I do this action.) Thx for help.
struct
typedef struct
{
int fd;
FILE *fp; //filepointer for logs file
} my_thread_context_t;
main
int main(int argc, char *argv[]) {
FILE * fp;
fp = fopen("Serverlog.log","w");//delete last file
fclose(fp);
fp = fopen("Serverlog.log","a+");
my_thread_context_t ctx;
//bind server
//listen
while(1) {
//accept
ctx.fp = fp;// add file pointer to structure
int check = pthread_create(&thread, NULL,handle_client,&ctx);
//other code
}
hadle_client function
void * handle_client(void * void_context){
my_thread_context_t * context = (my_thread_context_t *) void_context;
FILE *fp; //pointer na soubor s logy
fp = context->fp;//get the filepointer for logs
//other code ..
}
It looks like many threads can access your my_thread_context_t::fp and that is exactly the problem. The FILE* type is really a (opaque) pointer to structure used by the C library. This structure contents may get corrupted (or non-consistent) when more than one thread modifies it. You must synchronize access to your my_thread_context_t::fp. I would suggest creating a mutex (see pthread_mutex_init, pthread_mutex_lock, pthread_mutex_unlock) inside your ctx structure. Each thread should lock it before it starts writing to a file and unlock it when its finished writing - a separate (inline) function for this puprpose would be even better design, e.g:
typedef struct
{
int fd;
pthread_mutex_t mutex;
FILE *fp; //filepointer for logs file
} my_thread_context_t;
my_thread_context_t ctx;
fp = fopen("Serverlog.log","a+");
pthread_mutex_init(&ctx.mutex);
ctx.fp = fp;
inline void write_to_log(my_thread_context_t* pCtx,const char* pcszMessage)
{
pthread_mutex_lock(&(pCtx->mutex));
/* here write to the log file */
pthread_mutex_unlock(&(pCtx->mutex));
}
If you do it this way, it is not only safe, but you also do not have to invoke fflush
after every write (unless you want your log to be always in sync).
Don't forget to call pthread_mutex_destroy after all threads are finished (e.g. after all neccesary ptrhead_joins).
I need to get the name of a file from a given file descriptor, inside a small linux kernel module that I wrote. I tried the solution given at Getting Filename from file descriptor in C, but for some reason, it prints out garbage values (on using readlink on /proc/self/fd/NNN as mentioned in the solution). How can I do it?
Don't call SYS_readlink - use the same method that procfs does when one of those links is read. Start with the code in proc_pid_readlink() and proc_fd_link() in fs/proc/base.c.
Broadly, given an int fd and a struct files_struct *files from the task you're interested in (which you have taken a reference to), you want to do:
char *tmp;
char *pathname;
struct file *file;
struct path *path;
spin_lock(&files->file_lock);
file = fcheck_files(files, fd);
if (!file) {
spin_unlock(&files->file_lock);
return -ENOENT;
}
path = &file->f_path;
path_get(path);
spin_unlock(&files->file_lock);
tmp = (char *)__get_free_page(GFP_KERNEL);
if (!tmp) {
path_put(path);
return -ENOMEM;
}
pathname = d_path(path, tmp, PAGE_SIZE);
path_put(path);
if (IS_ERR(pathname)) {
free_page((unsigned long)tmp);
return PTR_ERR(pathname);
}
/* do something here with pathname */
free_page((unsigned long)tmp);
If your code is running in process-context (eg. invoked through a syscall) and the file descriptor is from the current process, then you can use current->files for the current task's struct files_struct *.