write() to sysfs entry /sys/bus/pci/devices/.../driver/remove_id fails - c

Seeing the write() function failing on /sys/bus/pci/devices/.../driver/remove_id file returning -1 with errno equal to 19 (ENODEV).
But, same works fine via command line. I have checked the file permission which seems to be fine (--w-------) for user to perform write on this file.
int fp = 0;
int buffer_length = 0;
int bytes_written = 0;
fp = open(cmd_buf, O_WRONLY); // where cmd_buf will hold this string
// "/sys/bus/pci/devices/.../driver/remove_id"
if (fp == -1)
{
return -1;
}
// where inbuf will be a char * pointing to pci vendor device id like
// this, "XXXX YYYY"
bytes_written = write(fp, in_buf, sizeof(in_buf));
printf(" bytes_written : %d \n ", bytes_written);
Seeing bytes_written equal to -1 and errno shows 19.
Please let me know if you find something wrong with the code snippet?

You do not provide enough information to pinpoint the problem.
However, here is an example program, example.c, that shows that it is your implementation that has the bug:
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <stdio.h>
/* Write string 'data' to existing file or device at 'path'.
Returns 0 if success, errno error code otherwise.
*/
int write_file(const char *path, const char *data)
{
const char *const ends = (data) ? data + strlen(data) : data;
ssize_t n;
int fd;
/* NULL or empty path is invalid. */
if (!path || !*path)
return errno = EINVAL;
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC | O_NOCTTY | O_CLOEXEC, 0666);
if (fd == -1)
return errno; /* errno already set by open(). */
/* Write the contents of data. */
while (data < ends) {
n = write(fd, data, (size_t)(ends - data));
if (n > 0) {
/* Wrote n bytes. */
data += n;
} else
if (n != -1) {
/* C Library bug: Should never occur. */
close(fd);
return errno = EIO;
} else {
/* Error in errno. */
const int saved_errno = errno;
close(fd);
return errno = saved_errno;
}
}
if (close(fd) == -1) {
/* It is possible for close() to report a delayed I/O error. */
return errno;
}
/* Success. */
return 0;
}
static void usage(const char *argv0)
{
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv0);
fprintf(stderr, " %s FILE CONTENTS\n", argv0);
fprintf(stderr, "\n");
fprintf(stderr, "This does the same thing as 'echo -n \"CONTENTS\" > FILE'.\n");
fprintf(stderr, "\n");
}
int main(int argc, char *argv[])
{
if (argc < 2) {
usage((argv && argv[0] && argv[0][0]) ? argv[0] : "(this)");
return EXIT_SUCCESS;
} else
if (argc > 3) {
usage((argv && argv[0] && argv[0][0]) ? argv[0] : "(this)");
return EXIT_FAILURE;
} else
if (!strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
usage((argv && argv[0] && argv[0][0]) ? argv[0] : "(this)");
return EXIT_SUCCESS;
}
if (write_file(argv[1], argv[2])) {
fprintf(stderr, "%s: %s.\n", argv[1], strerror(errno));
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
Compile it using e.g. gcc -Wall -Wextra -O2 example.c -o example, and run using e.g. ./example /sys/bus/pci/devices/.../driver/remove_id "vendor_id device_id".
Run it without arguments, or with -h or --help as the only argument, and it will print usage information to standard error.
The program essentially does what echo -n "vendor_id device_id" > /sys/bus/pci/devices/.../drivers/remove_id does.
If it is successful, it will not output anything, just return success (exit status 0). If there is any kind of an error, it will report it to standard error.
If you know the target path is always a device or a pseudo-file (like those in /sys or /proc), use fd = open(path, O_WRONLY | O_NOCTTY | O_CLOEXEC); instead. O_CLOEXEC means that if the process forks at any point, this particular file descriptor is not copied to the child process. O_NOCTTY means that if the path is a tty device, and the current process does not have a controlling terminal, the kernel is NOT to make the opened device the controlling terminal.
echo -n uses O_CREAT | O_TRUNC, so that if the target path exists and is a normal file, it is truncated, but if it does not exist, it is created. It does not affect opening existing character devices and pseudofiles. Whenever O_CREAT is used, there must be a third parameter, which affects the access mode of the file created. This mode is usually 0666, allowing read and write access as moderated by the current umask. One can obtain the current umask using mode_t mask = umask(0); umask(mask);. Access mode bits set in umask are always zero in the final access mode, and access mode bits clear in umask are taken from the third parameter of the open() command when the file is created.

Two possible problems:
Using sizeof(in_buf) in the write() system call writes the string "vendorId deviceId" and may be more garbage data behind it if in_buf[] is bigger than 10 chars.
Perhaps in_buf is not a table but a pointer and so, sizeof(in_buf) will return 4 or 8 (the size of the pointer respectively for 32 or 64 bits systems) but not the length of the string it points to.
So, in both cases (in_buf defined as a table or a pointer), strlen(in_buf) instead of sizeof(in_buf) is the most secured solution for the length of the data to write provided that the string is terminated by '\0'.

Related

Implementing the cp command using read/write system calls [duplicate]

This question already has an answer here:
Using open(), read() and write() system calls to copy a file
(1 answer)
Closed last year.
I am trying to implement the cp command only using read/write system calls.
Here is my code:
/**
* cp file1 file 2
*/
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
int main(int argc, char *argv[])
{
int errsv;
char contents[1024];
int fd_read, fd_write;
fd_read = open(argv[1], O_RDONLY);
if (fd_read == -1)
{
errsv = errno;
printf("Error occured: %d\n", errsv);
}
read(fd_read, contents, sizeof(contents));
fd_write = open(argv[2], O_CREAT | O_WRONLY | O_TRUNC, 0744);
if (fd_write == -1)
{
errsv = errno;
printf("Error occured: %d\n", errsv);
}
write(fd_write, contents, sizeof(contents));
close(fd_read);
close(fd_write);
return 0;
}
I tested the code using the commands:
cc test.c
./a.out file1 file2
Here is my file1:
dummy text
dummy text
After running the code, although file2 contains the text from file1, it also has some gibberish characters. [not keeping this here.]
Why is this so?
You need to call read() and write() in a loop to copy the entire file. read() returns 0 when you reach EOF, or a negative result if there's an error, then you can end the loop.
read() returns the number of bytes that were read, which may be less than the size of the buffer. You need to use that number when calling write(), otherwise you'll write extra characters to the output file. These will be unitialized characters on the first iteration, and on other iterations they'll be left over characters from previous iterations.
int main(int argc, char *argv[])
{
char contents[1024];
int fd_read, fd_write;
fd_read = open(argv[1], O_RDONLY);
if (fd_read == -1)
{
perror("open input file");
exit(1);
}
fd_write = open(argv[2], O_CREAT | O_WRONLY | O_TRUNC, 0744);
if (fd_write == -1)
{
perror("open output file");
exit(1)
}
int n_read;
while ((n_read = read(fd_read, contents, sizeof(contents))) > 0) {
write(fd_write, contents, n_read);
}
close(fd_read);
close(fd_write);
return 0;
}
write(fd_write, contents, strlen(contents));
Strlen returns the filled entries number but sizeof returns the buffer size which is 1024

Towards understanding availability of xdg-open

I want to open an image, and in Windows I do:
#include <windows.h>
..
ShellExecute(NULL, "open", "https://gsamaras.files.wordpress.com/2018/11/chronosgod.png", NULL, NULL, SW_SHOWNORMAL);
I would like to use a Linux approach, where it's so much easier to run something on the fly. Example:
char s[100];
snprintf(s, sizeof s, "%s %s", "xdg-open", "https://gsamaras.files.wordpress.com/2018/11/chronosgod.png");
system(s);
In my Ubuntu, it works. However, when running that in Wandbox (Live Demo), or in any other online compiler, I would most likely get an error:
sh: 1: xdg-open: not found
despite the fact that these online compilers seem to live in Linux (checked). I don't expect the online compiler to open a browser for me, but I did expect the code to run without an error. Ah, and forget Mac (personal laptop, limiting my machines).
Since I have no other Linux machine to check, my question is: Can I expect that this code will work in most of the major Linux distributions?
Maybe the fact that it failed on online compilers is misleading.
PS: This is for my post on God of Time, so no worries about security.
Although Antti Haapala already completely answered the question, I thought some comments about the approach, and an example function making safe use trivial, might be useful.
xdg-open is part of desktop integration utilities from freedesktop.org, as part of the Portland project. One can expect them to be available on any computer running a desktop environment participating in freedesktop.org. This includes GNOME, KDE, and Xfce.
Simply put, this is the recommended way of opening a resource (be it a file or URL) when a desktop environment is in use, in whatever application the user prefers.
If there is no desktop environment in use, then there is no reason to expect xdg-open to be available either.
For Linux, I would suggest using a dedicated function, perhaps along the following lines. First, a couple of internal helper functions:
#define _POSIX_C_SOURCE 200809L
#define _GNU_SOURCE
//
// SPDX-License-Identifier: CC0-1.0
//
#include <stdlib.h>
#include <unistd.h>
#include <limits.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <dirent.h>
#include <fcntl.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
/* Number of bits in an unsigned long. */
#define ULONG_BITS (CHAR_BIT * sizeof (unsigned long))
/* Helper function to open /dev/null to a specific descriptor.
*/
static inline int devnullfd(const int fd)
{
int tempfd;
/* Sanity check. */
if (fd == -1)
return errno = EINVAL;
do {
tempfd = open("/dev/null", O_RDWR | O_NOCTTY);
} while (tempfd == -1 && errno == EINTR);
if (tempfd == -1)
return errno;
if (tempfd != fd) {
if (dup2(tempfd, fd) == -1) {
const int saved_errno = errno;
close(tempfd);
return errno = saved_errno;
}
if (close(tempfd) == -1)
return errno;
}
return 0;
}
/* Helper function to close all except small descriptors
specified in the mask. For obvious reasons, this is not
thread safe, and is only intended to be used in recently
forked child processes. */
static void closeall(const unsigned long mask)
{
DIR *dir;
struct dirent *ent;
int dfd;
dir = opendir("/proc/self/fd/");
if (!dir) {
/* Cannot list open descriptors. Just try and close all. */
const long fd_max = sysconf(_SC_OPEN_MAX);
long fd;
for (fd = 0; fd < ULONG_BITS; fd++)
if (!(mask & (1uL << fd)))
close(fd);
for (fd = ULONG_BITS; fd <= fd_max; fd++)
close(fd);
return;
}
dfd = dirfd(dir);
while ((ent = readdir(dir)))
if (ent->d_name[0] >= '0' && ent->d_name[0] <= '9') {
const char *p = &ent->d_name[1];
int fd = ent->d_name[0] - '0';
while (*p >= '0' && *p <= '9')
fd = (10 * fd) + *(p++) - '0';
if (*p)
continue;
if (fd == dfd)
continue;
if (fd < ULONG_MAX && (mask & (1uL << fd)))
continue;
close(fd);
}
closedir(dir);
}
closeall(0) tries hard to close all open file descriptors, and devnullfd(fd) tries to open fd to /dev/null. These are used to make sure that even if the user spoofs xdg-open, no file descriptors are leaked; only the file name or URL is passed.
On non-Linux POSIXy systems, you can replace them with something more suitable. On BSDs, use closefrom(), and handle the first ULONG_MAX descriptors in a loop.
The xdg_open(file-or-url) function itself is something along the lines of
/* Launch the user-preferred application to open a file or URL.
Returns 0 if success, an errno error code otherwise.
*/
int xdg_open(const char *file_or_url)
{
pid_t child, p;
int status;
/* Sanity check. */
if (!file_or_url || !*file_or_url)
return errno = EINVAL;
/* Fork the child process. */
child = fork();
if (child == -1)
return errno;
else
if (!child) {
/* Child process. */
uid_t uid = getuid(); /* Real, not effective, user. */
gid_t gid = getgid(); /* Real, not effective, group. */
/* Close all open file descriptors. */
closeall(0);
/* Redirect standard streams, if possible. */
devnullfd(STDIN_FILENO);
devnullfd(STDOUT_FILENO);
devnullfd(STDERR_FILENO);
/* Drop elevated privileges, if any. */
if (setresgid(gid, gid, gid) == -1 ||
setresuid(uid, uid, uid) == -1)
_Exit(98);
/* Have the child process execute in a new process group. */
setsid();
/* Execute xdg-open. */
execlp("xdg-open", "xdg-open", file_or_url, (char *)0);
/* Failed. xdg-open uses 0-5, we return 99. */
_Exit(99);
}
/* Reap the child. */
do {
status = 0;
p = waitpid(child, &status, 0);
} while (p == -1 && errno == EINTR);
if (p == -1)
return errno;
if (!WIFEXITED(status)) {
/* Killed by a signal. Best we can do is I/O error, I think. */
return errno = EIO;
}
switch (WEXITSTATUS(status)) {
case 0: /* No error. */
return errno = 0; /* It is unusual, but robust to explicitly clear errno. */
case 1: /* Error in command line syntax. */
return errno = EINVAL; /* Invalid argument */
case 2: /* File does not exist. */
return errno = ENOENT; /* No such file or directory */
case 3: /* A required tool could not be found. */
return errno = ENOSYS; /* Not implemented */
case 4: /* Action failed. */
return errno = EPROTO; /* Protocol error */
case 98: /* Identity shenanigans. */
return errno = EACCES; /* Permission denied */
case 99: /* xdg-open does not exist. */
return errno = ENOPKG; /* Package not installed */
default:
/* None of the other values should occur. */
return errno = ENOSYS; /* Not implemented */
}
}
As already mentioned, it tries hard to close all open file descriptors, redirects the standard streams to /dev/null, ensures the effective and real identity matches (in case this is used in a setuid binary), and passes success/failure using the child process exit status.
The setresuid() and setresgid() calls are only available on OSes that have saved user and group ids. On others, use seteuid(uid) and setegid() instead.
This implementation tries to balance user configurability with security. Users can set the PATH so that their favourite xdg-open gets executed, but the function tries to ensure that no sensitive information or privileges are leaked to that process.
(Environment variables could be filtered, but they should not contain sensitive information in the first place, and we don't really know which ones a desktop environment uses. So better not mess with them, to keep user surprises to a minimum.)
As a minimal test main(), try the following:
int main(int argc, char *argv[])
{
int arg, status;
if (argc < 2 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
fprintf(stderr, " %s FILE-OR-URL ...\n", argv[0]);
fprintf(stderr, "\n");
fprintf(stderr, "This example program opens each specified file or URL\n");
fprintf(stderr, "xdg-open(1), and outputs success or failure for each.\n");
fprintf(stderr, "\n");
return EXIT_SUCCESS;
}
status = EXIT_SUCCESS;
for (arg = 1; arg < argc; arg++)
if (xdg_open(argv[arg])) {
printf("%s: %s.\n", argv[arg], strerror(errno));
status = EXIT_FAILURE;
} else
printf("%s: Opened.\n", argv[arg]);
return status;
}
As the SPDX license identifier states, this example code is licensed under Creative Commons Zero 1.0. Use it any way you wish, in any code you want.
The xdg-open is part of the xdg-utils. They're almost always installed with the GUI desktop of any Linux distribution.
A Linux distribution can be installed without any Graphical User Interface, on servers say, and most probably then they would lack xdg-open.
Instead of system, you could - and should - use fork + exec - if exec fails then xdg-open could not be executed.
The online compilers most probably don't have any Desktop GUI installed on them, thus the lack of that utility.

C's open overwriting file to be opened and fdopen returning NULL

I'm trying to write a secure version of fopen that prevents the opening of symbolically linked files. Below is my code taken from here.
/*
Secure fopen
*/
enum { FILE_MODE = 0600 };
FILE *secure_fopen(char *filename, char* mode)
{
int fd;
FILE *f;
unlink(filename);
if (strncmp(mode, "w", 1) == 0) {
fd = open(filename, O_WRONLY|O_CREAT|O_EXCL, FILE_MODE);
}
else if (strncmp(mode, "r", 1) == 0) {
fd = open(filename, O_RDONLY|O_CREAT|O_EXCL);
}
else {
fd = open(filename, O_RDWR|O_CREAT|O_EXCL, FILE_MODE);
}
if (fd == -1) {
perror("Failed to open the file");
return NULL;
}
/* Get a FILE*, as they are easier and more efficient than file descriptors */
f = fdopen(fd, mode);
if (f == NULL) {
perror("Failed to associate file descriptor with a stream");
return NULL;
}
return f;
}
There are two issues with this code: one - it's overwriting the file pointed to by filename and two - it's returning NULL, but the NULL file pointer doesn't get caught in the final check:
if (f == NULL) {
perror("Failed to associate file descriptor with a stream");
return NULL;
}
Does anybody have any insight into way these two things are happening?
First, O_CREAT creates a file if it does not exist, and O_CREAT|O_EXCL creates the file and fails if it already exists.
Second, (strncmp(mode, "w", 1) == 0) is equivalent to (mode[0] == 'w'), which is probably not what you intended. You probably meant (strchr(mode, "w")) instead.
Consider the following implementation (a full example program):
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
/* Internal flags used by custom_fopen(): */
#define FM_R (1<<0) /* r, w+: For reading */
#define FM_W (1<<1) /* w, r+: For writing */
#define FM_TRUNC (1<<2) /* w, w+: Truncate */
#define FM_CREAT (1<<3) /* w, r+: Create if necessary */
#define FM_EXCL (1<<4) /* x: Fail if already exists */
#define FM_APPEND (1<<5) /* a: Append */
#define FM_CLOEXEC (1<<6) /* e: Close-on-exec() */
#define FM_SYMLINK (1<<7) /* s: Fail if last path component is a symlink */
#define FM_RW (FM_R | FM_W) /* r+, w+ */
FILE *custom_fopen(const char *path, const char *mode)
{
const char *fdmode;
int fm, flags, fd, saved_errno;
FILE *ret;
if (!path || !*path || !mode) {
errno = EINVAL;
return NULL;
}
switch ((strchr(mode, 'r') ? 1 : 0) +
(strchr(mode, 'w') ? 2 : 0) +
(strchr(mode, 'a') ? 4 : 0) +
(strchr(mode, '+') ? 8 : 0)) {
case 1: fdmode = "r"; fm = FM_R; break;
case 2: fdmode = "w"; fm = FM_W | FM_CREAT | FM_TRUNC; break;
case 4: fdmode = "a"; fm = FM_W | FM_CREAT | FM_APPEND; break;
case 9: fdmode = "r+"; fm = FM_RW | FM_CREAT; break;
case 10: fdmode = "w+"; fm = FM_RW | FM_CREAT | FM_TRUNC; break;
case 12: fdmode = "a+"; fm = FM_RW | FM_CREAT | FM_APPEND; break;
default:
/* Invalid combination of 'r', 'w', 'a', and '+'. */
errno = EINVAL;
return NULL;
}
if (strchr(mode, 'x')) {
if (fm & FM_CREAT)
fm |= FM_EXCL;
else {
/* 'rx' does not make sense, and would not work anyway. */
errno = EINVAL;
return NULL;
}
}
if (strchr(mode, 'e'))
fm |= FM_CLOEXEC;
if (strchr(mode, 's'))
fm |= FM_SYMLINK;
/* Verify 'mode' consists of supported characters only. */
if (strlen(mode) != strspn(mode, "rwa+xesb")) {
errno = EINVAL;
return NULL;
}
/* Map 'fm' to 'flags' for open(). */
switch (fm & FM_RW) {
case FM_R: flags = O_RDONLY; break;
case FM_W: flags = O_WRONLY; break;
case FM_RW: flags = O_RDWR; break;
default:
errno = EINVAL;
return NULL;
}
if (fm & FM_TRUNC) flags |= O_TRUNC;
if (fm & FM_CREAT) flags |= O_CREAT;
if (fm & FM_EXCL) flags |= O_EXCL;
if (fm & FM_APPEND) flags |= O_APPEND;
if (fm & FM_CLOEXEC) flags |= O_CLOEXEC;
if (fm & FM_SYMLINK) flags |= O_NOFOLLOW;
/* Open the file. If we might create it, use mode 0666 like fopen() does. */
if (fm & FM_CREAT)
fd = open(path, flags, 0666);
else
fd = open(path, flags);
/* Failed? */
if (fd == -1)
return NULL; /* errno set by open() */
/* Convert the file descriptor to a file handle. */
ret = fdopen(fd, fdmode);
if (ret)
return ret;
/* Failed. Remember the reason for the failure. */
saved_errno = errno;
/* If we created or truncated the file, unlink it. */
if (fm & (FM_EXCL | FM_TRUNC))
unlink(path);
/* Close the file descriptor. */
close(fd);
/* Return, recalling the reason for the failure. */
errno = saved_errno;
return NULL;
}
int main(int argc, char *argv[])
{
FILE *handle;
if (argc != 3 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
fprintf(stderr, " %s PATH MODE\n", argv[0]);
fprintf(stderr, "\n");
return EXIT_FAILURE;
}
handle = custom_fopen(argv[1], argv[2]);
if (!handle) {
const int err = errno;
fprintf(stderr, "custom_fopen(\"%s\", \"%s\") == NULL, errno = %d: %s.\n",
argv[1], argv[2], err, strerror(err));
return EXIT_FAILURE;
}
if (fclose(handle)) {
const int err = errno;
fprintf(stderr, "fclose(custom_fopen(\"%s\", \"%s\")) failed, errno = %d: %s.\n",
argv[1], argv[2], err, strerror(err));
return EXIT_FAILURE;
}
printf("custom_fopen(\"%s\", \"%s\"): Success.\n", argv[1], argv[2]);
return EXIT_SUCCESS;
}
The #define _POSIX_C_SOURCE 200809L tells your C library (at least those compatible to GNU C) to expose POSIX.1-2008 features (like open()).
The behaviour of r, w, a, r+, w+, and a+ modes are as described in man 3 fopen. At least one of these must be in mode. (The + does not need to immediately follow the letter, though.)
The above implementation additionally supports b (does nothing, per POSIX), x (fail if creating a new file but it already exists), s (fail if the filename part of the path is a symlink), and e (close the descriptor at exec, not leaking it to child processes).
The first switch statement handles the primary mode, ignoring the order of the characters. It essentially checks which of the four characters rwa+ exist in mode, and only accepts the sane combinations. The strchr(mode, 'c') calls return a nonzero pointer (logical true) if and only if mode contains 'c'.
The if clause following it detects x in the mode. The combination of x and r is not allowed, because it makes no sense. (POSIX.1 says the behaviour with open() with O_RDONLY | O_EXCL is undefined.)
The (strlen(mode) == strspn(mode, "rwa+xesb")) verifies that mode consists only of letters r, w, a, +, x, e, s, and b; they can be repeated, or in any order. This check rejects unsupported characters.
The second switch statement and if clauses map fm to flags. We do this, because the O_ constants may not be single bits, which means tests like (flags & O_RDONLY), (flags & O_WRONLY), and (flags & O_RDWR) are not reliable, and in fact will not work the way one might expect. Instead, we use fm and our own single-bit FM_ constants that we can treat as masks, and just map them to corresponding values of flags later on. (Simply put, fm tracks the features we want, and we only assign the corresponding set of flags to flags, and never examine flags.)
If we might create the file, we use mode 0666 (rw-rw-rw-), as modified by user's umask. (Typically, normal users have an umask of 002, 007, 022, or 077, which result in new files getting modes 0664 (rw-rw-r--), 0660 (rw-rw----), 0644 (rw-r--r--), or 0600 (rw-------), respectively.)
This is exactly what fopen() does, too.
When we have the open file descriptor, we must still associate it with a stream handle. We do that using fdopen(). Note that we decided on the correct mode for this call in the first switch statement. If this call succeeds, the stream will "own" the file descriptor, and all we need to do is to return the stream handle fdopen() returned.
If fdopen() fails, we need to close the file descriptor. We may also decide to remove/unlink the file. The above code removes the file if we are sure it did not exist previously (e), or if we truncated it (w, w+) as then the data it might have contained would be lost anyway.
The test program takes two command-line parameters: a path or a file name, and the mode string. The program does a custom_fopen(pathorfilename, modestring) call, and reports the results. (If the custom_fopen() call succeeds, it also checks that the corresponding fclose() call succeeds, because sometimes issues related to the file descriptor (or incompatible mode flags for the open()/fdopen() calls) may only be observed at the first operation done, or at stream close fime.
I've only lightly tested the above function and program, and bugs are always a possibility. If you find a bug or have an issue with the code, let me know in a comment, so I can verify and fix if necessary.

Why stat and fstat return the st_size == 0?

I was testing a code from APUE, in chapter 14(Advanced I/O) of memory map file, the fstat() always return the fdin's st_size as zero, and I tried stat() instead, and also get the same result. I list the code below(I have removed the apue.h dependencies):
#include <fcntl.h>
#include <sys/mman.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#define COPYINCR (1024*1024*1024) /* 1GB */
int main(int argc, char *argv[]) {
if (argc != 3) {
printf("usage: %s <fromfile> <tofile>", argv[0]);
exit(1);
}
int fdin, fdout;
if ((fdin = open(argv[1], O_RDONLY)) < 0) {
printf("can not open %s for reading", argv[1]);
exit(1);
}
if ((fdout = open(argv[2] /* typo fix */, O_RDONLY | O_CREAT | O_TRUNC)) < 0) {
printf("can not open %s for writing", argv[2]);
exit(1);
}
struct stat sbuf;
if (fstat(fdin, &sbuf) < 0) { /* need size fo input file */
printf("fstat error");
exit(1);
}
// always zero, and cause truncate error (parameter error)
printf("input_file size: %lld\n", (long long)sbuf.st_size);
if (ftruncate(fdout, sbuf.st_size) < 0) { /* set output file size */
printf("ftruncate error");
exit(1);
}
void *src, *dst;
off_t fsz = 0;
size_t copysz;
while (fsz < sbuf.st_size) {
if (sbuf.st_size - fsz > COPYINCR)
copysz = COPYINCR;
else
copysz = sbuf.st_size - fsz;
if (MAP_FAILED == (src = mmap(0, copysz, PROT_READ,
MAP_SHARED, fdin, fsz))) {
printf("mmap error for input\n");
exit(1);
}
if (MAP_FAILED == (dst = mmap(0, copysz,
PROT_READ | PROT_WRITE,
MAP_SHARED, fdout, fsz))) {
printf("mmap error for output\n");
exit(1);
}
memcpy(dst, src, copysz);
munmap(src, copysz);
munmap(dst, copysz);
fsz += copysz;
}
return 0;
}
And then I have tried the Python os.stat, it also get the zero result, why this happened? I have tried these and got the same result on Mac OS (Darwin kernel 13.4) and Ubuntu (kernel 3.13).
UPDATE:
Oh, there was a typo error, I should refer to fdout to argv[2], and the O_TRUNC flag certainly make the fdin to zero. Should I close or delete this question?
The reason why Python's os.stat() also return (stat.st_size == 0) is that I passed the same test file (argv[1]) to test, and the file has been previously truncated to zero (I haven't check its size using ls -lh before passing to os.stat()), and certainly os.stat() return zero.
Do not ask SO questions before you go to bed or in a rush.
Ok, the real problem is double open the same input file, and this does not cause any build or runtime error until the ftruncate().
The first open get a read-only fdin, the second open create a new file (fdout and truncated) to copy from fdin via memory map, and the second open truncated the first file (argv[1]), and cleaned all its content. But the fdin still working with fstat (and certainly), this make me hard to find the reason.
The second part is I always use the same file for testing (generated via dd) and have not checking the size, so the os.stat(/path/to/file) and stat(/path/to/file) also return st_size == 0, this makes me believe that this must be some os-level-prolicy defined the behaviour, and I rushed to Mac OS (using the same typo code), and got the same result (they really consistent on POSIX level, event the bug!), and at last, I came to SO for help.

memory mapped files

I wrote a code for writing the content to the mapped buffer which mapped by using the mmap() system call.
After I did some the changes in the mapped buffer,then I called the msync().It should update to the file on disk.
But,It doesn't made any changes to the file on disk.
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include<sys/mman.h>
#include<fcntl.h>
#define FILEMODE S_IRWXU | S_IRGRP | S_IROTH
#define MAX 150
main(int argc,char *argv[])
{
int fd,ret,len;
long int len_file;
struct stat st;
char *addr;
char buf[MAX];
if(argc > 1)
{
if((fd = open(argv[1],O_RDWR | O_APPEND | O_CREAT ,FILEMODE)) < 0)
perror("Error in file opening");
if((ret=fstat(fd,&st)) < 0)
perror("Error in fstat");
len_file = st.st_size;
/*len_file having the total length of the file(fd).*/
if((addr=mmap(NULL,len_file,PROT_READ|PROT_WRITE,MAP_PRIVATE,fd,0)) == MAP_FAILED)
perror("Error in mmap");
len = len_file;
while((fgets(buf,MAX,stdin)) != NULL)
{
strcat(addr+len,buf);
printf( "Val:%s\n",addr ) ; //Checking purpose
len = len + (strlen(buf));
}
if((msync(addr,len,MS_SYNC)) < 0)
perror("Error in msync");
if( munmap(addr,len) == -1)
printf("Error:\n");
printf("addr %p\n",addr);
}
else
{
printf("Usage a.out <filename>\n");
}
}
If you want your changes to be reflected in the on-disk file, you must map the file as MAP_SHARED, not MAP_PRIVATE.
Additionally, you cannot extend the file simply by writing beyond the end of the mapping. You must use ftruncate() to extend the file to the new size, then change the mapping to include the new portion of the file. The portable way to change the mapping is to unmap the mapping then recreate it with the new size; on Linux you can instead use mremap().
Your len and len_file variables should be of type size_t, and you should use memcpy() rather than strcat(), since you know exactly the length of the string, exactly where you want to copy it, and you don't want to copy the null-terminator.
The following modification of your code works on Linux (using mremap()) :
#define _GNU_SOURCE
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include<sys/mman.h>
#include<fcntl.h>
#define FILEMODE S_IRWXU | S_IRGRP | S_IROTH
#define MAX 150
int main(int argc,char *argv[])
{
int fd, ret;
size_t len_file, len;
struct stat st;
char *addr;
char buf[MAX];
if (argc < 2)
{
printf("Usage a.out <filename>\n");
return EXIT_FAILURE;
}
if ((fd = open(argv[1],O_RDWR | O_CREAT, FILEMODE)) < 0)
{
perror("Error in file opening");
return EXIT_FAILURE;
}
if ((ret = fstat(fd,&st)) < 0)
{
perror("Error in fstat");
return EXIT_FAILURE;
}
len_file = st.st_size;
/*len_file having the total length of the file(fd).*/
if ((addr = mmap(NULL,len_file,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0)) == MAP_FAILED)
{
perror("Error in mmap");
return EXIT_FAILURE;
}
while ((fgets(buf,MAX,stdin)) != NULL)
{
len = len_file;
len_file += strlen(buf);
if (ftruncate(fd, len_file) != 0)
{
perror("Error extending file");
return EXIT_FAILURE;
}
if ((addr = mremap(addr, len, len_file, MREMAP_MAYMOVE)) == MAP_FAILED)
{
perror("Error extending mapping");
return EXIT_FAILURE;
}
memcpy(addr+len, buf, len_file - len);
printf( "Val:%s\n",addr ) ; //Checking purpose
}
if((msync(addr,len,MS_SYNC)) < 0)
perror("Error in msync");
if (munmap(addr,len) == -1)
perror("Error in munmap");
if (close(fd))
perror("Error in close");
return 0;
}
Note that you've provided a mapping for the file that is exactly the size of the file. If you create the file in your call to open(2), it will have a length of 0, and I wouldn't be surprised if the kernel doesn't bother setting up any kind of memory mapping from a 0 length mapping. (Maybe it does? I've never tried...)
I would suggest using ftruncate(2) to extend the length of your file before performing the mapping. (Note that extending files using ftruncate(2) isn't very portable; not all platforms provide extending functionality and not all filesystem drivers support the extending functionality. See your system's manpage for details.)
You must use the MAP_SHARED mapping for your file modifications to be saved to disk.
Your use of perror(3) isn't quite correct; perror(3) will not terminate your program, so it will continue executing with incorrect assumptions:
if((ret=fstat(fd,&st)) < 0)
perror("Error in fstat");
Should read:
if((ret=fstat(fd,&st)) < 0) {
perror("Error in fstat");
exit(1);
}
(Or exit(EXIT_FAILURE) if you want to be more portable -- I find that a little harder on the eyes but I live in Linux-land.)
strcat(3) expects to find an ASCII NUL character (byte value 0x00, C representation '\0') -- the usual C end-of-string marker -- at the end of the dest string. Your file will not contain an ASCII NUL if you create it in this program -- its length is zero, after all -- and I don't know the consequences of trying to read a zero-byte file via mmap(2). If the file already exists and has data in it, it probably doesn't have an ASCII NUL encoded in the file. strcat(3) is almost certainly the wrong tool to write into your file. (No one wants ASCII NULs in their files anyway.) Try memcpy(3) instead.

Resources