I wonder if there is any way to lock and unlock a file in Linux when I open a file using fopen (not open)?
Based on Stack Overflow question C fopen vs open, fopen is preferred over open.
How can I implement my own file lock (if possible) by creating and deleting lock files?
I would strongly disagree with the claim that fopen is prefered over open. It's impossible to use fopen safely when writing a file in a directory that's writable by other users due to symlink vulnerabilities/race conditions, since there is no O_EXCL option. If you need to use stdio on POSIX systems, it's best to use open and fdopen rather than calling fopen directly.
Now, as for locking it depends on what you want to do. POSIX does not have mandatory locking like Windows, but if you just want to ensure you're working with a new file and not clobbering an existing file or following a symlink, use the O_EXCL and O_NOFOLLOW options, as appropriate. If you want to do cooperative locking beyond the initial open, use fcntl locks.
In Linux, if you need a file descriptor (e.g., to pass to a file-locking primitive), you can use fileno(FILE*) to retrieve it. After retrieving the file descriptor, you can use it as if it had been returned by open.
For example, instead of
int fd = open("myfile.txt", flags);
int result = flock(fd, LOCK_SH);
you could equally well do this:
FILE* f = fopen("myfile.txt", "r");
int result = flock(fileno(f)), LOCK_SH);
Note that fileno is defined in the POSIX standard, but not in C or C++ standards.
As for your second question, the Linux open() man page has this to say:
The solution for performing atomic file locking using a lockfile is to
create a unique file on the same file system (e.g., incorporating
hostname and pid), use link(2) to make a link to the lockfile. If
link() returns 0, the lock is successful. Otherwise, use stat(2) on
the unique file to check if its link count has increased to 2, in
which case the lock is also successful.
Files can be locked by using flock(). Its syntax is
#include <sys/file.h>
#define LOCK_SH 1 /* shared lock */
#define LOCK_EX 2 /* exclusive lock */
#define LOCK_NB 4 /* don't block when locking */
#define LOCK_UN 8 /* unlock */
int flock(int fd, int operation);
First file is opened using fopen() or open(). Then this opened file is locked using flock() as given below
int fd = open("test.txt","r");
int lock = flock(fd, LOCK_SH); // Lock the file . . .
// . . . .
// Locked file in use
// . . . .
int release = flock(fd, LOCK_UN); // Unlock the file . . .
Note that in below code fopen will fail (and return NULL) if the lock file /var/lock/my.lock doesn't exist.
FILE* f = fopen("/var/lock/my.lock", "r");
int result = flock(fileno(f)), LOCK_SH);
Use fopen with w+ if you need the lockfile to be created if it doesn't exist.
FILE* f = fopen("/var/lock/my.lock", "w+");
int result = flock(fileno(f)), LOCK_SH);
If you wish to implement your own lock simply, I suggest Rob's answer of using flock. If you wish to implement it in a complex manner, such as for high availability, you can try something like using a thread to touch a file at a regular interval. All other programs wanting to lock the file should also check the file to see if its update time has been updated in at another fixed, but larger interval (the larger part is important). This is probably overkill for most applications, but it handles things such as crashes, freezes, etc. much better than flock.
There is another way with open() function, but I'm not sure about this called locked file. I am using file permissions to open a file.
The code is here:
#include <stdio.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#define FILE_NAME "hello.txt"
int main()
{
int fd;
fd = open(FILE_NAME, O_CREAT, S_IRWXU);
// Error checking
if(fd == -1){
perror("[error]\n");
}
else{
printf("[file is opened]\n");
}
return 0;
}
I used a flag for permissions (third argument). This flag gives read, write and execute permissions to the user.
$ls -alh
total 24K
drwxrwxr-x 2 arien arien 4.0K Dec 28 20:56 .
drwxrwxr-x 18 arien arien 4.0K Dec 27 22:20 ..
-rwxrwxr-x 1 arien arien 8.5K Dec 28 20:56 fopen
-rw-rw-r-- 1 arien arien 290 Dec 28 20:56 fopen.c
-rwx------ 1 arien arien 0 Dec 28 20:55 hello.txt
A little tip: If you are using Ubuntu or Debian, you can see the description of functions with man [function_name] man pages of open() function.
The below code is not letting me lock the file using lockf,
flock works fine though
#include <iostream>
#include <unistd.h>
#include<thread>
#include <vector>
#include <sys/file.h>
#include <fcntl.h>
#include <string.h>
using namespace std;
void append()
{
FILE *fp=fopen("a.txt","a");
if(fp)
{
cout<<lockf(fileno(fp),F_LOCK,0)<<endl;
//flock(fileno(fp), LOCK_EX);
fprintf(fp,"abcdefghijklmnopqrstuvwxyz\n");fflush(fp);
sleep(1);
fprintf(fp,"^$^&%&*&^&*(*)_*)_()_*&***&(\n");fflush(fp);
fclose(fp);
}
else {
printf("null\n");
}
}
int main()
{
fclose(fopen("a.txt","w"));
//return 0;
vector<thread*> v;
//#pragma omp parallel for
for(int i=0;i<1000;++i)
{
v.push_back(new thread(append));
//append();
}
for(auto y:v)
{
y->join();
delete y;
}
return 0;
}
Related
I just discovered that a FILE* can not only refer to a regular file, but also to a directory. If the latter is the case, fread will fail with errno set to 21 (Is a directory).
Minimal repro can be tested here
#include <stdio.h>
#include <fcntl.h>
#include <assert.h>
#include <errno.h>
int main() {
char const* sz = ".";
int fd = open(sz, O_RDONLY | O_NOFOLLOW); // all cleanup omitted for brevity
FILE* file = fdopen(fd, "rb");
// I would like to test in this line if it is a directory
char buffer[21];
int const n = fread(buffer, 1, 20, file);
if (0 < n) {
buffer[n] = 0;
printf(buffer);
} else {
printf("Error %d", errno); // 21 = Is a directory
}
}
What is the proper way to detect early that my FILE* is referring to directory without trying to read from it?
EDIT to repel the duplicate flags:
I want to test on the FILE*, not the filename. Testing on filename only and then opening it later is a race condition.
Assuming a POSIX-like environment, if you have just the file stream (FILE *fp), then you are probably reduced to using fileno() and fstat():
#include <sys/stat.h>
struct stat sb;
if (fstat(fileno(fp), &sb) != 0)
…oops…
if (S_ISDIR(sb.st_mode))
…it is a directory…
else
…it is not a directory…
Assuming you are on a POSIX-based system, use stat() (if you wish to use the filename in sz before the call to open()) or fstat() (if you wish to use the descriptor fd after calling open()) to get a file status structure from the OS. The member of the structure named st_mode can be used with the POSIX API S_ISDIR(st_mode) to see if the file is a directory.
For more information, see: http://man7.org/linux/man-pages/man2/stat.2.html
Checking The fcntl.h man page:
header shall define the following symbolic constants as
file creation flags for use in the oflag value to open() and
openat(). The values shall be bitwise-distinct and shall be suitable
for use in #if preprocessing directives.
And the flag :
O_DIRECTORY Fail if not a directory.
I am running a process (Process A) which opens a file with read only access mode. Then it pauses, causing the process to stay running and keep the file descriptor open. Later, if any of the below is possible, it will resume after some time and continue operation.
I want to know if any of these are possible:
Can we create another process (Process B) with superuser privileges, which can access Process A's open file descriptor and change its access mode to read and write ?
Can we modify the file descriptor of Process A, from within the process (I mean within its code) from read only to read and write?
Can I create a loadable kernel module that access the process A using its process ID (PID) and check for open file descriptors and change their permissions to read and write?
I have searched the forums countless number of times, but didn't find anything specific to my problem. I also found out about the fcntl() system call. But this doesn't allow us to modify the status flag of a file descriptor.
You can't. Permissions are locked in place at the time when a file descriptor is opened and it would be almost impossible to ensure the security of the system if they could be changed at run time. You could make a kernel module that changes this, but it would pretty much be a death sentence to the stability and security of the system.
What you can do and what is normally done is to open the file again with different permissions and replace the file descriptor with dup2.
3 first, since it's easiest: Can it be done in the kernel? Certainly … in theory, you can do “almost anything” there.
Both #2 and #1 come down to the question as to whether the file descriptor is re-openable.
In the most common case — the fd refers to a regular file stream in the local filesystem, the pathname of which has a directory link to which has not been altered — you can simply open the same pathname from another process. EG: If A opens /home/user/foo.log read-only, then either A or B can simply open the same pathname read-write in future.
Since you're asking, I'll assume it's not that easy. Perhaps the pathname may have been altered (eg, the file may have been unlinked), or perhaps the fd is a reference to another type of stream, like a shell pipeline, FIFO, or network socket connection.
As you probably noticed, fcntl does not allow escalation of privileges:
F_SETFL (int)
Set the file status flags to the value specified by arg. File
access mode (O_RDONLY, O_WRONLY, O_RDWR) and file creation
flags (i.e., O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC) in arg are
ignored.
This is, of course, a security precaution to protect against escalation of privileges in a process that may have opened the file under temporarily elevated or changed permissions.
However, it seems that you may have a chance to open a new, duplicate stream based upon a pathname if you know the file descriptor number and the process ID of Process A.
The /proc filesystem contains virtual file entries which represent open streams. Under /proc/ pid /fd/ fd you can find a pathname to the currently-open stream. Given sufficient permissions, you can open that stream.
reader.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int
main (int argc, char** argv) {
FILE* f = fopen("/tmp/foo", "r");
while(1) {
sleep(10);
}
}
writer.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int
main (int argc, char** argv) {
/* sane PID passed in */
if (2 != argc) exit(1);
if (5 > strlen(argv[1])) exit(2);
for(size_t ci = 0; argv[ci]; ++ci) {
if (! ( ('0' <= argv[1][ci]) && (argv[1][ci] <= '9') ) ) exit(3);
}
/* note we know FD=3 so it's hard-coded */
char path[100];
int n = snprintf(path, 99, "/proc/%s/fd/3", argv[1]);
if (n < 0) exit (4);
FILE* f = fopen(path, "rw+");
fprintf(f, "written\n");
exit(0);
}
shell test
⇒ cc reader.c -o reader
⇒ cc writer.c -o writer
⇒ echo XXXXXXXXXXXX > /tmp/foo
⇒ cat /tmp/foo
XXXXXXXXXXXX
⇒ ./reader &
[1] 20709
⇒ ./writer 20709
⇒ cat /tmp/foo
written
XXXX
I'm trying to write the contents of a windows directory to a file using c. For example, if I had a directory of jpegs (i.e. a directory that contains multiple jpegs) and wanted to convert them to a .raw file, I have something like this:
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdlib.h>
typedef uint8_t BYTE;
#define BLOCK 512*sizeof(BYTE);
int main(void)
{
FILE * fd = fopen("C:\\jpegs", "r");
if (fd == NULL) {
fprintf(stderr, "Error opening device file.\n");
return EXIT_FAILURE;
}
int block = BLOCK;
FILE * fn = fopen("new.raw", "w+");
void * buff = malloc(block);
while(feof(fd) == 0) {
fread(buff,block,1,fd);
fwrite(buff,block,1,fn);
}
free(buff);
fclose(fd);
fclose(fn);
return 0;
}
The problem is I don't think windows directories are terminated with EOF. Does anyone have any ideas about how to solve this?
On Unix systems, although you can open a directory for reading, you can't really read from it unless you use the opendir(), readdir(), closedir() family of calls. You can't write to a directory on Unix; even superuser (root) can't do that. (The main reason for opening a directory, more usually with open() than fopen(), is so that you can use chdir() followed by fchdir() to get back to where you started, or use the various *at() functions, such as openat(), to reference the directory.)
On Windows, you'd at minimum need to use "rb" mode, but frankly, I'd not expect you to be able to do much with it. There are probably analogues to the Unix opendir() functions in the Windows API, and you should use those instead.
I am creating a file with full permission (777) using the open system call, but when I do ls -l I can see only permission as (755). Could you please tell why file permission is not 777?
Code
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
int main()
{
int fd;
/* Creates a file with full permission*/
fd = open("test", O_CREAT | O_RDWR | O_APPEND, 0777);
if (fd = -1)
{
return -1;
}
close(fd);
}
Output
$ ls -l
-rwxr-xr-x 1 ubuntu ubuntu 0 2012-09-19 11:55 test
There's a value maintained by the system called the umask; it is a property of the process, just like the PID (process ID) or EUID (effective user ID) is. It will be set to 022 (octal), which indicates that the system should remove the group and other write permission from files that are created.
You can call umask(0); before using open() so that the mode you specify in open() won't be altered. You should certainly do this to demonstrate that umask is the issue. However, it is generally best to let the user's choice of umask prevail — I for one get very stroppy if a program doesn't obey my umask setting; it tends not to be used again after I spot and verify the problem.
The shell also has a (built-in) command umask which you can use. The 022 value is a sensible default; most of the time, you do not want just anybody writing to your files.
The permissions of a created file are restricted by the process's current umask -- your current umask is 022, so group and world write are always disabled by default. (Which is a good thing, in most cases.) If you really want a group- and world-writable file, you will need to temporarily set umask(0) while creating this file (make sure to save the old value returned by the system call, and set it back afterwards!), or "manually" set the file's permissions using chmod().
umask will return the original value of the mask, so to reset it temporarily you just need to do
#include <sys/types.h>
#include <sys/stat.h>
mode_t old_mask = umask(0);
...
umask(old_mask);
though it would be perhaps preferable to use fchmod after opening the file - this will avoid the problem of changing process-global state:
fd = open("test", O_CREAT | O_RDWR | O_APPEND, 0777);
if (fd < 0) { ... }
int rv = fchmod(fd, 0777);
if (rv < 0) { /* fchmod errored */ }
here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main(int argc, char **argv) {
int pipefd[2];
int result;
FILE *in_file;
FILE *out_file;
result = pipe(pipefd);
in_file = fopen(argv[1], "rb");
out_file = fopen(argv[2], "wb");
result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
if (result == -1)
printf("%d - %s\n", errno, strerror(errno));
close(pipefd[0]);
close(pipefd[1]);
fclose(in_file);
fclose(out_file);
return 0;
}
When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?
Thanks!
From the splice manpage:
EINVAL Target file system doesn't support splicing; target file is
opened in append mode; neither of the descriptors refers to a
pipe; or offset given for non-seekable device.
We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.
What kind of file system(s) are you copying to/from?
Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.
The splice(2) system call is for copying between files and pipes and not between files, so it can not be used to copy between files, as has been pointed out by the other answers.
As of Linux 4.5 however a new copy_file_range(2) system call is available that can copy between files. In the case of NFS it can even cause server side copying.
The linked man page contains a full example program.