I'm trying to implement append command in my own shell.
I succeeded to append to existing file but whenever I'm trying to append to file doesn't exist it makes a file without any permission (not read and not write)
if (append) {
fd = open(outfile,'a');
lseek(fd,0,SEEK_END);
close (STDOUT_FILENO) ;
dup(fd);
close(fd);
/* stdout is now appended */
}
What should I do to make a file with permissions
?
The open() system call doesn't use a character constant to indicate 'append'. Read the POSIX specification for open() — and look at O_APPEND etc. You need more flags than just O_APPEND, and you need three arguments to open() if you want to create the file if it doesn't exist (O_CREAT), etc.
if (append)
{
fd = open(outfile, O_CREAT|O_APPEND|O_WRONLY, 0644);
if (fd < 0)
…deal with error…
}
You can write 0644 as S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH but the octal is shorter and (after 30+ years practice) a lot easier to read. You can add write permissions for group (S_IWGRP) and others (S_IWOTH) if you like (0666 in octal), but unless you know you want group members and others to modify the files, it is safer to omit those — for all it goes against historical precedent. Users can and should set the shell umask value to 022 to prevent group and others from being able to write to files by default, but there's no harm (IMO) in being secure without that.
Related
I am trying to implement my own shell in which i have to make a shell feature by which i will able to append text to file by using >>.here is my code....
int filedesc = open(inputargs[limit-1],O_WRONLY | O_APPEND);
//printf("%s\n",inputargs[limit-1]);
if(filedesc < 0) {
printf("Error opening file\n");
}
else{
dup2(filedesc,1);
}
filedesc always returning -1.
Transferring an edited form of my relevant comments into an answer.
Using O_APPEND is correct — and you do need O_WRONLY too — and maybe you should add O_CREAT in case it doesn't exist yet (but you definitely don't want O_TRUNC). I suppose O_RDWR would also work (in place of O_WRONLY), but it is not a good idea.
Does the file exist to be appended to? If not, add O_CREAT and a file mode argument. Tradition says "use 0666" (aka S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH — there's a reason people prefer typing the octal number!). I'd be sorely tempted to not grant write to group or others (using 0644), even though you can (and I do) use umask 022 to suppress those permissions.
What do you get from perror(inputargs[limit-1]); (instead of or as well as the printf() message? That will tell you why the system thinks it can't open the file. And don't forget that the system is right — it can't open the file. Your code has a problem — it is not yet clear what that is. It might be EPERM (no permission); it might be ENOENT (no such file or directory); it might be something else, but those two are the most likely.
Getting "permission denied" from perror(inputargs[limit-1]);
So, you don't have permission to write on the file — what are you going to do about that? Change the file name to one you have permission to write on, or set the permissions on the file you have so that you can write (append) to it? Or something else? If you, somewhere along the line, create the file using O_CREAT but don't specify the third (optional) argument to open(), you get quasi-random weird permissions on the file that you create. What does ls -l file-to-append-to tell you? (Note that the third argument is mandatory when O_CREAT is one of the options used in the call to open() — it is optional and ignored otherwise.)
Note that you should always write error messages in a shell to standard error and not to standard output. Even if there's I/O redirection on standard error, the shell still writes the error messages to standard error, and the redirection points it where the user wanted it to go.
I'm a newbie linux programmer that ordered to be a sitin for my colleague.
I opened up a file with this line:
err = open("path/foo.txt", O_RDWR | O_CREAT,0777);
And now I write to it, and hexdump the file, the output showed that I appended
new content instead of overwriting the original.
How do I overwrite?
Also if you ask me "wth is "open"?" I'd refer to the newbie defense and say I don't know. The closest thing I know is fopen but I don't know what library/framework my colleague is using. Posix perhaps?
If you open the manual for open(2) via man 2 open, you will find a list of accepted flags.
You should be reading man pages in any case and it's not hard, the only things you need to remember is that you can search for keywords by pressing /, n/N for jumping between things that were found, and q to exit when you're done, everything else is superfluous and may only speed things up a little.
The flag you're looking for is
O_TRUNC
If the file already exists and is a regular file and the access mode allows
writing (i.e., is O_RDWR or O_WRONLY) it will be truncated to length 0.
If the file is a FIFO or terminal device file, the O_TRUNC flag is ignored.
Otherwise, the effect of O_TRUNC is unspecified.
So all you have to do is add this flag whenever you want to overwrite the file completely.
But if you can, you should be using fopen(3), in which case the mode you're looking for is w+, and it is equivalent to O_RDWR | O_CREAT | O_TRUNC in open(2).
open(2) is a low level system call and reading from a raw file descriptor efficiently and correctly is not that simple, FILE* you get from fread(3) on the other hand is an implementation that handles most of details like buffering for you, and unless you can do better, you shouldn't be avoiding it.
By the way, 0777 file permission is rarely needed and doesn't apply to regular files, I'd recommend you open normal text files with 0666, which does everything you need except does not enable it to be executed as a program.
I'm new to Linux, still struggling to understand how permisson control work in Linux. The open function prototype is sth like :
int open(char *filename, int flags, mode_t mode);
Let's I have the following code:
fd = open("foo.txt", O_CREAT|O_RDWR, S_IRUSR)
and let's say the file "foo.txt" didn't exist before, so the above statment will create a file called "foo.txt", and current process who executes this open statment can read and write this file. But after this process terminates, If another process starts and tries to open this file. Below is my question:
Q1-Since the file was created with S_IRUSR(owner can read this file) in the first open call, does it mean that even I as owner of the file, if I start a new process to open this file again, I can only read this file and I cannot write this file, is my understanding correct?
If my understanding is correct, is it sensible/practicable that owners create sth that they cannot have full access to it later?
Q2-If my above understanding is correct, then in the second call to open by a new process. I can only call like:
fd = open("foo.txt", O_RDONLY) // uses flags like O_WRONLY or O_RDWR will throw an error?
since the first open specified the mode as S_IRUSR, which maps to O_RDONLY in the subsequent calls, is my understanding correct?
Correct, if you create the file with permissions S_IRUSR (often written in octal as 0400), then you will not be able to open the file for writing. Attempting to do so will fail and set errno to EACCES.
This is quite practical as it gives you a way to protect files you do not want to accidentally overwrite, as long as the permissions stay as they are. However, as the owner, you have the power to change the permissions later, using the chmod() system call. So it's not as though you have permanently lost the ability to write that file; you can give yourself back that ability whenever you want.
I've been looking for quite a while and cannot find the answer to my question.
I'm trying to reproduce a shell in C, with full redirections. In order to do this, I wanted to open the file before executing my command.
For example, in ls > file1 > file2, I use dup2(file1_fd, 1) and dup2(file2_fd, 1) and then I execute ls to fill the files, but it seems a standard output can only be open once so only file2 will be filled, because it was the last one to be duplicated.
Is there a way to redirect standard output to multiple file?
Is there something I am missing? Thanks!
Is there a way to redirect standard output to multiple files?
Many file descriptors cannot be made one file descriptor. You need to write into each file descriptor separately. This is what tee utility does for you.
What you are asking for is the exact reason why the tee command exists (you can take a look at its source code here).
You cannot duplicate a file descriptor using dup2() multiple times. As you already saw, the last one overwrites any previous duplication. Therefore you cannot redirect the output of a program to multiple files directly using dup2().
In order to do this, you really need multiple descriptors, and therefore you would have to open both files, launch the command using popen() and then read from the pipe and write to both files.
Here is a very simple example of how you could do it:
#include <stdio.h>
#include <stdlib.h>
#define N 4096
int main(int argc, const char *argv[]) {
FILE *fp1, *fp2, *pipe;
fp1 = fopen("out1.txt", "w");
if (fp1 == NULL) {
perror("fopen out1 failed");
return 1;
}
fp2 = fopen("out2.txt", "w");
if (fp2 == NULL) {
perror("fopen out2 failed");
return 1;
}
// Run `ls -l` just as an example.
pipe = popen("ls -l", "r");
if (pipe == NULL) {
perror("popen failed");
return 1;
}
size_t nread, nwrote;
char buf[N];
while ((nread = fread(buf, 1, N, pipe))) {
nwrote = 0;
while (nwrote < nread)
nwrote += fwrite(buf + nwrote, 1, nread - nwrote, fp1);
nwrote = 0;
while (nwrote < nread)
nwrote += fwrite(buf + nwrote, 1, nread - nwrote, fp2);
}
pclose(pipe);
fclose(fp2);
fclose(fp1);
return 0;
}
The above code is only to give a rough estimate on how the whole thing works, it doesn't check for some errors on fread, fwrite, etc: you should of course check for errors in your final program.
It's also easy to see how this could be extended to support an arbitrary number of output files (just using an array of FILE *).
Standard output is not different from any other open file, the only special characteristic is for it to be file descriptor 1 (so only one file descriptor with index 1 can be in your process) You can dup(2) file descriptor 1 to get, let´s say file descriptor 6. That's the mission of dup() just to get another file descriptor (with a different number) than the one you use as source, but for the same source. Dupped descriptors allow you to use any of the dupped descriptors indifferently to output, or to change open flags like close on exec flag or non block or append flag (not all are shared, I'm not sure which ones can be changed without affecting the others in a dup). They share the file pointer, so every write() you attempt to any of the file descriptors will be updated in the others.
But the idea of redirection is not that. A convention in unix says that every program will receive three descriptors already open from its parent process. So to use forking, first you need to consider how to write notation to express that a program will receive (already opened) more than one output stream (so you can redirect any of them properly, before calling the program) The same also applies for joining streams. Here, the problem is more complex, as you'll need to express how the data flows might be merged into one, and this makes the merging problem, problem dependant.
File dup()ping is not a way to make a file descriptor to write in two files... but the reverse, it is a way to make two different file descriptors to reference the same file.
The only way to do what you want is to duplicate write(2) calls on every file descriptor you are going to use.
As some answer has commented, tee(1) command allows you to fork the flow of data in a pipe, but not with file descriptors, tee(1) just opens a file, and write(2)s there all the input, in addition to write(2)`ing it to stdout also.
There's no provision to fork data flows in the shell, as there's no provision to join (in paralell) dataflows on input. I think this is some abandoned idea in the shell design by Steve Bourne, and you'll probably get to the same point.
BTW, just study the possibility of using the general dup2() operator, which is <n>&m>, but again, consider that, for the redirecting program, 2>&3 2>&4 2>&5 2>&6 mean that you have pre-opened 7 file descriptors, 0...6 in which stderr is an alias of descpritors 3 to 6 (so any data written to any of those descriptors will appear into what was stderr) or you can use 2<file_a 3<file_b 4<file_c meaning your program will be executed with file descriptor 2 (stderr) redirected from file_a, and file descriptors 3 and 4 already open from files file_b and file_c. Probably, some notation should be designed (and it doesn't come easily to my mind now, how to devise it) to allow for piping (with the pipe(2) system call) between different processes that have been launched to do some task, but you need to build a general graph to allow for generality.
If you look at this code block below by taking into consideration the last parameter "0", Does write line work properly ?
filename = argv[1];
string = "Example string";
if (stat(argv[1], &buf) != 0)
{
fd = open(filename, O_WRONLY | O_CREAT, 0);
if (fd < 0)
{
perror(filename);
exit(1);
}
write(fd, string, strlen(string));
close(fd);
}
else
{
print("%s file exists\n", filename);
}
From the manpage:
mode specifies the permissions to use in case a new file is created. This argument must be supplied when O_CREAT is specified in flags; if O_CREAT is not specified, then mode is ignored. The effective permissions are modified by the process's umask in the usual way: The permissions of the created file are (mode & ~umask). Note that this mode applies only to future accesses of the newly created file; the open() call that creates a read-only file may well return a read/write file descriptor.
The following symbolic constants are provided for mode:
S_IRWXU 00700 user (file owner) has read, write and execute permission
S_IRUSR 00400 user has read permission
S_IWUSR 00200 user has write permission
S_IXUSR 00100 user has execute permission
S_IRWXG 00070 group has read, write and execute permission
S_IRGRP 00040 group has read permission
S_IWGRP 00020 group has write permission
S_IXGRP 00010 group has execute permission
S_IRWXO 00007 others have read, write and execute permission
S_IROTH 00004 others have read permission
S_IWOTH 00002 others have write permission
S_IXOTH 00001 others have execute permission
So, specifying a mode of zero, you will create a file with the permissions of 0 & ~umask, i.e. a file without any permissions.
What exactly the filesystem makes of this is not in the domain of the open() or write() functions.
It is valid,
This is from open(2) Linux manual pages
The mode argument specifies the file mode bits be applied when a new file is created. This argument must be supplied when O_CREAT or O_TMPFILE is specified in flags; if neither O_CREAT nor O_TMPFILE is specified, then mode is ignored. The effective mode is modified by the process's umask in the usual way: in the absence of a default ACL, the mode of the created file is (mode & ~umask). Note that this mode applies only to future accesses of the newly created file; the open() call that creates a read-only file may well return a read/write file descriptor.
In theory then, your access to the file will be valid until you call close() as I understand the part I highlighted in the above excerpt.
Interesting question. POSIX says:
The argument following the oflag argument does not affect whether the file is open for reading, writing, or for both.
Which means that since you're handling the error return from open, if you reach the write line the behavior is well defined.
To expand a bit why this works. On most filesystems on unix-like systems, the meta-data related to a file should not affect already open file descriptors. You can for example remove a file that you have opened. This is in fact done quite commonly with temporary files, so that you don't need to remember to delete them on exit. The same applies to permissions or even ownership of the file. In fact, you can chroot while holding a file open and you can still write to it without actually being able to see it. You can even use file descriptor passing to give an open file descriptor to another process that wouldn't be allowed to open that file. This is quite commonly used for privilege separation. The permissions you had when creating a file descriptor are valid regardless of the changes to permissions later. So your question is a very interesting edge case because it asks if the filesystem permissions of the file are set before or after we create a file descriptor for it and POSIX seems to be clear on that.
I can only think of two exceptions to that right now. First is when someone forcibly remounts a filesystem to read-only in that case the kernel will go through horrifying gymnastics to invalidate your file descriptor which will make all its operations fail. Second one is AFS where your permissions are actually checked when you close the file (or, when the last user of the file on your local system closes it which sends it to the server), which leads to hilarious problems where your time-limited access tokens were valid when you opened a file but aren't valid any longer when you close it. This is also why close returns errors (but that's another rant).
This is why I mentioned error handling above. Even though POSIX says that it should not have an effect, I could see AFS or certain other file systems refusing to open such a file.