I am stuck/struggling with a problem I am trying in C(Linux) using API calls(only) to copy multiple input files via command line into one output file. I have searched the Internet for answers but none seem to solve.
My program allows me to specify multiple input files and one output file via the command line. For example:
./archiver file1.txt file2 file3 file4 outputfile
I read these parameters using argc/argv. For some reason when I do ls -l, ./archiver and outputfile have the same number of bytes, thus meaning none of my input files have been copied to my outputfile, just whatever was in memory (when I do cat outputfile it shows a bunch of these )
None of the contents from my input files are in my output files.
Please could you help me as after those bunch of "" I don't know what to do I have tried reading up on malloc() etc. but I don't know how to implement that or if thats even relevant here.
Any help is appreciated, thanks for your time.
file_desc_in = open(argv[i],O_RDONLY,0);
//NEED a loop to copy multiple files in...
while (!eof) {
bytes_read = read(file_desc_in, &buffer, sizeof(buffersize));
if (bytes_read > 1)
bytes_written = write(file_desc_out, &i, bytes_read);
else {
eof=1;
}
I haven't included the errors but I do have them. Thanks for replying immediately.
It'd help to see your code. There's not a lot here to go on, but I'm going to take a wild guess. I suspect you're copying the file specified by argv[0] (your program) and not getting the rest. I don't think I can do any better with what you've given.
You say you are only using API calls. What API are you talking about? The POSIX API? The standard C file I/O API?
If you are just combining input files, you don't really need to write a C program to do it. Since you are running Linux, try using the shell command cat input1 input2 input3 > output.
If you must write a C program to do it, start simple. Before you actually do any file I/O, make sure that you can interpret the input arguments correctly. Have your program simply read in the command-line input and print out something like this:
Input files: file1.txt file2.txt file2.txt
Output files: outputfile.txt
That way, you can verify that your CLI parsing code works correctly before you start worrying about file I/O. It's much easier to debug things one piece at a time.
Your outer loop needs to open each filename, and close it at the end of the loop. You close the output file at the very end, after all the input files are read.
You should also learn the difference between open, read, write and fopen, fread, fwrite.
Related
steps:
Let's say I have a C program inputFileName.c
I run inputFileName with input redirection such as ./inputFileName < file
How can I print the name of the file in my C program that I have typed in the terminal as an input redirection file?
The input redirection is a function of the shell. Your inputFileName executable see this as standard input. Depending on the exact operating system, you may be able to use system-specific functions to get the information you want, but there is not a standard means of doing so.
Input redirection can be achieved not only with the '<' symbol, but also with '|'.
program < filename
is equivalent to
cat filename | program
From there, one could go to
cat file1 file2 file3 | program
You begin to see why the initial 'stdin' for an executable cannot and does not have a "filename" associated with it.
If input comes from a pipe, there can't be an associated filename. Also if the file has been deleted or moved before closing the file descriptor, there is no associated filename. A file can have multiple names. In that case there are multiple filenames.
Given that, the "associated filename" of a file descriptor doesn't really make much sense. And even if you could get that info, using the filename in any way might make race conditions an issue.
The linux kernel does try to track an associated filename if a file descriptor was created by opening a file. But the keyword here is "tries".
If you are running Linux, you can find the filenname for standard input as a symlink under "/proc/self/fd/0". Just remember that you should not rely on that name for anything more than debug or display purposes.
I'm quite new to C programming, but I'm starting to get the hang of it.
For a school assignment, I have written a code in C to play Blackjack.
One of the requirements for the assignment, is that is has to have an option that saves the entire output (everything that's shown in cmd when the program is build and run).
I can't find how to do this anywhere.
Is there a way to write the entire output into a file?
Thanks in advance!
There are 3 ways to achieve what you want.
Use a file. This is the way I recommend. You will need the functions fopen to open a file and enter link description here to print the data in the file. I suggest you read the documentation of the functions in the links and look at the examples.
Redirect stdout to a file using freopen. This basically puts everything that you see now in the console in a file, by adding just one line.
Redirect the output of the program in a file. This is more a shell thing than a C programming technique, but I think it is worth mentioning. In an *NIX environment, the command ./a.out > file.txt will redirect the output of a.out to a file called file.txt in a similar manner freopen does.
You can pipe the stdout and stderr to a file when you build and run. For example, on Linux using the bash shell you can do try this (where the "build_script" is what you use to build and the "a.exe" is the program name):
$ ./build_script >& build_out.txt
$ ./a.exe >& run_out.txt &
The ">&" tells bash to pipe both stdout and stderr to the file. The final "&" in the second line tells bash to run the program in the background. It's hard to tell if this answer will suit your purposes since it's not clear exactly how the game is played from what you have posted. E.g., if you need to use stdin/stdout to play the game then maybe piping the "a.exe" stdout to a file might not work...
I'm assuming simple output redirection is not an option:
$ app > file.txt
Probably, you are using printf to print data to console. Instead of printf, you can use fprintf. fprintf can write data to an arbitrary file, just like printf does to the standard output (which is also a file), in this case the console.
You must first open the file where you will write the output. The command fopen will do this for you:
// this will open or create the file as text to write data.
FILE *f = fopen("my-file.txt", "w");
With the f variable (which you should check for NULL in case of error), you can pass it to fprintf to write data:
fprintf(f, "my super string: %s", string);
Note that despite the first argument being a FILE*, everything else behaves like printf. Actually you can think of printf as a wrapper where the first argument of the fprintf is always stdout.
Do not forget to close your file after you write data!
fclose(f);
This can be done once, after all the data is written to file.
Check the man pages for more info about these commands.
There are more complex (not that much actually) ways of accomplishing this, like using freopen, but I'm leaving this out of the answer. If you need more, update your answer.
EDIT
In your comment, you said you must save or not the output to a file at the end of the program. Well, the file management stuff above you still be usefull. The changes are the following:
You must store the output somewhere in order to decide whether to write to a file or not at the end of the program. Probably you are doing some data formatting with printf. You will have to change your calls from printf to snprintf. This command will write your data to a string, just as printf does prior to output it to the stdout, but it will skip the print-to-output- part.
Then, store the buffer at a list of strings and at the end of the program you write this list to the file or not.
This has some complications: you need a list of strings (a dynamically allocated array of arrays will be enough, actually); how big must your buffer be?
snprintf will return the size required to print the passed data to the buffer, no matter if the buffer is larger or smaller then the given one. But, if it is smaller, you will have to increase its size (reallocating it) and call again snprintf:
char *buffer = malloc(sizeof(char) * 41); // 40 chars + \0!
int size;
if ( size = snprintf(buffer, 41, "your format here", args) > 40 ) {
buffer = realloc(buffer, sizeof(char) * (size + 1));
snprintf(buffer, size + 1, "your format here", args);
}
// now your buffer have the correct data!
// print it to stdout!
puts(buffer);
// store it at the string list.
// execise!
Is left as an exercise wrapping this piece of code in a command to avoid repeat it everywhere you print anything. Also, it is an exercise to create the list of strings.
When you decide to write data to file, instead of use fprintf you can use fputs, as data is already formatted.
I don't know if there is an easier way. I think not.
I am working in OSX and using bash for my shell. I have a script which calls an executable hundreds of times, and each call is independent of the other. Therefore I am going to run this code in parallel. However, each call to the executable appends output to a community text file on a new line.
The ordering of the text file is not of importance (although it would be nice, but totally not worth over complicating since I can just use unix sort command), but what is, is that every call of the executable properly printed to the file. My concern is that if I run the script in parallel that the by some freak accident, two threads will check out the text file, print to it and then save different copies back to the original directory of the text file. Thus nullifying one of the writes to the file.
Does this actually happen, or is my understanding of printing to a file flawed? I don't fully know if this would also be a case by case bases so I will provide some mock code of what is being done in my program below.
Script:
#!/bin/sh
abs=$1
input=$(echo "$abs" | awk '{print 0.004 + 0.005*$1 }')
./program input
"./program":
~~Normal .c file stuff here~~
~~VALUE magically calculated here~~
~~run number is pulled out of input and assigned to index for sorting~~
FILE *fpp;
fpp = fopen("Doc.txt","a");
fprintf(fpp,"%d, %.3f\n", index, VALUE);
fclose(fpp);
~Closing events of program.c~~
Commands to run script in parallel in bash:
printf "%s\n" {0..199} | xargs -P 8 -n 1 ./program
Thanks for any help you guys can offer.
A write() call (like fwrite()) with the append flag set in open() (like during fopen()) is guaranteed to avoid the race condition you describe.
O_APPEND
If set, the file offset shall be set to the end of the file prior to each write.
From: POSIX specifications for open:
opengroup.org open
Race conditions are what you are thinking of.
Not 100% sure but if you simple append to the end of the file rather than opening it and editing it should be right
If you have the option, make your program write to standard output instead of directly to a file. Then you can let the shell merge the output of your programs:
printf "%s\n" {0..199} | parallel -P 8 -n 1 ./program > merged_output.txt
Yeah, that looks like a recipe for disaster. If those processes both hit opening the file at the roughly the same time, only one will "take".
I suggest either (easier) writing to separate files then catting them together when the processing is done, or (harder) sending all results to a consumer process that will write the file for everyone.
Input files to test project 2. Intended to be used via redirection.
My professor gave us a txt file to use to test if our program works. It reads in ~1000 numbers (so we wouldn't have to manually enter them). But I don't know the linux command on how to use this txt file.
ccarri7#ubuntu:~/C$ ls
ccarri7lab2 ccarri7lab2.c lab2input.txt
ccarri7#ubuntu:~/C$
This is the folder where my executable/source/txt file are.
./ccarri7lab2 < lab2input.txt
will use the text in lab2input.txt as arguments for ccarri7lab2
http://linux.about.com/od/itl_guide/a/gdeitl42t01.htm
Did you try
ccarri7lab2 < lab2input.txt
Hope this is what you want, else you can give some more info.
I am working with a text file, which contains a list of processes under my programs control, along with relevant data.
At some point, one of the processes will finish, and thus will need to be removed from the file (as its no longer under control).
Here is a sample of the file contents (which has enteries added "randomly"):
PID=25729 IDLE=0.200000 BUSY=0.300000 USER=-10.000000
PID=26416 IDLE=0.100000 BUSY=0.800000 USER=-20.000000
PID=26522 IDLE=0.400000 BUSY=0.700000 USER=-30.000000
So for example, if I wanted to remove the line that says PID=26416.... how could I do that, without writing the file over again?
I can use external unix commands, however I am not very familiar with them so please if that is your suggestion, give an example.
Thanks!
Either you keep the contents of the file in temporary memory and then rewrite the file. Or you could have a file for each of the PIDs with the relevant information in them. Then you simply delete the file when it's no longer running. Or you could use a database for this instead.
As others have already pointed out, your only real choice is to rewrite the file.
The obvious way to do that with "external UNIX commands" would be grep -v "PID=26416" (or whatever PID you want to remove, obviously).
Edit: It is probably worth mentioning that if the lines are all the same length (as you've shown here) and order doesn't matter, you could delete a line more efficiently by copying the last line into the space being vacated, then shorten the file so eliminate what had been the last line. This will only work if they really are all the same length though (e.g., if you got a PID of '1', you'd need to pad it to the same length as the others in the file).
The only way is by copying each character that comes after the deleted line down over the characters that are deleted.
It is far more efficient to simply rewrite the file.
how could I do that, without writing the file over again?
You cannot. Filesystems (perhaps besides more esoteric record based ones) does not support insertion or deletion.
So you'll have to write the lines to a temporary file up till the line you want to delete, skip over that line, and write the rest of the lines to the file. When done, rename/copy the temp file to the original filename
Why are you maintaining these in a text file? That's not the best model for such a task. But, if you're stuck with it ... if these lines are guaranteed to all be the same length (it appears that way from the sample), and if the order of the lines in the file doesn't matter, then you can write the last line over the line for the process that has died and then shorten the file by one line with the (f)truncate() call if you're on a POSIX system: see Jonathan Leffler's answer in How to truncate a file in C?
But note carefully netrom's answer, which gives three different better ways to maintain this info.
Also, if you stick with a text file (preferably written from scratch each time from data structures you maintain, as per netrom's first suggestion), and you want to be sure that the file is always well formed, then write the new data into a temp file on the same device (putting it in the same directory is easiest) and then do a rename() call, which is an atomic operation.
You can use sed:
sed -i.bak -e '/PID=26416/d' test
-i is for editing in place. It also creates a back-up file with the new extension .bak
-e is for specifying the pattern. The /d indicates all lines matching the pattern should be deleted.
test is the filename
The unix command for it is:
grep -v "PID=26416" myfile > myfile.tmp
mv myfile.tmp myfile
The grep -v part outputs the file without the rows with the search term.
The > myfile.tmp part creates a new temp file for this output.
The mv part renames the temp file to the original file.
Note that we are rewriting the file here, and moreover, we can lose data if someone write something to file between the two commands.