It looks like this question is pretty simple but I can't find the clear solution for copying files in C without platform dependency.
I used a system() call in my open source project for creating a directory, copying files and run external programs. It works very well in Mac OS X and other Unix-ish systems, but it fails on Windows. The problem was:
system( "cp a.txt destination/b.txt" );
Windows uses backslashes for path separator. (vs slashes in Unix-ish)
Windows uses 'copy' for the internal copy command. (vs cp in Unix-ish)
How can I write a copying code without dependency?
( Actually, I wrote macros to solve this problems, but it's not cool. http://code.google.com/p/npk/source/browse/trunk/npk/cli/tests/testutil.h, L22-56 )
The system() function is a lot more trouble than it's worth; it invokes the shell in a seperate proccess, and should usually be avoided.
Instead fopen() a.txt and dest/b.text, and use getc()/putc() to do the copying (because the standard library is more likely to do page-aligned buffering than you)
FILE *src = fopen("a.txt", "rb");
FILE *dst = fopen("dest/b.txt", "wb");
int i;
for (i = getc(src); i != EOF; i = getc(src))
{
putc(i, dst);
}
fclose(dst);
fclose(src);
You need to use the C standard library functions in stdio.h.
In particular, fopen, fread, fwrite, and fclose will be sufficient.
Be sure to include the b ("binary") option in the flags to fopen.
[edit]
Unfortunately, the file names themselves (forward-slashes vs. back-slashes) are still platform dependent. So you will need some sort of #ifdef or similar to deal with that.
Or you can use a cross-platform toolkit.
Use the standard C library stdio.h. First open input file for reading using fopen(inputFilename, "rb") and open output file for writing using fopen(outputFilename, "wb"), copy the content using fread and fwrire. Then close both files using fclose.
Related
These terms may not be 100% accurate, but I'm using the GCC compiler and POSIX library. I have C code compiled with the SQLite amalgamation file to a single executable.
In the user interface that exchanges JSON messages with the C program, I'd like to make it possible for users to copy the SQLite database files they create through the C program, and copy a full directory/folder.
Thus far, I've been able to rename and move files and folders programmatically.
I've read many questions and answers here, at Microsoft's C runtime library, and other places but I must be missing the fundamental points. I'm using regular old C, not C++ or C#.
My question is are there POSIX functions similar to rename(), _mkdir(), rmdir(), remove(), _stat(), that allow for programmatic copying of files and folders in Windows and Linux?
If not, can one just make a new folder and/or file and fread/fwrite the bytes from the original file to the new file?
I am primarily concerned with copying SQLite database files, although I wouldn't mind knowing the answer in general also.
Is this answer an adequate method?
Is the system() function a poor method? It seems to work quite well. However, it took awhile to figure out how to stop the messages, such as "copied 2 files" from being sent to stdout and shutting down the requesting application since it's not well-formed JSON. This answer explains how and has a link to Microsoft "Using command redirection operators". A /q in xcopy may or may not be necessary also, but certainly didn't do the job alone.
Thank you very much for any direction you may be able to provide.
The question that someone suggested as an answer and placed the little submission box on this question is one that I had already linked to in my question. I don't mean to be rude but, if it had answered my question, I would not have written this one. Thank you whoever you are for taking the time to respond, I appreciate it.
I don't see how that would be a better option than using system() because with the right parameters all the sub-directories and files of a single parent folder can be copied in one statement without having to iterate through all of them manually. Is there any reason why it would not be better to use system() apart from the fact that code will need to be different for each OS?
Handling errors are a bit different because system() doesn't return an errno but an exit code; however, the errors can be redirected from stderr to a file and pulled from there, when necessary
rename(): posix
_mkdir(): not posix. You want mkdir which is. mkdir takes two arguments, the second of which should usually be 077.
rmdir(): posix
remove(): posix
_stat(): not posix, you want stat() which is.
_stat and _mkdir are called as such on the Windows C library because they're not quite compatible with the modern Unix calls. _mkdir is missing an argument, and _stat looks like a very old version of the Unix call. You'll have trouble on Windows with files larger than 2GB.
You could do:
#ifdef _WIN32
int mkdir(const char *path, int mode) { return _mkdir(path); } /* In the original C we could have #defined this but that doesn't work anymore */
#define stat _stat64
#endif
but if you do so, test it like crazy.
In the end, you're going to be copying stuff with stdio; this loop works. (beware the linked answer; it has bugs that'll bite ya.)
int copyfile(const char *src, const char *dst)
{
const int bufsz = 65536;
char *buf = malloc(bufsz);
if (!buf) return -1; /* like mkdir, rmdir, return 0 for success, -1 for failure */
FILE *hin = fopen(src, "rb");
if (!hin) { free(buf); return -1; }
FILE *hout = fopen(dst, "wb");
if (!hout) { free(buf); fclose(hin); return -1; }
size_t buflen;
while ((buflen = fread(buf, 1, bufsz)) > 0) {
if (buflen != fwrite(buf, 1, buflen)) {
fclose(hout);
fclose(hin);
free(buf);
return -1; /* IO error writing data */
}
}
free(buf);
int r = ferror(hin) ? -1 : 0; /* check if fread had indicated IO error on input */
fclose(hin);
return r | (fclose(hout) ? -1 : 0); /* final case: check if IO error flushing buffer -- don't omit this it really can happen; calling `fflush()` won't help. */
}
I'm trying to learn File I/O concepts in C programming language. I'm using GNU / Linux ( Ubuntu 16.04 LTS ) and my IDE is eclipse 3.8. when I try to write in a file through fprintf() method, it doesn't create any files or if the file is even created, it doesn't write in it. I tried to fix the problem by using fflush() or setbuf(file_pointer, NULL) methods as is suggested here but still no change. I guess I'm writing the address of the file in a wrong way.
Here is the code:
#include <stdio.h>
int main(void){
FILE *file_pointer;
file_pointer=fopen("~/.textsfiless/test.txt","w+");
setbuf(file_pointer,NULL);
fprintf(file_pointer,"Testing...\n");
fclose(file_pointer);
return EXIT_SUCCESS;
}
Can someone explain what's wrong here?
On Linux, the ~ in ~/.textsfiless/test.txt is not expanded by the C library fopen... When you use ~ on the command line, it is expanded by your shell (but not by the program using it, started by the shell doing some execve(2)...) into your home directory; the expansion is called globbing. Read glob(7). You are very unlikely to have a directory named ~.
You should read Advanced Linux Programming
So you should check if fopen failed (it is very likely that it did fail). If you want to get a file in the home directory, you'll better use getenv(3) with "HOME" (or perhaps getpwuid(3) & getuid(2)...). See environ(7)
Perhaps a better code might be:
char*homedir = getenv("HOME");
if (!homedir) { perror("getenv HOME"); exit(EXIT_FAILURE); };
char pathbuf[512]; /// or perhaps PATH_MAX instead of 512
snprintf(pathbuf, sizeof(pathbuf),
"%s/.textsfiless/test.txt", homedir);
FILE *file_pointer = fopen(pathbuf, "r");
if (!file_pointer) { perror(pathbuf); exit(EXIT_FAILURE); };
and so on.
Notice that you should check against failures most C standard library (& POSIX) functions. The perror(3) function is useful to report errors to the user on stderr.
(pedantically, we should even test that snprintf(3) returns a length below sizeof(pathbuf) or use and test against failure asprintf(3) instead; I leave that test as an exercise to the reader)
More generally, read the documentation of every external function that you are using.
Beware of undefined behavior (your code is probably having some, e.g. fprintf to a NULL stream). Compile your code with all warnings & debug info (so gcc -Wall -g) and use the gdb debugger. Read What every C programmer should know about undefined behavior.
BTW, look into strace(1) and try it on your original (faulty) program. You'll learn a lot about the system calls used in it.
Most likely your call to fopen() fails. You don't have any checking in your program to ensure fopen even worked. It may not have, and this could be due to a variety of things, like you spelling the path wrong, wrong file or process permissions, etc.
To see what really happened, you should check fopen's return value:
#include <stdio.h>
int main(void){
FILE *file_pointer;
file_pointer=fopen("~/.textsfiless/test.txt","w+");
if (file_pointer == NULL) {
printf("Opening the file failed.");
return EXIT_FAILURE;
}
setbuf(file_pointer,NULL);
fprintf(file_pointer,"Testing...\n");
fclose(file_pointer);
return EXIT_SUCCESS;
}
Edit: Since your comment, you getting the path wrong is most certainly what happened. If you're executing your program from the current directory, and your file is in a folder called "textfiless" in your current directory and your file is called "test.txt", then you'd call fopen like this:
file_pointer=fopen("/textsfiless/test.txt","w+");
I want to generate a temporary file with a "nice" name like
my-app-Mar27-120357-Qf3K0a.html
while following the best practices for security.
POSIX offers me mkstemp(3) which takes a filename template (typically something like /tmp/my-app-XXXXXX) but it has two problems:
I need to choose the output directory myself. When I see glibc tempnam(3) (which is deprecated for a security reason) considers many factors, I wish to let the library function choose it.
There's no extension in the file name
The second item can be addressed by mkstemps(3) which takes a number of characters to keep as a user-defined extension. In my case, I can pass my-app-Mar27-120357-XXXXXX.html and 5
but it has its own problems:
I still need to choose the output directory
It isn't perfectly portable. NetBSD seems to lack it.
So I'm considering to use the deprecated tempnam(3) to generate a filename with the output directory path, overwrite the filename part with X and feed it to mkstemp(3), and then rename the file to my preferred format. So the problem lies in the last step, renaming without overwrite; is it possible in POSIX?
Or could there be any better alternatives?
Let mkstemp make the file it wants to make, in the POSIX-compliant way that it wants to. Use symlink to make a symbolic link from a source file and path of your choice to a destination that matches whatever comes from using mkstemp. Remove the symbolic link when you're done.
Another approach is to simply munge the template and add your path. We describe such a function in the BEDOPS toolkit here, used by the sort-bed application to allow the end user to specify where temporary intermediate files are stored: https://github.com/bedops/bedops/blob/6da835468565dfc30a3fcb65807e91fcf133ea2b/applications/bed/sort-bed/src/SortDetails.cpp#L115
FILE *
createTmpFile(char const* path, char** fileName)
{
FILE* fp;
int fd;
char* tmpl;
if (path == NULL)
{
fileName = NULL;
return tmpfile();
}
tmpl = static_cast<char*>( malloc(1 + strlen(path) + L_tmpnam) );
strcpy(tmpl, path);
strcpy(tmpl+strlen(path), "/sb.XXXXXX");
fd = mkstemp(tmpl);
if(fd == -1)
{
fprintf(stderr, "unable to create temp file!\n");
return NULL;
}
fp = fdopen(fd, "wb+");
*fileName = static_cast<char*>( malloc(strlen(tmpl) + 1) );
strcpy(*fileName, tmpl);
free(tmpl);
return fp;
}
This uses the L_tmpnam macro, part of the stdio library, to set the number of characters that the variable tmpl (the filename, ultimately) can store.
This compiles and works under Linux and OS X (BSD) hosts and also uses POSIX routines.
It is more complex than my other solution but it might work better for your use case.
I've got a basic program that's designed to copy the functionality of bash's cp command. I'm developing a copy for UNIX and for Windows. My UNIX version works fine, however, I'm finding that Windows doesn't have support for the "wx" mode option for fopen(), as in the following line:
file2 = fopen(argv[2], "wx");
Is there an alternative way to mirror the wx functionality mode for fopen here?
(wx allows for opening a file with write access, but will return an error if a file with the same filename already exists--meaning you won't override the existing file. See here.
note: attempting to run the program in Developer Command Prompt for VS2013
The short answer is that you cannot pass "wx" or any equivalent to fopen that will yield a CreateFile with CREATE_NEW. fopen simply does not accept any parameter combination to yield that - it's very limited. You can see the source code yourself for fopen in the Visual Studio CRT code base!
However you can instead call CreateFile directly. This is probably the best approach.
Alternatively you can call _open (http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx) which will take the parameter _O_EXCL which can yield CREATE_NEW and thus will cause it to fail if the file exists as you want.
From the CRT:
case _O_CREAT | _O_EXCL:
case _O_CREAT | _O_TRUNC | _O_EXCL:
filecreate = CREATE_NEW;
break;
How to move a particular file from one folder to another folder?
What I have tried,
#include <stdio.h>
int main() {
FILE *tFile;
if (tFile != NULL)
tFile = NULL;
if ((tFile = fopen("TempFile.txt", "rw")) == NULL) {
return -1;
}
mv("TempFile.txt", "../MST");
printf("Done Succesfully\n");
return 0;
}
Error :
test.c:17:2: warning: no newline at end of file
/tmp/ccKLWYNa.o(.text+0x5e): In function `main':
: undefined reference to `mv'
collect2: ld returned 1 exit status
Please guide me how can I do this.
You really should read Advanced Linux Programming and syscalls(2)
To move (from C) a file from one place to another in the same file system just use the rename(2) syscall.
At the very least, for your particular example, you'll need to code:
char* srcpath = "TempFile.txt"; // assume it is a variable path
char destpath[1024];
snprintf (destpath, sizeof(destpath), "../MST/%s", srcpath);
if (rename (srcpath, destpath)) {
// something went wrong
if (errno == EXDEV) {
// copy data and meta data
} else { perror("rename"); exit(EXIT_FAILURE); };
}
else { // the rename succeeded
}
If you really want to mv TempFile.txt ../MST/TempFile.txt specifically for TempFile.txt only you could just call rename("TempFile.txt", "../MST/TempFile.txt") and handle the error cases like I suggest. If you are sure that ../MST/lie in the same file system than . then EXDEV should not happen and you don't need to handle it particularly (but you do need to handle errors).
If you want to move a file between two different file systems, you have to copy the data (and perhaps some of the meta-data) yourself (and then remove e.g. with unlink(2)) the original source file). You could detect that situation by various means: you could just try the rename and if errno (see errno(3)) is EXDEV you need to copy the file. Or you could use stat(2) to query the source file(and the destination directory) meta-data -e.g. its size and its file system.
Of course, you need to understand what are files on Linux (or Posix), in particular what is an inode.... See inode(7) and credentials(7)
You could have used system with /bin/mv (but be careful about strange characters -like spaces or semicolons- in the file paths, you need to escape them to avoid code injection), apparently you don't want to.
You should play with strace(1) (or perhaps also ltrace) on mv in various situations to understand what it is doing. Also, study the source code of GNU coreutils which provides /bin/mv notably in mv.c ...
Some extra C or C++ libraries may provide you with functions to move files (in the same filesystem they should do a rename, in different file systems they copy the source file data and perhaps some meta-data and unlink the source, so cannot be atomic), e.g. in C g_file_move (from Gio with Glib from Gnome), or in C++ copy_file -followed by remove in Boost, etc etc....
PS. For temporary files see tmpfile(3), mkstemp(3), etc...