I'm writing a program in C that sends files across the network using sockets. This works fine for files - they are read into a buffer and then written onto the socket. They are picked up at the other end by reversing this process.
However, how can this apply to directories? I also want to copy directories, keeping the permissions the same (so I don't think mkdir will work). At the moment when I try to run this on a directory, it says the size is -1. How is a directory represented?
To be clear, for example, if I want my program to copy /tmp across the network, it will do this:
/tmp/1.txt - OK
/tmp/2.txt - OK
/tmp/dir/ - Skip
/tmp/dir/3.txt - Can't write to path
There are several possibilities. It would fit fairly will with what you have already to tar the directory to transfer, send the resulting archive across the network, and untar on the other side.
Alternatively, you can walk the directory tree recursively. For each directory you need transfer only the name and whichever attributes you want to preserve, but then you must list the directory contents (probably via readdir()) and transfer each member.
By the way, don't neglect to think about how you're going to handle links, both symbolic ones and hard ones. And if you want your program to be really robust then consider also what to do with special files such as device files and FIFOs.
I guess it is homework, otherwise why not use FTP, scp, rsync, unison etc.
To test if a file path is a plain file, a device, a directory, etc etc... use
stat(2)
To read a directory, use opendir(3) then loop on readdir(3) (then of course closedir). You don't need to know how a directory is represented.
You probably should be interested in nftw(3) to recursively traverse a file tree.
To make one directory, use mkdir(2)
You should read Advanced Linux Programming
BTW, this answer contains useful information too...
Related
My goal is to, inside my C program, make a new directory. Within this directory, I want to create 15 simple text files. The part that I am stuck on is how to generate the 15 text files inside the newly created directory. I have created the new directory like this:
mkdir("new_dir", 0755);
But I am unsure of how to create a text file within it (in the same program). Any tips for this?
I am guessing you are on some POSIX system. The C11 standard (read n1570) does not know about directories (an abstraction provided by your operating system). If you are on Windows, it has a different WinAPI (you should then use CreateDirectory)
First, your call to mkdir(2) could fail (for a variety of reasons, including the fact that the directory did already exist). And very probably, you actually want to create the directory in the home directory, or document that you are creating it in the current working directory (e.g. leave the burden of some appropriate and prior cd shell builtin to your user). Practically speaking, the directory path should be computed at runtime as a string (perhaps using snprintf(3) or asprintf(3)).
So if you wanted to create a directory in the home directory of the user (remember that ~/foo/ is expanded by the shell during globbing, see glob(7)...; you need to fetch the home directory from environ(7)), you would code something like:
char pathbuf[256];
snprintf(pathbuf, sizeof(pathbuf), "%s/new_dir", getenv("HOME"));
to compute that string. Of course, you need to handle failure (of getenv(3), or of snprintf). I am leaving these checks as an exercise. You might want to keep the result of getenv("HOME") in some automatic variable.
Then you need to make the directory, and check against failure. At the very least (using perror(3) and see errno(3)):
if (mkdir (pathbuf, 0750)) { perror(pathbuf); exit(EXIT_FAILURE); }
BTW, the mode passed to mkdir might not allow every other user to write or access it (if it did, you could have some security vulnerability). So I prefer 0750 to yours 0755.
At last you need to create files inside it, perhaps using fopen(3) before writing into them. So some code like
int i = somenumber();
snprintf(pathbuf, sizeof(pathbuf),
"%s/new_dir/file%d.txt", getenv("HOME"), i);
FILE* f = fopen(pathbuf, "w");
if (!f) { perror(pathbuf); exit(EXIT_FAILURE); };
As Jonathan Leffler wisely commented, there are other ways.
My recommendation is to document some convention. Do you want your program to create a directory in the current working directory, or to create it in some "absolute" path, perhaps related to the home directory of your user? If your program is started by some user (and is not setuid or doesn't have root permissions, see credentials(7)) it is not permitted to create directories or files at arbitrary places (see hier(7)).
If on Linux, you'll better read some system programming book like ALP or newer. If you use a different OS, you should read the documentation of its system API.
I want to copy some files from one folder to other folder and before copying or replacing the files I want to take backup from destination folder only those file which are I am trying to copy in to source file to destination file.
This is what I want to do but is this possible in pure C-language?.
First of all you should say on which platform you want to do that.
I'm not that familiar with Windows so what i'll be saying only applies to Linux (maybe it works on windows as well but I doubt it).
Have a look on man opendir and man readdir and those are the only things you need to do what you want (you obviously will also need to open/read/write and close but I imagine that you know that).
There's no native way in C to do that. You need platform-specific code. The simplest way to achieve that is to use the system() function
system("copy C:\\Windows\\notepad.exe D:\\"); // Windows
or
system("cp ~/myfile.txt ~/mycopiedfile.txt"); // *nix
But for better performance and/or more control over the result you need to call the corresponding APIs if available (like CopyFile() on Windows), or copy the file on your own by reading from source to a buffer with fread() then write to the destination with fwrite()
See Is there a POSIX function to copy a file?
I'm trying to use a system call to display the contents of a directory. I've been pointed in the direction of vfs_readdir, but I have no clue of how to use it or what to pass to it to get the contents of a directory. All I want to do is be able to list files in a directory similar to how ls works. (I eventually intend to store this in some sort of buffer, but for now just being able to print the contents of a dir would be enough).
I think you probably have to open the directory using filp_open.
For the "flags" argument, you proably need to put some combination of the LOOKUP_ flags listed in include/linux/namei.h
You can see what build_open_flags does here: http://lxr.linux.no/#linux+v3.1.5/fs/open.c#L876 to provide flags to filp_open.
As far as I know, filp_open IS the correct way of opening a file in kernel-space. However, doing so is discouraged.
Provided you do so from the context of a "normal" thread belonging to a user process, I think you will be ok.
I'm intending to create a programme that can permanently follow a large dynamic set of log files to copy their entries over to a database for easier near-realtime statistics. The log files are written by diverse daemons and applications, but the format of them is known so they can be parsed. Some of the daemons write logs into one file per day, like Apache's cronolog that creates files like access.20100928. Those files appear with each new day and may disappear when they're gzipped away the next day.
The target platform is an Ubuntu Server, 64 bit.
What would be the best approach to efficiently reading those log files?
I could think of scripting languages like PHP that either open the files theirselves and read new data or use system tools like tail -f to follow the logs, or other runtimes like Mono. Bash shell scripts probably aren't so well suited for parsing the log lines and inserting them to a database server (MySQL), not to mention an easy configuration of my app.
If my programme will read the log files, I'd think it should stat() the file once in a second or so to get its size and open the file when it's grown. After reading the file (which should hopefully only return complete lines) it could call tell() to get the current position and next time directly seek() to the saved position to continue reading. (These are C function names, but actually I wouldn't want to do that in C. And Mono/.NET or PHP offer similar functions as well.)
Is that constant stat()ing of the files and subsequent opening and closing a problem? How would tail -f do that? Can I keep the files open and be notified about new data with something like select()? Or does it always return at the end of the file?
In case I'm blocked in some kind of select() or external tail, I'd need to interrupt that every 1, 2 minutes to scan for new or deleted files that shall (no longer) be followed. Resuming with tail -f then is probably not very reliable. That should work better with my own saved file positions.
Could I use some kind of inotify (file system notification) for that?
If you want to know how tail -f works, why not look at the source? In a nutshell, you don't need to periodically interrupt or constantly stat() to scan for changes to files or directories. That's what inotify does.
I have a build system tool that is using getcwd() to get the current working directory. That's great, except that sometimes people have spaces in their paths, which isn't supported by the build system. You'd think that you could just make a symbolic link:
ln -s "Directory With Spaces" DirectoryWithoutSpaces
And then be happy. But unfortunately for me, getcwd() resolves all the symbolic links. I tried to use getenv("PWD"), but it is not pointing at the same path as I get back from getcwd(). I blame make -C for not updating the environment variable, I think. Right now, getcwd() gives me back a path like this:
/Users/carl/Directory With Spaces/Some/Other/Directories
And getenv("PWD") gives me:
/Users/carl/DirectoryWithoutSpaces
So - is there any function like getcwd() that doesn't resolve the symbolic links?
Edit:
I changed
make -C Some/Other/Directories
to
cd Some/Other/Directories ; make
And then getenv("PWD") works.. If there's no other solution, I can use that.
According to the Advanced Programming in the UNIX Environment bible by Stevens, p.112:
Since the kernel must maintain knowledge of the current working directory, we should be able to fetch its current value. Unfortunately, all the kernel maintains for each process is the i-node number and device identification for the current working directory. The kernel does not maintain the full pathname of the directory.
Sorry, looks like you do need to work around this in another way.
There is no way for getcwd() to determine the path you followed via symbolic links. The basic implementation of getcwd() stats the current directory '.', and then opens the parent directory '..' and scans the entries until it finds the directory name with the same inode number as '.' has. It then repeats the process upwards until it finds the root directory, at which point it has the full path. At no point does it ever traverse a symbolic link. So the goal of having getcwd() calculate the path followed via symlinks is impossible, whether it is implemented as a system call or as a library function.
The best resolution is to ensure that the build system handles path names containing spaces. That means quoting pathnames passed through the shell. C programs don't care about the spaces in the name; it is only when a program like the shell interprets the strings that you run into problems. (Compilers implemented as shell scripts that run pre-processors often have problems with pathnames that contain spaces - speaking from experience.)