Let me explain what I'm trying to realize:
I have a encrypted tar file. I can decrypt it in memory, but obviously I can't write the decrypted data back to hard disk as a real file. The decrypted data is structured as a char* buffer in memory; how can I untar it in memory?
I can't find answer with libtar library.
I also tried to untar it with execlp("tar", "tar", "-xvO", (void*)0).
But it didn't work as I thought.
Anyone can give me a hint of the best solution? Thanks!
I suspect that libtar is the answer.
Using libtar, you can specify your own functions for opening/closing, reading and writing.
From the manpage:
int tar_open(TAR **t, char *pathname, tartype_t *type, int oflags,
int mode, int options);
The tar_open() function opens a tar archive file corresponding to the
filename named by the pathname argument. The oflags argument must be
either O_RDONLY or O_WRONLY.
The type argument specifies the access methods for the given file
type. The tartype_t structure has members named openfunc(), closefunc(),
readfunc() and writefunc(), which are pointers to the functions for opening,
closing, reading, and writing the file, respectively. If type is NULL,
the file type defaults to a normal file, and the standard open(),
close(), read(), and write() functions are used.
I made an example, how to read file contents from an in-memory tar. The is_file_in_tar() function returns the length and the starting position of the named file if it is stored in the tar:
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <sys/mman.h>
struct tar {
char name[100]; char _unused[24];
char size[12]; char _padding[376];
} *tar;
int is_file_in_tar( struct tar *tar, char *name, char **start, int *length ){
for( ; tar->name[0]; tar+=1+(*length+511)/512 ){
sscanf( tar->size, "%o", length);
if( !strcmp(tar->name,name) ){ *start = (char*)(tar+1); return 1; }
}
return 0;
}
int main(){
int fd=open( "libtar-1.2.11.tar", O_RDONLY );
tar=mmap(NULL, 808960, PROT_READ, MAP_PRIVATE, fd, 0);
char *start; int length; char name[]="libtar-1.2.11/TODO";
if( is_file_in_tar(tar,name,&start,&length) ) printf("%.*s",length,start);
}
You can execute tar utility redirected to stdout. (tar --to-stdout). You should run it using forkpty() or popen() in order to read the output.
I've done for that with this code. Try it!
FILE*fp;
if( fp = popen("/bin/tar -xv -C /target/dir", "w") )
{
fwrite(tar_buffer,1,tar_size,fp);
pclose(fp);
printf("Untar End %d Save file\n", tar_size);
}
Just untar to in-memory tmpfs using a normal untar operation.
Related
Open() system call doesn't work in this code. However, they work normally if not used with combination to directory pointer. here i have used file->d_name to access the string base address to open the file but it doesn't work and prints error.
#include<stdio.h>
#include<string.h>
#include<fcntl.h>
#include<sys/stat.h>
#include<dirent.h>
#include<unistd.h>
#include<sys/dir.h>
int main()
{
DIR* d=opendir("DDD");
struct dirent* file;
int fd;
char WBUFFER[]="IO OS system calls\n";
char RBUFFER[100001];
while((file=readdir(d))!=NULL)
if(strlen(file->d_name)>=10)
{
if((fd=open(file->d_name,O_RDWR,0))==-1)
printf("error\n");
read(fd,RBUFFER,101);
printf("%s",RBUFFER);
close(fd);
}
else if(strlen(file->d_name)>=3)
{
if((fd=open(file->d_name,O_RDWR,0))==-1)
printf("error2\n");
write(fd,WBUFFER,50);
close(fd);
}
}
file->d_name contains just the file name, not the relative or absolute path which open(2) requires. That's why open() fails (unless you happen to have files with the same name in your current directory as the directory DDD has).
You need to prepend the directory name to file->d_name using snprintf(), like:
char buf[PATH_MAX];
snprintf(buf, sizeof buf, "DDD/%s", file->d_name);
and use buf in your open() calls.
So the dirent->d_name contains only the file name, not the path. If you're using a modern Linux or POSIX-2008 compliant system, a somewhat nice (as in, race-free) way of doing what you're doing is something like (shortened semi pseudo-code, filling in the details left as an exercise for the reader):
int dfd = open("DDD", ...);
DIR *d = fdopendir(dfd);
...
while ((file = readdir(d)) != NULL) {
int fd = openat(dfd, file->d_name, ...);
...
}
I've found on google code that was over 50 lines long and that's completely unnecessary for what I'm trying to do.
I want to make a very simple cp implementation in C.
Just so I can play with the buffer sizes and see how it affects performance.
I want to use only Linux API calls like read() and write() but I'm having no luck.
I want a buffer that is defined as a certain size so data from file1 can be read into buffer and then written to file2 and that continues until file1 has reached EOF.
Here is what I tried but it doesn't do anything
#include <stdio.h>
#include <sys/types.h>
#define BUFSIZE 1024
int main(int argc, char* argv[]){
FILE fp1, fp2;
char buf[1024];
int pos;
fp1 = open(argv[1], "r");
fp2 = open(argv[2], "w");
while((pos=read(fp1, &buf, 1024)) != 0)
{
write(fp2, &buf, 1024);
}
return 0;
}
The way it would work is ./mycopy file1.txt file2.txt
This code has an important problem, the fact that you always write 1024 bytes regardless of how many you read.
Also:
You don't check the number of command line arguments.
You don't check if the source file exists (if it opens).
You don't check that the destination file opens (permission issues).
You pass the address of the array which has a different type than the pointer to the first element to the array.
The type of fp1 is wrong, as well as that of fp2.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
char buffer[1024];
int files[2];
ssize_t count;
/* Check for insufficient parameters */
if (argc < 3)
return -1;
files[0] = open(argv[1], O_RDONLY);
if (files[0] == -1) /* Check if file opened */
return -1;
files[1] = open(argv[2], O_WRONLY | O_CREAT | S_IRUSR | S_IWUSR);
if (files[1] == -1) /* Check if file opened (permissions problems ...) */
{
close(files[0]);
return -1;
}
while ((count = read(files[0], buffer, sizeof(buffer))) != 0)
write(files[1], buffer, count);
return 0;
}
Go to section 8.3 of the K&R "The C Programming Language". There you will see an example of what you want to accomplish. Try using different buffer sizes and you will end up seeing a point where the performance tops.
#include <stdio.h>
int cpy(char *, char *);
int main(int argc, char *argv[])
{
char *fn1 = argv[1];
char *fn2 = argv[2];
if (cpy(fn2, fn1) == -1) {
perror("cpy");
return 1;
}
reurn 0;
}
int cpy(char *fnDest, char *fnSrc)
{
FILE *fpDest, *fpSrc;
int c;
if ((fpDest = fopen(fnDest, "w")) && (fpSrc = fopen(fnSrc, "r"))) {
while ((c = getc(fpSrc)) != EOF)
putc(fpDest);
fclose(fpDest);
fclose(fpSrc);
return 0;
}
return -1;
}
First, we get the two file names from the command line (argv[1] and argv[2]). The reason we don't start from *argv, is that it contains the program name.
We then call our cpy function, which copies the contents of the second named file to the contents of the first named file.
Within cpy, we declare two file pointers: fpDest, the destination file pointer, and fpSrc, the source file pointer. We also declare c, the character that will be read. It is of type int, because EOF does not fit in a char.
If we could open the files succesfully(if fopen does not return NULL), we get characters from fpSrc and copy them onto fpDest, as long as the character we have read is not EOF. Once we have seen EOF, we close our file pointers, and return 0, the success indicator. If we could not open the files, -1 is returned. The caller can check the return value for -1, and if it is, print an error message.
Good question. Related to another good question:
How can I copy a file on Unix using C?
There are two approaches to the "simplest" implementation of cp. One approach uses a file copying system call function of some kind - the closest thing we get to a C function version of the Unix cp command. The other approach uses a buffer and read/write system call functions, either directly, or using a FILE wrapper.
It's likely the file copying system calls that take place solely in kernel-owned memory are faster than the system calls that take place in both kernel- and user-owned memory, especially in a network filesystem setting (copying between machines). But that would require testing (e.g. with Unix command time) and will be dependent on the hardware where the code is compiled and executed.
It's also likely that someone with an OS that doesn't have the standard Unix library will want to use your code. Then you'd want to use the buffer read/write version, since it only depends on <stdlib.h> and <stdio.h> (and friends).
<unistd.h>
Here's an example that uses function copy_file_range from the unix standard library <unistd.h>, to copy a source file to a (possible non-existent) destination file. The copy takes place in kernel space.
/* copy.c
*
* Defines function copy:
*
* Copy source file to destination file on the same filesystem (possibly NFS).
* If the destination file does not exist, it is created. If the destination
* file does exist, the old data is truncated to zero and replaced by the
* source data. The copy takes place in the kernel space.
*
* Compile with:
*
* gcc copy.c -o copy -Wall -g
*/
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <unistd.h>
/* On versions of glibc < 2.27, need to use syscall.
*
* To determine glibc version used by gcc, compute an integer representing the
* version. The strides are chosen to allow enough space for two-digit
* minor version and patch level.
*
*/
#define GCC_VERSION (__GNUC__*10000 + __GNUC_MINOR__*100 + __gnuc_patchlevel__)
#if GCC_VERSION < 22700
static loff_t copy_file_range(int in, loff_t* off_in, int out,
loff_t* off_out, size_t s, unsigned int flags)
{
return syscall(__NR_copy_file_range, in, off_in, out, off_out, s,
flags);
}
#endif
/* The copy function.
*/
int copy(const char* src, const char* dst){
int in, out;
struct stat stat;
loff_t s, n;
if(0>(in = open(src, O_RDONLY))){
perror("open(src, ...)");
exit(EXIT_FAILURE);
}
if(fstat(in, &stat)){
perror("fstat(in, ...)");
exit(EXIT_FAILURE);
}
s = stat.st_size;
if(0>(out = open(dst, O_CREAT|O_WRONLY|O_TRUNC, 0644))){
perror("open(dst, ...)");
exit(EXIT_FAILURE);
}
do{
if(1>(n = copy_file_range(in, NULL, out, NULL, s, 0))){
perror("copy_file_range(...)");
exit(EXIT_FAILURE);
}
s-=n;
}while(0<s && 0<n);
close(in);
close(out);
return EXIT_SUCCESS;
}
/* Test it out.
*
* BASH:
*
* gcc copy.c -o copy -Wall -g
* echo 'Hello, world!' > src.txt
* ./copy src.txt dst.txt
* [ -z "$(diff src.txt dst.txt)" ]
*
*/
int main(int argc, char* argv[argc]){
if(argc!=3){
printf("Usage: %s <SOURCE> <DESTINATION>", argv[0]);
exit(EXIT_FAILURE);
}
copy(argv[1], argv[2]);
return EXIT_SUCCESS;
}
It's based on the example in my Ubuntu 20.x Linux distribution's man page for copy_file_range. Check your man pages for it with:
> man copy_file_range
Then hit j or Enter until you get to the example section. Or search by typing /example.
<stdio.h>/<stdlib.h> only
Here's an example that only uses stdlib/stdio. The downside is it uses an intermediate buffer in user-space.
/* copy.c
*
* Compile with:
*
* gcc copy.c -o copy -Wall -g
*
* Defines function copy:
*
* Copy a source file to a destination file. If the destination file already
* exists, this clobbers it. If the destination file does not exist, it is
* created.
*
* Uses a buffer in user-space, so may not perform as well as
* copy_file_range, which copies in kernel-space.
*
*/
#include <stdlib.h>
#include <stdio.h>
#define BUF_SIZE 65536 //2^16
int copy(const char* in_path, const char* out_path){
size_t n;
FILE* in=NULL, * out=NULL;
char* buf = calloc(BUF_SIZE, 1);
if((in = fopen(in_path, "rb")) && (out = fopen(out_path, "wb")))
while((n = fread(buf, 1, BUF_SIZE, in)) && fwrite(buf, 1, n, out));
free(buf);
if(in) fclose(in);
if(out) fclose(out);
return EXIT_SUCCESS;
}
/* Test it out.
*
* BASH:
*
* gcc copy.c -o copy -Wall -g
* echo 'Hello, world!' > src.txt
* ./copy src.txt dst.txt
* [ -z "$(diff src.txt dst.txt)" ]
*
*/
int main(int argc, char* argv[argc]){
if(argc!=3){
printf("Usage: %s <SOURCE> <DESTINATION>\n", argv[0]);
exit(EXIT_FAILURE);
}
return copy(argv[1], argv[2]);
}
Another way to ensure portability in general while still working with a Unix-like C API is to develop with GNOME (e.g. GLib, GIO)
https://docs.gtk.org/glib/
https://docs.gtk.org/gio/
This question already has answers here:
Reading a text file backwards in C
(5 answers)
Closed 9 years ago.
I am supposed to create a program that takes a given file and creates a file with reversed txt. I wanted to know is there a way i can start the read() from the end of the file and copy it to the first byte in the created file if I dont know the exact size of the file?
Also i have googled this and came across many examples with fread, fopen, etc. However i cant use those for this project i can only use read, open, lseek, write, and close.
here is my code so far its not much but just for reference:
#include<stdio.h>
#include<unistd.h>
int main (int argc, char *argv[])
{
if(argc != 2)/*argc should be 2 for correct execution*/
{
printf("usage: %s filename",argv[0[]);}
}
else
{
int file1 = open(argv[1], O_RDWR);
if(file1 == -1){
printf("\nfailed to open file.");
return 1;
}
int reversefile = open(argv[2], O_RDWR | O_CREAT);
int size = lseek(argv[1], 0, SEEK_END);
char *file2[size+1];
int count=size;
int i = 0
while(read(file1, file2[count], 0) != 0)
{
file2[i]=*read(file1, file2[count], 0);
write(reversefile, file2[i], size+1);
count--;
i++;
lseek(argv[2], i, SEEK_SET);
}
I doubt that most filesystems are designed to support this operation effectively. Chances are, you'd have to read the whole file to get to the end. For the same reasons, most languages probably don't include any special feature for reading a file backwards.
Just come up with something. Try to read the whole file in memory. If it is too big, dump the beginning, reversed, into a temporary file and keep reading... In the end combine all temporary files into one. Also, you could probably do something smart with manual low-level manipulation of disk sectors, or at least with low-level programming directly against the file system. Looks like this is not what you are after, though.
Why don't you try fseek to navigate inside the file? This function is contained in stdio.h, just like fopen and fclose.
Another idea would be to implement a simple stack...
This has no error checking == really bad
get file size using stat
create a buffer with malloc
fread the file into the buffer
set a pointer to the end of the file
print each character going backwards thru the buffer.
If you get creative with google you can get several examples just like this.
IMO the assistance you are getting so far is not really even good hints.
This appears to be schoolwork, so beware of copying. Do some reading about the calls used here. stat (fstat) fread (read)
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
int main(int argc, char **argv)
{
struct stat st;
char *buf;
char *p;
FILE *in=fopen(argv[1],"r");
fstat(fileno(in), &st); // get file size in bytes
buf=malloc(st.st_size +2); // buffer for file
memset(buf, 0x0, st.st_size +2 );
fread(buf, st.st_size, 1, in); // fill the buffer
p=buf;
for(p+=st.st_size;p>=buf; p--) // print traversing backwards
printf("%c", *p);
fclose(in);
return 0;
}
I need to delete the last n characters from a file using C code. At fist I was trying to use '\b', but it returns a Segmentation Fault. I have seen interesting answers to similar questions here and here, but I would prefer to use mmap function to do this, if it's possible. I know it could be simpler to truncate the file by creating a temp file, and writing chars to temp until some offset of the original file. The problem is I don't seem to understand how to use mmap function to do this, can't see what parameters I need to pass to that function, specially address, length and offset. From what I've read, I should use MAP_SHARED in flags and PROT_READ|PROT_WRITE in protect.
The function definition says:
void * mmap (void *address, size_t length, int protect, int flags, int filedes, off_t offset)
Here is my main:
int main(int argc, char * argv[])
{
FILE * InputFile;
off_t position;
int charsToDelete;
if ((InputFile = fopen(argv[1],"r+")) == NULL)
{
printf("tdes: file not found: %s\n",argv[1]);
}
else
{
charsToDelete = 5;
fseeko(InputFile,-charsToDelete,SEEK_END);
position = ftello(InputFile);
printf("Pos: %d\n",(int)position);
int i;
//for(i = 0;i < charsToDelete;i++)
//{
// putc(InputFile,'\b');
//}
}
fclose(InputFile);
return 0;
}
Why not use:
#include <unistd.h>
#include <sys/types.h>
int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length);
like for instance:
charsToDelete = 5;
fseeko(InputFile,-charsToDelete,SEEK_END);
position = ftello(InputFile);
ftruncate(fileno(InputFile), position);
Read all but n bytes from the file and write to a temporary file, close the original file, rename temporary file as original file.
Or use e.g. truncate or similar function if you have it.
Also, failure to open the file doesn't have to be that it can't be found, You should check errno on failure to see what the error is. Use e.g. strerror to get a printable string from the error code.
Unfortunately, mmap does not allow you to change size of underlying file object.
Instead, I would recommend to simply truncate your file, use something like this:
truncate(filename, new_length);
If I have a buffer which contains the data of a file, how can I get a file descriptor from it?
This is a question derived from how to untar file in memory
I wrote a simple example how to make filedescriptor to a memory area:
#include <unistd.h>
#include <stdio.h>
#include <string.h>
char buff[]="qwer\nasdf\n";
int main(){
int p[2]; pipe(p);
if( !fork() ){
for( int buffsize=strlen(buff), len=0; buffsize>len; )
len+=write( p[1], buff+len, buffsize-len );
return 0;
}
close(p[1]);
FILE *f = fdopen( p[0], "r" );
char buff[100];
while( fgets(buff,100,f) ){
printf("from child: '%s'\n", buff );
}
puts("");
}
Not possible in plain C. In plain C all file access happens via FILE * handles and these can only be created with fopen() and freopen() and in both cases must refer to a file path. As C tries to be as portable as possible, it limits I/O to the absolute bare minimum that probably all systems can support in some way.
If you have POSIX API available (e.g. Linux, macOS, iOS, FreeBSD, most other UNIX systems), you can use fmemopen():
char dataInMemory[] = "This is some data in memory";
FILE * fileDescriptor = fmemopen(dataInMemory, sizeof(dataInMemory), "r");
This is a true file handle that can be used with all C file API. It should also allow seeking, something not possible if you work with pipes as pipes support no seeking (you can emulate forward seeking but there is no way to ever seek backwards).
You can't. Unlike C++, the C model of file I/O isn't open to extension.