My problem is to deal with sparse file reads and understand where the extents of the file are to perform some logic around it.
Since, there is no direct API call to figure these stuff out, I decided to use ioctl api to do this. I got the idea from how cp command deals with problems of copying over sparse files by going through their code and ended up seeing this.
https://github.com/coreutils/coreutils/blob/df88fce71651afb2c3456967a142db0ae4bf9906/src/extent-scan.c#L112
So, I tried to do the same thing in my sample program running in user space and it errors out with "Invalid argument". I am not sure what I am missing or if this is even possible from userspace. I am running on ubuntu 14.04 on an ext4 file system. Could this be a problem with device driver supporting these request modes underneath?
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/fcntl.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/fs.h>
#include "fiemap.h" //This is from https://github.com/coreutils/coreutils/blob/df88fce71651afb2c3456967a142db0ae4bf9906/src/fiemap.h
int main(int argc, char* argv[]) {
int input_fd;
if(argc != 2){
printf ("Usage: ioctl file1");
return 1;
}
/* Create input file descriptor */
input_fd = open (argv [1], O_RDWR);
if (input_fd < 0) {
perror ("open");
return 2;
}
union { struct fiemap f; char c[4096]; } fiemap_buf;
struct fiemap *fiemap = &fiemap_buf.f;
int s = ioctl(input_fd, FS_IOC_FIEMAP, fiemap);
if (s == 0) {
printf("ioctl success\n");
} else {
printf("ioctl failure\n");
char * errmsg = strerror(errno);
printf("error: %d %s\n", errno, errmsg);
}
/* Close file descriptors */
close (input_fd);
return s;
}
As you're not properly setting the fiemap_buf.f parameters before invoking ioctl(), it is likely that the EINVAL is coming from the fiemap invalid contents than from the FS_IOC_FIEMAP request identifier support itself.
For instance, the ioctl_fiemap() (from kernel) will evaluate the fiemap.fm_extent_count in order to determine if it is greater than FIEMAP_MAX_EXTENTS and return -EINVAL in that case. Since no memory reset nor parameterization is being performed on fiemap, this is very likely the root cause of the problem.
Note that from the coreutils code you referenced, it performs the correct parameterization of fiemap before calling ioctl():
fiemap->fm_start = scan->scan_start;
fiemap->fm_flags = scan->fm_flags;
fiemap->fm_extent_count = count;
fiemap->fm_length = FIEMAP_MAX_OFFSET - scan->scan_start;
Note fiemap is not recommended as you have to be sure to pass FIEMAP_FLAG_SYNC which has side effects. The lseek(), SEEK_DATA and SEEK_HOLE interface is the recommended one, though note that will, depending on file system, represent unwritten extents (allocated zeros) as holes.
Related
I'm using a library for my project. This library sometimes prints some messages to stdout. This is a problem for me because the messages are mixed up along with the application messages. It will be useful to a stop this behaviour or have them printed to a different window. I'm using C Language and Mingw32 enviroment. How can I do this? Thanks.
You might be able to (nonportably) swap the stdout with another stream:
#include <stdio.h>
FILE *devnull;
#define SWAPSTDOUT() do{ FILE *tmp = stdout; stdout = devnull; devnull = tmp; }while(0)
int main(void)
{
/*program initialization*/
if(0==(devnull= fopen("/dev/null", "r"))) return 1;
fputs("your code 0\n",stdout);
SWAPSTDOUT();
fputs("library code 0\n",stdout); //should be silent
SWAPSTDOUT();
fputs("your code 1\n", stdout);
}
Unfortunately, that's unlikely to work with functions that hardcode stdout (e.g., printf or puts).
If you're on a POSIX platform, you might have freopen but that won't help much if you can't save the original stream. However, on POSIX you could fflush(stdout) and then shuffle the underlying file descriptors, which should be quite reliable:
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
int devnull, tmp;
int swapstdout(void);
int main(void)
{
/*program initialization*/
if(0>(devnull=open("/dev/null", O_RDONLY))) return EXIT_FAILURE;
if(0>(tmp=dup(devnull))) return EXIT_FAILURE; //reserve a fd spot
fputs("your code 0\n",stdout);
if(0>swapstdout()) return EXIT_FAILURE:
fputs("library code 0\n",stdout); //should be silent
if(0>swapstdout()) return EXIT_FAILURE:
fputs("your code 1\n", stdout);
}
int swapstdout(void)
{
if(0>fflush(stdout)) return -1;
if(0>dup2(STDOUT_FILENO,tmp)) return -1; /*really shouldn't happen*/
if(0>dup2(devnull,STDOUT_FILENO)) return -1; /*really shouldn't happen*/
if(0>tmp=dup(devnull)) return -1; /*really shouldn't happen unless we're multithreaded and another thread steals the fd spot*/
}
Either solution depends on your code being single threaded.
In any case, well behaved library functions should leave files they don't own alone, unless you explicitly request them to do something with such files.
I am new to uvlib. Is it normal to call uv_run twice if one wants to avoid blocking inside function? If not, then which instruments are available, except threads? Here I just open and close file.
#include <uv.h>
#include <stdio.h>
#include <fcntl.h>
#include <conio.h>
#ifdef _WIN32
#include <conio.h>
#include <Windows.h>
#define Sleep(x) Sleep(x)
#else
#include <unistd.h>
#define Sleep(x) sleep(x)
#endif
uv_loop_t* loop;
uv_fs_t open_req;
uv_fs_t close_req;
void open_cb(uv_fs_t*);
void close_cb(uv_fs_t*);
const char *filename = "C:/c/somedata.txt";
int main(int argc, char **argv) {
int r;
loop = uv_loop_new();
r = uv_fs_open(loop, &open_req, filename, O_RDONLY, S_IREAD, open_cb);
if (r < 0) {
printf("Error at opening file: %s\n", uv_strerror(r));
}
printf("in main now\n");
uv_run(loop, UV_RUN_DEFAULT);
uv_loop_close(loop);
return 0;
}
void open_cb(uv_fs_t* req) {
int result = req->result;
if (result < 0) {
printf("Error at opening file: %s\n", uv_strerror(result));
} else {
printf("Successfully opened file.\n");
}
uv_fs_req_cleanup(req);
uv_fs_close(loop, &close_req, result, close_cb);
uv_run(loop, UV_RUN_DEFAULT);
Sleep(5000);
printf("ok now\n");
}
void close_cb(uv_fs_t* req) {
int result = req->result;
printf("in close_cb now\n");
if (result < 0) {
printf("Error at closing file: %s\n", uv_strerror(result));
} else {
printf("Successfully closed file.\n");
}
}
Set aside your example, libuv offers the opportunity to run the loop more than once.
See the documentation for further details.
In particular, uv_run function accepts a parameter of type uv_run_mode.
Possible values are:
UV_RUN_DEFAULT: it doesn't stop unless you explicitly stop it and until there exists at least on referenced or active resource on the loop.
UV_RUN_ONCE: poll for I/O once and execute all the functions that are ready to be served. It has the drawback that it is blocking if there are no pending callbacks.
UV_RUN_NOWAIT: this is probably the one you are looking for, similar to the previous one, but it doesn't block if there are no pending callbacks.
Note that with both UV_RUN_ONCE and UV_RUN_NOWAIT you'll have to run the loop more than once.
Return value usually indicates if there are some other pending callbacks. In this case, the loop must be run sooner or later in the future.
The last mode, UV_RUN_NOWAIT, is probably the one you are looking for.
As an example, it can be used in scenarios where the client has its own loop and cannot block on the libuv's one.
Is it normal to run the loop more than once?
Well, yes, but it mostly depends on your actual problem if it's right.
It's hard to say from a 100 line snippet on SO.
I want to shred some temp files produced by my C program before the files are removed.
Currently I am using
system("shred /tmp/datafile");
system("rm /tmp/datafile");
from within my program, but I think instead of calling the system function is not the best way (correct me if I am wrong..) Is there any other way I can do it? How do I shred the file from within my code itself? A library, or anything? Also, about deletion part, is this answer good?
Can I ask why you think this is not the best way to achieve this? It looks like a good solution to me, if it is genuinely necessary to destroy the file contents irretrievably.
The advantage of this way of doing it are:
the program already exists (so it's faster to develop); and
the program is already trusted.
The second is an important point. It's possible to overstate the necessity of elaborately scrubbing files (Peter Gutmann, in a remark quoted on the relevant wikipedia page, has described some uses of his method as ‘voodoo’), but that doesn't matter: in any security context, using a pre-existing tool is almost always more defensible than using something home-made.
About the only criticism I'd make of your current approach, using system(3), is that since it looks up the shred program in the PATH, it would be possible in principle for someone to play games with that and get up to mischief. But that's easily dealt with: use fork(2) and execve(2) to invoke a specific binary using its full path.
That said, if this is just a low-impact bit of tidying up, then it might be still more straightforward to simply mmap the file and quickly write zeros into it.
You can use the following code:
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdio.h>
#define BUF_SIZE 4096
#define ABS_FILE_PATH "/tmp/aaa"
int main()
{
//get file size
struct stat stat_buf;
if (stat(ABS_FILE_PATH, &stat_buf) == -1)
return errno;
off_t fsize = stat_buf.st_size;
//get file for writing
int fd = open(ABS_FILE_PATH, O_WRONLY);
if (fd == -1)
return errno;
//fill file with 0s
void *buf = malloc(BUF_SIZE);
memset(buf, 0, BUF_SIZE);
ssize_t ret = 0;
off_t shift = 0;
while((ret = write(fd, buf,
((fsize - shift >BUF_SIZE)?
BUF_SIZE:(fsize - shift)))) > 0)
shift += ret;
close(fd);
free(buf);
if (ret == -1)
return errno;
//remove file
if (remove(ABS_FILE_PATH) == -1)
return errno;
return 0;
}
There are probably several problems with the code below. Found it online after searching for a way to get keyboard input in linux. I've verified the correct event for keyboard input. The reason it seems fishy to me is regardless of what i put in the filepath, it always seems to pass the error check (the open call returns something greater than 0). Something is obviously wrong, so suggestions are welcome.
This won't run correctly unless you run the exe as su.
When i want to read in my keystroke, do i just use something like fgets on the file descriptor in an infinite while loop(would that even work)? I want it to be constantly polling for keyboard inputs. Any tips on decoding the inputs from the keyboard event?
Thanks again! This project of mine may be overly ambitious, as it's been a really long time since i've done any coding.
#include <stdio.h>
#include <stdlib.h>
#include <stddef.h>
#include <fcntl.h>
#include <linux/input.h>
#include <unistd.h>
// Edit this line to reflect your filepath
#define FILE_PATH "/dev/input/event4"
int main()
{
printf("Starting KeyEvent Module\n");
size_t file; //will change this to int file; to make it possible to be negative
const char *str = FILE_PATH;
printf("File Path: %s\n", str);
error check here
if((file = open(str, O_RDONLY)) < 0)
{
printf("ERROR:File can not open\n");
exit(0);
}
struct input_event event[64];
size_t reader;
reader = read(file, event, sizeof(struct input_event) * 64);
printf("DO NOT COME HERE...\n");
close(file);
return 0;
}
the problem is here:
size_t file;
size_t is unsigned, so it will always be >=0
it should have been:
int file;
the open call returns something greater than 0
open returns int, but you put in in an unsigned variable (size_t is usually unsigned), so you fail to detect when it is <0
I am trying to read data from a bluetooth barcode scanner (KDC300) using C. Here is the code I have so far, and the program successfully establishes a bluetooth connection to the scanner, but when a barcode is scanned, no input is displayed on the screen (Eventually more will be done with the data, but we have to get it working first, right).
Here is the program:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <termios.h>
#include <sys/ioctl.h>
int main (int argc, const char * argv[]) {
// define vars
int STOP = 0;
//char buf[255];
if(argv[1])
{
int fd = open("/dev/tty.KDC1", O_RDONLY);
if(fd == -1)
{
printf("%s", strcat("Unable to open /dev/tty.", argv[1]));
}
int res;
while(STOP == 0)
{
while((res = read(fd,buf,255)) == 0);
{
if(res > 0)
{
buf[res]=0;
printf("%s:%d\n", buf, res);
if(buf[sizeof(buf)]=='\n') break;
}
}
}
}
return 0;
}
If anyone has any ideas, I am at a loss on this so far. If it is any help, I can run screen /dev/tty.KDC1 and any barcodes scanned on the scanner appear in the terminal, I just can't do anything with the data.
Jud
This line:
while((res = read(fd,buf,255)) == 0);
Does not do what you think it does. That's a while loop with an empty body.
#tommieb75,
the strcat statement was from the first "go" at the program, I took a variable from argv[1] and appended it to the /dev/tty.* so you could select which device you wanted to monitor.
I am not sure why I had commented out buf, probably stems from looking at the code too much / trying different approaches and forgetting where I was (not much of a C programmer, which is how I can get lost in 30 LOC).
#caf, Good catch on the extra semi-colon after the while loop, unfortunately, even after correcting it, the program doesn't behave correctly.
I am researching the problem further. I can verify (with osx packetlogger) that the computer is getting the data, but the but the buffer never has any data placed in it.
-Jud
---------------Edit--------------
I solved the problem after a little trial and error. Adding the following code to setup the serial connection solved everything:
struct termios theTermios;
memset(&theTermios, 0, sizeof(struct termios));
cfmakeraw(&theTermios);
cfsetspeed(&theTermios, 115200);
theTermios.c_cflag = CREAD | CLOCAL; // turn on READ
theTermios.c_cflag |= CS8;
theTermios.c_cc[VMIN] = 0;
theTermios.c_cc[VTIME] = 10; // 1 sec timeout
ioctl(fileDescriptor, TIOCSETA, &theTermios);
Thanks to the other answers for getting me to this point.
Here is the best info I've found.
The C program on there using termios worked just by adding
#include<string.h>
And changing the baudrate to match my needs.
In your code
printf("%s", strcat("Unable to open /dev/tty.", argv[1]));
Why did you do that? It would be easier to do it this way:
printf("%s: Unable to open /dev/tty.KDC1", argv[0]);
Why the parameter referencing to the command line?
res = read(fd,buf,255)
Why did you have buf declaration commented out above?