inotify event IN_MODIFY occurring twice for tftp put - c

I am using inotify to listen to modifications to a file.
When I test file modification, program is working fine.
# echo "test" > /tftpboot/.TEST
Output:
Read 16 data
IN_MODIFY
But when I do tftp put, two events are generated:
tftp> put .TEST
Sent 6 bytes in 0.1 seconds
tftp>
Output:
Read 16 data
IN_MODIFY
Read 16 data
IN_MODIFY
Any idea how to avoid the duplicate notification?
Code is given below:
#include <sys/inotify.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <iostream>
using namespace std;
int main(int argc, const char *argv[])
{
int fd = inotify_init();
if (fd < 0)
{
cout << "inotify_init failed\n";
return 1;
}
int wd = inotify_add_watch(fd, "/tftpboot/.TEST", IN_MODIFY);
if (wd < 0)
{
cout << "inotify_add_watch failed\n";
return 1;
}
while (true)
{
char buffer[sizeof(struct inotify_event) + NAME_MAX + 1] = {0};
ssize_t length;
do
{
length = read( fd, buffer, sizeof(struct inotify_event));
cout << "Read " << length << " data\n";
}while (length < 0);
if (length < 0)
{
cout << "read failed\n";
return 1;
}
struct inotify_event *event = ( struct inotify_event * ) buffer;
if ( event->mask & IN_ACCESS )
cout << "IN_ACCESS\n";
if ( event->mask & IN_CLOSE_WRITE )
cout << "IN_CLOSE_WRITE\n";
if ( event->mask & IN_CLOSE_NOWRITE )
cout << "IN_CLOSE_NOWRITE\n";
if ( event->mask & IN_MODIFY )
cout << "IN_MODIFY \n";
if ( event->mask & IN_OPEN )
cout << "IN_OPEN\n";
}
inotify_rm_watch( fd, wd );
close (fd);
return 0;
}

try using IN_CLOSE_WRITE instead
Q: What is the difference between IN_MODIFY and IN_CLOSE_WRITE?
The
IN_MODIFY event is emitted on a file content change (e.g. via the
write() syscall) while IN_CLOSE_WRITE occurs on closing the changed
file. It means each change operation causes one IN_MODIFY event (it
may occur many times during manipulations with an open file) whereas
IN_CLOSE_WRITE is emitted only once (on closing the file).
Q: Is it better to use IN_MODIFY or IN_CLOSE_WRITE?
It varies from
case to case. Usually it is more suitable to use IN_CLOSE_WRITE
because if emitted the all changes on the appropriate file are safely
written inside the file. The IN_MODIFY event needn't mean that a file
change is finished (data may remain in memory buffers in the
application). On the other hand, many logs and similar files must be
monitored using IN_MODIFY - in such cases where these files are
permanently open and thus no IN_CLOSE_WRITE can be emitted.
source

Related

Filter repetitive inotify events about the same file in C

I'm developing a basic dropbox program. Very simple. I'm kinda stuck with the inotify thing. I need events that notifies about file creation, modification and deletion. I also need to filter out temporary files. I assume I need to be careful to not filter out truthful events, like if a user modifies the same file two or more times in a row.
If a modify a simple text file, I get three separate events: IN_CLOSE_WRITE test.c, IN_MOVED_TO test.c and IN_CLOSE_WRITE test.c. My expected result would be a single notify event about this file. Either one it's fine. All of them sends the same signal to server, to upload the file.
If I copy a file 'test.c': IN_CREATE test (copy).c and IN_CLOSE_WRITE test (copy).c. See the example above for the expected result.
Here's my code so far.
#include <stdio.h>
#include <sys/inotify.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#define EVENT_BUF_LEN 4096
#define EVENT_SIZE sizeof(struct inotify_event)
typedef struct
{
int length, fd, wd;
char buffer[EVENT_BUF_LEN] __attribute__((aligned(__alignof__(struct inotify_event))));
} notification;
notification inotify;
void getNotified(char *pathname)
{
char command[256];
notification inotify;
inotify.fd = inotify_init();
inotify.wd = inotify_add_watch(inotify.fd, pathname, IN_CLOSE_WRITE | IN_CREATE | IN_MOVED_TO | IN_MOVED_FROM);
while (1)
{
inotify.length = read(inotify.fd, inotify.buffer, EVENT_BUF_LEN);
int i = 0;
while (i < inotify.length)
{
struct inotify_event *event = (struct inotify_event *)&inotify.buffer[i];
// Filtering out temporary files.
if (strstr(event->name, ".gout") != NULL)
{
i += (EVENT_SIZE + event->len);
continue;
}
if (event->mask & IN_CLOSE_WRITE)
strcpy(command, "IN_CLOSE_WRITE ");
else if (event->mask & IN_CREATE)
strcpy(command, "IN_CREATE ");
else if (event->mask & IN_MOVED_FROM)
strcpy(command, "IN_MOVED_FROM ");
else if (event->mask & IN_MOVED_TO)
strcpy(command, "IN_MOVED_TO ");
strcat(command, event->name);
printf("%s\n", command);
i += (EVENT_SIZE + event->len);
}
}
inotify_rm_watch(inotify.fd, inotify.wd);
close(inotify.fd);
}
int main(void)
{
getNotified(".");
return 0;
}

io_uring user_data field is always zero

I'm playing around with io_uring, https://kernel.dk/io_uring.pdf, to see if it can be used for async file I/O for logging. This is a simple program that opens a file, stats the file, and then reads the first 4k from the file. This program runs to completion successfully when the file exists and is readable. But the user_data field in the completion queue entry is always zero. The documentation for io_uring says:
user_data is common across op-codes, and is untouched by the kernel. It's simply copied to the completion event, cqe, when a completion event is posted for this request.
Since the completions are not ordered the user_data field is needed to match completions with submissions. If the field is always zero then how can it be used?
#include <iostream>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
#include <liburing.h>
#include <stdlib.h>
int main() {
struct io_uring ring;
// see man io_uring_setup for what this does
auto ret = io_uring_queue_init(64, &ring, 0);
if (ret) {
perror("Failed initialize uring.");
exit(1);
}
std::cout << "I/O uring initialized successfully. " << std::endl;
auto directory_fd = open("/tmp", O_RDONLY);
if (directory_fd < 0) {
perror("Failed to open current directory.");
exit(1);
}
struct io_uring_sqe *submission_queue_entry = io_uring_get_sqe(&ring);
submission_queue_entry->user_data = 100;
io_uring_prep_openat(submission_queue_entry, directory_fd, "stuff", O_RDONLY, 0);
submission_queue_entry = io_uring_get_sqe(&ring);
submission_queue_entry->user_data = 1000;
struct statx statx_info;
io_uring_prep_statx(submission_queue_entry, directory_fd, "stuff", 0, STATX_SIZE, &statx_info);
//TODO: what does this actually return?
auto submit_error = io_uring_submit(&ring);
if (submit_error != 2) {
std::cerr << strerror(submit_error) << std::endl;
exit(2);
}
int file_fd = -1;
uint32_t responses = 0;
while (responses != 2) {
struct io_uring_cqe *completion_queue_entry = 0;
auto wait_return = io_uring_wait_cqe(&ring, &completion_queue_entry);
if (wait_return) {
std::cerr << "Completion queue wait error. " << std::endl;
exit(2);
}
std::cout << "user data " << completion_queue_entry->user_data << " entry ptr " << completion_queue_entry << " ret " << completion_queue_entry->res << std::endl;
std::cout << "size " << statx_info.stx_size << std::endl;
io_uring_cqe_seen(&ring, completion_queue_entry);
if (completion_queue_entry->res > 0) {
file_fd = completion_queue_entry->res;
}
responses++;
}
submission_queue_entry = io_uring_get_sqe(&ring);
submission_queue_entry->user_data = 66666;
char buf[1024 * 4];
io_uring_prep_read(submission_queue_entry, file_fd, buf, 1024 * 4, 0);
io_uring_submit(&ring);
struct io_uring_cqe* read_entry = 0;
auto read_wait_rv = io_uring_wait_cqe(&ring, &read_entry);
if (read_wait_rv) {
std::cerr << "Error waiting for read to complete." << std::endl;
exit(2);
}
std::cout << "Read user data " << read_entry->user_data << " completed with " << read_entry->res << std::endl;
if (read_entry->res < 0) {
std::cout << "Read error " << strerror(-read_entry->res) << std::endl;
}
}
Output
I/O uring initialized successfully.
user data 0 entry ptr 0x7f4e3158c140 ret 5
size 1048576
user data 0 entry ptr 0x7f4e3158c150 ret 0
size 1048576
Read user data 0 completed with 4096
What happens if you try and set user_data after your calls to io_uring_prep_openat()/io_uring_prep_statx()?
I ask this because doing a Google search for io_uring_prep_statx suggests it comes from liburing library.
Searching the liburing source for io_uring_prep_openat leads us to a definition of io_uring_prep_openat() in liburing.h:
static inline void io_uring_prep_openat(struct io_uring_sqe *sqe, int dfd,
const char *path, int flags, mode_t mode)
{
io_uring_prep_rw(IORING_OP_OPENAT, sqe, dfd, path, mode, 0);
sqe->open_flags = flags;
}
Searching the liburing source for io_uring_prep_statx leads to a definition of io_uring_prep_statx():
static inline void io_uring_prep_statx(struct io_uring_sqe *sqe, int dfd,
const char *path, int flags, unsigned mask,
struct statx *statxbuf)
{
io_uring_prep_rw(IORING_OP_STATX, sqe, dfd, path, mask,
(__u64) (unsigned long) statxbuf);
sqe->statx_flags = flags;
}
Chasing the calls gets us to the definition of io_uring_prep_rw:
static inline void io_uring_prep_rw(int op, struct io_uring_sqe *sqe, int fd,
const void *addr, unsigned len,
__u64 offset)
{
sqe->opcode = op;
sqe->flags = 0;
sqe->ioprio = 0;
sqe->fd = fd;
sqe->off = offset;
sqe->addr = (unsigned long) addr;
sqe->len = len;
sqe->rw_flags = 0;
sqe->user_data = 0;
sqe->__pad2[0] = sqe->__pad2[1] = sqe->__pad2[2] = 0;
}
PS: I notice you have a comment that says
//TODO: what does this actually return?
auto submit_error = io_uring_submit(&ring);
Well, if we search the liburing repo for "int io_uring_submit" we come across the following in src/queue.c:
/*
* Submit sqes acquired from io_uring_get_sqe() to the kernel.
*
* Returns number of sqes submitted
*/
int io_uring_submit(struct io_uring *ring)
This ultimately chains calls down to io_uring_enter() syscall (raw man page) so you can read that for more detail.
Update: The questioner says moving the assignment solved their problem so I invested some time thinking about the text they quoted. Upon further reading I have picked up a subtlety (emphasis added):
user_data is common across op-codes, and is untouched by the kernel. It's simply copied to the completion event, cqe, when a completion event is posted for this request.
There's a similar statement earlier in the document (again emphasis added):
The cqe contains a user_data field. This field is carried from the
initial request submission, and can contain any information that the the application needs to identify said request. One common use case is to have it be the pointer of the original request. The kernel will not touch this field, it's simply carried straight from submission to completion event.
The statement applies to io_uring kernel syscalls but io_uring_prep_openat() / io_uring_prep_statx() are liburing functions. liburing is a userspace helper library so the statements above about user_data do not have to apply to all liburing functions.
If the field is always zero then how can it be used?
The field is being zeroed by certain liburing preparation helper functions. In this case it can be only be set (and retain the new value) after those helper function have been called. The io_uring kernel syscalls behave per the quote.

How does /proc/{pid}/stack work in linux? [duplicate]

I'm trying to port a program from Windows to Linux.
I encountered a problem when I found out that there isn't a "real" ReadProcessMemory counterpart on Linux; I searched for an alternative and I found ptrace, a powerful process debugger.
I quickly coded two small console applications in C++ to test ptrace, before using it in the program.
TestApp
This is the tracee; it keeps printing two integers every 50 milliseconds while increasing their value by 1 every time.
#include <QCoreApplication>
#include <QThread>
#include <iostream>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
int value = 145;
int i = 0;
do {
cout << "i: " << i << " " << "Value: " << value << endl;
value++;
i++;
Sleeper::msleep(50);
} while (true);
return a.exec();
}
MemoryTest
This is the tracer; it asks for the process name and retrieves the PID using the command pidof -s, then ptrace attaches to the process and retrieves the memory address' value every 500 milliseconds, for 10 times.
#include <QCoreApplication>
#include <QThread>
#include <iostream>
#include <string>
#include <sys/ptrace.h>
#include <errno.h>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
char process_name[50];
cout << "Process name: ";
cin >> process_name;
char command[sizeof(process_name) + sizeof("pidof -s ")];
snprintf(command, sizeof(command), "pidof -s %s", process_name);
FILE* shell = popen(command, "r");
char pidI[sizeof(shell)];
fgets(pidI, sizeof(pidI), shell);
pclose(shell);
pid_t pid = atoi(pidI);
cout << "The PID is " << pid << endl;
long status = ptrace(PTRACE_ATTACH, pid, NULL, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
unsigned long addr = 0x012345; // Example address, not the true one
int i = 0;
do {
status = ptrace(PTRACE_PEEKDATA, pid, addr, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
i++;
Sleeper::msleep(500);
} while (i < 10);
status = ptrace(PTRACE_DETACH, pid, NULL, NULL);
cout << "Status: " << status << endl;
cout << "Error: " << errno << endl;
return a.exec();
}
Everything works fine, but TestApp is paused (SIGSTOP) until ptrace detaches from it.
Also, when it attaches to the process, the status is 0 and the error is 2; the first time it tries to retrieve the memory address value it fails with status -1 and error 3. Is it normal?
Is there a way to prevent ptrace from sending the SIGSTOP signal to the process?
I already tried using PTRACE_SEIZE instead of PTRACE_ATTACH, but it doesn't work: status -1 and error 3.
Update: Using Sleeper in MemoryTest before the "do-while" loop fixes the problem of the first memory address value retrieval, even if the value of seconds, milliseconds or microseconds is 0. Why?
After a lot of research I'm pretty sure that there isn't a way to use ptrace without stopping the process.
I found a real ReadProcessMemory counterpart, called process_vm_readv, which is much more simple.
I'm posting the code in the hope of helping someone who is in my (previous) situation.
Many thanks to mkrautz for his help coding MemoryTest with this beautiful function.
#include <QCoreApplication>
#include <QThread>
#include <sys/uio.h>
#include <stdint.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <iostream>
using namespace std;
class Sleeper : public QThread
{
public:
static void usleep(unsigned long usecs){QThread::usleep(usecs);}
static void msleep(unsigned long msecs){QThread::msleep(msecs);}
static void sleep(unsigned long secs){QThread::sleep(secs);}
};
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
char process_name[50];
cout << "Process name: ";
cin >> process_name;
char command[sizeof(process_name) + sizeof("pidof -s ")];
snprintf(command, sizeof(command), "pidof -s %s", process_name);
FILE* shell = popen(command, "r");
char pidI[sizeof(shell)];
fgets(pidI, sizeof(pidI), shell);
pclose(shell);
pid_t pid = atoi(pidI);
cout << "The PID is " << pid << endl;
if (pid == 0)
return false;
struct iovec in;
in.iov_base = (void *) 0x012345; // Example address, not the true one
in.iov_len = 4;
uint32_t foo;
struct iovec out;
out.iov_base = &foo;
out.iov_len = sizeof(foo);
do {
ssize_t nread = process_vm_readv(pid, &out, 1, &in, 1, 0);
if (nread == -1) {
fprintf(stderr, "error: %s", strerror(errno));
} else if (nread != in.iov_len) {
fprintf(stderr, "error: short read of %li bytes", (ssize_t)nread);
}
cout << foo << endl;
Sleeper::msleep(500);
} while (true);
return a.exec();
}
Davide,
Have you had a look at the /proc filesystem? It contains memory map files that can be used to peek at the full process space. You can also write in the space to set a breakpoint. There is a wealth of other information in /proc as well.
The PTRACE_CONT command can be used to continue a process. Generally, the target will be paused with a PTRACE_ATTACH when the debugger attaches.
The man page says PTRACE_SIEZE should not pause the process. What flavor and version of Linux are you using? PTRACE_SIEZE has been around for quite awhile so I'm not sure why you are having trouble there.
I note the addr value is set to 0x12345. Is this a valid address in the target space? Or was that just an example? How is the stack address of interest (&value) communicated between the two processes?
I'm not too sure about the return codes. Generally a 0 means all is well, the errno may just be a hangover value from the last error.
--Matt

How to use inotify in C?

I searched for questions related to inotify, and this one is somewhat different...
I use the following code to monitor change of one file (not directory). In testing, the read() does return when I save the target file which means it works. But event->mask is 32768 which is not IN_MODIFY and name is empty. Another issue: it cannot monitor continuously. When I change the file the second time, it has no response. Thank you for the help!
#include <sys/inotify.h>
#include <unistd.h>
#include <stdio.h>
#define EVENT_SIZE (sizeof (struct inotify_event))
#define BUF_LEN (16 * (EVENT_SIZE + 16))
int main()
{
int fd;
fd = inotify_init();
if (fd < 0)
perror("inotify_init()");
int wd;
wd = inotify_add_watch(fd, "target.txt", IN_MODIFY);
if (wd < 0)
perror("inotify_add_watch");
char buf[BUF_LEN];
int len;
start:
len = read(fd, buf, BUF_LEN);
if (len > 0)
{
int i = 0;
while (i < len)
{
struct inotify_event *event;
event = (struct inotify_event *) &buf[i];
printf("wd=%d mask=%x cookie=%u len=%u\n",
event->wd, event->mask,
event->cookie, event->len);
if (event->mask & IN_MODIFY)
printf("file modified %s", event->name);
if (event->len)
printf("name=%s\n", event->name);
i += EVENT_SIZE + event->len;
}
}
goto start;
return 0;
}
The 0x8000 corresponds to IN_IGNORED. Its presence in the mask indicates that the inotify watch had been removed because the file had been removed. Your editor probably removed the old file and put a new file in its place. Changing the file a second time had no effect because the watch had been removed.
The name is not being returned because you are not watching a directory.
From the inotify man page.
The name field is only present when an event is returned for a file inside a watched directory; it identifies the file pathname relative to the watched directory.
...
IN_IGNORED -- Watch was removed explicitly (inotify_rm_watch(2)) or automatically (file was deleted, or file system was unmounted).
event->mask 32768 is equivalent to 0x8000 that is IN_IGNORED
For more information : "/usr/include/linux/inotify.h"
if (event->mask & IN_IGNORED) {
/*Remove watch*/ inotify_rm_watch(fileDescriptor,watchDescriptor)
/*Add watch again*/ inotify_add_watch
}

inotify file in C

I am trying to run an example of inotify in C..but it's not working.
I want to monitor modifications to a file (the file is tmp.cfg), but it doesn't work..I don't know if i'm running it correctly, because I understand how to monitor directories, but not a single file
HereĀ“s the example:
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/inotify.h>
#include <unistd.h>
#define EVENT_SIZE ( sizeof (struct inotify_event) )
#define BUF_LEN ( 1024 * ( EVENT_SIZE + 16 ) )
int main( int argc, char **argv )
{
int length, i = 0;
int fd;
int wd;
char buffer[BUF_LEN];
fd = inotify_init();
if ( fd < 0 ) {
perror( "inotify_init" );
}
wd = inotify_add_watch( fd, "/home/name/tmp.cfg",
IN_MODIFY | IN_CREATE | IN_DELETE );
length = read( fd, buffer, BUF_LEN );
if ( length < 0 ) {
perror( "read" );
}
while ( i < length ) {
struct inotify_event *event = ( struct inotify_event * ) &buffer[ i ];
if ( event->mask & IN_CREATE ) {
printf( "The file %s was created.\n", event->name );
}
else if ( event->mask & IN_DELETE ) {
printf( "The file %s was deleted.\n", event->name );
}
else if ( event->mask & IN_MODIFY ) {
printf( "The file %s was modified.\n", event->name );
}
i += EVENT_SIZE + event->len;
}
( void ) inotify_rm_watch( fd, wd );
( void ) close( fd );
return 0;
}
Once i run it, if i write something on the file and then save it, nothing happens.
i've tryed debugging it..the problem seems to be the if ( event->mask & IN_MODIFY ), as it doesn't recognize it as a modification
You have 2 issues. First, as far as I can tell, inotify does not really work on files - it needs directory name to watch.
Second, you missed if (event->len) { inside while loop.
This code works for me for creating, deleting and modifying files in current directory:
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/inotify.h>
#include <unistd.h>
#define EVENT_SIZE (sizeof(struct inotify_event))
#define BUF_LEN (1024 * (EVENT_SIZE + 16))
int main(int argc, char **argv) {
int length, i = 0;
int fd;
int wd;
char buffer[BUF_LEN];
fd = inotify_init();
if (fd < 0) {
perror("inotify_init");
}
wd = inotify_add_watch(fd, ".",
IN_MODIFY | IN_CREATE | IN_DELETE);
length = read(fd, buffer, BUF_LEN);
if (length < 0) {
perror("read");
}
while (i < length) {
struct inotify_event *event =
(struct inotify_event *) &buffer[i];
if (event->len) {
if (event->mask & IN_CREATE) {
printf("The file %s was created.\n", event->name);
} else if (event->mask & IN_DELETE) {
printf("The file %s was deleted.\n", event->name);
} else if (event->mask & IN_MODIFY) {
printf("The file %s was modified.\n", event->name);
}
}
i += EVENT_SIZE + event->len;
}
(void) inotify_rm_watch(fd, wd);
(void) close(fd);
return 0;
}
It doesn't work on a single file because, when we use a editor to modify file, the editor opens a copy of the file and when we save the edited version from the text editor, the existing file is deleted and a new file of the same name is created with the modifications.
When the old file is deleted, the watch created on that file becomes invalid and it is deleted automatically.
You can see the old file being replaced by the new file if you monitor the parent directory.
There are two ways to solve it, monitor the parent directory and print the message when modifications is done to the particular that you want to watch.
Else create a new watch on the file whenever modifications are made. When the old file is deleted, IN_DELETE_SELF event is triggered.
event->name will be non-empty only when you watch a directory, as it will contain the name of the file on which the event has occurred in the watched directory.
I think you're not using your user name, which is your home directory, and you're not checking the return of inotify_add_watch which probably fails:
"/home/name/tmp.cfg"
Edit: okay second problem, you shouldn't print name because
The name field is only present when an event is returned for a file
inside a watched directory;
Edit2: third problem, the file must exist before you run the program because you add a watch on the file, I suggest you check the error from inotify_add_watch
In watching a file, if the file is manipulated by an editor which you might do to edit it and create a change, it is likely to be doing some operations that results in the original file you asked to watch being deleted. Hence the notifications will stop if you only watch the one file.

Resources