I'm watching a directory by calling ReadDirectoryChangesW synchronously. When a new file is available, I try to access it immediately with CreateFile with GENERIC_READ and FILE_SHARE_READ, but this gives me ERROR_SHARING_VIOLATION. The process that put the file in the watched directory does not finish writing by the time I try to read it.
Is there any way to reliably wait until the file is available for reading? I can put the method into a loop like the one below, but I'm hoping there's a better way.
while ((hFile = CreateFile (path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL)) == INVALID_HANDLE_VALUE)
{
if (GetLastError() == ERROR_SHARING_VIOLATION)
Sleep (500);
else
break; // some other error occurred
}
if (hFile == INVALID_HANDLE_VALUE)
{
// deal with other error
return 0;
}
ReadFile (...);
I don't think there is a notification for the kind of event you're looking for, but as an improvement, I'd suggest progressive delays. This way you will get fast response times for stuff like a drag/drop and won't hog the CPU with a tight loop if the user keeps the file open for an hour in Excel.
int delay= 10;
while ((hFile = CreateFile (path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL)) == INVALID_HANDLE_VALUE)
{
if (GetLastError() == ERROR_SHARING_VIOLATION) {
Sleep (delay);
if (delay<5120) // max delay approx 5.Sec
delay*= 2;
}
else
break; // some other error occurred
}
There's no user-mode API for notifications on a closed file that I'm aware of. The loop you've proposed is really probably the best way. The only other thing you could do would be to watch for CloseFile in a filter driver ala Process Monitor, but yuck...
As #Matt Davis said, there is unfortunately no user-mode API but there is a workaround that depending on your use-case (I've written mine below) may do just what you want.
What worked for me in the past was registering for FILE_NOTIFY_CHANGE_LAST_WRITE instead of FILE_NOTIFY_CHANGE_FILE_NAME when calling ReadDirectoryChangesW:
ZeroMemory(&overlapped, sizeof(OVERLAPPED));
overlapped.hEvent = hChangeEvent;
// ...
ReadDirectoryChangesW(hSpoolPath,
eventBuffer,
EVENT_BUF_LENGTH,
FALSE,
FILE_NOTIFY_CHANGE_LAST_WRITE, // <----
NULL,
&overlapped,
NULL);
// ...
HANDLE events[2];
events[0] = hChangeEvent;
events[1] = hCancelEvent;
DWORD wRc = WaitForMultipleObjects(2, events, FALSE, DIRECTORY_WATCH_TIMEOUT);
The last write time gets updated as soon as the owning process closes the handle after creating the file and writing to it.
My use-case was one process that received HTTP-requests via TCP/IP and wrote the HTTP-body into a directory, where another process picked it up as soon as the receiving process was finished writing (and consequently closing the handle) it. The http-server was the only process that wrote to that directory, so I could rely on the create-write-close pattern.
If you know something about how the file is created, maybe wait until the file stops growing for X seconds, or wait until a sentinel file is deleted. Or sense the state of the program which creates them.
Related
My scenario: I have a C program in Windows that, in running time, corresponds to process A; it needs to start another process (process B) and monitor it so that, under some external event (say, lock file removed) it can terminate B and all its eventual children (processes started by B).
My approach was to place B into a job created with CreateJobObject, so that process A can terminate it (together with its children) with TerminateJobObject - and then it can terminate itself.
HANDLE jobHandle = CreateJobObject(NULL, NULL); // creates job
...
res=CreateProcess(NULL,cmdline, .... &pi); // creates process B
AssignProcessToJobObject(jobHandle,pi.hProcess); // add process B to job
...
if(...) {
TerminateJobObject(jobHandle,exitCode); // terminate job: process B and children
....
}
This works. Except that, under certain circumstances [*] the process A happens to be already included in a job. In this case CreateJobObject(NULL, NULL) does not create a new job, but it returns the current one - which is not what I want.
How do create a wholly new job?
I don't want to rely on nested jobs, because I want to support Windows 7.
[*] I'm looking at you, Eclipse - but that does not matter much now.
From comments:
The solution was to add the flag CREATE_BREAKAWAY_FROM_JOB to the CreateProcess call:
res=CreateProcess(NULL,cmdline, NULL, // Process handle not inheritable
NULL, // Thread handle not inheritable
FALSE, // Set handle inheritance to FALSE
CREATE_BREAKAWAY_FROM_JOB, // don't place inside old job
NULL, // Use parent's environment block
NULL, // Use parent's starting directory
&si, // Pointer to STARTUPINFO structure
&pi ); // Pointer to PROCESS_INFORMATION structure
Two caveats:
For this to work, the old job must have the JOB_OBJECT_LIMIT_BREAKAWAY_OK flag enabled - it was in my case.
I had wrongly believed that, in my original code, the call CreateJobObject(NULL, NULL) returned the old job and that's why the new process ended in the old job - if that were true, then this solution would not work. But it was not true, what happened was that AssignProcessToJobObject failed (my fault for not checking return code) because the newly created process was already placed in the old job. CreateJobObject(NULL, NULL) returns a new job.
I'm writing a plugin for my statusbar to print MPD state, currently using the libmpdclient library. It has to be robust to properly handle lost connections in case MPD is restarted, but simple checking with mpd_connection_get_error on existing mpd_connection object does not work – it can detect the error only when the initial mpd_connection_new fails.
This is a simplified code I'm working with:
#include <stdio.h>
#include <unistd.h>
#include <mpd/client.h>
int main(void) {
struct mpd_connection* m_connection = NULL;
struct mpd_status* m_status = NULL;
char* m_state_str;
m_connection = mpd_connection_new(NULL, 0, 30000);
while (1) {
// this check works only on start up (i.e. when mpd_connection_new failed),
// not when the connection is lost later
if (mpd_connection_get_error(m_connection) != MPD_ERROR_SUCCESS) {
fprintf(stderr, "Could not connect to MPD: %s\n", mpd_connection_get_error_message(m_connection));
mpd_connection_free(m_connection);
m_connection = NULL;
}
m_status = mpd_run_status(m_connection);
if (mpd_status_get_state(m_status) == MPD_STATE_PLAY) {
m_state_str = "playing";
} else if (mpd_status_get_state(m_status) == MPD_STATE_STOP) {
m_state_str = "stopped";
} else if (mpd_status_get_state(m_status) == MPD_STATE_PAUSE) {
m_state_str = "paused";
} else {
m_state_str = "unknown";
}
printf("MPD state: %s\n", m_state_str);
sleep(1);
}
}
When MPD is stopped during the execution of the above program, it segfaults with:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fb2fd9557e0 in mpd_status_get_state () from /usr/lib/libmpdclient.so.2
The only way I can think of to make the program safe is to establish a new connection in every iteration, which I was hoping to avoid. But then what if the connection is lost between individual calls to libmpdclient functions? How often, and more importantly how exactly, should I check if the connection is still alive?
The only way I could find that really works (beyond reestablishing a connection with each run) is using the idle command. If mpd_recv_idle (or mpd_run_idle) returns 0, it is an error condition, and you can take that as a cue to free your connection and run from there.
It's not a perfect solution, but it does let you keep a live connection between runs, and it helps you avoid segfaults (though I don't think you can completely avoid them, because if you send a command and mpd is killed before you recv it, I'm pretty sure the library still segfaults). I'm not sure if there is a better solution. It would be fantastic if there was a reliable way to detect if your connection was still alive via the API, but I can't find anything of the sort. It doesn't seem like libmpdclient is well-built for very long-lived connections that have to deal with mpd instances that go up and down over time.
Another lower-level option is to use sockets to interact with MPD through its protocol directly, though in doing that you'd likely reimplement much of libmpdclient itself anyway.
EDIT: Unfortunately, the idle command does block until something happens, and can sit blocking for as long as a single audio track will last, so if you need your program to do other things in the interim, you have to find a way to implement it asynchronously or in another thread.
Assuming "conn" is a connection created with "mpd_connection_new":
if (mpd_connection_get_error(conn) == MPD_ERROR_CLOSED) {
// mpd_connection_get_error_message(conn)
// will return "Connection closed by the server"
}
You can run this check after almost any libmpdclient call, including "mpd_recv_idle" or (as per your example) "mpd_run_status".
I'm using libmpdclient 2.18, and this certainly works for me.
I am facing this strange problem.
To delete a file unlink() API is called in my code. This call removes the file and succeeds on non-windows platforms. On windows it succeeds (returns 0) but doesn't remove the file.
To experiment I added a loop to call same API repeatedly. In second iteration I got an Permission denied error, Error code =13. Though read/write attributes are set on file and program has full permission to access the file.
I then called DeleteFile() instead of unlink() API. To my surprise I see the same result,call succeeded i.e. returned 1 but file is not removed physically.
I checked through unlocker utility, no other program is accessing the file except the program which is trying to remove this file.
Does anyone has idea what else could be wrong ?
Edit1:
Just to ensure file was not opened at the time of removing it. I saved the handle when file was created and tried to close before removing the file but I got error "'UNOPENED' (Errcode: 9 - Bad file descriptor)". Thus I conclude the file was not open at the time of removing it.
Edit2
As requested, here is the simplified version of code used to create and remove the file.
// Code to create the file
int create_file(const char* path)
{
HANDLE osfh; /* OS handle of opened file */
DWORD fileaccess; /* OS file access (requested) */
DWORD fileshare; /* OS file sharing mode */
DWORD filecreate; /* OS method of opening/creating */
DWORD fileattrib; /* OS file attribute flags */
SECURITY_ATTRIBUTES SecurityAttributes;
SecurityAttributes.nLength= sizeof(SecurityAttributes);
SecurityAttributes.lpSecurityDescriptor= NULL;
SecurityAttributes.bInheritHandle= !(oflag & _O_NOINHERIT);
fileaccess= GENERIC_WRITE;
fileshare= FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
filecreate= CREATE_NEW;
if ((osfh= CreateFile(path, fileaccess, fileshare, &SecurityAttributes,
filecreate, fileattrib, NULL)) == INVALID_HANDLE_VALUE)
{
// error handling
}
}
//Code to delete the file -
int remove_file (const char* name)
{
if ((err = unlink(name)) == -1)
{ //Error handling }
}
Edit3
As pointed by Joachim Pileborg and icabod, that DeleteFile() does not remove file if it is still open. As suggested by Remy Lebeau, to use process explorer. I found that one handle to file was indeed open when I closed that from process explorer file deleted like a charm :)
I had also mentioned in the Edit1 when I tried to close the file I got an error. It happened because the file descriptor I get from createfile() is not the actual handle returned by CreateFile() API instead a logical mapped handle due to underlying code complexities to support other non-windows platforms. Anyways, now I understood the root cause of problem but I was expecting if a file with open handle is passed to DeleteFile() API then it should fail in first attempt rather succeed and wait for open handles to close.
Assuming that you call your Createfile function, then later call your remove_file function... you still have a handle open to the file. The WinAPI function CreateFile, if it succeeds, keeps a handle open on the file. In your provided code, you don't close that handle.
From the documentation on DeleteFile:
The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED.
My guess is that you still have a handle open, and when you close that handle the file will be deleted.
However, your sample code is incomplete, so it is difficult to tell.
Inside a Windows C/C++ programm, I need to process a text file. So I just need to open the file for reading. However, I do not want to do that while the file is still being written to by another process. I also know that this other process will eventually close the file and never write to it agin.
Looking at similar questions on stackoverflow, the typical answer is "try and open the file for writing - if that fails then try again later").
Now in this case, my process does not have write access to the file at all. So checking if the file can be opened for writing is not an option . It will always fail irrespective of any other process having write access or not.
As Hans Passant and Igor Tandetnik said you just need to pass the appropriate sharing flag to CreateFile. As the MSDN documentation for CreateFile says:
FILE_SHARE_WRITE 0x00000002
Enables subsequent open operations on a file or device to request write access.
Otherwise, other processes cannot open the file or device if they request write access.
If this flag is not specified, but the file or device has been opened for write access
or has a file mapping with write access, the function fails.
You'll want to use code like the following:
HANDLE handle = CreateFile(name, GENERIC_READ, FILE_SHARE_READ, NULL,
OPEN_EXISTING, 0, NULL);
if (handle == INVALID_HANDLE_VALUE) {
DWORD errcode = GetLastError();
if (errcode == ERROR_SHARING_VIOLATION) {
printf("%s: sharing violation\n", name);
} else {
printf("%s: CreateFile failed, error code = %lu\n", name, errcode);
}
} else {
printf("%s: CreateFile succeeded\n", name);
}
This code in unable to tell if the ERROR_SHARING_VIOLATION occurred because the other process has the file open for writing or because the another process didn't use FILE_SHARE_READ when opening the file. In the later case any attempt to read from the file will fail with a sharing violation. The FILE_SHARE_READ flag is passed to prevent sharing violations in the case when the file already been opened and FILE_SHARE_READ was used. You could also add FILE_SHARE_DELETE but I assume you'd consider that the same as write access.
I have an application which uses libuv library. it runs default loop:
uv_run(uv_default_loop());
How can the application be gracefully exited in case of a failure? Currently I am doing it like in the following example:
uv_tcp_t* tcp = malloc(sizeof(uv_tcp_t));
int r = uv_tcp_init(uv_default_loop(), tcp);
if (r) {
free(tcp);
uv_loop_delete(default_loop);
exit(EXIT_FAILURE);
}
Should uv_loop_delete function be called? What does it do? Does it drop all pending callback functions? Does it close all currently opened TCP connections? Do I have to do it manually before exiting?
P.S.: Can't add the tag 'libuv' (less than 1500 reputation). Can somebody create and add it?
Declaration of uv_loop_delete is here and source code is here. It looks like this:
void uv_loop_delete(uv_loop_t* loop) {
uv_ares_destroy(loop, loop->channel);
ev_loop_destroy(loop->ev);
#if __linux__
if (loop->inotify_fd == -1) return;
ev_io_stop(loop->ev, &loop->inotify_read_watcher);
close(loop->inotify_fd);
loop->inotify_fd = -1;
#endif
#if HAVE_PORTS_FS
if (loop->fs_fd != -1)
close(loop->fs_fd);
#endif
}
It will, effectively, clean every file descriptor it's possible to clean. It will close TCP connection, Inotify connections, Socket used to read events, Pipe fds, etc, etc.
=> Yes, this function will close everything you have opened through libuv.
NB: Anyway, when your application exit, your Operating System will clean and close everything you have left open, without any mercy.