kqueue on directory not trigger when file within is modified - c

I have used kquque to monitor desktop with:
flags - EV_ADD | EV_CLEAR
fflags - NOTE_DELETE | NOTE_WRITE | NOTE_EXTEND | NOTE_ATTRIB | NOTE_LINK | NOTE_RENAME | NOTE_REVOKE
filter - EVFILT_VNODE
However when I edit a .js file on desktop with sublime2 software, it doesnt trigger a notification :(
Please advise
Here is my js-ctypes code:
var rez_fd = ostypes.API('kqueue')();
console.info('rez_fd:', rez_fd.toString(), uneval(rez_fd));
if (ctypes.errno != 0) {
throw new Error('Failed rez_fd, errno: ' + ctypes.errno);
}
this.kq = rez_fd;
this.path = OS.Constants.Path.desktopDir;
// Open a file descriptor for the file/directory that you want to monitor.
var event_fd = ostypes.API('open')(this.path, OS.Constants.libc.O_EVTONLY);
console.info('event_fd:', event_fd.toString(), uneval(event_fd));
if (ctypes.errno != 0) {
throw new Error('Failed event_fd, errno: ' + ctypes.errno);
}
// The address in user_data will be copied into a field in the event.If you are monitoring multiple files,you could,for example,pass in different data structure for each file.For this example,the path string is used.
var user_data = ctypes.cast(ctypes.char.array()(this.path), ctypes.void.ptr);
// Set the timeout to wake us every half second.
var timeout = ostypes.TYPE.timespec();
var useSec = 0;
var useNsec = 500000000;
timeout.tv_sec = useSec; // 0 seconds
timeout.tv_nsec = useNsec; // 500 milliseconds
// Set up a list of events to monitor.
var fflags = vnode_events = ostypes.CONST.NOTE_DELETE | ostypes.CONST.NOTE_WRITE | ostypes.CONST.NOTE_EXTEND | ostypes.CONST.NOTE_ATTRIB | ostypes.CONST.NOTE_LINK | ostypes.CONST.NOTE_RENAME | ostypes.CONST.NOTE_REVOKE; // ostypes.TYPE.unsigned_int
var events_to_monitor = ostypes.TYPE.kevent.array(ostypes.CONST.NUM_EVENT_FDS)();
var filter = ostypes.CONST.EVFILT_VNODE;
var flags = ostypes.CONST.EV_ADD | ostypes.CONST.EV_CLEAR;
EV_SET(events_to_monitor.addressOfElement(0), event_fd, filter, flags, fflags, 0, user_data);
// Handle events
var event_data = ostypes.TYPE.kevent.array(ostypes.CONST.NUM_EVENT_SLOTS)(); // 1 slot
var num_files = 1; // ostypes.TYPE.int
var continue_loop = 40; // Monitor for twenty seconds. // ostypes.TYPE.int
while (--continue_loop) {
var event_count = ostypes.API('kevent')(this.kq, ctypes.cast(events_to_monitor.address(), ostypes.TYPE.kevent.ptr), ostypes.CONST.NUM_EVENT_SLOTS, ctypes.cast(event_data.address(), ostypes.TYPE.kevent.ptr), num_files, timeout.address());
console.info('event_count:', event_count.toString(), uneval(event_count));
if (ctypes.errno != 0) {
throw new Error('Failed event_count, errno: ' + ctypes.errno + ' and event_count: ' + cutils.jscGetDeepest(event_count));
}
if (cutils.jscEqual(event_data.addressOfElement(0).contents.flags, ostypes.CONST.EV_ERROR)) {
throw new Error('Failed event_count, due to event_data.flags == EV_ERROR, errno: ' + ctypes.errno + ' and event_count: ' + cutils.jscGetDeepest(event_count));
}
if (!cutils.jscEqual(event_count, '0')) {
console.log('Event ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.ident) + ' occurred. Filter ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.filter) + ', flags ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.flags) + ', filter flags ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.fflags) + ', filter data ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.data) + ', path ' + cutils.jscGetDeepest(event_data.addressOfElement(0).contents.udata /*.contents.readString()*/ ));
} else {
// No event
}
// Reset the timeout. In case of a signal interrruption, the values may change.
timeout.tv_sec = useSec; // 0 seconds
timeout.tv_nsec = useNsec; // 500 milliseconds
}
ostypes.API('close')(event_fd);

I just realized that you're not monitoring the .js file, you're monitoring its directory. That makes everything a lot less mysterious.
The short version is: If you open a file and write it, that doesn't change anything on the directory. If you atomically save a file, that does change the directory, but Sublime 2 doesn't atomically save by default.
So, to watch for any changes to any file in the directory, you need to enumerate all files in the directory and add them all to the kqueue,* as well as the directory.
Watching the directory will catch atomic saves (and new files being created); watching the files will catch overwrites. (Files being unlinked will trigger both.) If you're worried about performance… well, kqueue is designed to handle switching on 10000 file descriptors, and neither UFS nor HFS+ is a good filesystem for hundreds of thousands of directory entries in the same directory, so you're probably OK… but you may want to add some code that warns or aborts if the directory turns out to be massively huge.
If you want to understand why this is necessary, you have to think about how the two different kinds of saves work.
A write just writes to a file descriptor. That file descriptor could have one directory entry link on the filesystem—but it could just as easily have none (e.g., it was created in a temporary namespace, or you just unlinked the file after creating it), or many (e.g., you've created hard links to it). So it can't actually update "the directory entry for the file", because there is no such thing.
An atomic save, on the other hand, works by creating a new temporary file, writing to that, and then rename-ing the temporary over the original file. That rename has to update the directory, replacing the entry pointing at the old file with an entry pointing at the new file. (Of course it also sends a DELETE notification for the file itself, because the file is losing a link. And you'll also usually sends an ATTRIB, because most apps want the new file to have the same extended attributes, extra forks, etc.)
* There is an obvious race condition here: if a file is moved or deleted between the readdir and adding it to the kqueue, you'll get an error. You may want to handle that error by generating an immediate notification, or maybe you just want to ignore it—after all, from the user perspective, it's not much different from the case where someone deletes a file between the time your program starts and the time you do the readdir.

Related

Error loading physionet ECG database on MATLAB

I'm using this code to load the ECG-ID database into MATLAB:
%% Initialization
clear all; close all; clc
%% read files from folder A
% Specify the folder where the files live.
myFolder = 'Databases\ECG_ID';
% Check to make sure that folder actually exists. Warn user if it doesn't.
if ~isfolder(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder;)
uiwait(warndlg(errorMessage);)
myFolder = uigetdir(; % Ask for a new one.)
if myFolder == 0
% User clicked Cancel
return;
end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '**/rec_*'; % Change to whatever pattern you need.)
theFiles = dir(filePattern;)
for k = 1 : length(theFiles)
baseFileName = theFiles(k.name;)
fullFileName = fullfile(theFiles(k.folder, baseFileName);)
fprintf(1, 'Now reading %s\n', fullFileName;)
% Now do whatever you want with this file name,
% such as reading it in as an image array with imread()
[sig, Fs, tm] = rdsamp(fullFileName, [1],[],[],[],1;)
end
But I keep getting this error message:
Now reading C:\Users\******\Documents\MATLAB\Databases\ECG_ID\Person_01\rec_1.atr
Error using rdsamp (line 203)
Could not find record: C:\Users\******\Documents\MATLAB\Databases\ECG_ID\Person_01\rec_1.atr. Search path is set to: '.
C:\Users\******\Documents\MATLAB\mcode\..\database\ http://physionet.org/physiobank/database/'
I can successfully load one signal at a time (but I can't load the entire database using the above code) using this command:
[sig, Fs, tm] = rdsamp('Databases\ECG_ID\Person_01\rec_1');
How do I solve this problem? How can I load all the files in MATLAB?
Thanks in advance.

Apache Camel consumer template to copy file, cannot copy one file twice

Hi I am using apache camel 2.15.2. I have got a consumer template so that I can copy file with dynamic file names:
if (fileInfo != null) {
filename = fileInfo.getFileName();
String camelUri = "file://" + fileInfo.getCopyFilePath() + "/?fileName=RAW("
+ filename + ")&noop=false&idempotent=false&readLock=changed";
System.out.println("Camel uri: " + camelUri);
logger.info("Camel uri: " + camelUri);
Exchange ex = consumerTemplate.receive(camelUri);
....
As you can see, I have set noop, and idempotent explicitly to achieve copying same file more than once. But, it does not do that. It hangs on receive method for subsequent tries to copy a file with same name. It can copy that, only if we restart the application. Any suggestions would be much appreciated. It might be something similar to this issue, but I do not have access to that solution. Thanks in advance.
When I debugged through Camel code, it seems, it is calling EventDrivenPollingConsumer's receive method, and hangs when calls queue.take() (line 110, EventDrivenPollingConsumer). And, even inside that, 'count' variable is zero in ArrayBlockingQueue:
while (count == 0)
notEmpty.await();
Added this, just in case it helps anyone having a clue.
Ok, If I call 'consumerTemplate.doneUoW(ex)', it does copy multiple times. But, at the same time it was deleting (actually moving to .camel folder) the source file, which I did not want to! Then, had to set noop=true:
if (fileInfo != null) {
filename = fileInfo.getFileName();
String camelUri = "file://" + fileInfo.getCopyFilePath() + "/?fileName=RAW("
+ filename + ")&noop=true&idempotent=false&readLock=none";
System.out.println("Camel uri: " + camelUri);
logger.info("Camel uri: " + camelUri);
Exchange ex = consumerTemplate.receive(camelUri);
// consumerTemplate.r
logger.info("File received: " + fileInfo.getFileName());
exchange.getOut().setBody(ex.getIn().getBody());
consumerTemplate.doneUoW(ex);
}
Now, it works as expected.

File Open Mechanism

I'm trying to create an application. The application gives the user 2 combo boxes. Combo Box 1 gives the first part of the file name the user wants, and Combo Box 2 gives the second part of the file name. E.g. Combo box 1 option 1 is 1 and Combo Box 2 option 1 is A; the selected file is 1_A.txt.
I have a load button which is to use the file name , and open a file with that name. If no file exists, the application opens a dialog saying "No Such File Exists"
from PySide import QtGui, QtCore
from PySide.QtCore import*
from PySide.QtGui import*
class MainWindow(QtGui.QMainWindow):
def __init__(self,):
QtGui.QMainWindow.__init__(self)
QtGui.QApplication.setStyle('cleanlooks')
#PushButtons
load_button = QPushButton('Load',self)
load_button.move(310,280)
run_Button = QPushButton("Run", self)
run_Button.move(10,340)
stop_Button = QPushButton("Stop", self)
stop_Button.move(245,340)
#ComboBoxes
#Option1
o1 = QComboBox(self)
l1 = QLabel(self)
l1.setText('Option 1:')
l1.setFixedSize(170, 20)
l1.move(10,230)
o1.move(200, 220)
o1.setFixedSize(100, 40)
o1.insertItem(0,'')
o1.insertItem(1,'A')
o1.insertItem(2,'B')
o1.insertItem(3,'test')
#Option2
o2 = QComboBox(self)
l2 = QLabel(self)
l2.setText('Option 2:')
l2.setFixedSize(200, 20)
l2.move(10,290)
o2.move(200,280)
o2.setFixedSize(100, 40)
o2.insertItem(0,'')
o2.insertItem(1,'1')
o2.insertItem(2,'2')
o2.insertItem(3,'100')
self.fileName = QLabel(self)
self.fileName.setText("Select Options")
o1.activated.connect(lambda: self.fileName.setText(o1.currentText() + '_' + o2.currentText() + '.txt'))
o2.activated.connect(lambda: self.fileName.setText(o1.currentText() + '_' + o2.currentText() + '.txt'))
load_button.clicked.connect(self.fileHandle)
def fileHandle(self):
file = QFile(str(self.fileName.text()))
open(file, 'r')
if __name__ == '__main__':
import sys
app = QtGui.QApplication(sys.argv)
window = MainWindow()
window.setWindowTitle("Test11")
window.resize(480, 640)
window.show()
sys.exit(app.exec_())
The error I'm getting is TypeError: invalid file: <PySide.QtCore.QFile object at 0x031382B0> and I suspect this is because the string described in the file handle isn't being inserted in the QFile properly. Can someone please help
The Python open() function doesn't have any knowledge of objects of type QFile. I doubt you actually need to construct a QFile object though.
Instead, just open the file directly via open(self.fileName.text(), 'r'). Preferably, you would do:
with open(self.fileName.text(), 'r') as myfile:
# do stuff with the file
unless you need to keep the file open for a long period of time
I came up with a solution also.
def fileHandle(self):
string = str(self.filename.text())
file = QFile()
file.setFileName(string)
file.open(QIODevice.ReadOnly)
print(file.exists())
line = file.readLine()
print(line)
What this does is that it takes the string of the filename field. Creates the file object. Names the file object the string, and then opens the file. I have exists to check if the file is there, and after reading the test document i have, ti seemed to work as I wanted.
Thanks anyway #three_pineapples, but I'm going to use my solution :P

Python Simple PiggyBank Program

This is my Python Program that I have been having some issues with:
-- coding: cp1252 --
from time import gmtime, strftime
print("Welcome to the PiggyBank version 1.")
num_write = int(input("How much money would you like to store in your PiggyBank?"))
f = open("PiggyBanks_Records.txt", "w")
current_time = strftime("%Y-%m-%d %H:%M:%S", gmtime())
convert_1 = str(current_time)
convert_2 = str(int(num_write))
add_1 = ("\n" + convert_1 + " £" + convert_2)
add_2 = ("\n" + add_1) #Tried to make it so new line is added every time the program is run
final_record = str(add_2)
print("Final file written to the PiggyBank: " + final_record)
#Write to File
f.write(final_record)
f.close()
Right now whenever the program writes to the file it over-writes. I would preferably would like to keep, like a history of the amounts added. If anyone can help so the string that needs to be written to the .txt file goes down by one line and essentially keeps going for ever. I am also open to any suggestion on how I can shorten this code.
You need to open your file with append mode :
f = open("PiggyBanks_Records.txt", "a")
Using the 'w' write option with open automatically looks for the specified file, and deletes its contents if it already exists (which you can read about here) or creates it if it doesn't. Use 'a' instead to add / append to the file.

Groovy - create file issue: The filename, directory name or volume label syntax is incorrect

I'm running a script made in Groovy from Soap UI and the script needs to generate lots of files.
Those files have also in the name two numbers from a list (all the combinations in that list are different), and there are 1303 combinations
available and the script generates just 1235 files.
A part of the code is:
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
where $file is actually that part of the file name which include those 2 combinations from that list:
file = "abc" + "-$firstNumer"+"_$secondNumber"
For those file which are not created is a message returned:"The filename, directory name or volume label syntax is incorrect".
I've tried puting another path:
filename = "D:\\rez\\" + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
and also:
File parentFolder = new File("D:\\rez\\");
File targetFile = new File(parentFolder, "$file"+"_OK.txt");
targetFile.createNewFile();
(which I've found here: What are possible reasons for java.io.IOException: "The filename, directory name, or volume label syntax is incorrect")
but nothing worked.
I have no ideea where the problem is. Is strange that 1235 files are created ok, and the rest of them, 68 aren't created at all.
Thanks,
My guess is that some of the files have illegal characters in their paths. Exactly which characters are illegal is platform specific, e.g. on Windows they are
\ / : * ? " < > |
Why don't you log the full path of the file before targetFile.createNewFile(); is called and also log whether this method succeeded or not, e.g.
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
println "attempting to create file: $targetFile"
if (targetFile.createNewFile()) {
println "Successfully created file $targetFile"
} else {
println "Failed to create file $targetFile"
}
When the process is finished, check the logs and I suspect you'll see a common pattern in the ""Failed to create file...." messages
File.createNewFile() returns false when a file or directory with that name already exists. In all other failure cases (security, I/O) it throws an exception.
Evaluate createNewFile()'s return value or, additionally, use the File.exists() method:
File file = new File("foo")
// works the first time
createNewFile(file)
// prints an error message
createNewFile(file)
void createNewFile(File file) {
if (!file.createNewFile()) {
assert file.exists()
println file.getPath() + " already exists."
}
}

Resources