These are files in the file system as:
vgg16_weights_tf_dim_ordering_tf_kernels_0.h5
vgg16_weights_tf_dim_ordering_tf_kernels_1.h5
vgg16_weights_tf_dim_ordering_tf_kernels_2.h5
vgg16_weights_tf_dim_ordering_tf_kernels_3.h5
vgg16_weights_tf_dim_ordering_tf_kernels_4.h5
vgg16_weights_tf_dim_ordering_tf_kernels_5.h5
an attempt to open 'vgg16_weights_tf_dim_ordering_tf_kernels_0.h5' causes an error that suggests Family driver should be used
opening with
f = h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_0.h5','r', driver='family')
causes error "file name not unique"
opening with
f = h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_%d.h5','r', driver='family')
causes error "Unable to open file (Family member size should be 102400000. but the size from file access property is 2147483647)"
After some research I found the right way of opening a family of files as:
h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_%d.h5','r', driver='family', memb_size=102400000)
The default memb_size is 2**31-1 (i.e. 2147483647). If file family is written with member size specified as 102400000, one should also open it with the same.
Related
I am using fswebcam to capture images from a USB device.
The command is fswebcam -d /dev/v4l/by-id/device-id-of-usb-webcam-index0 /home/myhome/output.jpg.
Usually, it works fine.
But once it gave the following error:
Failed to open /dev/v4l/by-id/device-id-of-usb-webcam-index0: Invalid argument
[1m--- Opening /dev/v4l/by-id/device-id-of-usb-webcam-index0...
[0m[0mTrying source module v4l2...
[0m[31mError opening device: /dev/v4l/by-id/device-id-of-usb-webcam-index0
[0m[31mopen: Invalid argument
[0m[0mTrying source module v4l1...
[0m[31mError opening device: /dev/v4l/by-id/device-id-of-usb-webcam-index0
[0m[31mopen: Invalid argument
[0m[31mUnable to find a source module that can read /dev/v4l/by-id/device-id-of-usb-webcam-index0.
I tried to see when this happens by reading fswebcam's source code at https://github.com/fsphil/fswebcam/blob/master/src_v4l2.c#L817
Basically the code s->fd = open(src->source, O_RDWR | O_NONBLOCK); fails with errno being set to EINVAL.
But I am wondering when would this happen.
I tried making a symlink to an non-existing file, but it didn't work (prints open: No such file or directory)
Is there a way to make the open function return with errno EINVAL?
I'm making some extensions to the kernel module nandsim, and I'm having trouble finding the correct way to test if a file exists before opening it. I've read this question, which covers how the basic open/read/write operations go, but I'm having trouble figuring out if and how the normal open(2) flags apply here.
I'm well aware that file reading and writing in kernel modules is bad practice; this code already exists in the kernel and is already reading and writing files. I am simply trying to make a few adjustments to what is already in place. At present, when the module is loaded and instructed to use a cache file (specified as a string path when invoking modprobe), it uses filp_open() to open the file or create it if it does not exist:
/* in nandsim.c */
...
module_param(cache_file, charp, 0400);
...
MODULE_PARM_DESC(cache_file, "File to use to cache nand pages instead of memory");
...
struct file *cfile;
cfile = filp_open(cache_file, O_CREAT | O_RDWR | O_LARGEFILE, 0600);
You might ask, "what do you really want to do here?" I want to include a header for the cache file, such that it can be reused if the system needs to be reset. By including information about the nand page geometry and page count at the beginning of this file, I can more readily simulate a number of error conditions that otherwise would be impossible within the nandsim framework. If I can bring down the nandsim module during file operations, or modify the backing file to model a real-world fault mode, I can recreate the net effect of these error conditions.
This would allow me to bring the simulated device back online using nandsim, and assess how well a fault-tolerant file system is doing its job.
My thought process was to modify it as follows, such that it would fail trying to force creation of a file which already exists:
struct file *cfile;
cfile = filp_open(cache_file, O_CREAT | O_EXCL | O_RDWR | O_LARGEFILE, 0600);
if(IS_ERR(cfile)){
printk(KERN_INFO "File didn't exist: %ld", PTR_ERR(cfile));
/* Do header setup for first-time run of NAND simulation */
}
else{
/* Read header and validate against system parameters. Recover operations */
}
What I'm seeing is an error, but it is not the one I would have expected. It is reporting errno 14, EFAULT (bad address) instead of errno 17 EEXIST (File exists). I don't want to run with this because I would like this to be as idiomatic and correct as possible.
Is there some other way I should be doing this?
Do I need to somehow specify that the file path is in user address space? If so, why is that not the case in the code as it was?
EDIT: I was able to get a reliable error by trying to open with only O_RDWR and O_LARGEFILE, which resulted in ENOENT. It is still not clear why my original approach was incorrect, nor what the best way to accomplish my goal is. That said, if someone more experienced could comment on this, I can add it as a solution.
Indeed, filp_open expects a file path which is in kernel address space. Proof is the use of getname_kernel. You can mimic this for your use case with something like this:
struct filename *name = getname(cache_file);
struct file *cfile = ERR_CAST(name);
if (!IS_ERR(name)) {
cfile = file_open_name(name, O_CREAT | O_EXCL | O_RDWR | O_LARGEFILE, 0600);
if (IS_ERR(cfile))
return PTR_ERR(cfile);
putname(name);
}
Note that getname expects a user-space address and is the equivalent of getname_kernel.
When trying to run a MEEP simulation the following error appears:
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
#000: H5F.c line 509 in H5Fopen(): unable to open file
major: File accessibilty
minor: Unable to open file
#001: H5Fint.c line 1498 in H5F_open(): unable to open file: time = Fri Nov 15 16:56:54 2019
, name = '*.h5', tent_flags = 0
major: File accessibilty
minor: Unable to open file
#002: H5FD.c line 734 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#003: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = '*.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
h5topng error: error opening HD5 file
rm: *.h5: No such file or directory
Could somebody enlighten me on how to fix this/understand the error?
Thanks in advance
Disclaimer: I have never worked with MEEP.
The error seems to indicate taht the file doesn't exist:
#003: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = '*.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
So this appears to be the core issue.
Going further down the rabbit hole it seems that the file name passed to H5FD_sec2_open() is *.h5 which is most likely an invalid file name. I expect you want to pass something such as foo.h5 or relative/path/to/foo.h5 or /absolute/path/to/foo.h5 to the H5FD_sec2_open() function.
The wildcard is something that is usually not handled/interpreted by these kinds of functions. It's a higher level concept and requires "more filesystem access" to do anything useful with it as whoever receives that wildcard has to get a list of files & directories and figure out which file system entries match this wildcard.
Therefore, my answer would be: Make sure that you pass a valid file path to the corresponding file opening function.
Furthermore, you might want to extend your program so that it checks whether it's a valid file before you pass it to the corresponding MEEP function which allows you to have better error and user feedback control.
I am facing this strange problem.
To delete a file unlink() API is called in my code. This call removes the file and succeeds on non-windows platforms. On windows it succeeds (returns 0) but doesn't remove the file.
To experiment I added a loop to call same API repeatedly. In second iteration I got an Permission denied error, Error code =13. Though read/write attributes are set on file and program has full permission to access the file.
I then called DeleteFile() instead of unlink() API. To my surprise I see the same result,call succeeded i.e. returned 1 but file is not removed physically.
I checked through unlocker utility, no other program is accessing the file except the program which is trying to remove this file.
Does anyone has idea what else could be wrong ?
Edit1:
Just to ensure file was not opened at the time of removing it. I saved the handle when file was created and tried to close before removing the file but I got error "'UNOPENED' (Errcode: 9 - Bad file descriptor)". Thus I conclude the file was not open at the time of removing it.
Edit2
As requested, here is the simplified version of code used to create and remove the file.
// Code to create the file
int create_file(const char* path)
{
HANDLE osfh; /* OS handle of opened file */
DWORD fileaccess; /* OS file access (requested) */
DWORD fileshare; /* OS file sharing mode */
DWORD filecreate; /* OS method of opening/creating */
DWORD fileattrib; /* OS file attribute flags */
SECURITY_ATTRIBUTES SecurityAttributes;
SecurityAttributes.nLength= sizeof(SecurityAttributes);
SecurityAttributes.lpSecurityDescriptor= NULL;
SecurityAttributes.bInheritHandle= !(oflag & _O_NOINHERIT);
fileaccess= GENERIC_WRITE;
fileshare= FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
filecreate= CREATE_NEW;
if ((osfh= CreateFile(path, fileaccess, fileshare, &SecurityAttributes,
filecreate, fileattrib, NULL)) == INVALID_HANDLE_VALUE)
{
// error handling
}
}
//Code to delete the file -
int remove_file (const char* name)
{
if ((err = unlink(name)) == -1)
{ //Error handling }
}
Edit3
As pointed by Joachim Pileborg and icabod, that DeleteFile() does not remove file if it is still open. As suggested by Remy Lebeau, to use process explorer. I found that one handle to file was indeed open when I closed that from process explorer file deleted like a charm :)
I had also mentioned in the Edit1 when I tried to close the file I got an error. It happened because the file descriptor I get from createfile() is not the actual handle returned by CreateFile() API instead a logical mapped handle due to underlying code complexities to support other non-windows platforms. Anyways, now I understood the root cause of problem but I was expecting if a file with open handle is passed to DeleteFile() API then it should fail in first attempt rather succeed and wait for open handles to close.
Assuming that you call your Createfile function, then later call your remove_file function... you still have a handle open to the file. The WinAPI function CreateFile, if it succeeds, keeps a handle open on the file. In your provided code, you don't close that handle.
From the documentation on DeleteFile:
The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED.
My guess is that you still have a handle open, and when you close that handle the file will be deleted.
However, your sample code is incomplete, so it is difficult to tell.
I've coded a SIC assembler and everything seems to be working fine except for the I/O aspect of it.
I've loaded the object code into memory (converted char format into machine representation), but when I call SICRun(); to execute the code, I get an error stating "devf1 cannot be found".
I know this is related to the input/output device instructions in the source code.
The c file states that it depends on external files, most notably, Dev[6]. Am I supposed to create this myself? My instructor did not give us any other files to work with. Any insight?
Example: TD OUTPUT ;TEST OUTPUT DEVICE
This directory contains the source code (source.asm), header file (sic.h) and the SIC simulator (sicengine.c)
From the sicengine.c source file it looks as though the devf1 (also dev2/dev3) file is expected to exist so this 'input device' can be read from (fopen is passed "r" as a parameter):
if (opcode == 216) { /* RD */
/* ... */
if ((Dev[Devcode] = fopen(SICFile[Devcode],"r")) == NULL) {
printf("cannot open file %s\n", SICFile[Devcode]);
exit(1);
}
The comment in the code about depending on file Dev[6] is ambiguous. It really means the names of the files in the Dev array, which are devf1, devf2 and devf3 (input devices) and devf04, devf05 and devf05 (output devices).
I would suggest creating files devf1, devf1 and devf3.