How to read bytes from a file - file

I'm attempting to read a file into a list of bytes in prolog, using swipl version 8.0.3.
:- use_module(library(readutil)).
try_read_byte(File):-
open(File, read, Stream),
get_byte(Stream, B),
print(B).
try_read_char(File):-
open(File, read, Stream),
get_char(Stream, C),
print(C).
try_read_char succeeds, but when I call try_read_byte, I get an error:
ERROR: No permission to read bytes from TEXT stream `<stream>(0x56413a06a2b0)'
ERROR: In:
ERROR: [9] get_byte(<stream>(0x56413a06a2b0),_9686)
ERROR: [8] try_read_byte("test.pl") at /home/justin/code/decompile/test.pl:5
ERROR: [7] <user>
From reviewing the source code/documentation (https://www.swi-prolog.org/pldoc/man?section=error), it seems as if this is something like a type error, but I'm not able to figure out what to do based on that.

To read binary, you need to specify the option type(binary). Otherwise get_byte raises a permission_error as specified by the standard (section 8.13.1.3). This can be confusing as the user might be checking for correct permissions, which has nothing to do with the actual source of the problem.
try_read_byte(File):-
open(File, read, Stream, [type(binary)])),
get_byte(Stream, B),
write(B).
I just tried this with Scryer Prolog, and it prints the first byte of the file as expected.

Related

How to fix "HDF5-DIAG: Error unable to open file"?

When trying to run a MEEP simulation the following error appears:
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
#000: H5F.c line 509 in H5Fopen(): unable to open file
major: File accessibilty
minor: Unable to open file
#001: H5Fint.c line 1498 in H5F_open(): unable to open file: time = Fri Nov 15 16:56:54 2019
, name = '*.h5', tent_flags = 0
major: File accessibilty
minor: Unable to open file
#002: H5FD.c line 734 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#003: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = '*.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
h5topng error: error opening HD5 file
rm: *.h5: No such file or directory
Could somebody enlighten me on how to fix this/understand the error?
Thanks in advance
Disclaimer: I have never worked with MEEP.
The error seems to indicate taht the file doesn't exist:
#003: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = '*.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
So this appears to be the core issue.
Going further down the rabbit hole it seems that the file name passed to H5FD_sec2_open() is *.h5 which is most likely an invalid file name. I expect you want to pass something such as foo.h5 or relative/path/to/foo.h5 or /absolute/path/to/foo.h5 to the H5FD_sec2_open() function.
The wildcard is something that is usually not handled/interpreted by these kinds of functions. It's a higher level concept and requires "more filesystem access" to do anything useful with it as whoever receives that wildcard has to get a list of files & directories and figure out which file system entries match this wildcard.
Therefore, my answer would be: Make sure that you pass a valid file path to the corresponding file opening function.
Furthermore, you might want to extend your program so that it checks whether it's a valid file before you pass it to the corresponding MEEP function which allows you to have better error and user feedback control.

How to read a file in Crystal?

I recently picked up on Crystal after being a Rubyist for a while, and I can't seem to find anything about the File class. I want to open and read a file, but it gives me an error.
file = File.open("ditto.txt")
file = file.read
tequila#tequila-pc:~/code$ crystal fileopen.cr
Error in fileopen.cr:2: wrong number of arguments for 'File#read' (given 0, expected 1)
Overloads are:
- IO::Buffered#read(slice : Bytes)
- IO#read(slice : Bytes)
file = file.read
^~~~
You're probably looking for IO#gets_to_end which reads the entire file as a String. But you might as well use File.read
file_content = File.read("ditto.txt")
IO#read is a more low-level method which allows to read pieces of an IO into a byte slice.

simple_copy example on pmem.io

I have created the emulated device given at http://pmem.io/2016/02/22/pm-emulation.html, successfully.
It shows the device correctly:
:~/Prakash/nvml/src/examples/libpmem$ mount | grep pmem
/dev/pmem0 on /mnt/pmemd type ext4 (rw,relatime,dax,errors=continue,data=ordered)
However, when I execute the simple_copy sample given with pmem nvml, it gives this error:
amd#amd:~/Prakash/nvml/src/examples/libpmem$ ./simple_copy logs
/dev/pmem0 pmem_map_file: File exists
amd#amd:~/Prakash/nvml/src/examples/libpmem$ ./simple_copy logs
/dev/pmem0/logs pmem_map_file: Not a directory
Am I not using the program correctly?
Also, I have mounted the device as dax and I clearly see the performance advantage with
:~/Prakash/nvml/src/examples/libpmem$ sudo dd if=/dev/zero of=/dev/pmem0 bs=2G count=1
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 0.910729 s, 2.4 GB/s
:~/Prakash/nvml/src/examples/libpmem$ sudo dd if=/dev/zero of=/mnt/pmem0/test bs=2G count=1
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 6.39032 s, 336 MB/s
from the errors posted, is seems reasonable to believe:
without the appropriate option, it will not create a directory
without the appropriate option, it will not replace a file
If you open the example you are referring to, you will see the following:
if ((pmemaddr = pmem_map_file(argv[2], BUF_LEN,
PMEM_FILE_CREATE|PMEM_FILE_EXCL,
0666, &mapped_len, &is_pmem)) == NULL) {
perror("pmem_map_file");
exit(1);
}
This is the part that is giving you trouble. To understand why, let's look at the man 7 libpmem. You can find the relevant part here.
This is the paragraph we are interested in:
The pmem_map_file() function creates a new read/write mapping for a
file. If PMEM_FILE_CREATE is not specified in flags, the entire
existing file path is mapped, len must be zero, and mode is ignored.
Otherwise, path is opened or created as specified by flags and mode,
and len must be non-zero. pmem_map_file() maps the file using mmap(2),
but it also takes extra steps to make large page mappings more likely.
So, the pmem_map_file function effectively calls open(2) and then mmap(2). In the simple_copy.c example we can observe that the flags which were used are: PMEM_FILE_CREATE and PMEM_FILE_EXCL, and as we can learn from the manpage, they roughly translate to O_CREAT and O_EXCL respectively.
This means that the error messages are correct and you've received them because in your first attempt you've provided an existing file, whilst on the second attempt you tried a directory.
There's an in-depth explanation of libpmem here.

How do I read a 'family' of h5 files using python?

These are files in the file system as:
vgg16_weights_tf_dim_ordering_tf_kernels_0.h5
vgg16_weights_tf_dim_ordering_tf_kernels_1.h5
vgg16_weights_tf_dim_ordering_tf_kernels_2.h5
vgg16_weights_tf_dim_ordering_tf_kernels_3.h5
vgg16_weights_tf_dim_ordering_tf_kernels_4.h5
vgg16_weights_tf_dim_ordering_tf_kernels_5.h5
an attempt to open 'vgg16_weights_tf_dim_ordering_tf_kernels_0.h5' causes an error that suggests Family driver should be used
opening with
f = h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_0.h5','r', driver='family')
causes error "file name not unique"
opening with
f = h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_%d.h5','r', driver='family')
causes error "Unable to open file (Family member size should be 102400000. but the size from file access property is 2147483647)"
After some research I found the right way of opening a family of files as:
h5py.File('../input/keras-models/vgg16_weights_tf_dim_ordering_tf_kernels_%d.h5','r', driver='family', memb_size=102400000)
The default memb_size is 2**31-1 (i.e. 2147483647). If file family is written with member size specified as 102400000, one should also open it with the same.

TiffLib leaks handles to invalid files

If I try to open an invalid TIFF file with TIFFOpen(), the function returns NULL. For some reason, the error handler isn't called. However, the file remains open, so I can't delete/overwrite it from the same process.
I tried using TIFFFdOpen(), so that I can close the handle myself, but for some reason it gives me this error on valid TIFFs: "Cannot read TIFF header". This time the error is passed via the error handler.
How can I solve either of these problems?
Update: I'm talking about problems in TIFFOpen() itself, not in functions called later on. For example, they might occur if the TIFF file has size zero.
Perhaps you have an old or broken libtiff? I tried with libtiff-3.8.2 and it seems to work OK:
$ head -c 1000000 sample.tif > broken.tif
$ tiffinfo broken.tif
TIFFReadDirectory: broken.tif: Can not read TIFF directory count.
$ strace tiffinfo broken.tif
...
open("broken.tif", O_RDONLY) = 3
...
write(2, "broken.tif: Can not read TIFF dir"...
...
close(3) = 0
exit_group(0) = ?
Ie. there's a sane error and no leak (explicit close before the exit). I get the same behaviour with my own code.

Resources