How to read a binary file entirely and quickly in Ada? - file

Question

I would like to read the content of a binary file of several MB and store it into a buffer. Here's my function prototype (I can change it if needed):
procedure GET_BIN_CONTENT_FROM_PATH(PATH : in UNBOUNDED_STRING;
CONTENT : out UNBOUNDED_STRING);
Until now I've tried two methods, both using the Direct_IO package. In the first method, I was reading the file character by character; it worked, but it was awfully slow. In order to speed up the process, I tried to read the file MB by MB:
procedure GET_BIN_CONTENT_FROM_PATH (PATH : in UNBOUNDED_STRING;
CONTENT : out UNBOUNDED_STRING) is
BIN_SIZE_LIMIT : constant NATURAL := 1000000;
subtype FILE_STRING is STRING (1 .. BIN_SIZE_LIMIT);
package FILE_STRING_IO is new ADA.DIRECT_IO (FILE_STRING);
FILE : FILE_STRING_IO.FILE_TYPE;
BUFFER : FILE_STRING;
begin
FILE_STRING_IO.OPEN (FILE, MODE => FILE_STRING_IO.IN_FILE,
NAME => TO_STRING (C_BASE_DIR & PATH));
while not FILE_STRING_IO.END_OF_FILE (FILE) loop
FILE_STRING_IO.READ (FILE, ITEM => BUFFER);
APPEND (CONTENT, BUFFER);
end loop;
FILE_STRING_IO.CLOSE (FILE);
end GET_BIN_CONTENT_FROM_PATH;
Unfortunately, it seems that the READ operation won't happen if there is less than 1MB remaining in the file. As a result, big files (>1MB) get truncated, and little ones are not read at all. It's especially visible when working on images.
So, my question is: What's the correct method to read a binary file both quickly and entirely?
Thanks in advance.

Answer 1

Make the Bin_Size equal to Ada.Directories.Size(my_file), and read it in one go.
If it's too big for stack allocation (you'll get Storage_Error) allocate it with New, and use the rename trick
my_image : bin_array renames my_image_ptr.all;
so that nothing else need know...
But if it's only a few MB, that won't be necessary.

Answer 2

Ada.Streams.Stream_IO.Read reads into a Stream_Element_Array and tells you the last element read; if the array isn't filled (because you've reached the end of file), Last will be less than Item'Last.
A purist will note that Ada.Streams.Stream_Element'Size may not be the same as Character'Size, but for any normal processor chip it will be, so you can do unchecked conversion between the used part of the Stream_Element_Array and a String of the same size before appending to your Content.

Answer 3

There are a number of "correct" ways, but here's one that you might like. Especially when reading large files, an efficient way to read an entire file is to map the memory using mmap.
Depending on your licensing needs, you could be open to a third party, GPLd solution. AdaCore provides the GNATColl collection, which provides a nice interface for mmap. You can map the entire file and copy the contents.
declare
File : Mapped_File;
Str : Str_Access;
begin
File := Open_Read ("/tmp/file_on_disk");
Read (File); -- read the whole file
Str := Data (File);
for S in 1 .. Last (File) loop
Put (Str (S));
end loop;
Close (File);
end;
If your system doesn't support the mmap call, the library falls back to a read(2) implementation.

Answer 4

As others have mentioned, Ada.Streams.Stream_IO.Read is the way to go. Here's an example I put together on my system. Assuming you have sufficient memory available for dynamic allocation, this is able to read files larger than the stack size.
I haven't dug into the internals of the Stream.IO.Read code, but I suspect that the Stream_IO package is using a 4k block of memory (allocated from the heap) to buffer read operations.
with Ada.Directories; use Ada.Directories;
with Ada.Direct_IO;
with Ada.Unchecked_Deallocation;
with Ada.Streams.Stream_IO;
procedure Read_Input_File is
type Byte is mod 2 ** 8;
type Byte_Array is array (File_Size range <>) of Byte;
type Byte_Array_Access is access Byte_Array;
procedure Delete is new Ada.Unchecked_Deallocation
(Byte_Array, Byte_Array_Access);
function Read_Binary_File (Filename : String)
return Byte_Array_Access
is
package SIO renames Ada.Streams.Stream_IO;
Binary_File_Size : File_Size := Ada.Directories.Size (Filename);
Binary_File_Data : Byte_Array_Access;
S : SIO.Stream_Access;
File : SIO.File_Type;
begin
-- Allocate memory from the heap
Binary_File_Data := new Byte_Array (1 .. Binary_File_Size);
SIO.Open (File, SIO.In_File, Filename);
S := SIO.Stream (File);
-- Read entire file into the buffer
Byte_Array'Read (S, Binary_File_Data.all);
SIO.Close (File);
return Binary_File_Data;
end;
File_Data : Byte_Array_Access;
begin
File_Data := Read_Binary_File ("File_Name.bin");
-- Do something with data
Delete (File_Data);
end Read_Input_File;