I am working with fortran 90. I have 50 .dat files that correspond to 50 time steps. Files have a similar name, for instance tstep01.dat, tstep02.dat, tstep03.dat, etc. I have to read sequentially the name of these files. Files are loacted in the output directory that is in the same directory as my script. I want to get the name of files in order to pass it to a subroutine that generates animation. The subroutine gets this name to read the data and creates .png frames subsequently. I have already tried this:
character(len = 14) :: data_name !data name
nframes = 50 !number of timesteps
do i = 1, nframes
write(data_name, '(output/("/tstep", I2.2, ".dat"))') i
end do
but I got this error :
write(data_name, ('output/("/tstep", I2.2, ".dat")')) i
1
Error: Nonnegative width required in format string at (1)
I think the problem is with the output/, but I don't know what is the correct way for defining the directory of files. Your help would be appreciated.
Related
i have a text file which contain several lines in the end of the file I have the following line: "Total: 235267878"
my question is: How do I retrieve the specific value (235267878) and set it to a variable?
Thanks!
Since you didn't specify how long your file can be, we can iterate through the file:
with open('test.txt', 'r') as file:
for line in file:
if 'Total:' in line:
totalValue = line.split(':')[-1].strip()
print(totalValue)
In this solution I assume that the line we are looking for always has the form Total: {number}. We open the file in the read only mode and iterate through the lines (In my exmaple the file is called test.txt). After the line containing the total value is found, we split it and remove possible white spaces to get the number. The variable totalValue contains the number you are looking for.
Could someone help me in writing a program that has to compile all the files in the directory and report error, if any. For which my program has to get the list of all files under the folder with its full path and store it in a temp-table and then it has to loop through the temp table and compile the files.
Below is a very rough start.
Look for more info around the COMPILE statement and the COMPILER system handle in the online help (F1).
Be aware that compiling requires you to have a developer license installed. Without it the COMPILE statement will fail.
DEFINE VARIABLE cDir AS CHARACTER NO-UNDO.
DEFINE VARIABLE cFile AS CHARACTER NO-UNDO FORMAT "x(30)".
ASSIGN
cDir = "c:\temp\".
INPUT FROM OS-DIR(cDir).
REPEAT:
IMPORT cFile.
IF cFile MATCHES "*..p" THEN DO:
COMPILE VALUE(cDir + cFile) SAVE NO-ERROR.
IF COMPILER:ERROR THEN DO:
DISPLAY
cFile
COMPILER:GET-MESSAGE(1) FORMAT "x(60)"
WITH FRAME frame1 WIDTH 300 20 DOWN.
END.
END.
END.
INPUT CLOSE.
Since the comment wouldn't let me paste this much into it... using INPUT FROM OS-DIR returns all of the files and directories under a directory. You can use this information to keep going down the directory tree to find all sub directories
OS-DIR documentation:
Sometimes, rather than reading the contents of a file, you want to read a list of the files in a directory. You can use the OS–DIR option of the INPUT FROM statement for this purpose.
Each line read from OS–DIR contains three values:
*The simple (base) name of the file.
*The full pathname of the file.
*A string value containing one or more attribute characters. These characters indicate the type of the file and its status.
Every file has one of the following attribute characters:
*F — Regular file or FIFO pipe
*D — Directory
*S — Special device
*X — Unknown file type
In addition, the attribute string for each file might contain one or more of the following attribute characters:
*H — Hidden file
*L — Symbolic link
*P — Pipe file
The tokens are returned in the standard ABL format that can be read by the IMPORT or SET statements.
What is the maximum length allowed for filenames? And is the max different for different operating system? I'm asking because I have trouble creating or deleting files, and I suspect the error was because of long file names.
1. Creating:
I wrote a program that will read a xml source and save a copy of the file. The xml contains hundreds of <Document>, and each have childnode <Name> and <Format>, the saved file is named based on what I read in the xml. For example, if I have the code below, I will save a file called test.txt
<Document>
<Name>test</Name>
<Format>.txt</Format>
</Document>
I declared a counter in my code, and I found out not all files are successfully saved. After going through the large xml file, I found out the program fail to save the files whose <Name> are like a whole paragraph long. I modify my code to save as a different name if <Name> is longer than 15 characters, and it went through no problem. So I think the issue was that the filename is too long.
2. Deleting
I found a random file on my computer, and I was not able to delete it. The error says that the file name was too long, even if I rename the file to 1 character. The file doesn't take up much space, but it was just annoying being there and not doing anything.
So my overall question is: What is the maximum and minimum length for filenames? Does it differ based on the operating system? And how can I delete the file I mentioned in 2?
It depends on the filesystem. Have a look here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
255 characters is a common maximum length these days.
I have about 500 HDF5 files each of about 1.5 GB.
Each of the files has the same exact structure, which is 7 compound (int,double,double) datasets and variable number of samples.
Now I want to concatenate all this files by concatenating each of the datasets so that at the end I have a single 750 GB file with my 7 datasets.
Currently I am running a h5py script which:
creates a HDF5 file with the right datasets of unlimited max
open in sequence all the files
check what is the number of samples (as it is variable)
resize the global file
append the data
this obviously takes many hours,
would you have a suggestion about improving this?
I am working on a cluster, so I could use HDF5 in parallel, but I am not good enough in C programming to implement something myself, I would need a tool already written.
I found that most of the time was spent in resizing the file, as I was resizing at each step, so I am now first going trough all my files and get their length (it is variable).
Then I create the global h5file setting the total length to the sum of all the files.
Only after this phase I fill the h5file with the data from all the small files.
now it takes about 10 seconds for each file, so it should take less than 2 hours, while before it was taking much more.
I get that answering this earns me a necro badge - but things have improved for me in this area recently.
In Julia this takes a few seconds.
Create a txt file that lists all the hdf5 file paths (you can use bash to do this in one go if there are lots)
In a loop read each line of txt file and use label$i = h5read(original_filepath$i, "/label")
concat all the labels label = [label label$i]
Then just write: h5write(data_file_path, "/label", label)
Same can be done if you have groups or more complicated hdf5 files.
Ashley's answer worked well for me. Here is an implementation of her suggestion in Julia:
Make text file listing the files to concatenate in bash:
ls -rt $somedirectory/$somerootfilename-*.hdf5 >> listofHDF5files.txt
Write a julia script to concatenate multiple files into one file:
# concatenate_HDF5.jl
using HDF5
inputfilepath=ARGS[1]
outputfilepath=ARGS[2]
f = open(inputfilepath)
firstit=true
data=[]
for line in eachline(f)
r = strip(line, ['\n'])
print(r,"\n")
datai = h5read(r, "/data")
if (firstit)
data=datai
firstit=false
else
data=cat(4,data, datai) #In this case concatenating on 4th dimension
end
end
h5write(outputfilepath, "/data", data)
Then execute the script file above using:
julia concatenate_HDF5.jl listofHDF5files.txt final_concatenated_HDF5.hdf5
I have a .bin file that contains records in CSV format what I want to do is assign each record in the bin file a sequence number. So the first record in the bin file would be assigned 0 and so on. These will be placed into a bianry index file such as (username, seq #).
If I have the bin file already created with records in it, how do I go through the bin file and index each record? Thanks for any help!
You would read lines from file A, and write lines to file B. Keep a count of each line you read and use that to generate the sequence number column when writing to file B.
To be honest, I would probably use Excel or a spreadsheet app if the file is already in CSV format