Big array problem - arrays

I have a code that worked fine until now with 3 million atom-sized static arrays.
For practical reasons, I have to go now to 10 million atom sized arrays. At first, my compiler did not allow me to do this, but I managed to find a way around with the following flags ifort -mcmodel medium -shared-intel -traceback kubo.f. It runs, but something very strange is happening. My matrix contains 11 609 198 elements.
I check the value of my coordinates as follows (the value of 4 669 671 is the first time it goes wrong):
print*, x(4669671),y(4669671),zcoord(4669671)
followed by several lines where the value of x, y and zcoord is not changed or anything. Then, I enter a loop on these 3 vectors where the value of x, y and zcoord will be used but not changed. I print the 3 values again, and suddenly, the 3 values are changed?!
Is there something I'm missing for large arrays?
EDIT : Here the complete code (as I don't know what is a race condition, I don't know If I'm allowed to delete some parts to make it more readable) :
open(1,FILE='fort.10')
read(1,*)NAT1
write(*,*)'Lecture de Nat1=',NAT1
read(1,*)
do i=1,nsites
read(1,*)parcon(i),x(i),y(i),zcoord(i)
enddo
print*, x(4663659),y(4663659),zcoord(4663659)
print*, x(4663663),y(4663663),zcoord(4663663)
!HERE
print*, x(4669671),y(4669671),zcoord(4669671)
print*, x(4673254),y(4673254),zcoord(4673254)
iflag=0
iflagg=0
impurityCounter=0
C4Counter=0
do i=1,nsites
nvo=0
if(i.le.(nsites-93998)) then
jj=i-10000
jjj=i+10000
do j=jj,jjj,1
if((j.gt.0).and.(j.le.(nsites-93998))) then
dist=dsqrt((x(j)-x(i))**2+(y(j)-y(i))**2
. +(zcoord(j)-zcoord(i))**2)
if((dist.lt.(1.11*aCC))
. .and.(j.ne.i).and.(dist.gt.0.1)) then
nvo=nvo+1
v(i,nvo)=j
if(i.eq.4663660) then
!THERE
print*, dist,j,x(j),y(j),zcoord(j)
endif
endif
endif
enddo
jjjj=nsites-93998+1
do j=jjjj,nsites,1
dist=dsqrt((x(j)-x(i))**2+(y(j)-y(i))**2
. +(zcoord(j)-zcoord(i))**2)
if((dist.lt.(1.11*aCC)).and.(j.ne.i).and.(dist.gt.0.1)) then
nvo=nvo+1
v(i,nvo)=j
endif
enddo
else
do j=1,nsites
dist=dsqrt((x(j)-x(i))**2+(y(j)-y(i))**2
. +(zcoord(j)-zcoord(i))**2)
if((dist.lt.(1.11*aCC)).and.(j.ne.i).and.(dist.gt.0.1)) then
nvo=nvo+1
v(i,nvo)=j
endif
enddo
endif
if ((nvo.eq.2).AND.(parcon(i).eq.'C')) then
iflag=iflag+1
vpb(iflag)=i
endif
if((nvo.eq.1).AND.(parcon(i).eq.'C')) then
iflagg=iflagg+1
vpbb(iflagg)=i
endif
!count the number of impurities
if((nvo.eq.2).AND.(parcon(i).eq.'O1')) then
impurityCounter=impurityCounter+1
impurityVector(impurityCounter)=i
endif
if((nvo.eq.2).AND.(parcon(i).eq.'O2')) then
impurityCounter=impurityCounter+1
impurityVector(impurityCounter)=i
endif
!If nvo equals 4, there is a BAD counting!
if(nvo.eq.4) then
print*, v(i,1)
print*, v(i,2)
print*, v(i,3)
print*, v(i,4)
print*, x(i), y(i)
endif
if(nvo.eq.5) then
C4Counter=C4Counter+1
C4Vector(C4Counter)=i
endif
enddo
I added !HERE and !THERE to show you where are the two places I print the x, y and zcoord of the element 4669671...

I'm not familiar with ifort, but I assume it has an option for checking array bounds. Turn it on.
Variables changing their value without you actually assigning something to them are often a sign that some other variable is referenced outside of its declared bounds.

Do you get any error message besides "killed"? Maybe with bounds checking the memory usage is once again too large. A common problem with large arrays is exceeding the available stack space ... see Stack overflow in Fortran 90. How are the variables declared? Are all the integers at least four bytes to hold these large values? If you are overwriting memory from exceeding an array bound in this block of code, it has to be from storage to an array in this block (v as suggested by #Jonathan Dursi, vpb, vpbb) ... obvious, but you can insert your own index checking code if the compiler option bounds checking still results in an executable that is too large. Place an IF statement before each array assignment in the code that is executed from before to after the problem occurs.

Related

Segmentation fault - invalid memory reference in Fortran

Lately I received the following error in my Fortran code
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0x2AD9B0F8FE08
#1 0x2AD9B0F8EF90
#2 0x2AD9B12D44AF
#3 0x401A3E in MAIN__ at tstreadin.f90:?
and my code as follows
Program www
implicit none
integer ::i,j,rows,cols,row
real(kind=8) ::x,y,z
real(kind=8),allocatable::mat(:,:),xrange(:),yrange(:)
real(kind=8),allocatable::pot_bar(:,:),acc_bar_x(:,:),acc_bar_y(:,:)
real(kind=8),allocatable::pot_sph(:,:),acc_sph_x(:,:),acc_sph_y(:,:)
rows=2250000
cols=8
row=1500
allocate(mat(cols,rows))
allocate(xrange(row),yrange(row),pot_bar(row,row))
allocate(acc_bar_x(row,row),acc_bar_y(row,row))
allocate(pot_sph(row,row),acc_sph_x(row,row),acc_sph_y(row,row))
open(24,file='pot.txt',status='old',form='Formatted', access='SEQUENTIAL')
do i=1,rows,1
read(24,*)mat(:,i)
enddo
close(24)
do i=1,rows,row
xrange(i)=mat(1,i)
enddo
do i=1,row,1
yrange(i)=mat(2,i)
enddo
do i=1,row,1
do j=1,row,1
pot_bar(j,i)=mat(3,j+(i-1)*1500)
acc_bar_x(j,i)=mat(4,j+(i-1)*1500)
acc_bar_y(j,i)=mat(5,j+(i-1)*1500)
pot_sph(j,i)=mat(6,j+(i-1)*1500)
acc_sph_x(j,i)=mat(7,j+(i-1)*1500)
acc_sph_x(j,i)=mat(8,j+(i-1)*1500)
enddo
enddo
print*,xrange
print*,yrange
end Program www
I want to input data from the ASCII profile to the arrays,so I write the code for test. This is my first time using Fortran, and I completely can't understand why the error appeared.
The array xrange only has 1500 elements allocated. However in the following
do i=1,rows,row
xrange(i)=mat(1,i)
enddo
you are attempting to access an element of xrange with an index far greater than 1500 (rows >> 1500). Hence the invalid memory reference.

How to split a file with multiple columns into multiple files with two columns each using Fortran 90

I have a physics simulation program that generates a file with six columns, one for the time, and other five for physical properties. I need to make a Fortran 90 program that read this file, and generates five files with two columns, one for the time and another for a physical property.
I have used F90 before, but I only know how to generate files and write on them, but I have no idea how to modify a file and generate more files with data from a file.
I don't expect to have the problem solved, I just want to know where to find information. Any advice will be useful.
I don't know a priori how many rows the program will generate
Here is an example which is not been tested...
It is a bit of a kindergarten approach, but it may be helpful. You could avoid the 6 array altogether, but it is often better to have the variables as separate arrays as it makes it vectorise better with contiguous memory layout. One could also read those into the 6 arrays, and avoid the 6xN array.
PROGRAM ABC
IMPLICIT NONE
REAL, DIMENSION(:,:) :: My_File_Data
REAL, DIMENSION(:) :: My_Data1
REAL, DIMENSION(:) :: My_Data2
REAL, DIMENSION(:) :: My_Data3
REAL, DIMENSION(:) :: My_Data4
REAL, DIMENSION(:) :: My_Data5
REAL, DIMENSION(:) :: My_Data6
INTEGER :: Index, LUN, I, IO_Status
OPEN(NEWUNIT=LUN, FILE='abc.dat')
Index = 0
FirstPass: DO WHILE(.TRUE.)
READ(UNIT=LUN,*, IO_Status)
IF(IO_Status /= 0) EXIT
Index = Index + 1
ENDDO FirstPass
REWIND(LUN)
ALLOCATE(My_File_Data(Index))
ALLOCATE(My_Data1(Index))
ALLOCATE(My_Data2(Index))
ALLOCATE(My_Data3(Index))
ALLOCATE(My_Data4(Index))
ALLOCATE(My_Data5(Index))
ALLOCATE(My_Data6(Index))
SecondPass: DO I = 1, Index
READ(UNIT=LUN,*) My_File_Data(:,I)
Index = Index + 1
ENDDO SecondPass
DO I = 1, Index
Data1(I) = My_File_Data(1,I)
ENDDO
! What follows is more elegant...
Data2(:) = My_File_Data(2,:) !Where the first (:) is redundant... It seems more readable, but there are some reasons not to use it... (LTR)
Data3 = My_File_Data(3,:)
Data4 = My_File_Data(4,:)
Data5 = My_File_Data(5,:)
Data6 = My_File_Data(6,:)
DEALLOCATE(My_File_Data)
!Etc
The first step is to read in the data. In the following instructions, we will first loop over the file and count the number of rows, nrows. This value will be used to allocate a data array to the necessary size. We then return to the beginning of the file and read in our data in a second loop.
Declare an integer variable to act as a file handle/reference.
Declare an allocatable array of reals (floats) to hold the data.
Loop over the file to count the number of lines in the file. Remove header lines from the count.
Allocate the data array to the proper size, (nrows,nvalues).
Return to the beginning of the file. Repeat the loop over each of the rows, reading all values from the row into your data array.
Close the file.
The next step is to create 5 new files, each containing the time and one of the 5 property measurements:
Loop over each of the 5 properties contained in data.
For each jth property, open a new file.
Loop over the data array, writing the time and jth property to a new line.
Close the file.
Here is working code you can use or modify to suit your needs:
program SO
implicit none
integer :: i, j, nrows, nvalues, funit, ios
real, allocatable, dimension(:,:) :: data
character(len=10), dimension(5) :: outfiles
!! begin
nvalues = 5
nrows = 0
open(newunit=funit, file='example.txt', status='old', iostat=ios)
if (ios /= 0) then
print *, 'File could not be opened.'
call exit
else
do
read(funit,*,iostat=ios)
if (ios == 0) then
nrows = nrows + 1
elseif (ios < 0) then !! End of file (EOF).
exit !! The 'exit' stmt breaks out of the loop.
else !! Error if > 0.
print *, 'Read error at line ', nrows + 1
call exit() !! The 'exit' intrinsic ends the program.
endif !! We we may pass an optional exit code.
enddo
endif
nrows = nrows - 1 !! 'nrows-1': Remove column headers from count.
if (allocated(data)) deallocate(data) !! This test follows standard "best practices".
allocate(data(nrows,nvalues+1))
rewind(funit)
read(funit, *) !! Skip column headers.
do i = 1,nrows
read(funit, *) data(i,:) !! Read values into array.
enddo
close(funit)
!! Output file names.
outfiles = ['prop1.txt', 'prop2.txt', 'prop3.txt', 'prop4.txt', 'prop5.txt']
do j = 1,nvalues
open(newunit=funit, file=outfiles(j), status='replace', iostat=ios)
if (ios /= 0) then
print *, 'Could not open output file: ',outfiles(j)
call exit()
endif
write(funit,"(a)") "time "//outfiles(j)(1:5)
do i = 1,nrows
write(funit,"(f0.0,t14,es14.6)") data(i,1), data(i,j+1)
enddo
close(funit)
enddo
end program SO
All the other answers want to read in everything at once. I think that's too much of a bother.
Firstly, I'd check if I even needed Fortran for that. The Linux command cut can be used very effectively here. For example, if your data is comma separated, you could simply use
for i in {1..5}; do
cut -d, -f1,$((i+1)) data.txt > data${i}.txt;
done
to do the whole thing.
If you need Fortran, here's how I'd go about it:
Open all files
In a permanent loop, read in the whole row at once.
If you encounter an error, it's probably EOF, so exit the loop
Write the data to the output files.
Here's some basic code:
program split
implicit none
integer :: t, d(5), u_in, u_out(5)
integer :: i
integer :: ios
open(newunit=u_in, file='data.txt', status="old", action="read")
open(newunit=u_out(1), file='temperature.txt', status='unknown', action='write')
open(newunit=u_out(2), file='pressure.txt', status='unknown', action='write')
open(newunit=u_out(3), file='pair_energy.txt', status='unknown', action='write')
open(newunit=u_out(4), file='ewald_energy.txt', status='unknown', action='write')
open(newunit=u_out(5), file='pppm_energy.txt', status='unknown', action='write')
read(u_in, *) ! omit the column names
write(u_out(1), *) "Time Temperature"
write(u_out(2), *) "Time Pressure"
write(u_out(3), *) "Time Pair Energy"
write(u_out(4), *) "Time Ewald Energy"
write(u_out(5), *) "Time PPPM Energy"
do
read(u_in, *, iostat=ios) t, d
if (ios /= 0) exit
do i = 1, 5
write(u_out(i), *) t, d(i)
end do
end do
close(u_in)
do i = 1, 5
close(u_out(i))
end do
end program split
Cheers

openmp fortran reduction and critical not working for array

I am currently trying to get a fortran FE (finite element) code to work with openmp. I have a loop over all elements, ie that I want to work in parallel. Here is a simplified part of the code that is not working
!$omp parallel do default(none) shared(nelm,A,res,enod) private(ie,Fe,B,edof)
do ie=1,nelm
call calcB(B,A(:,ie))
call calcFe(Fe,B)
write(*,*) Fe !writes Fe=40d0, this is correct
call getEdof(edof,enod(:,ie))
!$OMP CRITICAL
res(edof)=res(edof)+fe
!$OMP END CRITICAL
enddo
!$omp end parallel do
The purpose of the code is to calculate a force Fe and then adding it to res at edof. The force is calculated with calcFe, and the calculated force is correct, but the resulting res is incorrect after the loop.
If I replace calcFe with simply Fe=40d0 then add it to res the result is correct after the loop
!$omp parallel do default(none) shared(nelm,A,res,enod) private(ie,Fe,B,edof)
do ie=1,nelm
call calcB(B,A(:,ie))
Fe=40d0
call getEdof(edof,enod(:,ie))
!$OMP CRITICAL
res(edof)=res(edof)+fe
!$OMP END CRITICAL
enddo
!$omp end parallel do
What causes this error? In both cases Fe=40d0 is declared private but only one of them gives the correct result. Instead of using !$ CRITICAL I could use reduction but it gives the same error. In the program several large and sparse matrices are also used but the are passive/ not used during the loop. My supervisor has had problems with using openmp and sparse matrices before and suspects that they are using the same memory. If the error is not apparent what debugger is best to use? Im a novice to both fortran ,openmp and programing in general.
Im using ifort to compile and my OS is ubuntu.
EDIT: Added simplified code that you can run, although this code works
In the code there are two loops, on parallel and one serial, to they should give the same result, res and res2
program main
use omp_lib
implicit none
integer :: ie, nelm,enod(4,50*50),edof(12),i,j,k
double precision ::B(12,8),fe(12),A(12,12,2500),res(2601*3),res2(2601*3),finish,start
!creates enod
i=1
do j=1,50
ie=j
do k=1,50
nelm=k
enod(:,i)=(/ 51*(nelm-1)+1+ie-1, 51*(nelm-1)+1+ie, 51*(nelm)+1+ie-1, 51*(nelm)+1+ie /)
i=i+1
end do
end do
A=1d0
res2=0d0
nelm=2500
start=omp_get_wtime()
!$omp parallel do default(none) shared(nelm,A,enod) private(ie,fe,edof,B) reduction(+:res2)
do ie=1,nelm
call calcB(B,A(:,:,ie))
call calcFe(fe,B) !the calculated fe is always 2304
!can write fe=2304 to get correct result with real code
call getEdof(edof,enod(:,ie))
res2(edof)=res2(edof)+fe
end do
!$omp end parallel do
finish=omp_get_wtime()
write(*,*) 'time: ', finish-start
res=0d0
nelm=2500
start=omp_get_wtime()
do ie=1,nelm
call calcB(B,A(:,:,ie))
call calcFe(fe,B)
call getEdof(edof,enod(:,ie))
res(edof)=res(edof)+fe
end do
finish=omp_get_wtime()
write(*,*) 'time: ', finish-start
write(*,*) 'difference: ',sum(res2-res)
write(*,*) sum(res2)
stop
end program main
subroutine calcB(B,A)
double precision ::B(12,8),A(12,12),C(12)
integer ::gp
C=1d0
do gp=1,8
B(:,gp)=matmul(A,C)
end do
end subroutine calcB
subroutine calcFe(fe,B)
double precision ::fe(12),B(12,8),D(12,12)
integer ::gp
fe=0d0
D=2d0
do gp=1,8
fe=fe+matmul(D,B(:,gp))
end do
end subroutine calcFe
subroutine getEdof(edof,enod)
implicit none
integer,intent(in) :: enod(4)
integer,intent(out):: edof(12)
edof=0
edof(1:3) =(/ enod(1)*3-2, enod(1)*3-1, enod(1)*3 /)
edof(4:6) =(/ enod(2)*3-2, enod(2)*3-1, enod(2)*3 /)
edof(7:9) =(/ enod(3)*3-2, enod(3)*3-1, enod(3)*3 /)
edof(10:12)=(/ enod(4)*3-2, enod(4)*3-1, enod(4)*3 /)
end subroutine getedof
And the make file
FF = ifort -O3 -openmp
OBJ1 = main.f90
ls: $(FORT_OBJS)
$(FF) -o exec $(OBJ1)
Unfortunately this piece of code works, so i'm unable to replicate the error. res2 and res are calculated in serial and parallel. In my real program I have put all values to 1d0 in order to get a constant fe. The calulated fe is correct, if I add a write(*,*) fe after calcFe I see that the values are correct. I then add these values to res2 and compare them with the serial res. They are then different by a large margin, so there is no numerical roundoff error. If I simply declare fe=2304 in my main program I get the correct answer even though fe already is 2304 when write is used.
In the my real program all the subroutines are in different modules, do I need to take any special care because of this?
Also in one of the modules some global variables are used, they are read only but since they are not declared in the subroutine they are not automaticly made private? This should be no issue since I put all variables used to to calulate fe to a constant, the global variables are not used directly to calculate fe
Solved it, it started working when I added -openmp to the makefile for my modules. Apparently the modules needs to be compiled with -openmp and not just the main file.

MPI collective output 5 noncontiguous 3D arrays in special form

During the realization of the course work I have to write MPI program to solve PDE continuum mechanics. (FORTRAN)
In the sequence program file is written as follows:
do i=1,XX
do j=1,YY
do k=1,ZZ
write(ifile) R(i,j,k)
write(ifile) U(i,j,k)
write(ifile) V(i,j,k)
write(ifile) W(i,j,k)
write(ifile) P(i,j,k)
end do
end do
end do
In the parallel program, I write the same as follows:
/ parallelization takes place only along the axis X /
call MPI_TYPE_CREATE_SUBARRAY(4, [INT(5), INT(ZZ),INT(YY), INT(XX)], [5,ZZ,YY,PDB(iam).Xelements], [0, 0, 0, PDB(iam).Xoffset], MPI_ORDER_FORTRAN, MPI_FLOAT, slice, ierr)
call MPI_TYPE_COMMIT(slice, ierr)
call MPI_FILE_OPEN(MPI_COMM_WORLD, cFileName, IOR(MPI_MODE_CREATE, MPI_MODE_WRONLY), MPI_INFO_NULL, ifile, ierr)
do i = 1,PDB(iam).Xelements
do j = 1,YY
do k = 1,ZZ
dataTmp(1,k,j,i) = R(i,j,k)
dataTmp(2,k,j,i) = U(i,j,k)
dataTmp(3,k,j,i) = V(i,j,k)
dataTmp(4,k,j,i) = W(i,j,k)
dataTmp(5,k,j,i) = P(i,j,k)
end do
end do
end do
call MPI_FILE_SET_VIEW(ifile, offset, MPI_FLOAT, slice, 'native', MPI_INFO_NULL, ierr)
call MPI_FILE_WRITE_ALL(ifile, dataTmp, 5*PDB(iam).Xelements*YY*ZZ, MPI_FLOAT, wstatus, ierr)
call MPI_BARRIER(MPI_COMM_WORLD, ierr)
It works well. But I'm not sure about using an array dataTmp. What solution will be faster and more correct? What about using 4D array like the dataTmp in the whole program? Or, maybe, I should create 5 special mpi_types with different displacemet.
Using dataTmp is fine, if you have the memory space. your MPI_FILE_WRITE_ALL call will be the most expensive part of this code.
You've done the hard part, setting an MPI-IO file view. if you want to get rid of dataTmp, you could create an MPI datatype to describe the arrays (probably using MPI_Type_hindexed and MPI_Get_address)), then use MPI_BOTTOM as the memory buffer.
If I/O speed is an issue and you have the option, I'd suggest changing the file format - or alternately, how the data is laid out in memory - to be more closely lined up: in the serial code, writing data in this transposed and interleaved way is going to be very slow:
program testoutput
implicit none
integer, parameter :: XX=512, YY=512, ZZ=512
real, dimension(:,:,:), allocatable :: R, U, V, W, P
integer :: timer
integer :: ifile
real :: elapsed
integer :: i,j,k
allocate(R(XX,YY,ZZ), P(XX,YY,ZZ))
allocate(U(XX,YY,ZZ), V(XX,YY,ZZ), W(XX,YY,ZZ))
R = 1.; U = 2.; V = 3.; W = 4.; P = 5.
open(newunit=ifile, file='interleaved.dat', form='unformatted', status='new')
call tick(timer)
do i=1,XX
do j=1,YY
do k=1,ZZ
write(ifile) R(i,j,k)
write(ifile) U(i,j,k)
write(ifile) V(i,j,k)
write(ifile) W(i,j,k)
write(ifile) P(i,j,k)
end do
end do
end do
elapsed=tock(timer)
close(ifile)
print *,'Elapsed time for interleaved: ', elapsed
open(newunit=ifile, file='noninterleaved.dat', form='unformatted',status='new')
call tick(timer)
write(ifile) R
write(ifile) U
write(ifile) V
write(ifile) W
write(ifile) P
elapsed=tock(timer)
close(ifile)
print *,'Elapsed time for noninterleaved: ', elapsed
deallocate(R,U,V,W,P)
contains
subroutine tick(t)
integer, intent(OUT) :: t
call system_clock(t)
end subroutine tick
! returns time in seconds from now to time described by t
real function tock(t)
integer, intent(in) :: t
integer :: now, clock_rate
call system_clock(now,clock_rate)
tock = real(now - t)/real(clock_rate)
end function tock
end program testoutput
Running gives
$ gfortran -Wall io-serial.f90 -o io-serial
$ ./io-serial
Elapsed time for interleaved: 225.755005
Elapsed time for noninterleaved: 4.01700020
As Rob Latham, who knows more than a few things about this stuff, says, your transposition for the parallel version is fine - it does the interleaving and transposing explicitly in memory, where it's much faster, and then blasts it out to disk. It's about as fast as the IO is going to get.
You can definitely avoid the dataTmp array by writing one or five individual data types to do the transposition/interleaving for you on the way out to disk via the MPI_File_write_all routine. That will give you a bit more of a balance in between in terms of memory usage and performance. You won't be explicitly defining a big 3-D array, but the MPI-IO code will improve performance over looping over individual elements by doing a fair bit of buffering, meaning that a certain amount of memory is being set aside to do the writing efficiently. The good news is that the balance will be tunable by setting MPI-IO hints in the Info variable; the bad news is that the code is likely to be less clear than what you have now.

Automatically printing the structures and variables in C

I am working with 4-5 .c files (around 2000 to 5000 lines each) which include several
headers.Currently I do not have any debug prints which would help me debug the program
during its course of execution.
My question is :-
Is there a way (or some existing tool) to parse the .c files and add new set of print
statements for all the variables in the current scope in the .c file ? Just the same way as
VC++ allows us to see Locals and globals etc. I need them printed at each step. Also,
pointers should be dereferenced.
For ex. lets say at one point in the .c file, there are 10 global variables and 3 locals.
I need to generate the smart printfs to print these 13 variables at that point. Later in
the program if there are 20 variables, i should be able to print the 20 variables etc.
The included header files contain all the relevant declarations for each of these
variables (which can be structures/pointers/arrays or some combinations etc etc.)
I was trying to achieve this via perl script.
What I did is, I generated the preprocessed file (.i file) and i tried parsing it via perl
and then generate individual print functions specific to each variable, but after half a
days' effort i realized that its just too time consuming.
Is there a tool that already does that ? If not this, anything close to it should be good
enough (On which i can apply some perl processing etc)
My goal is that after the program execution,at each step during the program execution, I should be able to see the variables(valid at that scope) without having to invoke the debugger.
I am allowed to process the .c files and re-write them again etc. etc.
Hope my question is clear and thanks for your replies.
Assuming that your C program can be interpreted by Frama-C's value analysis, which is far from a given, you could use that to obtain a log of the values of all living variables at each point of the program or at points of interest.
Consider the following program:
int x = 1;
main(){
int l;
x=2;
Frama_C_dump_each();
l=3;
Frama_C_dump_each();
{
int blocklocal = l + 1;
Frama_C_dump_each();
x = blocklocal + 1;
Frama_C_dump_each();
}
Frama_C_dump_each();
return 0;
}
Running frama-c -val -slevel 1000000000 -no-results t.c on this program produces the log:
[value] Values of globals at initialization
x ∈ {1}
[value] DUMPING STATE of file t.c line 7
x ∈ {2}
=END OF DUMP==
[value] DUMPING STATE of file t.c line 9
x ∈ {2}
l ∈ {3}
=END OF DUMP==
[value] DUMPING STATE of file t.c line 12
x ∈ {2}
l ∈ {3}
blocklocal ∈ {4}
=END OF DUMP==
[value] DUMPING STATE of file t.c line 14
x ∈ {5}
l ∈ {3}
blocklocal ∈ {4}
=END OF DUMP==
[value] DUMPING STATE of file t.c line 16
x ∈ {5}
l ∈ {3}
=END OF DUMP==
The Frama_C_dump_each() statements were inserted by me manually, but you could also nudge the interpreter so that it dumps a state automatically at each statement.
For this approach to work, you need the entire source code of your program, including standard library functions (strlen(), memcpy(), …) and you must hard-code the values of the input at the beginning of the main() function. Otherwise, it will behave as the static analyzer that it really is instead of behaving as a C interpreter.
You could also use the GUI to observe values of variables in your program, but if it is not linear, statements that are visited several times either because of function calls or because of loops will show all the values that can be taken during execution.

Resources