Getting weird crashes in mixed fortran/C program - c

I am trying to replace some graphics code in a set of fortran programs (not my own code). I got one of the simpler ones ('psvdraw') to work just fine, replacing the fortran postscript-generating code with C calls that call the Cairo library (graphic_output.c). I have been able to successfully work out the cross-language calls without too much trouble.
However, when trying to get the second, larger program ('pssect') to work, calling the same C code, I get Segmentation Faults, or in some cases, the program flow goes back to a fortran routine 'error' (I do not call this, or any fortran routine in my C code).
In trying to diagnose this problem, I linked in a bunch of the fortran code from pssect into psvdraw ('biglib.f'), and got the same errors. Note that none of this added code is actually called! Also the errors occur right at the first call from fortan into the c code. So: psvdraw with biglib.f linked in fails, but psvdraw without it succeeds.
Here are relevant bits of the makefile:
Makefile
COMP77 = gfortran
FFLAGS = -C -v -pedantic -Wunused -fno-underscoring
CC = gcc-4
CFLAGS = -v -pedantic -Wunused
CAIRO_INCLUDE = /sw/include/cairo
CAIRO_LIB = /sw/lib
# PSVDRAW Make setup that works:
psvdraw: psvdraw.o graphic_output.o tlib.o pscom.o
$(COMP77) $(FFLAGS) $#.o graphic_output.o tlib.o pscom.o -L$(CAIRO_LIB) -lcairo -o $#
# PSVDRAW Make setup with errors:
#psvdraw: psvdraw.o graphic_output.o tlib.o pscom.o biglib.o
# $(COMP77) $(FFLAGS) $#.o graphic_output.o pscom.o tlib.o biglib.o -L$(CAIRO_LIB) -lcairo -o $#
pssect: pssect.o graphic_output.o pscom.o tlib.o biglib.o
$(COMP77) $(FFLAGS) $#.o graphic_output.o pscom.o tlib.o biglib.o -L$(CAIRO_LIB) -lcairo -o $#
pssect.o: pssect.f
$(COMP77) $(FFLAGS) -c pssect.f
psvdraw.o: psvdraw.f
$(COMP77) $(FFLAGS) -c psvdraw.f
pscom.o: pscom.f
$(COMP77) $(FFLAGS) -c pscom.f
tlib.o: tlib.f
$(COMP77) $(FFLAGS) -c tlib.f
biglib.o: biglib.f
$(COMP77) $(FFLAGS) -c biglib.f
graphic_output.o: graphic_output.c
$(CC) $(CFLAGS) $(INCL) -c -I$(CAIRO_INCLUDE) graphic_output.c
.c.o:
$(CC) $(CFLAGS) $(INCL) -c $<
.f.o:
$(FC) $(FFLAGS) $(INCL) -c $<
Here is the offending fortran code: Note that the problem occurs right at the beginning of the program:
Beginning of pssect.f:
PROGRAM PSSECT
implicit none
include 'perplex_parameters.h'
integer jop0, ier99
logical vertex, output, first
character*100 fname, yes*1
integer iop0
logical debug
common / basic /iop0, debug
integer isec,icopt,ifull,imsg,io3p
common/ cst103 /isec,icopt,ifull,imsg,io3p
c----------------------------------------------------------------------
c Look for the "debug_yes" file to turn on debugging messages
PRINT *,'Pre-PSOPEN1'
call psopen ('plot2')
PRINT *,'Post-PSOPEN1'
And here is part of the c code that gets called and produces a fault:
Part of graphic_output.c:
char dmh_debug = 0;
#define DEBUGPRINT(x) if (dmh_debug) {printf x;};
void psopen(char *fname, int fnamelen) {
printf("Debug opened\n");
char *outFileName;
char outputType[255];
char pageWidthString[255];
char pageHeightString[255];
/* Set debug status based upon presence of file named 'debug_yes' in directory */
FILE *debugFile = fopen("debug_yes", "r");
if (debugFile == NULL) {
dmh_debug = 0;
} else {
dmh_debug = 1;
}
fclose(debugFile);
dmh_debug = 1;
DEBUGPRINT(("Debug closed\n"));
fname[fnamelen]='\0';
fname = trim(fname);
outFileName = malloc((strlen(fname) + 50) * sizeof(char));
strcpy(outFileName, fname);
DEBUGPRINT(("Found file name:%s of length: %lu\n", fname, strlen(fname)));
[...]
Results of running the program
pnr-rethington:source dave$ ./pssect
Pre-PSOPEN1
Debug opened
Segmentation fault

If linking in unused code triggers the problem, that would tend to indicate that somewhere (either in the Fortran code or the C code) you're overwriting memory that you shouldn't. Try running the compiled program under Valgrind - it should help pinpoint where this is happening.

I'd still suspect that you have a problem with ones of the calls between Fortran and C, resulting in an inconsistency. How are you making the calls? I think that the most reliable way is to use the ISO C Binding to specify to Fortran how the C routines should be called.
Or, you could consider a graphics package that has a Fortran interface or binding. Then you wouldn't have to work on an interface, since Fortran calls or a Fortran interface would already exist. I've been using DISLIN. Another possibility is PLplot.
With the current approach, I suggest examining the arguments at the entry to the C routines, to make sure that they are correct.

You are calling psopen from Fortran with a single argument, and the C routine expects two arguments. Maybe your Fortran compiler adds the length of the character string as a trailing argument after each string. Or maybe you got lucky on your first attempt and it just happened to work with a "random" value that the C routine found on the stack. In the second case, in another situation, a peculiar crash is likely. At best, this is a non-portable way to interface Fortran and C. You could try adding an integer length to your Fortran call and see what happens. Or print or use the debugger to see the value of the second argument at entry to the C-routine.
The ISO C Binding works much better. It is supported by numerous compilers (e.g., gfortran >= 4.3, ifort) and provides a defined and portable way of connecting Fortran and C. You specify an "interface" in your Fortran code so that the Fortran compiler generate the correct instructions for the C-call.
Still, it might be easier to use a plotting package that already provides a Fortran interface.

I notice from your make file that you are using gfortran. The combination of gfortran (>=4.3) & gcc supports the ISO C Binding. (Part of Fortran 2003.) If you include the interface example and call psopen with two arguments, it should work. The interface goes into the declaration of the Fortran program and is the Fortran description of the C routine psopen. Not guaranteed since I haven't tested it... Strings are a special case -- you match a scaler string in the Fortran program to a string array in the interface, because the C argument is an array of chars, or a pointer to chars.
interface To_psopen
subroutine psopen ( fname, fnamelen ) bind (C, name="psopen")
use iso_c_binding
implicit none
character (kind=c_char, len=1), dimension (100), intent (inout) :: fname
integer (c_int), value, intent (in) :: fnamelen
end subroutine psopen
end interface To_psopen

Running under the debugger (gdb) should tell you where the segfault happens. Compile all code with -O0 -g to get accurate information.

Related

Pointer is being being masked when calling a C function from Fortran

TL;DR
When I pass an array from Fortran to C, the array's address is incorrect in C. I've checked this by printing the address of the array in Fortran before the CALL, then stepping into the C function and printing the address of the argument.
The Fortran pointer: 0x9acd44c0
The C pointer: 0xffffffff9acd44c0
The upper dword of the C pointer has been set to 0xffffffff. I'm trying to understand why this is happening, and only happening on the HPC cluster and not on a development machine.
Context
I'm using a rather large scientific program written in Fortran/C++/CUDA. On some particular machine, I get a segfault when calling a C function from Fortran. I've found that a pointer is being passed to the C function with some bytes set incorrectly.
Code Snippets
Every Fortran file in the program includes a common header file which sets up some options and declares the common blocks.
IMPLICIT REAL*8 (A-H,O-Z)
COMMON/NBODY/ X(3,NMAX), BODY(NMAX)
COMMON/GPU/ GPUPHI(NMAX)
The Fortran call site looks like this:
CALL GPUPOT(NN,BODY(IFIRST),X(1,IFIRST),GPUPHI)
And the C function, which is compiled by nvcc, is declared like so:
extern "C" void gpupot_(int *n,
double m[],
double x[][3],
double pot[]);
GDB Output
I found from debugging that the value of the pointer to pot is incorrect; so any attempt to access that array will segfault.
When I ran the program with gdb, I put a break point just before the call to gpupot and printed the value of the GPUPHI variable:
(gdb) p &GPUPHI
$1 = (PTR TO -> ( real(kind=8) (1050000))) 0x9acd44c0 <gpu_>
I then let the debugger step into the gpupot_ C function, and inspected the value of the pot argument:
(gdb) p pot
$2 = (double *) 0xffffffff9acd44c0
All of the other arguments have the correct pointer values.
Compiler options
The compiler options that are set for gfortran are:
-fPIC -O3 -ffast-math -Wall -fopenmp -mcmodel=medium -march=native -mavx -m64
And nvcc is using the following:
-ccbin=g++ -Xptxas -v -ftz=true -lineinfo -D_FORCE_INLINES \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_35,code=compute_35 -Xcompiler \
"-O3 -fPIC -Wall -fopenmp -std=c++11 -fPIE -m64 -mavx \
-march=native" -std=c++14 -lineinfo
For debugging, the -O3 is replaced with -g -O0 -fcheck=all -fstack-protector -fno-omit-frame-pointer, but the behaviour (crash) remains the same.
This is prefaced by my top comments [and yours].
It looks like you're getting an [unwanted] sign extension of the address.
gfortran is being built with -mcmodel=medium but C does not.
With that option, larger symbols/arrays will be linked above 2GB [which has the sign bit set]
So, add the option to both or leave it off both to fix the problem.

Receiving *** caught segfault ***, 'memory not mapped' and mangled results after passing values from R to C

I'm trying to implement a simple integration of R with C. Initially it's simple: I want to pass values from R to a C function built into a .o shared library via .C or .Call function. The C function should simply print the values passed in via printf.
Here's my .Call method:
.Call("test", as.integer(5), as.character("A"), as.character("string_test"))
And my C code:
#include <stdio.h>
void test(int integer, char character, char **str) {
printf("Integer: %i\nChar: %c\nString: %s\n", integer, character, *str);
}
But when I call the C function from R via console (RStudio crashes) with gdb enabled, I receive:
Integer: 1466480376
Char: �
Float: -100407552.000000
String:
***caught segfault ***
address 0x20000090, cause 'memory not mapped'
Traceback:
1: .Call("test", as.integer(5), as.character("A"), as.character("string_test"))
As if it were not enough, as we can see the values passed in are printed very strangely.
Details of what I did, step by step:
I built the .o shared library with gcc:
gcc -shared -o func_teste.o -fPIC func_teste.c
And prepared it for dynamic loading in R environment:
$ R CMD SHLIB func_teste.o
gcc -m64 -I/usr/include/R -DNDEBUG -I/usr/local/include -fpic -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -c func_teste.c -o func_teste.o
gcc -m64 -shared -L/usr/lib64/R/lib -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o func_teste.so func_teste.o -L/usr/lib64/R/lib -lR
And finally, inside R console, i ran:
>dyn.load('func_teste.o')
>.Call("test", as.integer(5), as.character("A"), as.character("string_test"))
Does anyone have idea why this is happening?
R offers two main functions for interfacing from C code (and hence C++ code, or any other language able to use a C interface):
- .C() is the older interface using int*, double*, ... and alike
- .Call() is the newer, more powerful interface using SEXP objects
Now, .Call() looks more complicated but it is so much more powerful as well as safer. There is near universal consensus that .C() should no longer be used (see various discussions on the r-devel list and other places).
The main downside with .Call() is that you need to learn how to pack and unpack your values. Or ... you cheat and let Rcpp do it for you. So with that, here is one-line solution of the OP's example:
> library(Rcpp)
> cppFunction("void mytest(int i, char c, std::string str) { printf(\"Integer: %i Char: %c String: %s\\n\", i, c, str.c_str()); }")
> mytest(42L, 'Q', "a boat")
Integer: 42 Char: Q String: a boat
>
I made the char* a string. Note that cppFunction() requires escaping of strings, you may want to look into sourceCpp() and packages for real work. The Rcpp documentation has details.
Don't as.character on "string_test".
Read more here: http://mazamascience.com/WorkingWithData/?p=1067

How to write a C program that compiles other C programs using GCC?

I want my program to do the same thing as the terminal commands below:
gcc program1.c -o p1 funcs.c
gcc program2.c -o p1 funcs.c
This is what I've been experimenting with: Making a C program to compile another
I got as far so calling my program (./"programName") in the terminal that it replaced the need for me too type gcc but needing me too type in the rest.
You can use the Linux and POSIX APIs so read first Advanced Linux Programming and intro(2)
You could just run a compilation command with system(3), you can use snprintf(3) to build the command string (but beware of code injection); for example
void compile_program_by_number (int i) {
assert (i>0);
char cmdbuf[64];
memset(cmdbuf, 0, sizeof(cmdbuf));
if (snprintf(cmdbuf, sizeof(cmdbuf),
"gcc -Wall -g -O program%d.c fun.c -o p%d",
i, i)
>= sizeof(cmdbuf)) {
fprintf(stderr, "too wide command for #%d\n", i);
exit(EXIT_FAILURE);
};
fflush(NULL); // always useful before system(3)
int nok = system(cmdbuf);
if (nok) {
fprintf(stderr, "compilation %s failed with %d\n", cmdbuf, nok);
exit(EXIT_FAILURE);
}
}
BTW, your program could generate some C code, then fork + execve + waitpid the gcc compiler (or make), then perhaps even dlopen(3) the result of the compilation (you'll need gcc -fPIC -shared -O foo.c -o foo.so to compile a dlopenable shared object...). MELT is doing exactly that. (and so does my manydl.c which shows that you can do a big lot of dlopen-s ...)
You can use exec family function or you can directly execute shell command by using system() method
There is build in functionality in Makefile.
All you would have to call is make.
How to: http://www.cs.colby.edu/maxwell/courses/tutorials/maketutor/
Great stackoverflow question: How do I make a simple makefile for gcc on Linux?

Compiling multiple files in C

I recently asked this question about compiling multiple files in C so that a file main.c can reference a file modules.c. The answer ended up being to make the modules file into a header file and having main import it.
I have now been told that this is an incorrect way to do it, as C supports modular compilation. My Makefile is below, and this is supposedly supposed to be correct, but I receive errors for each function call in main.c -- warning: implicit declaration of function X.
What do I need to do to compile this correctly, with two .c files rather than a .c and .h file? The main.c file has a main() function that needs to be able to call the functions in modules.c.
Makefile:
#################################################################
# Variables
# -- allows C-source and assembly-source files mix. Again, the
# -- indented lines start with a TAB(^I) and not spaces..
#################################################################
CFLAGS = -g -Wall -Werror
LDFLAGS =
CC = gcc
LD = gcc
TARG = driver
OBJS = modules.o main.o
#################################################################
# Rules for make
#################################################################
$(TARG): $(OBJS)
$(LD) $(LDFLAGS) $(OBJS) -o $(TARG)
%.o: %.c %.s
$(CC) $(CFLAGS) -c $<
clean:
rm -f *.o *˜ $(TARG)
print:
pr -l60 Makefile modules.c main.c | lpr
#################################################################
# Dependencies -- none in this program
#################################################################
You've already gotten feedback about using GCC and Makefiles, and it's been noted that the typical way to accomplish your task would be two .c files and one .h file. But it's not required to have a .h file if you use function declarations (which is arguably simpler, just less maintainable and useful), as demonstrated by the following below example.
main.c:
void moduleFunc1(int); // extern keyword required for vars, not for functions
int main()
{
moduleFunc1(100);
return 0;
}
module.c:
#include <stdio.h>
void moduleFunc1(int value)
{
printf("%d\n", value);
}
To compile:
gcc main.c module.c
Edit: After having looked at the assignment you linked, my best guess is actually still that function declarations are what you are looking for. To quote from the assignment, under "Others", #7:
A function should be declared in the module/function where
it is called and not in global scope. Say A calls B and C does
not call it then B should be declared in A only.
In my example, the function declaration is in the module where it's called and seems to meet the A-B-C example. (The confusing part is the global scope comment, but I wouldn't say that the function declaration's scope is global. Observe that if you move the declaration below main(), for example, it messes things up. I haven't found something strictly authoritative for this point, though.)
Having read the assignment, could your instructor possibly mean the following?
main.c:
#include <stdio.h>
int main() {
int plus(int a, int b); /* declaration */
printf("%d ", plus(4, 5));
exit(0);
}
module.c:
int plus(int a, int b) {
return a + b;
}
gcc -Wall -Wextra main.c module.c
The thing is though, that plus() is available in the global namespace. So I am a bit lost.
Just an aside:
3. int next = 234;
printf("%6d ", next);
will print value of next, right justified in 6 columns
6. Use separate statements for declaration and initialization
of a variable as:
int xval;
xval = 100;
Do as I say, not as I do!
You can do this a few ways, but regardless of which you choose, if main.c calls functions from module.c, then main.c must #include a header which declares prototypes for those functions.
The first and simplest way is to just do this:
gcc -Wall -g main.c module.c -o myprogram
The second and more ornate way is to build module.c first as an object file. The primary purpose of this method is to save time when developing/debugging/compiling large programs with multiple parts -- rather than having to recompile the whole thing, you can just recompile the parts the have changed. It also allows you to easily mix and match parts. This is easiest to do with a makefile:
myprogram: main.c module.o
CC $(CFLAGS) main.c module.o -o myprogram
module.o:
CC $(CFLAGS) -c module.c
Notice the "myprogram" target from the makefile works with (prereq) module.o whereas the plain gcc method works with module.c.
If, as per your assignment, you can't use a header or global declarations, you can declare prototypes inside functions:
void somefunc () {
char *whatever (int x); // prototype
printf("%s\n", whatever(12));
}
Is fine, and presuming whatever() is defined somewhere, will work when you compile and run it.

How do I link object files in C? Fails with "Undefined symbols for architecture x86_64"

So I'm trying trying to use a function defined in another C (file1.c) file in my file (file2.c). I'm including the header of file1 (file1.h) in order to do this.
However, I keep getting the following error whenever I try to compile my file using gcc:
Undefined symbols for architecture x86_64:
"_init_filenames", referenced from:
_run_worker in cc8hoqCM.o
"_read_list", referenced from:
_run_worker in cc8hoqCM.o
ld: symbol(s) not found for architecture x86_64
I've been told I need to "link the object files together" in order to use the functions from file1 in file2, but I have no clue what that means :(
I assume you are using gcc, to simply link object files do:
$ gcc -o output file1.o file2.o
To get the object-files simply compile using
$ gcc -c file1.c
this yields file1.o and so on.
If you want to link your files to an executable do
$ gcc -o output file1.c file2.c
The existing answers already cover the "how", but I just wanted to elaborate on the "what" and "why" for others who might be wondering.
What a compiler (gcc) does: The term "compile" is a bit of an overloaded term because it is used at a high-level to mean "convert source code to a program", but more technically means to "convert source code to object code". A compiler like gcc actually performs two related, but arguably distinct functions to turn your source code into a program: compiling (as in the latter definition of turning source to object code) and linking (the process of combining the necessary object code files together into one complete executable).
The original error that you saw is technically a "linking error", and is thrown by "ld", the linker. Unlike (strict) compile-time errors, there is no reference to source code lines, as the linker is already in object space.
By default, when gcc is given source code as input, it attempts to compile each and then link them all together. As noted in the other responses, it's possible to use flags to instruct gcc to just compile first, then use the object files later to link in a separate step. This two-step process may seem unnecessary (and probably is for very small programs) but it is very important when managing a very large program, where compiling the entire project each time you make a small change would waste a considerable amount of time.
You could compile and link in one command:
gcc file1.c file2.c -o myprogram
And run with:
./myprogram
But to answer the question as asked, simply pass the object files to gcc:
gcc file1.o file2.o -o myprogram
Add foo1.c , foo2.c , foo3.c and makefile in one folder
the type make in bash
if you do not want to use the makefile, you can run the command
gcc -c foo1.c foo2.c foo3.c
then
gcc -o output foo1.o foo2.o foo3.o
foo1.c
#include <stdio.h>
#include <string.h>
void funk1();
void funk1() {
printf ("\nfunk1\n");
}
int main(void) {
char *arg2;
size_t nbytes = 100;
while ( 1 ) {
printf ("\nargv2 = %s\n" , arg2);
printf ("\n:> ");
getline (&arg2 , &nbytes , stdin);
if( strcmp (arg2 , "1\n") == 0 ) {
funk1 ();
} else if( strcmp (arg2 , "2\n") == 0 ) {
funk2 ();
} else if( strcmp (arg2 , "3\n") == 0 ) {
funk3 ();
} else if( strcmp (arg2 , "4\n") == 0 ) {
funk4 ();
} else {
funk5 ();
}
}
}
foo2.c
#include <stdio.h>
void funk2(){
printf("\nfunk2\n");
}
void funk3(){
printf("\nfunk3\n");
}
foo3.c
#include <stdio.h>
void funk4(){
printf("\nfunk4\n");
}
void funk5(){
printf("\nfunk5\n");
}
makefile
outputTest: foo1.o foo2.o foo3.o
gcc -o output foo1.o foo2.o foo3.o
make removeO
outputTest.o: foo1.c foo2.c foo3.c
gcc -c foo1.c foo2.c foo3.c
clean:
rm -f *.o output
removeO:
rm -f *.o
Since there's no mention of how to compile a .c file together with a bunch of .o files, and this comment asks for it:
where's the main.c in this answer? :/ if file1.c is the main, how do
you link it with other already compiled .o files? – Tom Brito Oct 12
'14 at 19:45
$ gcc main.c lib_obj1.o lib_obj2.o lib_objN.o -o x0rbin
Here, main.c is the C file with the main() function and the object files (*.o) are precompiled. GCC knows how to handle these together, and invokes the linker accordingly and results in a final executable, which in our case is x0rbin.
You will be able to use functions not defined in the main.c but using an extern reference to functions defined in the object files (*.o).
You can also link with .obj or other extensions if the object files have the correct format (such as COFF).

Resources