undefined symbol: PyExc_ImportError when embedding Python in C - c

I'm developing a C shared library that makes a call to a python script.
When I run the application I get this error:
Traceback (most recent call last):
File "/home/ubuntu/galaxy-es/lib/galaxy/earthsystem/gridftp_security/gridftp_acl_plugin.py", line 2, in <module>
import galaxy.eggs
File "/home/ubuntu/galaxy-es/lib/galaxy/eggs/__init__.py", line 5, in <module>
import os, sys, shutil, glob, urllib, urllib2, ConfigParser, HTMLParser, zipimport, zipfile
File "/usr/lib/python2.7/zipfile.py", line 6, in <module>
import io
File "/usr/lib/python2.7/io.py", line 60, in <module>
import _io
ImportError: /usr/lib/python2.7/lib-dynload/_io.so: undefined symbol: PyExc_ImportError
If I try to import the module io from console works fine instead:
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import galaxy.eggs
>>>
During the compilation of library I've used this compiler option as suggest here :
Embedding python in C, undefined symbol: PyExc_ImportError
In addition I've added also the compiler options obtained from
python-config --includes|--libs|--cflags|--ldflags
Here you can find the log of makefile of library http://pastebin.com/348rhBjM
Thanks a lot, any help will be apreciated.

#user1515248 solution is a link-only solution which are discouraged. i am writing this answer to expand on the links he gave and provide a more fleshed out answer (that also backs up the link he gave).
The link, https://mail.python.org/pipermail/new-bugs-announce/2008-November/003322.html, says:
I have been given the following workaround: in mylib.c, before
PyInitialize() I can call dlopen("libpython2.5.so", RTLD_LAZY | RTLD_GLOBAL);
This works, but I believe that lib-dynload/*.so should depend on
libpython2.5.so.1 so this hack should not be necessary.
I am using Ubuntu 8.04 with Python version 2.5.2-2ubuntu4.1.
All I had to do was add a single line of code:
// new line of code
void*const libpython_handle = dlopen("libpython2.6.so", RTLD_LAZY | RTLD_GLOBAL);
PyInitialize();
p.s.
I am on CentOS-6.
p.p.s.
My PyInitialize() is wrapped in a class and so dlopen()/PyInitialize() is done in the constructor and dlclose()/PyFinalize() is done in the destructor.

I've found the solution. Maybe can be useful for someone else.
It's a bug of python as written here http://mail.python.org/pipermail/new-bugs-announce/2008-November/003322.html
I've used the solution posted here http://www.cilogon.org/gsi-c-authz

I use such workaround: explicit linking of plugins from lib-dynload directory (it's simply, then explicit dlopen in code). Example with datetime.so:
cmake:
SET ( CMAKE_SHARED_LINKER_FLAGS "/usr/lib/python2.7/lib-dynload/datetime.so" )
or just add /usr/lib/python2.7/lib-dynload/datetime.so as linker parameter to
gcc in command line:
g++ -shared -o libfoo.so foo.o -lbar -lzab /usr/lib/python2.7/lib-dynload/datetime.so

Related

`gcc.exe' failed in phase `C pre-processor' error message in /dist-newstyle/ folder

I have this project on a subject 'Languages and Compilers' Where we write parsers in Haskell. At the moment I'm working on this 2-Arrow project where we use the 'alex' and 'happy' libraries to create a parser.hs and lexer.hs file from a lexer.x and parser.y file. While trying to build my framework (cabal build in the project directory) I get the following error message:
C:\\ ... \2-Arrow\dist-newstyle\build\x86_64-windows\ghc-9.2.4\assignment-arrow-0.1.0.0\x\arrow\build\arrow\arrow-tmp\Lexer.hs:1:1: error:
`gcc.exe' failed in phase `C pre-processor'. (Exit code: 1)
|
1| {-# OPTIONS_GHC -fno-warn-unused-binds -fno-warn-missing-signatures #-}
| ^
C:\\ ... \2-Arrow\dist-newstyle\build\x86_64-windows\ghc-9.2.4\assignment-arrow-0.1.0.0\x\arrow\build\arrow\arrow-tmp\Parser.hs:1:1: error:
`gcc.exe' failed in phase `C pre-processor'. (Exit code: 1)
|
1 | {-# OPTIONS_GHC -w #-}
| ^
If they were error messages originating from the code I wrote I would at least have a direction to look for solutions, but these errors come from the /dist-newstyle/ folder. Which, in my understanding, is a folder for intermediate files used in building the program. So I'm kind of at a dead end at the moment, since reinstalling 'happy' and 'alex', and updating cabal to latest version all did not change anything. Does anyone have any advice for resolving this error? Any help is greatly appreciated!!
(project framework from uni: https://www.cs.uu.nl/docs/vakken/b3tc/downloads-2018/2-Arrow.zip (depends on 'happy' and 'alex' packages). At first I thought the problem could lay in the framework, but I checked with the professor, he stated that others had no issues, so I don't think that's the problem)
I wasn't able to duplicate the problem with a fresh GHCup installation on Windows 10, but the error message suggests there might be an issue with an installed version of the GNU C Compiler (GCC) which the Glasgow Haskell Compiler (GHC) invokes to apply the C pre-processor to certain Haskell source files, including the Lexer.hs source file that's being generated by Alex from the Lexer.x file, and the Parser.hs source file that's being generated by Happy from the Parser.y file.
Try creating a simple test.hs program that invokes the C pre-processor:
{-# LANGUAGE CPP #-}
#define HELLO "Hello, world!"
main = putStrLn HELLO
and try compiling with ghc test.hs. Assuming that fails with the same error as above, try ghc -v test.hs, and you should find lines in the output starting with:
*** C pre-processor
"C:\...\gcc.exe" "-E" "-undef" ...blah blah blah...
An excerpt from my successful compilation follows. Maybe you can spot what's going wrong with your installation. Note that mine invokes the GHCup-installed version of GCC and directs the output to a temporary file area where it's then compiled by GHC. If the wrong GCC is invoked or the temporary file area isn't writable for some reason, that might lead to the problem you're seeing.
*** C pre-processor:
"C:\ghcup\ghc\9.2.5\lib\../mingw/bin/gcc.exe" "-E" "-undef" "-traditional" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\base-4.16.4.0\include" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\ghc-bignum-1.2\include" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\rts-1.0.2\include" "-include" "C:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\rts-1.0.2\include\ghcversion.h" "-Dmingw32_BUILD_OS" "-Dx86_64_BUILD_ARCH" "-Dmingw32_HOST_OS" "-Dx86_64_HOST_ARCH" "-D__GLASGOW_HASKELL_TH__" "-D__SSE__" "-D__SSE2__" "-D__IO_MANAGER_WINIO__=1" "-D__IO_MANAGER_MIO__=1" "-includeC:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_2.h" "-x" "assembler-with-cpp" "test.hs" "-o" "C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_1.hscpp"
!!! systool:cpp: finished in 31.25 milliseconds, allocated 0.311 megabytes
!!! Chasing dependencies: finished in 46.88 milliseconds, allocated 2.719 megabytes
Stable obj: {}
Stable BCO: {}
Ready for upsweep
[NONREC
ModSummary {
ms_hs_date = 2022-12-18 20:54:57.2679991 UTC
ms_mod = Main,
ms_textual_imps = [(Nothing, Prelude)]
ms_srcimps = []
} []]
*** Deleting temp files:
Deleting: C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_2.h
compile: input file C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_1.hscpp
*** Checking old interface for Main (use -ddump-hi-diffs for more details):
[1 of 1] Compiling Main ( test.hs, test.o )
I went to my seminar and asked the TA for help. After some time he proposed that maybe the folder location was the problem. Lo and behold, after moving the folder from my OneDrive folder (which is also saved locally) to my C:\\ directory, the project finally compiled! I have no idea why this would be the case, so I have posted another question here Folder location makes Haskell project incompileable.

Go compile returns duplicate symbols for architecture x86_64 error when I import 2 different packages which use C package via Cgo

Here is my code:
package main
import (
kusb "github.com/karalabe/usb"
tusb "github.com/trezor/trezord-go/usb"
)
func main() {
kusb.Enumerate(0, 0)
tusb.InitHIDAPI(nil)
}
When I compile (I'm using go mod to manage the packages), it returns this error:
duplicate symbol _libusb_dev_mem_alloc in:
/var/folders/fm/1rln65d94mn45s0h5l78tdyh0000gp/T/go-link-624554542/000002.o
/var/folders/fm/1rln65d94mn45s0h5l78tdyh0000gp/T/go-link-624554542/000020.o
ld: 136 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Why?
Some investigation i had:
The both packages use the same hidapi and libusb C packages in order to interact with usb devices.
Those C packages are identical, hence it defines the same functions so i think it is directly related to the error.
in trezord-go/usb, they include .C file, not the header file.
It is very counterintuitive to me because in the perspective of package users, I shouldn't need to worry about how a C package is used in the internal of the package, only the exposed types, functions and its behaviors.
Can anyone really explain what is going on here and how can I import both of them? They do different functions, eventhough they use the same C package.
From here: https://www.repustate.com/blog/go-duplicate-symbols-for-architecture-x86_64/
"What does this mean? Well, it means we're trying to link the same symbol name (in our case, a method) from two (or more) different source files. The fix was easy: rename one of the methods by updating the header file, the source file (.c or .cpp file) and lastly, updating your references to the symbol in your Go code, if it is directly referenced there."
Will it help ?
I was running into the same issue for hours and finally found the fix on a google groups channel
A package you import could be using cgo, you don't have to be using it directly
...
You can try CGO_ENABLED=0 go build and if it works then it is cgo related.
This was the charm that i was looking for! Hope this works for you too.

How to import a haskell module that uses FFI without refering to the c object?

I'm trying to write a haskell module that wraps a bunch of c functions.
I want to be able to import this module like any other haskell module without referring to the c object files.
I can't find any examples about how to do this.
This is what I've tried. I have a c file "dumbCfunctions.c":
double addThree(double x) {
return x+3;
}
and a haskell file with a module defined in it "Callfunctions.hs"
module Callfunctions (
addThree
) where
import Foreign.C
foreign import ccall "addThree" addThree :: Double -> Double
main = print $ addThree 4
I can make an executable doing:
ghc --make -o cf_ex Callfunctions.hs dumbCfunctions.o
Which correctly gives me 7.
I can also import it into ghic by calling ghci with
shane> ghci dumbCfunctions.o
Prelude> :l Callfunctions.hs
[1 of 1] Compiling Callfunctions ( Callfunctions.hs, interpreted )
Ok, modules loaded: Callfunctions.
*Callfunctions> addThree 3
6.0
But I want to be able to treat it like any other module with out referring to "dumbCfunctions.o":
shane> ghci
Prelude> :l Callfunctions.hs
[1 of 1] Compiling Callfunctions ( Callfunctions.hs, interpreted )
Ok, modules loaded: Callfunctions.
*Callfunctions> addThree 3
But now I get the error
ByteCodeLink: can't find label
During interactive linking, GHCi couldn't find the following symbol:
addThree
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session. Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
glasgow-haskell-bugs#haskell.org
This makes sense because I haven't refereed to the object anywhere. So I must be able to do something better by first compiling the module, but I couldn't find out how to do this. I must be looking in the wrong places.
You can create a library through Cabal, and cabal install it.
This would link the C code inside your Haskell library. Later on, when you load the module, you will not need to manually load the C parts.

cannot find symbol "Embeddedrcall_Init"

I am trying to create a dll file using swig for an embeddedR C Program in windows environment. I am using the below commands:
C:\swigwin-3.0.12\Examples\r\Z>swig -c++ -tcl embeddedRCall.i
C:\swigwin-3.0.12\Examples\r\Z>gcc -c embeddedRCall.c -I/swigwin-3.0.12/Examples/r/Z
C:\swigwin-3.0.12\Examples\r\Z>gcc -c embeddedRCall_wrap.c -I/Tcl/include/tcl8.6 -I/swigwin-3.0.12/Examples/r/Z
C:\swigwin-3.0.12\Examples\r\Z>gcc -shared embeddedRCall.o embeddedRCall_wrap.o -o embeddedRCall.dll -L/Tcl/lib -L/R/R-3.3.2/bin/i386 -lR -lRblas -lRiconv -lRlapack -ltcl86
% load embeddedRCall
cannot find symbol "Embeddedrcall_Init"
I was able to load other example.dll files with tclsh
However I was unable to figure out the reason-- I am already using tcl 32 bit
My module file name is and module name is embeddedRcall
Am I missing something???
I am relatively new to TCL can someone please help me.
You should have an exported (extern "C") function symbol in your library called something like Embeddedrcall_Init; it is the entry point that lets Tcl install the library into a specific interpreter instance. (It has to be found explicitly because it takes an argument.) By default, the name of the function is found by munging the name of the library (strip version number, case convert, append _Init) but the determination of the name can be overridden by the optional second argument to load.
To be more exact, if the entry is actually called EmbeddedRCall_Init, you would have to load it with:
load embeddedRCall EmbeddedRCall
# The _Init suffix is fixed when loading into a standard interp
Note the case difference! (Also, we recommend using fully qualified path names to loaded libraries, as it avoids some complexities in the dlopen() system.)

gsoap client compile/link error

Now I am writing a program to call a web service. I write testMain.c. The others are generated by wsdl2h and soapcpp2.
My compiling command is like this:
gcc -Wall -g -c -L. soapC.c soapClient.c stdsoap2.c testMain.c
gcc -o testMain -L/usr/lib -lgsoap -lgsoapck -lgsoapssl soapC.o soapClient.o stdsoap2.o testMain.o
And I get these errors. Please help me.
stdsoap2.o: In function `soap_print_fault':
/test/stdsoap2.c:16279: undefined reference to `soap_check_faultsubcode'
/test/stdsoap2.c:16281: undefined reference to `soap_check_faultdetail'
stdsoap2.o: In function `soap_sprint_fault':
/test/stdsoap2.c:16341: undefined reference to `soap_check_faultdetail'
collect2: ld returned 1 exit status
Recent versions of GCC/ld/the GNU toolchain require that the object and library files be specified in a certain order, so that symbols can be found by the linker in the same order they depend on each other. This means that libraries should go to the end of the command line; your second line (when you're linking) should be
gcc -o testMain -L/usr/lib soapC.o soapClient.o stdsoap2.o testMain.o -lgsoap -lgsoapck -lgsoapssl
instead.
I search the web, and found a post which is very similar with my problem. I use this solution and have solved the problem. http://www.mail-archive.com/gsoap#yahoogroups.com/msg01022.html
You should not need to link stdsoap2.o to your project because it's already included in libgsoap (given through the gcc linker option -lgsoap). Try to exclude stdsoap2.c from your project. From the gSOAP FAQ:
I get a link error with gcc/g++ (GNU GCC). What should I do? For C
apps: use soapcpp2 option -c to generate C code, use only the
package's .c files, link with libgsoap.a (-lgsoap) or use the lib's
source stdsoap2.c (and dom.c when applicable).
I had the same problem with gsoap-2.8.16 compiled from source. (That version was shipped with CentOS 6.)
First I checked for a missing library. According to nm used on all static libraries provided by gsoap-2.8.16:
for X in /usr/local/lib/libgsoap*.a ; do echo $X; nm $X | grep soap_check_faultdetail; done`
it turned out that none of the libraries provided the missing symbols.
A brief look at the source code revealed that the expected return type of both methods soap_check_faultdetail and soap_check_faultsubcode was const char*, and that these were used to generate error messages.
It looked to me as if these are meant to be callbacks that the client must provide. Maybe their implementation is WSDL-dependent and would be supplied by the gsoap code generation utilities - that I don't know, see the answer from #ChristianAmmer above or below.
Anyway, since I knew the symbols were nowhere supplied, and that null-terminated strings were probably acceptable here, I just supplied my own no-op implementation:
// gsoap-missing-symbols.cpp
extern "C" {
const char* soap_check_faultdetail() { return 0; }
const char* soap_check_faultsubcode() { return 0; }
}
This is a brute-force solution. If you follow this solution, you should maybe check for linker warnings in the future; maybe some mechanism (eg. from the gsoap code generator) will supply conflicting implementations later during development.
For later versions of gsoap, I believe these symbols are no longer used and can be dropped (or renamed), see soap_check_faultX in https://www.genivia.com/changelog.html.

Resources