I am trying to build a program in python using Cython and PyInstaller. Before starting, I built this test program. However, the two modules aren't working together at all. I have tried them both separately, and they work.
I looked at this question but that doesn't work.
I also tried using cython to compile it to c and then use gcc, but that doesn't work either (I added the python.h file, and gcc just aborts after complaining that the keyword doesn't exist)
#hello.pyx
import pygame
def say_hello_to(name):
print("Hello %s!" % name)
#hello.py
from hello import say_hello_to
say_hello_to('Me')
#setup.py
from setuptools import setup
from Cython.Build import cythonize
setup(
name='Hello world app',
ext_modules=cythonize("hello.pyx"),
zip_safe=False,
)
All of this works in CPython, with :
python3 setup.py build_ext --inplace
Then
python3 hello.py
But PyInstaller keeps complaining it can't find the module hello. I tried add-data and add-binaries, as well as editing the spec file, and it still doesn't work, saying:
invalid add_data_or_binary value: 'home:Achyut-BK:hello.cpython-38-x86_64-linux-gnu.so'
Related
I have this project on a subject 'Languages and Compilers' Where we write parsers in Haskell. At the moment I'm working on this 2-Arrow project where we use the 'alex' and 'happy' libraries to create a parser.hs and lexer.hs file from a lexer.x and parser.y file. While trying to build my framework (cabal build in the project directory) I get the following error message:
C:\\ ... \2-Arrow\dist-newstyle\build\x86_64-windows\ghc-9.2.4\assignment-arrow-0.1.0.0\x\arrow\build\arrow\arrow-tmp\Lexer.hs:1:1: error:
`gcc.exe' failed in phase `C pre-processor'. (Exit code: 1)
|
1| {-# OPTIONS_GHC -fno-warn-unused-binds -fno-warn-missing-signatures #-}
| ^
C:\\ ... \2-Arrow\dist-newstyle\build\x86_64-windows\ghc-9.2.4\assignment-arrow-0.1.0.0\x\arrow\build\arrow\arrow-tmp\Parser.hs:1:1: error:
`gcc.exe' failed in phase `C pre-processor'. (Exit code: 1)
|
1 | {-# OPTIONS_GHC -w #-}
| ^
If they were error messages originating from the code I wrote I would at least have a direction to look for solutions, but these errors come from the /dist-newstyle/ folder. Which, in my understanding, is a folder for intermediate files used in building the program. So I'm kind of at a dead end at the moment, since reinstalling 'happy' and 'alex', and updating cabal to latest version all did not change anything. Does anyone have any advice for resolving this error? Any help is greatly appreciated!!
(project framework from uni: https://www.cs.uu.nl/docs/vakken/b3tc/downloads-2018/2-Arrow.zip (depends on 'happy' and 'alex' packages). At first I thought the problem could lay in the framework, but I checked with the professor, he stated that others had no issues, so I don't think that's the problem)
I wasn't able to duplicate the problem with a fresh GHCup installation on Windows 10, but the error message suggests there might be an issue with an installed version of the GNU C Compiler (GCC) which the Glasgow Haskell Compiler (GHC) invokes to apply the C pre-processor to certain Haskell source files, including the Lexer.hs source file that's being generated by Alex from the Lexer.x file, and the Parser.hs source file that's being generated by Happy from the Parser.y file.
Try creating a simple test.hs program that invokes the C pre-processor:
{-# LANGUAGE CPP #-}
#define HELLO "Hello, world!"
main = putStrLn HELLO
and try compiling with ghc test.hs. Assuming that fails with the same error as above, try ghc -v test.hs, and you should find lines in the output starting with:
*** C pre-processor
"C:\...\gcc.exe" "-E" "-undef" ...blah blah blah...
An excerpt from my successful compilation follows. Maybe you can spot what's going wrong with your installation. Note that mine invokes the GHCup-installed version of GCC and directs the output to a temporary file area where it's then compiled by GHC. If the wrong GCC is invoked or the temporary file area isn't writable for some reason, that might lead to the problem you're seeing.
*** C pre-processor:
"C:\ghcup\ghc\9.2.5\lib\../mingw/bin/gcc.exe" "-E" "-undef" "-traditional" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\base-4.16.4.0\include" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\ghc-bignum-1.2\include" "-IC:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\rts-1.0.2\include" "-include" "C:\ghcup\ghc\9.2.5\lib\x86_64-windows-ghc-9.2.5\rts-1.0.2\include\ghcversion.h" "-Dmingw32_BUILD_OS" "-Dx86_64_BUILD_ARCH" "-Dmingw32_HOST_OS" "-Dx86_64_HOST_ARCH" "-D__GLASGOW_HASKELL_TH__" "-D__SSE__" "-D__SSE2__" "-D__IO_MANAGER_WINIO__=1" "-D__IO_MANAGER_MIO__=1" "-includeC:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_2.h" "-x" "assembler-with-cpp" "test.hs" "-o" "C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_1.hscpp"
!!! systool:cpp: finished in 31.25 milliseconds, allocated 0.311 megabytes
!!! Chasing dependencies: finished in 46.88 milliseconds, allocated 2.719 megabytes
Stable obj: {}
Stable BCO: {}
Ready for upsweep
[NONREC
ModSummary {
ms_hs_date = 2022-12-18 20:54:57.2679991 UTC
ms_mod = Main,
ms_textual_imps = [(Nothing, Prelude)]
ms_srcimps = []
} []]
*** Deleting temp files:
Deleting: C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_2.h
compile: input file C:\Users\buhr\AppData\Local\Temp\ghc7128_0\ghc_1.hscpp
*** Checking old interface for Main (use -ddump-hi-diffs for more details):
[1 of 1] Compiling Main ( test.hs, test.o )
I went to my seminar and asked the TA for help. After some time he proposed that maybe the folder location was the problem. Lo and behold, after moving the folder from my OneDrive folder (which is also saved locally) to my C:\\ directory, the project finally compiled! I have no idea why this would be the case, so I have posted another question here Folder location makes Haskell project incompileable.
I am using Windows to follow the Kotlin-Native libcurl example here https://kotlinlang.org/docs/tutorials/native/curl.html
Unfortunately, this has been a bigger challenge then I could have imagined. I have been learning how to compile C binaries on windows from source using tools such as CMake and have followed this tutorial to build libcurl on Windows: https://jonnyzzz.com/blog/2018/10/29/kn-libcurl-windows/
After quite a few days of trial and error with the .def file for cinterop on Windows, and using compilerOpts to point to my header file for curl in my built library, I have hit a brick wall.
I can't seem to get Gradle to recognize the dll file. Task ':linkDebugExecutableMingw' keeps failing. I have no idea where I am supposed to put the dll file. I have tried putting it in to the build output folder next to the exe, but it still fails to compile, with this error:
C:\Users\yous\.konan\dependencies\msys2-mingw-w64-x86_64-clang-llvm-lld-compiler_rt-8.0.1\bin\ld: C:\Users\yous\AppData\Local\Temp\konan_temp6326583690522756621\result.o:out:(.rdata$.refptr.knifunptr_sample35_curl_easy_strerror[.refptr.knifunptr_sample35_curl_easy_strerror]+0x0): undefined reference to `knifunptr_sample35_curl_easy_strerror'
And a similar list of header file errors, then the final message is:
e: C:\Users\yabde\.konan\dependencies\msys2-mingw-w64-x86_64-clang-llvm-lld-compiler_rt-8.0.1/bin/clang++ invocation reported errors
This is after I succesfully located the header files. I believe this is an issue with the dll file not being included.
Here is my relevant Gradle build script:
mingwX64("mingw") {
compilations.main {
cinterops {
libcurl {
// Def-file describing the native API.
// The default path is src/nativeInterop/cinterop/<interop-name>.def
defFile project.file("src/nativeInterop/cinterop/libcurl.def")
// Package to place the Kotlin API generated.
packageName 'libcurl'
// Options to be passed to compiler by cinterop tool.
compilerOpts '-Isrc\\nativeInterop\\cinterop\\libcurl-vc-x64-release-dll-ipv6-sspi-winssl\\include -o libcurl'
//linkerOpts '-Isrc\\nativeInterop\\cinterop\\libcurl-vc-x64-release-dll-ipv6-sspi-winssl\\bin'
// Directories for header search (an analogue of the -I<path> compiler option).
//includeDirs.allHeaders("path1", "path2")
// Additional directories to search headers listed in the 'headerFilter' def-file option.
// -headerFilterAdditionalSearchPrefix command line option analogue.
includeDirs.headerFilterOnly("src/nativeInterop/cinterop/libcurl-vc-x86-release-dll-ipv6-sspi-winssl/include")
// A shortcut for includeDirs.allHeaders.
//includeDirs("include/directory", "another/directory")
}
}
}
binaries {
executable {
// Change to specify fully qualified name of your application's entry point:
entryPoint = 'sample.main'
// Specify command-line arguments, if necessary:
//runTask?.args('-Lsrc/nativeInterop/cinterop/libcurl-vc-x86-release-dll-ipv6-sspi-winssl/bin')
runTask?.args('')
}
}
}
I believe this issue may be with linker options to locate the dll, but uncommenting the linkerOpts line does not help.
I think I have given up on Kotlin/Native and will be sticking to C for data-oriented native programming...
EDIT:
Also, note that any text commented out is things that I have attempted
Ok so, this is my def file:
headers = curl/curl.h
headerFilter = curl/*
compilerOpts.linux = -I/usr/include -I/usr/include/x86_64-linux-gnu
linkerOpts.osx = -L/opt/local/lib -L/usr/local/opt/curl/lib -lcurl
linkerOpts.linux = -L/usr/lib/x86_64-linux-gnu -lcurl
linkerOpts.mingw = -Llibcurl-vc-x86-release-dll-ipv6-sspi-winssl/lib
Ok so, this is my project structure:
I followed the same steps (jonyzzz blog). I created a Libcurl static example for Kotlin-Native. Support for gzip and SSL. Compile to standalone exe, no need for dll
https://github.com/carlosrafp/Libcurl-Kotlin-Native-standalone
I built the static libraries supplied on the repository with mingw64 (gzip and libcurl) and msys2/mingw (openssl)
I went through this question several times when looking for a solution for statically linking libcurl with my binary.
In the end, I ended up using WinInet for the HTTP communication on Windows. Linux and macOS binary still rely on ktor clients with libcurl.
See my example of how to use WinInet API with Kotlin MPP: https://github.com/localazy/kotlin-mpp-wininet
For a simple HTTP communication, WinInet API is good enough and the resulting binary is much smaller and in my case, it has no additional dependencies.
I'm quite new to python that I always use by writing script in spyder and running them in its Ipython console with python3.6.2 .
I'm trying to write a simple module from a "swig_example.c" file following a couple of swig tutorial (http://www.swig.org/tutorial.html, http://www.swig.org/Doc1.3/Python.html#Python_nn6).
My aim is to be able to run a script "main_python.py" which should look like:
import swig_test
print(swig_test.fact(4))
where fact is a function defined in the original c source.
The source file is "swig_example.c":
/* File: swig_example.c */
#include "swig_example.h"
int fact(int n) {
if (n == 0) {
return 1;
}
else {
return n * fact(n-1);
}
}
The header file is as simple as:
/* File: swig_example.h */
int fact(int n);
and the interface one:
/* File: swig_example.i */
%module swig_test
%{
#include "swig_example.h"
%}
%include "swig_example.h"
When in the terminal I run:
swig -python swig_example.i
a "swig_example_wrap.c" and a "swig_test.py" files are created.
How should I proceed to have my "main_python.py" working?
(It now returns a "No module named '_swig_test' error).
I would like to have some script (maybe using distutils?) so that each time I modify the .c source I can easily update the module without changing the "main_python.py" file.
If you have any solution which uses Xcode instead of spyder it would be well accepted.
I think that the question could be useful to many that are new to python (and Mac actually...) and try to use it while not throwing away their previous works...
EDIT:
I -partially- solved the problem. Now the main point remain Spyder.
I create the files ".c", ".h" and ".i" the way I described. Then, following this post (Python.h not found using swig and Anaconda Python) I create, in the same folder, my "setup.py" file:
from distutils.core import setup, Extension
example_module = Extension('_example', sources=['example.c','example.i'])
setup(name='example', ext_modules=[example_module], py_modules= .["example"])
Then, In anaconda navigator I open the terminal of the environment I'm working in, move to the right folder and run:
python setup.py build_ext --inplace
If now I open spyder everything works the desired way. But If I now want to modify my C source, say add a new function, problems arises. I modify the ".c", ".h" and ".i" files annd thenn re-run in the terminal the previous line. The "example.py" file result to e correctly modified (it innncludes the attribute of the new function), but when try to import the module in spyder (import example) changes are nnot registered and an error message "_example has no attribute "new function" is given in the Ipython console unless I restart Spyder itself.
Is there faster way to fix it? (maybe this is the interaction mentioned in the comments... )
Thank you all :-)
I'm trying to write a haskell module that wraps a bunch of c functions.
I want to be able to import this module like any other haskell module without referring to the c object files.
I can't find any examples about how to do this.
This is what I've tried. I have a c file "dumbCfunctions.c":
double addThree(double x) {
return x+3;
}
and a haskell file with a module defined in it "Callfunctions.hs"
module Callfunctions (
addThree
) where
import Foreign.C
foreign import ccall "addThree" addThree :: Double -> Double
main = print $ addThree 4
I can make an executable doing:
ghc --make -o cf_ex Callfunctions.hs dumbCfunctions.o
Which correctly gives me 7.
I can also import it into ghic by calling ghci with
shane> ghci dumbCfunctions.o
Prelude> :l Callfunctions.hs
[1 of 1] Compiling Callfunctions ( Callfunctions.hs, interpreted )
Ok, modules loaded: Callfunctions.
*Callfunctions> addThree 3
6.0
But I want to be able to treat it like any other module with out referring to "dumbCfunctions.o":
shane> ghci
Prelude> :l Callfunctions.hs
[1 of 1] Compiling Callfunctions ( Callfunctions.hs, interpreted )
Ok, modules loaded: Callfunctions.
*Callfunctions> addThree 3
But now I get the error
ByteCodeLink: can't find label
During interactive linking, GHCi couldn't find the following symbol:
addThree
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session. Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
glasgow-haskell-bugs#haskell.org
This makes sense because I haven't refereed to the object anywhere. So I must be able to do something better by first compiling the module, but I couldn't find out how to do this. I must be looking in the wrong places.
You can create a library through Cabal, and cabal install it.
This would link the C code inside your Haskell library. Later on, when you load the module, you will not need to manually load the C parts.
I am trying to write a Cython wrapper for a C Library. I have read through the docs carefully but I must be missing something since I cannot get the following simple Cython code to work.
I created a shared library from the following:
mathlib.c
#include "mathlib.h"
int add_one(int x){
return x + 1;
}
mathlib.h
extern int add_one(int x);
Then I created the library like so:
gcc -c mathlib.c
gcc -shared -o libmathlib.so mathlib.o -lm
My Cython files are mathlib.pyx, cmathlib.pyd, and setup.py
cymathlib.pyx
from mathlib cimport add_one
def add_one_c(int x):
print add_one(x)
return x
mathlib.pyd
cdef extern from "mathlib.h":
int add_one(int x)
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
ext_modules = cythonize([Extension("cymathlib", ["cymathlib.pyx"])], libraries ["mathlib"])
)
The module cymathlib.so is created but when I attempt to import it in Python I get the following error: ImportError: dlopen(./cymathlib.so, 2): Symbol not found: _add_one Referenced from: ./cymathlib.so Expected in: flat namespace in ./cymathlib.so"
It looks like something went wrong with your Extension specification, which should be something like:
ext_modules = cythonize([Extension("cymathlib",
["cymathlib.pyx"],
libraries=["mathlib"],
library_dirs=["/place/path/to/libmathlib.so/here"]
)])
To be able to use the module, it must be able to find libmathlib.so at run time, since it will look in that file for the actual implementation of add_one. Apart from copying the file to /usr/lib or /usr/local/lib (and running ldconfig again), you can also set an environment variable to make sure the library can be found:
export LD_LIBRARY_PATH=/place/full/path/to/libmathlib.so/here
It's also possible to add the C code into the python module you're creating (so no libmathlib.so will need to be compiled or used anymore). You just need to add the mathlib.c file to the list of cython sources:
ext_modules = cythonize([Extension("cymathlib",
["cymathlib.pyx","mathlib.c"]
)])