Writing R extensions in C (SHMGET crash in C) - c

I try to create a shared memory about cygwin in a Windows 7 environment. I compile it as normal c-program, all is working fine. If I try to call the function as R-Extension, the shmget-function is going crashed. The next step is to use semaphore. I hope this will run as well, if I solve this problem.
Is there any compiler option or something like else what I have to do? Maybe, have I to change the mingw to cygwin in R or what can I do. I'm really frustrated!
Actually I try to compile the extension with:
R CMD SHLIB wrappers.c IPC.c -Wall -pthread
And this is the problem child:
_smID = shmget(SHAREDMEMORYID, sizeof(struct sSharedMemory), IPC_CREAT |0666);
My R version:
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 1.0
year 2014
month 04
day 10
svn rev 65387
language R
version.string R version 3.1.0 (2014-04-10)
nickname Spring Dance
cygwin is up to date.

Related

Discrepancy in behavior of Linux loaders (ld-linux-x86-64) between Glibc 2.12 and Glibc 2.17

I'm trying to compile the same lib on two x86 separate machines.
Both use the same toolchain (exactly same set of files) but have different Glibc versions.
When I run command LD_DEBUG=libs /lib64/ld-linux-x86-64.so.2 --list ./libl2ps.so I notice the following discrepancy between the 2 Linux loaders:
Machine 1 (with Glibc 2.12):
19943: find library=libm.so.6 [0]; searching
19943: search path=/ebs/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64:...:/ebs/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/. (RPATH from file ./libl2ps.so)
19943: trying file=/ebs/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm.so.6
19943:
19943: find library=libgcc_s.so.1 [0]; searching
...
In this case the Linux loader selects lib libm.so.6 from the toolchain path based on RPATH of lib libl2ps.so.
Machine 2 (with Glibc 2.17):
10699: find library=libm.so.6 [0]; searching
10699: search path=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64:/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/lib64:/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib:/home/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/. (RPATH from file ./libl2ps.so)
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/lib64/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/./libm.so.6
10699: search cache=/etc/ld.so.cache
10699: trying file=/lib64/libm.so.6
As for Machine 1, the loader attempts from RPATH of libl2ps.so to select lib libm.so.6 from toolchain path but skip it for some reason and try further other paths. Finally it selects libm.so.6from the system path /lib64/.
The RPATH of the 2 libs lib2ps.so are exactly the same. The two files libm.so.6 are also exactly the same on both machines (checked with md5sum).
I don't understand this differences of behavior between the 2 Linux loaders.
Do you see any reason what would explain this discrepancy ?
Thank you very much for your answers.
Update:
Thank you yugr for your answer.
Output of readelf -h gives only differences on fields "Entry point address" and "Start of section headers" and there is no other differences so I think it will not help.
Regarding using dlopen()/dlerror(), I've done a little executable with the following statement:
dlopen("/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so", RTLD_LAZY);
On machine 1 it works as expected:
C++ dlopen demo
Opening libm-2.28.so...
Closing library...
On machine 2 it fails and dlerror() gives the following output:
Cannot open library: /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so: cannot open shared object file: No such file or directory
but the file libm-2-28.so really exists on my file system:
$ ls -l /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so
-rwxr-xr-x 1 frperies linseeusers_lte_espoo 1682944 Oct 5 13:50 /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic- linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so
This is very weird, what could lead to this situation ???
Thanks
Update 2:
That is true that I haven't pointed out that machine 1 is a RHEL6.8 distro while machine 2 is RHEL7.4 distro. I (naively?) didn't think this really matters...
On machine 1:
$ cat /proc/sys/kernel/osrelease
4.4.115-1.NSN.el6.x86_64
$ uname -a
Linux sq24-3 4.4.115-1.NSN.el6.x86_64 #1 SMP Mon Feb 12 12:35:46 CET 2018 x86_64 x86_64 x86_64 GNU/Linux
$ readelf -n libl2ps.so
Notes at offset 0x00000270 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: b598468830fdf2f61eda25553b9a367c4d28cdc9
On machine 2:
$ cat /proc/sys/kernel/osrelease
3.10.0-693.el7.x86_64
$ uname -a
Linux localhost.localdomain 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
$ readelf -n libl2ps.so
Displaying notes found at file offset 0x00000270 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 5829181bc0502233748149369108915ea7b10e8f
Does it help ?
Thanks
Update 3:
$ readelf -n libm.so.6
Notes at offset 0x00000238 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 0d84c7247dd76008c096719043e5592735a1c4bd
Notes at offset 0x0000025c with length 0x00000020:
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 4.4.0
So, how to interpret this ABI version number set to 4.4.0 ??
Thanks
Thank you yugr and Employed Russian for your answers!!
I will give it a try by upgrading my Kernel version on Machine 2.
Thanks
Regards
The error message that you see is the infamously confusing ENOENT errno. I see two instances of it in dl-load.c:
checking OS compatibility
loading non-setuid to setuid process
I suspect the first one fails in your case which would mean that OS kernel is incompatible between two machines. ld.so manpage indeed says that
Each shared object can inform the dynamic linker of the
minimum kernel ABI version that it requires. (This
requirement is encoded in an ELF note section that is viewable
via readelf -n as a section labeled NT_GNU_ABI_TAG.) At run
time, the dynamic linker determines the ABI version of the
running kernel and will reject loading shared objects that
specify minimum ABI versions that exceed that ABI version.
NT_GNU_ABI_TAG is 4.4.0 which means that you run a program expecting a minimum 4.4 kernel on a 3.10 kernel. Theoretically newer Glibc should run on older kernels as well but in your case Glibc was probly built with explicit --enable-kernel flag which prevents it's usage on kernels before 4.4 (see e.g. this explanation of --enable-kernel).
As a workaround, you may try to fool Glibc by overriding kernel version on machine 2 via
export LD_ASSUME_KERNEL=4.4.0
but it may not work if libm makes 4.4-specific syscalls that are not really present on 3.10.

Verification of poly1305-donna-16.h code

The RFC from May 2015 Y. Nir et al, ChaCha20 and Poly1305 for IETF Protocols
(https://www.rfc-editor.org/rfc/rfc7539) contains a reference to the MIT/Public domain C library https://github.com/floodyberry/poly1305-donna.
I am just porting the C code to Pascal. The 8 bit code works OK (self-test, example, and RFC test vectors).
The port from poly1305-donna-16.h using 16->32 bit multiples and 32 bit additions failed. After some testing I compiled the original source with DJGPP GCC 4.7.3, MS VC 6.0 and BC 3.1 and all three failed too (poly1305 self-test).
Questions: Does this C version (build with -DPOLY1305_16BIT) fail for other compilers too? Is there a known fix available? (The blog of the author Andrew Moon at https://floodyberry.wordpress.com/ is inactive since 6 years)
I can confirm a build failure on a pretty vanilla Fedora 22 system:
% gcc poly1305-donna.c -c -DPOLY1305_16BIT
% gcc example-poly1305.c -o ex poly1305-donna.o -DPOLY1305_16BIT
% ./ex
poly1305 self test: failed
Notice that the test succeeds when I omit -DPOLY1305_16BIT.
Also notice:
% uname -rmp
4.0.8-300.fc22.x86_64 x86_64 x86_64
% gcc --version
gcc (GCC) 5.1.1 20150618 (Red Hat 5.1.1-4)
I suggest you submit a bug report. Andrew has been responsive in the past.
EDIT:
Compiling with clang version 3.5.0 yields the same results as the above gcc test.

"Source file is more recent than executable" except it isn't

GDB is complaining that my source file is more recent than the executable, and it appears the debugging information is indeed related to an older version of the source file, because gdb is stopping on a blank line:
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) up
#1 0x00007ffff7ba2d88 in CBKeyPairGenerate (keyPair=0x602010) at library/src/CBHDKeys.c:246
warning: Source file is more recent than executable.
246
(gdb) list
241 if (versionBytes == CB_HD_KEY_VERSION_TEST_PUBLIC
242 || versionBytes == CB_HD_KEY_VERSION_TEST_PRIVATE)
243 return CB_NETWORK_TEST;
244
245 return CB_NETWORK_UNKNOWN;
246
247 }
248
249 uint8_t * CBHDKeyGetPrivateKey(CBHDKey * key) {
250
But the executable is more recent than the source file, see here:
$ ls -l library/src/CBHDKeys.c
-rw-r--r-- 1 matt matt 9249 Apr 29 22:40 library/src/CBHDKeys.c
$ ls -l bin/noLowerAddressGenerator
-rwxr-xr-x 1 matt matt 17845 Apr 30 15:52 bin/noLowerAddressGenerator
I tried rebuilding after make clean and ccache -C but the same problem occurs. When I updated the source file I only added whitespace, so the program logic remains equal.I feel that has something to do with it, but since I cleared the ccache and cleaned the build and bin directory with make clean I'm not sure what is going on.
Versions:
GNU Make 3.81
gcc (Debian 4.8.2-16) 4.8.2
GNU gdb (GDB) 7.6.2 (Debian 7.6.2-1)
ccache version 3.1.9
SolydXK - SMP Debian 3.13.5-1 (2014-03-04)
Perhaps you're not using the most recent compiled version of the code, if it's in a shared library. You could use ldd noLowerAddressGenerator to see the library dependencies of your program; I don't know if it's possible from within GDB to locate the relevant library, but there ought to be a way (please comment or edit if you know how).
If this is indeed the case, you might want to set environment LD_LIBRARY_PATH in GDB prior to running the program, to place your newly-built library ahead of any installed ones. You could look into setting the RPATH ELF variable when linking, but that's likely to be less help.
Another possibility is to run your debugger on a system where you know the library isn't installed. I've had good results using schroot to keep build/debug/install environments separated.

OpenSSL- Linux System requirements [duplicate]

This question already has answers here:
SSL_read failing with SSL_ERROR_SYSCALL error
(6 answers)
Closed 8 years ago.
Unfortunately, I was unable to find an answer to my question on google.
I have 2 devices with very similar hardware. On 1 device my SSL code runs flawlessly, on the second it doesn't. The major difference between both devices is the linux kernel version.
The code works on 2.6.24.6 but not 2.4.21 (Error when attempting to do handshake using ssl_connect() the error is SSL_ERROR_SYSCALL). My guess is, the latter kernel version is not supported.
Could someone point me out to where I can find more information about minimum system requirements for OpenSSL ?
In case, kernel 2.4.21 is not supported, is there any other library I could use?
To see a list of supported OSes and platforms, run Configure with a bogus a argument:
$ ./Configure XXX
Configuring for XXX
Usage: Configure [no-<cipher> ...] [enable-<cipher> ...] [experimental-<cipher> ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx] [no-hw-xxx|no-hw] [[no-]threads] [[no-]shared] [[no-]zlib|zlib-dynamic] [no-asm] [no-dso] [no-krb5] [sctp] [386] [--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]] [--test-sanity] os/compiler[:flags]
pick os/compiler from:
BC-32 BS2000-OSD BSD-generic32 BSD-generic64 BSD-ia64 BSD-sparc64 BSD-sparcv8
BSD-x86 BSD-x86-elf BSD-x86_64 Cygwin Cygwin-pre1.3 DJGPP MPE/iX-gcc OS2-EMX
OS390-Unix QNX6 QNX6-i386 ReliantUNIX SINIX SINIX-N UWIN VC-CE VC-WIN32
VC-WIN64A VC-WIN64I aix-cc aix-gcc aix3-cc aix64-cc aix64-gcc android
android-armv7 android-x86 aux3-gcc beos-x86-bone beos-x86-r5 bsdi-elf-gcc cc
cray-j90 cray-t3e darwin-i386-cc darwin-ppc-cc darwin64-ppc-cc
darwin64-x86_64-cc dgux-R3-gcc dgux-R4-gcc dgux-R4-x86-gcc dist gcc hpux-cc
hpux-gcc hpux-ia64-cc hpux-ia64-gcc hpux-parisc-cc hpux-parisc-cc-o4
hpux-parisc-gcc hpux-parisc1_1-cc hpux-parisc1_1-gcc hpux-parisc2-cc
hpux-parisc2-gcc hpux64-ia64-cc hpux64-ia64-gcc hpux64-parisc2-cc
hpux64-parisc2-gcc hurd-x86 iphoneos-cross irix-cc irix-gcc irix-mips3-cc
irix-mips3-gcc irix64-mips4-cc irix64-mips4-gcc linux-alpha+bwx-ccc
linux-alpha+bwx-gcc linux-alpha-ccc linux-alpha-gcc linux-aout linux-armv4
linux-elf linux-generic32 linux-generic64 linux-ia32-icc linux-ia64
linux-ia64-ecc linux-ia64-icc linux-ppc linux-ppc64 linux-sparcv8
linux-sparcv9 linux-x86_64 linux32-s390x linux64-s390x linux64-sparcv9 mingw
mingw64 ncr-scde netware-clib netware-clib-bsdsock netware-clib-bsdsock-gcc
netware-clib-gcc netware-libc netware-libc-bsdsock netware-libc-bsdsock-gcc
netware-libc-gcc newsos4-gcc nextstep nextstep3.3 osf1-alpha-cc osf1-alpha-gcc
purify qnx4 rhapsody-ppc-cc sco5-cc sco5-gcc solaris-sparcv7-cc
solaris-sparcv7-gcc solaris-sparcv8-cc solaris-sparcv8-gcc solaris-sparcv9-cc
solaris-sparcv9-gcc solaris-x86-cc solaris-x86-gcc solaris64-sparcv9-cc
solaris64-sparcv9-gcc solaris64-x86_64-cc solaris64-x86_64-gcc sunos-gcc
tandem-c89 tru64-alpha-cc uClinux-dist uClinux-dist64 ultrix-cc ultrix-gcc
unixware-2.0 unixware-2.1 unixware-7 unixware-7-gcc vos-gcc vxworks-mips
vxworks-ppc405 vxworks-ppc60x vxworks-ppc750 vxworks-ppc750-debug
vxworks-ppc860 vxworks-ppcgen vxworks-simlinux ...
NOTE: If in doubt, on Unix-ish systems use './config'.

Problem with loading compiled c code in R x64 using dyn.load

I recently went from a 32bit laptop to a 64bit desktop (both win7). I just found out that I get an error now when loading dll's using dyn.load. I guess this is a simple mistake and I am overlooking something.
For example, I write this simple c function (foo.c):
void foo( int *x) {*x = *x + 1;}
Then compile it in command prompt:
R CMD SHLIB foo.c
Then in 32bit R I can use it in R:
> dyn.load("foo.dll")
> .C("foo",as.integer(1))
[[1]]
[1] 2
but in 64bit R I get:
> dyn.load("foo.dll")
Error in inDL(x, as.logical(local), as.logical(now), ...) :
unable to load shared object 'C:/Users/Sacha/Documents/R/foo.dll':
LoadLibrary failure: %1 is not a valid Win32 application.
nd.
Edit:
For reference, R CMD can be forced in an architecture by using --arch 64x :
R --arch x64 CMD SHLIB foo.c
Quite clear actually, I knew I was making a rooky mistake:)
My guess is that you are compiling it to a 32 bit target. You need to build it on your 64 bit machine with 64 bit tools. You can't load a 32 bit DLL into a 64 bit process, and vice versa.
what i did is to compile with --arch x64 and --arch 32 once a time and manually put corresponding .dll ( with the same name ) under separate folders src-x64 and src-i386 respectively, these two folders are under the same directory where the folder src is.

Resources