Limiting shared library symbol visibility on Solaris - linker

With GCC (on Linux) I can easily limit the visibility of symbols in shared libraries.
What are my options on Solaris (10)?
Do these GCC features also work with GCC on Solaris (especially with a GCC that uses the Solaris linker/assembler)?
And does the Solaris Studio C-compiler/linker provide similar attributes/pragmas for controlling the visibility of symbols (i.e. for setting the default to hidden and explicitly marking symbols as visible)?

I've tested some declaration annotation methods on a Solaris 10 machine. Surprisingly, the Solaris Studio C compiler also supports the GCC-hidden-function attribute.
A GCC configured with Solaris as/ld correctly implements the visibiltiy function attribute.
Thus, using the GCC function attribute syntax should be the most convenient/portable method because it works on Linux/GCC, Solaris/GCC and Solaris/Sol-Studio.
See the following table for an overview of the effects setting a function visibility to hidden.
Results
.dynsym|.symtab
System Compiler Visibility nm readelf link-error
elfdump
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Linux Fedora 17 gcc-4.7.2 not specified T GLOBAL DEFAULT no
Linux Fedora 17 gcc-4.7.2 attr-hidden t -|LOCAL DEFAULT yes
Solaris 10 gcc-4.8 not specified GLOB GLOB D no
Solaris 10 gcc-4.8 attr-hidden LOCL -|LOCL H yes
Solaris 10 cc-12.3 attr-hidden LOCL -|LOCL H yes
Solaris 10 cc-12.3 __hidden LOCL -|LOCL H yes
Methods
main.c:
#include <stdio.h>
#include <stdlib.h>
#include "power3.h"
#include "power2.h"
int main(int argc, char **argv)
{
printf("Result: %d\n", power3(atoi(argv[1])));
// should result in a link error when symbol is hidden
printf("Result: %d\n", power2(atoi(argv[1])));
return 0;
}
power2.h:
#ifndef POWER2_H
#define POWER2_H
#if !defined(NO_HIDE)
#if defined(__GNUC__) || defined(FORCE_GCC)
__attribute__((visibility("hidden")))
#warning Using GNU-C function attribute
#elif defined(__SUNPRO_C)
__hidden
#warning Using SUNPRO-C qualifier
#endif
#endif
int
// GCC attribute also possible here
power2(int x);
#endif
power3.h:
#ifndef POWER3_H
#define POWER3_H
int power3(int x);
#endif
power3.c
#include "power3.h"
#include "power2.h"
int power3(int x)
{
return power2(x)*x;
}
Build commands:
cc -g -c -o main.o main.c
cc -g -fpic -c -o power3.po power3.c
cc -g -fpic -c -o power2.po power2.c
cc -shared -fpic -o libpower.so power3.po power2.po
cc -L$PWD -Wl,-R$PWD
Introspection:
Under Linux:
nm libpower.so | grep power
readelf --dyn-sym libpower.so | grep power
readelf -s libpower.so | grep 'FUNC.*power'
Under Solaris:
/usr/ccs/bin/nm libpower.so | grep 'FUNC.*power'
/usr/ccs/bin/elfdump -N .dynsym libpower.so | grep 'FUNC.*power'
elfdump -N .symtab libpower.so | grep 'FUNC.*power'
System details:
The Solaris 10 system is a SPARC machine and the GCC uses as/ld from /usr/ccs/bin. The Solaris Studio version is 12.3 with patches applied (2013/02/04).
Sources
Oracle Solaris Studio 12.3 C User's Guide. Section 2.2 Linker Scoping Specifiers, page 32 (__hidden). Section 2.9 Supported Attributes, page 40 (__visibility).
Solaris Studio 12.3 cc man page, cc(1) (__hidden)
Reducing Symbol Scope with Sun Studio C/C++ by Giri Mandalika, 2005/2006
GCC 4.8 manual, -fvisibility paragraph
Global switches
For completeness, visibility of functions (and other symbols) can also be influenced by other means:
GCC-method Sol equivalent effect
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
#pragma GCC visibility push(hidden) - everything between push/pop
#pragma GCC visibility pop - has default visibility hidden
#pragma GCC visibility push(default) - ~ default to default-visibility
#pragma GCC visibility pop -
-fvisibility=hidden -xldscope=hidden sets default visibility of
-fvisibility=default -xldscope=global a translation unit
The ELF standard also defines symbol visibilities internal and protected - which are also understood by the compilers, but which are less useful in general.

Another option is to use version script files. This works on Linux, Solaris/GCC and Solaris/CC.
Example
Consider a shared library where only a function power3() should be globally available. It uses power2() which is defined in another translation unit and power2() should be hidden.
Following version script specifies this:
$ cat vscript
{
global: power3;
local: *;
};
You can use one file for linking on Linux and Solaris - Linux/Solaris seem to understand the same syntactic constructs.
Build commands
System Compiler Link command
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Linux Fedora 17 gcc-4.7.2 cc -shared -Wl,--version-script,vscript -fpic -o libpower.so ...
Solaris 10 gcc-4.8* gcc -shared -Wl,-M,vscript -fpic -o libpower.so ...
Solaris 10 cc 12.3 cc -shared -M vscript -fpic -o libpower.so ...
Note that the GCC on Solaris is configured to use ld/as from /usr/ccs/bin.

Can't answer about gcc visibility on Solaris, but
In the Sun Studio there are
compiler option -xldscope, and the code attributes _global/_hidden
according to http://lists.qt.nokia.com/public/qt-interest/2010-January/017893.html
http://qt.gitorious.org/qt/qt/merge_requests/433:
SunStudio since version 8 (CC version 5.5) has had the ability to control symbol visibility in ABI via the linker option
-xldscope=[global|symbolic|hidden]
and via code "attributes" __global, __symbolic & __hidden.
-xldscope=global maps to GCC's -fvisibility=default
-xldscope=symbolic maps to GCC's -fvisibility=protected
-xldscope=hidden maps to GCC's -fvisibility=hidden
__global maps to GCC's __attribute__((visibility("default"))
__symbolic maps to GCC's __attribute__((visibility("protected"))
__hidden maps to GCC's __attribute__((visibility("hidden"))
And there is an overview from Sun: http://www.oracle.com/technetwork/server-storage/solaris/symbol-scope-140496.html
Reducing Symbol Scope with Sun Studio C/C++ by Giri Mandalika, May, 2005 (revised March 22, 2006)

Related

Coverity Scan fails to build <stdlib.h> with _GNU_SOURCE defined

The Coverity Scan Build Tool fails to compile any C file that includes <stdlib.h> on Ubuntu 18.04 when _GNU_SOURCE is defined:
$ cat > main.c
#include <stdlib.h>
int main() {
}
$
$ gcc -D_GNU_SOURCE=1 -o main main.c
$
$ /opt/cov-analysis/bin/cov-build --dir cov-int gcc -D_GNU_SOURCE=1 -o main main.c
Coverity Build Capture (64-bit) version 2017.07 on Linux 4.15.0-20-generic x86_64
...
[WARNING] Emitted 0 C/C++ compilation units (0%) successfully
...
$
The same build works perfectly on Ubuntu 16.04 or without _GNU_SOURCE defined:
$ /volatile/local/cov-analysis/bin/cov-build --dir cov-int gcc -o main main.c
Coverity Build Capture (64-bit) version 2017.07 on Linux 4.15.0-20-generic x86_64
...
Emitted 1 C/C++ compilation units (100%) successfully
...
$
How to get Coverity Scan to build C sources with _GNU_SOURCEdefined on Ubuntu 18.04?
For those interested file cov-int/build-log.txt can be found here:
https://gist.github.com/DimitriPapadopoulos/0dcd9018eed26401cc6095087d9cc1d5
After contacting Coverity support, it appears this is known bug. They suggested I work around it by switching from the default Ubuntu 18.04 compiler (GCC 7) to the previous version (GCC 6):
sudo apt install gcc-6
Indeed _Float32, _Float32x, _Float64, _Float64x and _Float128 were introduced in GCC 7.
Coverity is failing to define the types GCC would define, but then it's claiming to be GCC anyway. Here's a workaround: https://gist.github.com/vathpela/0cede6d6eb5b0ec0791c6afc4282c340#file-fix_coverity-h
Just be sure you do:
#include "fix_coverity.h"
before stdlib.h gets included, whether directly or indirectly.

Unable to use compiled netlib BLAS on mac OS X

I'm trying to make a repository collecting all the examples, tutorials and instructions I could find on the internet for C mathematical and algebra libraries (BLAS, CBLAS, LAPACK, CLAPACK, LAPACKE, ATLAS, openblas, GSL...). but it seems that I just can't get the compiled BLAS .a files working on mac OS X.
So far I have been able to compile BLAS and use it on ubuntu:
BLAS source code from netlib website downloaded and compiled (rename blas_LINUX.a to libblas.a)
Then I can compile the C file on ubuntu using the command below:
gcc foo.c path/to/libblas.a
On my mac OS X (EL Capitan), I can compile BLAS (changing LINUX in the make.inc to DARWIN), but when I try to compile a C code using the command above I get errors like below:
Undefined symbols for architecture x86_64:
"_ddot_", referenced from:
_main in foo-3a35db.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
("ddot" part differs for different functions)
possibilities:
Maybe I'm not compiling the library correctly on mac and there are some differences I'm not aware of
The builtin Accelerate framework of mac OS X is messing up with the compiling process
P.S. Guys I know BLAS/LAPACK is already built into the mac OS X Accelerate framework and I can easily compile using the command gcc foo.c -lblas or gcc foo.c -framework Accelerate but I want to use the compiled .a from netlib. I want to know why it works properly on ubuntu but not mac OS X?
P.S.2. Please notice that I can compile the source code successfully without any errors on mac OS X. I just can use it!
Example code: source
#include <stdio.h>
#include <stdlib.h>
double ddot_(const int *N, const double *a, const int *inca, const double *b, const int *incb);
int main(int argc, char **argv) {
double *a = (double *)malloc(3 * sizeof(double));
a[0] = 1.0;
a[1] = 2.0;
a[2] = 3.0;
// on the stack
double b[3] = {4.0, 5.0, 6.0};
int N = 3, one = 1; // one really doesn't look good in C
double dot_product = ddot_(&N, a, &one, b, &one);
printf(" The dot product is: %f \n", dot_product);
return 0;
}
(edit1) solution:
open make.inc
change the line OPTS = -O3 to OPTS = -O3 -pipe -c and make.
(edit2): better solution:
since I asked this question I have realised that I have been doing everything wrong. Netlib BLAS is actually a collection of fortran routines/subroutines/functions. and the Makefile in the source code just gives us a static library libblas.a which is a collection of all .o object files compiled with gfortran. when we want to compile a C code which want to call one of those routines, we also need to link to the gfortran library libgfortran.* so if you have gcc installed (brew install gcc). look for libgfrotran* (sudo find / -name "libgfortrn.*") and then link your gcc to this folder too. to make it easy I put a Makefile here:
all:
gcc -c foo.c
gcc -o bar.out foo.o -L path/to/libgfortran.*/ -lgfortran -L path/to/libblas.a -lblas
or alternatively compile the code directly with gfortran:
all:
gcc -c foo.c
gfortran -o bar.out foo.o -L path/to/libblas.a -lblas
or simply compile with:
gcc foo.c bar.out -L path/to/libblas.a -lblas -L path/to/libgfortran.*/ -lgfortran
the wonder is how/why the former solution actually worked and why on ubuntu you don't have to link to -lgfortran!
It seems you compiled BLAS library from Netlib with compiler options that changed the mangling scheme of Fortran routines.
By default, Netlib's make.inc uses gfortran to compile BLAS:
$ grep FORTRAN make.inc
# Modify the FORTRAN and OPTS definitions to refer to the
FORTRAN = gfortran
It is compiled without any flags:
gfortran -O3 -pipe -c ddot.f -o ddot.o
and you get the ddot() routine:
$ grep -i ddot ddot.o libblas.a
Binary file ddot.o matches
Binary file libblas.a matches
And you can find it with the command line tools:
$ nm ddot.o libblas.a | grep -i ddot
ddot.o:
0000000000000000 T _ddot_
libblas.a(ddot.o):
0000000000000000 T _ddot_
Your example compiles with the library:
cc ex.c libblas.a
or with the ddot.o file:
cc -pipe ex.c ddot.o
I cannot reproduce your problem. You should use the nm and grep commands to find out what happened to the name of the ddot() routine.
PS. Your code has extra semicolon ; after the end of definition of main().

Compiling SQLite for Windows (64-bit)

I have MinGW and I wish to compile the SQLite amalgamation source into a 64-bit dll. I'm fairly new to this sort of compilation and my efforts so far have resulted in failure. (I first started using the autoconf amalgamation and used the configure & make tool on Linux. But apparently that will never work for Windows binaries.)
Anyway, I've been told I need the following preprocessor defines:
Here are the compiler pre-processor defines I use for a 64-bit release build:
WIN64 NDEBUG
_WINDOWS
_USRDLL
NO_TCL
_CRT_SECURE_NO_DEPRECATE
THREADSAFE=1
TEMP_STORE=1
SQLITE_MAX_EXPR_DEPTH=0
Here are the compiler pre-processor defines I use for a 32-bit release build:
WIN32
NDEBUG
_WINDOWS
_USRDLL
NO_TCL
_CRT_SECURE_NO_DEPRECATE
THREADSAFE=1
TEMP_STORE=1
SQLITE_MAX_EXPR_DEPTH=0
I had no idea where to put these in. I eventually took an educated guess, made a new file (for neatness) called sqlite3w64.h and pasted in the following:
#define WIN64 NDEBUG
#define _WINDOWS
#define _USRDLL
#define NO_TCL
#define _CRT_SECURE_NO_DEPRECATE
#define THREADSAFE 1
#define TEMP_STORE 1
#define SQLITE_MAX_EXPR_DEPTH 0
I then compiled the source with the following command:
gcc sqlitew64.h sqlite3.h sqlite3ext.h shell.c sqlite3.c -o sqlite_x64.dll
What resulted was a 733KB DLL file. Nice! Did it actually work? Did it nuts - I got a BadImageFormatException. I also then tried doing an x86 compilation using the same method. Once again, I got a 733KB DLL file (that's odd?) and once again, I got a BadImageFormatException.
Help.
Update
Used the following command instead:
gcc -shared -DWIN64 -DNDEBUG -D_WINDOWS -D_USRDLL -DNO_TCL -D_CRT_SECURE_NO_DEPRECATE -DTHREADSAFE=1 -DTEMP_STORE=1 -DSQLITE_MAX_EXPR_DEPTH=0 -I. shell.c sqlite3.c -o sqlite_x64.dll -Wl,--out-implib,sqlite3.a
Resulted in a 740KB DLL file which still gives a BadImageFormatException.
Final Update
Turns out my MinGW build was 32-bit only. Getting a 64-bit version then allowed me to make SQLite for 64-bit. Adding the flag -m64 sets the compiler into 64-bit mode.
64-bit:
gcc -shared -DWIN64 -DNDEBUG -D_WINDOWS -D_USRDLL -DNO_TCL -D_CRT_SECURE_NO_DEPRECATE -DTHREADSAFE=1 -DTEMP_STORE=1 -DSQLITE_MAX_EXPR_DEPTH=0 -m64 -I. shell.c sqlite3.c -o sqlite3_x64.dll -Wl,--out-implib,sqlite3_x64.a
32-bit:
gcc -shared -DWIN32 -D_WINDOWS -D_USRDLL -DNO_TCL -D_CRT_SECURE_NO_DEPRECATE -DTHREADSAFE=1 -DTEMP_STORE=1 -DSQLITE_MAX_EXPR_DEPTH=0 -m32 -I. shell.c sqlite3.c -o sqlite3_x86.dll -Wl,--out-implib,sqlite3_x86.a
MinGW-64 Precompiled: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds/mingw-w64-bin_i686-mingw_20111220.zip/download?use_mirror=ignum
Installation Instructions: http://code.google.com/p/tonatiuh/wiki/InstallingMinGWForWindows64
You are compiling to an EXE. Calling it a DLL won't magically make it a DLL.
You need to pass special linker options to gcc to make it create DLLs.
Quoted from Mingw site: (Sort of, I replaced g++ with gcc)
gcc -c -DBUILDING_EXAMPLE_DLL example_dll.cpp
gcc -shared -o example_dll.dll example_dll.o -Wl,--out-implib,libexample_dll.a
The page also explains that functions you want your DLL to export, must be declared with __declspec(dllexport). (Further down, there is an example on how to export all global functions to the DLL, like usually happens in Unix.)
The -Wl argument to gcc is what tells gcc to pass on the further arguments --out-implib,libexample_dll.a to the linker.
I would also make 100% sure that the built DLL is actually a 64 bit DLL and not a 32 bit DLL. Do you have any way to check that? On Linux you can run the "file" command.
You can also try adding the -m64 option to the gcc commandline, that should force gcc to target the amd64 target.
If that doesn't work, you may have the wrong compiler altogether. Make sure you have the x86_64/amd64 version of the Mingw toolchain. Installation is as simple as finding the right ZIP, unpacking it, and setting the path.
If all of that fails, or if you just want to verify against a supposedly correctly compiled setup, try precompiled 64-bit binaries here or from here.
What would work in your case is this single link-and-compile command:
g++ -shared
-DWIN64
-DNDEBUG
-D_WINDOWS
-D_USRDLL
-DNO_TCL
-D_CRT_SECURE_NO_DEPRECATE
-DTHREADSAFE=1
-DTEMP_STORE=1
-DSQLITE_MAX_EXPR_DEPTH=0
-I.
shell.c sqlite3.c
-o sqlite_x64.dll
-Wl,--out-implib,libsqllite_x64.dll.a
The compile and link stage will be performed at once. The defined can be added on the commandline. The headers need not be compiled, but you need to pass the current directory as a header search directory, and specify the names of the dll and import file.

Using LLVM linker when using Clang & CMake

What's the best way to tell CMake to use the LLVM linker llvm-link instead of GNU ld as linker? When configuring a project with
CXX=clang++ cmake <args>
the default linker appears to be untouched, remaining usr/bin/ld (on Linux).
Is this possible without using a separate toolchain file?
This turns out to be unrelated to CMake: clang++ uses the system linker by default. For example,
echo "#include <atomic>\n int main() { return 0; }" \
| clang++ -x c++ -std=c++11 -stdlib=libc++ -
uses /usr/bin/ld to link the application. To change the linker to llvm-link, one needs to first emit LLVM byte code, and then call the linker, e.g.:
echo "#include <atomic>\n int main() { return 0; }" \
| clang++ -x c++ -std=c++11 -stdlib=libc++ -S -emit-llvm -o - - \
| llvm-link -o binary -
This bypasses /usr/bin/ld.
As of 3.4, clang looks for the linker (ld) at GCCInstallation.getParentLibPath() + "/../" + GCCInstallation.getTriple().str() + "/bin" before it looks for ld on the path. You should be able to put your linker in /usr/lib/gcc/<arch><sub>-<vendor>-<sys>-<abi>/<version>/ld and have it called by clang in 1 step. To specify this location manually, use the undocumented -B flag. Unfortunately, I don't believe there is a way to alter the name of the linker that is searched for so using ld.gold or lld is going to require a symlink at the aforementioned location.

How to compile a APR test script

Its a long time since I've used C but now I'm trying to compile a short script that gets server-stats from the Apache-Portable-Runtime (APR).
Header files located at /usr/include/apr-1/apr*.h and libs are located at /usr/lib64/libapr-1.*
Source files can be found on the official APR site http://apr.apache.org/docs/apr/1.3/files.html.
/* test.c */
#include <stdio.h>
#include <stdlib.h>
#include <apr_general.h>
#include <apr_time.h>
int main(int argc, const char *argv[])
{
apr_time_t t;
t = apr_time_now();
printf("The current time: %" APR_TIME_T_FMT "[us]\n", t);
return 0;
}
When I try and compile I get the following error (which I believe is a linking issue):
~> gcc -Wall $(apr-1-config --cflags --cppflags --includes --link-ld) test.c -o test.bin
/tmp/cc4DYD2W.o: In function `main':
test.c:(.text+0x10): undefined reference to `apr_time_now'
collect2: ld returned 1 exit status
My environment is gentoo:
~> uname -a
Linux alister 2.6.32.21-grsec-gt-r2 #1 SMP Tue Sep 7 23:54:49 PDT 2010\
x86_64 Intel(R) Xeon(R) CPU L5640 # 2.27GHz GenuineIntel GNU/Linux`
~> gcc -v
gcc version 4.3.4 (Gentoo 4.3.4 p1.1, pie-10.1.5)
~> emerge --search "%#^dev-lib.*apr"
* dev-libs/apr
Latest version installed: 1.3.9
* dev-libs/apr-util
Latest version installed: 1.3.9
Does anyone with more experience with C on Linux have any suggestions for me to get this working?
As always thanks in advance.
gcc -Wall -I/usr/include/apr-1 -L/usr/lib64 -lapr-1 test.c -o test.bin
-l specifies which shared library to link to, while -L specifies where to look for shared libraries.
APR provides a tool to make this easier, apr-1-config. Something like this should work:
gcc -Wall $(apr-1-config --cflags --cppflags --includes --link-ld) test.c -o test.bin
I finally got around to looking into this.
gcc mentions -l twice in different contexts:
Linker Options
object-file-name -llibrary ...
Directory Options
... -Idir -Ldir ...
so I moved the -llib after the object name (to get the 2nd context) and it compiled!
APR_CFG=$(apr-1-config --cflags --cppflags --includes --link-ld)
gcc -Wall test.c -o test.bin $APR_CFG
./test.bin
The current time: 1332999950442660[us]
I'm not fully sure I understand the linking order and why it didn't work before (if someone could shed some light on that it would be fantastic) but for now I have enough to continue.

Resources