I am writing a C (shared) library. It started out as a single translation unit, in which I could define a couple of static global variables, to be hidden from external modules.
Now that the library has grown, I want to break the module into a couple of smaller source files. The problem is that now I have two options for the mentioned globals:
Have private copies at each source file and somehow sync their values via function calls - this will get very ugly very fast.
Remove the static definition, so the variables are shared across all translation units using extern - but now application code that is linked against the library can access these globals, if the required declaration is made there.
So, is there a neat way for making private global variable shared across multiple, specific translation units?
You want the visibility attribute extension of GCC.
Practically, something like:
#define MODULE_VISIBILITY __attribute__ ((visibility ("hidden")))
#define PUBLIC_VISIBILITY __attribute__ ((visibility ("default")))
(You probably want to #ifdef the above macros, using some configuration tricks à la autoconfand other autotools; on other systems you would just have empty definitions like #define PUBLIC_VISIBILITY /*empty*/ etc...)
Then, declare a variable:
int module_var MODULE_VISIBILITY;
or a function
void module_function (int) MODULE_VISIBILITY;
Then you can use module_var or call module_function inside your shared library, but not outside.
See also the -fvisibility code generation option of GCC.
BTW, you could also compile your whole library with -Dsomeglobal=alongname3419a6 and use someglobal as usual; to really find it your user would need to pass the same preprocessor definition to the compiler, and you can make the name alongname3419a6 random and improbable enough to make the collision improbable.
PS. This visibility is specific to GCC (and probably to ELF shared libraries such as those on Linux). It won't work without GCC or outside of shared libraries.... so is quite Linux specific (even if some few other systems, perhaps Solaris with GCC, have it). Probably some other compilers (clang from LLVM) might support also that on Linux for shared libraries (not static ones). Actually, the real hiding (to the several compilation units of a single shared library) is done mostly by the linker (because the ELF shared libraries permit that).
The easiest ("old-school") solution is to simply not declare the variable in the intended public header.
Split your libraries header into "header.h" and "header-internal.h", and declare internal stuff in the latter one.
Of course, you should also take care to protect your library-global variable's name so that it doesn't collide with user code; presumably you already have a prefix that you use for the functions for this purpose.
You can also wrap the variable(s) in a struct, to make it cleaner since then only one actual symbol is globally visible.
You can obfuscate things with disguised structs, if you really want to hide the information as best as possible. e.g. in a header file,
struct data_s {
void *v;
};
And somewhere in your source:
struct data_s data;
struct gbs {
// declare all your globals here
} gbss;
and then:
data.v = &gbss;
You can then access all the globals via: ((struct gbs *)data.v)->
I know that this will not be what you literally intended, but you can leave the global variables static and divide them into multiple source files.
Copy the functions that write to the corresponding static variable in the same source file also declared static.
Declare functions that read the static variable so that external source files of the same module can read it's value.
In a way making it less global. If possible, best logic for breaking big files into smaller ones, is to make that decision based on the data.
If it is not possible to do it this way than you can bump all the global variables into one source file as static and access them from the other source files of the module by functions, making it official so if someone is manipulating your global variables at least you know how.
But then it probably is better to use #unwind's method.
Related
This question already has answers here:
C: Why do we include header files, which declare but don't define?
(5 answers)
Closed 4 years ago.
This is the question I still cannot answer by myself (probably due to lack of experience in C).
So I found this answer and the question is what's wrong with putting definitions into a header files? I we don't declare them as static it should be fine since they will have external linkage and linker will not complain.
Secondly, why should not we put static definition into a header file? Suppose we want to make some function sort of "compilation-unit private" that is not intended to have external linkage.
And what I was surprised about was why is it (rarely) possible to put inline function definition into header files?
Is it just by convention only?
There is not usually any point in putting information into a header unless it will be included in multiple other source files. That means that if the header defines some variables, each source file that includes the header will define those variables. And it is normally an error to define the same (global) variable more than once. So, putting variable definitions into a header normally defeats the reason for creating a header in the first place.
Note that C does have a 'one definition rule', similar to C++, but not as strong. There is a 'common' extension — complete with double entendre — that often allows you to get away with defining a variable in more than one file, but it doesn't work if the variable has a non-trivial initialization.
You can find a lot of information in How do I use extern to share variables between source files?, including information about the common extension.
You should not normally define static variables in a header because that means that each file that includes the header will define its own copy of the variables. This is usually not what's wanted. It tends to make programs bigger.
If you reliably work with compilers that support inline, there is no particular reason why you can't put static inline function definitions into a header. There is a risk that if the function cannot be inlined, then the compiler will generate a static function in the object code for each source file that uses the function, again increasing the size of the executable. You might be able to avoid that by using non-static inline function definitions. If the functions are small enough that they will be inlined, there is no particular reason not to put them in a header.
You can find more information about inline functions, with static or extern, in the Q&A on Stack Overflow:
extern inline
Is inline without static or extern ever useful in C99
What's the difference between static and static inline functions?
Note the rather extensive use of weasel words such as 'usually' and 'normally' in the answer above. You can, with care, find exceptional situations that warrant breaking the rules, but if you remember that headers are the glue that provide cross-checking between the code that uses some functionality (types, constants, functions, sometimes variables) and the code that implements the functionality, then you'll realize that it is good to follow these rules:
Headers declare types, constants, enumerations, functions, variables that are used by multiple source files.
Source files implement functions and define variables that are used by multiple source files.
The header files are used by the implementation code to ensure that the implementation matches the specification.
The header files are used by the consumer code to follow the source-level rules for the implementation.
Header files should not define variables.
Source files should not declare external variables; they should include the relevant header.
Headers may sensibly define static inline functions; you may even include plain inline function definitions in a header, but you have to be careful to instantiate the functions somewhere in case they are not inlined — and that's when extern inline comes into play. (Be aware that old GCC rules for inline functions are different from C99 standard rules.)
There are some rules that simply including a header won't enforce, such as required sequences for calling functions (don't call a free function before the corresponding allocate function, for example). But using headers wisely prevents a lot of errors.
See also:
Should I use #include in headers?
How to link multiple implementation files in C?
And many other questions, no doubt.
So I'm working on a "quick and dirty" profiler for firmware- I just need to know how long some functions take. Merely printing the time it takes every time will skew the results, as logging is expensive- so I am saving a bunch of results to an array and dumping that after some time.
When working in one compilation unit (one source file), I just had a bunch of static arrays storing the results. Now I need to do this across several files. I could "copy paste" the code, but that would be just ugly (Bear with me). If I put timing code in a seperate compilation unit, make static variables, and provide accessor functions in the header file, I will be incurring the overhead of function calls every time i want to access those static variables.
Is it possible to access static variables of a compilation unit directly?
I've always tried to encapsulate data, and not use global variables, but this situation calls for it simply due to speed concerns.
I hope this makes sense! Thank you!
EDIT: Alright, so it appears what I'm asking is impossible- do any of you see alternatives that essentially allow me to directly access data of another compilation unit?
EDIT2: Thank you for the answers Pablo and Jonathan. I ended up accepting Pablo's because I didn't have clear place to get the pointer to the static data (as per Jonathan) in my situation. Thanks again!
No, it's not possible to access static variables of a compilation unit from another one. static keyword precisely prevents that from happening.
If you need to access globals of one compilation unit from another, you can do:
file1.c:
int var_from_file1 = 10;
file2.c:
extern int var_from_file1;
// you can access var_from_file1 here
If you can remove the static keyword from your declarations, you should be fine. I understand that changing existing source code is not always an option (I.E. dealing with existing legacy compiled code).
To get at the static variables in a compilation unit C1 from another unit C2, some function in C1 must make pointers to the variables available to C2, or some non-static variable must contain a pointer to the static variables.
So, you could package the 'static variables' into a single structure, and then write a function that returns a pointer to that structure; you can call that function to gain access to the static variables.
Similar rules apply to static functions; if some function (or non-static variable) in the file makes the pointers to the functions available, then the static functions can be called indirectly from outside the file.
If access via pointers doesn't count as directly, then you are snookered; static hides and you can't unhide except by removing the keyword static from the variables when the module is compiled - maybe via the C preprocessor. Beware name clashes.
I have an interface with which I want to be able to statically link modules. For example, I want to be able to call all functions (albeit in seperate files) called FOO or that match a certain prototype, ultimately make a call into a function in the file without a header in the other files. Dont say that it is impossible since I found a hack that can do it, but I want a non hacked method. (The hack is to use nm to get functions and their prototypes then I can dynamically call the function). Also, I know you can do this with dynamic linking, however, I want to statically link the files. Any ideas?
Put a table of all functions into each translation unit:
struct functions MOD1FUNCS[]={
{"FOO", foo},
{"BAR", bar},
{0, 0}
};
Then put a table into the main program listing all these tables:
struct functions* ALLFUNCS[]={
MOD1FUNCS,
MOD2FUNCS,
0
};
Then, at run time, search through the tables, and lookup the corresponding function pointer.
This is somewhat common in writing test code. e.g., you want to call all functions that start with test_. So you have a shell script that grep's through all your .C files and pulls out the function names that match test_.*. Then that script generates a test.c file that contains a function that calls all the test functions.
e.g., generated program would look like:
int main() {
initTestCode();
testA();
testB();
testC();
}
Another way to do it would be to use some linker tricks. This is what the Linux kernel does for its initialization. Functions that are init code are marked with the qualifier __init. This is defined in linux/init.h as follows:
#define __init __section(.init.text) __cold notrace
This causes the linker to put that function in the section .init.text. The kernel will reclaim memory from that section after the system boots.
For calling the functions, each module will declare an initcall function with some other macros core_initcall(func), arch_initcall(func), et cetera (also defined in linux/init.h). These macros put a pointer to the function into a linker section called .initcall.
At boot-time, the kernel will "walk" through the .initcall section calling all of the pointers there. The code that walks through looks like this:
extern initcall_t __initcall_start[], __initcall_end[], __early_initcall_end[];
static void __init do_initcalls(void)
{
initcall_t *fn;
for (fn = __early_initcall_end; fn < __initcall_end; fn++)
do_one_initcall(*fn);
/* Make sure there is no pending stuff from the initcall sequence */
flush_scheduled_work();
}
The symbols __initcall_start, __initcall_end, etc. get defined in the linker script.
In general, the Linux kernel does some of the cleverest tricks with the GCC pre-processor, compiler and linker that are possible. It's always been a great reference for C tricks.
You really need static linking and, at the same time, to select all matching functions at runtime, right? Because the latter is a typical case for dynamic linking, i'd say.
You obviusly need some mechanism to register the available functions. Dynamic linking would provide just this.
I really don't think you can do it. C isn't exactly capable of late-binding or the sort of introspection you seem to be requiring.
Although I don't really understand your question. Do you want the features of dynamically linked libraries while statically linking? Because that doesn't make sense to me... to static link, you need to already have the binary in hand, which would make dynamic loading of functions a waste of time, even if you could easily do it.
I wonder about the use of the static keyword as scope limiting for variables in a file, in C.
The standard way to build a C program as I see it is to:
have a bunch of c files defining functions and variables, possibly scope limited with static.
have a bunch of h files declaring the functions and possibly variables of the corresponding c file, for other c files to use. Private functions and variables are not published in the h file.
every c file is compiled separately to an o file.
all o files are linked together to an application file.
I see two reasons for declaring a gobal as static, if the variable is not published in the h file anyway:
one is for readability. Inform future readers including myself that a variable is not accessed in any other file.
the second is to prevent another c file from redeclaring the variable as extern. I suppose that the linker would dislike a variable being both extern and static. (I dislike the idea of a file redeclaring a variable owned by someone else as extern, is it ok practice?)
Any other reason?
Same goes for static functions. If the prototype is not published in the h file, other files may not use the function anyway, so why define it static at all?
I can see the same two reasons, but no more.
When you talk about informing other readers, consider the compiler itself as a reader. If a variable is declared static, that can affect the degree to which optimizations kick in.
Redefining a static variable as extern is impossible, but the compiler will (as usual) give you enough rope to hang yourself.
If I write static int foo; in one file and int foo; in another, they are considered different variables, despite having the same name and type - the compiler will not complain but you will probably get very confused later trying to read and/or debug the code. (If I write extern int foo; in the second case, that will fail to link unless I declare a non-static int foo; somewhere else.)
Global variables rarely appear in header files, but when they do they should be declared extern. If not, depending on your compiler, you risk that every source file which includes that header will declare its own copy of the variable: at best this will cause a link failure (multiply-defined symbol) and at worst several confusing cases of overshadowing.
By declaring a variable static on file level (static within function has a different meaning) you forbid other units to access it, e.g. if you try to the variable use inside another unit (declared with extern), linker won't find this symbol.
When you declare a static function the call to the function is a "near call" and in theory it performs better than a "far call". You can google for more information. This is what I found with a simple google search.
If a global variable is declared static, the compiler can sometimes make better optimizations than if it were not. Because the compiler knows that the variable cannot be accessed from other source files, it can make better deductions about what your code is doing (such as "this function does not modify this variable"), which can sometimes cause it to generate faster code. Very few compilers/linkers can make these sorts of optimizations across different translation units.
If you declare a variable foo in file a.c without making it static, and a variable foo in file b.c without making it static, both are automatically extern which means the linker may complain if you initialise both, and assign the same memory location if it doesn't complain. Expect fun debugging your code.
If you write a function foo () in file a.c without making it static, and a function foo () in file b.c without making it static, the linker may complain, but if it doesn't, all calls to foo () will call the same function. Expect fun debugging your code.
My favorite usage of static is being able to store methods that I wont have to Inject or create an object to use, the way I see it is, Private Static Methods are always useful, where public static you have to put some more time in thinking of what it is your doing to avoid what crazyscot defined as, getting your self too much rope and accidentally hanging ones self!
I like to keep a folder for Helper classes for most of my projects that mainly consist of static methods to do things quickly and efficiently on the fly, no objects needed!
Is there any way to keep global variables visible only from inside a library while inaccessible from programs that access that library in C?
It's not that it is vital to keep the variable protected, but I would rather it if programs couldn't import it as it is nothing of their business.
I don't care about solutions involving macros.
If you use g++, you can use the linker facilities for that using attributes.
__attribute__((visibility("hidden"))) int whatever;
You can also mark everything as hidden and mark explicitly what is visible with this flag: -fvisibility=hidden
And then mark the visible variables with:
__attribute__((visibility("default"))) int whatever;
static int somelocalvar = 0;
that makes somelocalvar visible only from whithin the source file where it is declared (reference and example).
Inside the library implementation, declare your variables like that:
struct my_lib_variables
{
int var1;
char var2;
};
Now in the header for end-users, declare it like that:
struct my_lib_variables;
It declares the structure as an incomplete type. People who will use the header will be able to create a pointer to the struct, but that's all. The goal is that they have to write something like that:
#include "my_lib.h"
struct my_lib_variables* p = my_lib_init();
my_lib_do_something(p);
my_lib_destroy(p);
The libray code is able to modify the variables, but the library can't do it directly.
Or you can use global variables, but put the extern declarations inside a header which will not be used by the end-user.
You can use another header file for exporting functionality to outside modules than you have for the internal functionality and thus you don't have to declare globals that doesn't have to be accessible from outside the module.
Edit:
There is only linker problems if you declare things more than once. There is no need to keep all global data in one header file, in fact, there may be a wise reason top split it up into several smaller pieces for maintainability and different areas of responisiblity. Splitting up into header files for external data and internal data is one such reason and this should not be a problem since it is possible to include more than one header file into the same source file. And don't forget the guards in the header files, this way, collision in linking is mostly avoided.
#ifndef XXX_HEADER_FILE
#define XXX_HEADER_FILE
code
#endif