I'm trying to learn the glibc source, and I've found navigation to be quite formidable. I'm not referring to the code itself, but simply finding it: it seems to be a maze of macros and wrappers, such that just finding the actual code I want is quite tough.
Not only system dependent things, like setjmp, but even portable functions like fprintf: in each case, I struggle to find the true definition in C. It's easy to find the start, but it's usually an empty shell wrapping defines and macros.
This feels like a modern day equivalent of goto statements, with the spaghetti problem all over again.
How can I navigate glibc and find the actual implementation for lib functions?
Update
As an example, try looking up the definition of hidden_def in glibc. It's a macro taking you to hidden_def1, which is a macro taking you to hidden_def2, which is a macro taking you to hidden_asm, which is a macro taking you to hidden_asm1, at which point...
Moreover, each of these macros is defined in several different files, with other #defines controlling which definition is actually invoked.
This is not unusual: it seems to be de rigeur throughout the source code. How does anyone follow it? How do the GNU developers follow it?
How can I navigate glibc and find the actual implementation for lib functions?
Type the function in the search bar at https://code.woboq.org/userspace/glibc or at https://github.com/bminor/glibc . Navigate the results manually until you find the definition.
If you want to index the project locally, use cscope, ctags, GLOBAL tags or clangd to index the project and then use that tools specific interface to search for the definition.
As an example, try looking up the definition of hidden_def in glibc
Type hidden_def glibc into google. My first hit is woboq.org https://code.woboq.org/userspace/glibc/include/libc-symbols.h.html#550 .
I use firefox. I type ctrl+f and type hidden_def. Type Enter until I find # define hidden_def at https://code.woboq.org/userspace/glibc/include/libc-symbols.h.html#550 .
Then select __hidden_ver1 and type ctrl+c ctrl+f and ctrl+v and search for it. In the web browser. I type enter until I find https://code.woboq.org/userspace/glibc/include/libc-symbols.h.html#540 . __hidden_ver2 is just below on line 542.
For most cases all you need is a browser, google, coboq.org and github.org.
It's a macro taking you to hidden_def1
There are no such macros as you mentioned, at least at the version hosted at woboq.org.
How does anyone follow it?
While IDE is a powerful help, each project is unique and requires different settings, that take time to figure out. Mostly browsing the source code is grep (or faster alternatives, like ag, very useful for big projects like glibc) and going through the result list.
Not only system dependent things, like setjmp
Developers are (shoudl be :) sane people - in most cases a function named setjmp will be in a file named setjmp.c. or setjmp.S. Or in the same directory as setjmp.h. Or inside directory named stdlib or setjmp.
Type setjmp in github search bar. https://github.com/bminor/glibc/search?q=setjmp You see there are multiple definitions for each architecture powerpc s390 etc. But files are all named setjmp. Go back. Type "Go to file" on https://github.com/bminor/glibc . Search for a file named x86/setjmp. There are 3 implementations, the most standard one seems to be https://github.com/bminor/glibc/blob/master/sysdeps/x86_64/setjmp.S .
even portable functions like fprintf
As above, saerch for file named fprintf . You quickly find https://github.com/bminor/glibc/blob/master/stdio-common/fprintf.c .
Related
I tried to use vim for c program editing. Is there a way to auto write function skeleton defined in header file?
situations like
"my_code.h"
int temp(int*);
and "my_code.c"
<<< here auto write >>> like
int temp(int*) { return }
int main()
{
}
I'm using c.vim plug-in. I tried to find it, but I couldn't make it.
There are code completion scripts, yes.. However, this is not something you generally want. It works for simple things like basic C functions, and fails horribly beyond that (i.e. templates etc in c++). You don't save any time by using such plugins, and mastering vim motion/yank/paste commands provide the same result in the same amount of time, and you become more familiar with a modal editor. Is it that hard to copy-paste the function prototype and add some braces {/}?
If you want something to help as a reminder to write function definitions to go with function prototypes, consider using the taglist plugin.
snippets are like the built-in :abbreviate on steroids, usually with parameter insertions, mirroring, and multiple stops inside them. One of the first, very famous (and still widely used) Vim plugins is snipMate (inspired by the TextMate editor); unfortunately, it's not maintained any more; though there is a fork. A modern alternative (that requires Python though) is UltiSnips. There are more, see this list on the Vim Tips Wiki.
There are three things to evaluate: First, the features of the snippet engine itself, second, the quality and breadth of snippets provided by the author or others; third, how easy it is to add new snippets.
Additionally, there are also template plugins that pre-initialize a new, empty file with a skeleton, often including a file header and copyright statement. Search vim.org; you'll find plenty.
I've have a :GOTOIMPL command in lh-cpp that generates an empty function definition from a function declaration.
However, you'll have to execute the command on each function declaration and go back to the header file. I've never took the time to batch the process from an header file and no implementation file -- as this is not a use case I have as there exist other solutions...
IOW... there exist projects that do the job from the command-line (and which you could call from vim then) (like for instance https://github.com/Davidbrcz/header-expander), or even other plugins (like protodef: http://www.vim.org/scripts/script.php?script_id=2624).
I don't know whether it's a stupid question. But here it is: In Sublime Text I haven't found any way to see the definition/documentation of a C/C++ method, no matter it comes from my own project or from system library. "Go To Definition" simply doesn't do anything. This is especially inconvenient when I'm reading some code which uses a bunch of std library stuffs which I don't necessarily recall at first sight, such as strloul, getopt, in which case I have to look them up in terminal or online, inducing overhead.
Thanks in advance.
You should install the SublimeText CTags package. Once installed and you have built your tags, you can put the cursor on the function of interest and (on OS X) type ctrl+t,t to jump to the method, wherever it might be located. Typing ctrl+t,b will bring you back to where you initiated the search from.
Example:
Cursor in the print_matrix function. Press ctrl+t,t
And it jumps to another file with the function definition:
ctrl+t,b takes you back again.
i've been working for some time with an opensource library ("fast artificial neural network"). I'm using it's source in my static library. When i compile it however, i get hundreds of linker warnings which are probably caused by the fact that the library includes it's *.c files in other *.c files (as i'm only including some headers i need and i did not touch the code of the lib itself).
My question: Is there a good reason why the developers of the library used this approach, which is strongly discouraged? (Or at least i've been told all my life that this is bad and from my own experience i believe it IS bad). Or is it just bad design and there is no gain in this approach?
I'm aware of this related question but it does not answer my question. I'm looking for reasons that might justify this.
A bonus question: Is there a way how to fix this without touching the library code too much? I have a lot of work of my own and don't want to create more ;)
As far as I see (grep '#include .*\.c'), they only do this in doublefann.c, fixedfann.c, and floatfann.c, and each time include the reason:
/* Easy way to allow for build of multiple binaries */
This exact use of the preprocessor for simple copy-pasting is indeed the only valid use of including implementation (*.c) files, and relatively rare. (If you want to include some code for another reason, just give it a different name, like *.h or *.inc.) An alternative is to specify configuration in macros given to the compiler (e.g. -DFANN_DOUBLE, -DFANN_FIXED, or -DFANN_FLOAT), but they didn't use this method. (Each approach has drawbacks, so I'm not saying they're necessarily wrong, I'd have to look at that project in depth to determine that.)
They provide makefiles and MSVS projects which should already not link doublefann.o (from doublefann.c) with either fann.o (from fann.c) or fixedfann.o (from fixedfann.c) and so on, and either their files are screwed up or something similar has gone wrong.
Did you try to create a project from scratch (or use your existing project) and add all the files to it? If you did, what is happening is each implementation file is being compiled independently and the resulting object files contain conflicting definitions. This is the standard way to deal with implementation files and many tools assume it. The only possible solution is to fix the project settings to not link these together. (Okay, you could drastically change their source too, but that's not really a solution.)
While you're at it, if you continue without using their project settings, you can likely skip compiling fann.c, et. al. and possibly just removing those from the project is enough – then they won't be compiled and linked. You'll want to choose exactly one of double-/fixed-/floatfann to use, otherwise you'll get the same link errors. (I haven't looked at their instructions, but would not be surprised to see this summary explained a bit more in-depth there.)
Including C/C++ code leads to all the code being stuck together in one translation unit. With a good compiler, this can lead to a massive speed boost (as stuff can be inlined and function calls optimized away).
If actual code is going to be included like this, though, it should have static in most of its declarations, or it will cause the warnings you're seeing.
If you ever declare a single global variable or function in that .c file, it cannot be included in two places which both compile to the same binary, or the two definitions will collide. If it is included in even one place, it cannot also be compiled on its own while still being linked into the same binary as its user.
If the file is only included in one place, why not just make it a discrete compilation unit (and use its globals via extern declarations)? Why bother having it included at all?
If your C files declare no global variables or functions, they are header files and should be named as such.
Therefore, by exhaustive search, I can say that the only time you would ever potentially want to include C files is if the same C code is used in building multiple different binaries. And even there, you're increasing your compile time for no real gain.
This is assuming that functions which should be inlined are marked inline and that you have a decent compiler and linker.
I don't know of a quick way to fix this.
I don't know that library, but as you describe it, it is either bad practice or your understanding of how to use it is not good enough.
A C project that wants to be included by others should always provide well structured .h files for others and then the compiled library for linking. If it wants to include function definitions in header files it should either mark them as static (old fashioned) or as inline (possible since C99).
I haven't looked at the code, but it's possible that the .c or .cpp files being included actually contain code that works in a header. For example, a template or an inline function. If that is the case, then the warnings would be spurious.
I'm doing this at the moment at home because I'm a relative newcomer to C++ on Linux and don't want to get bogged down in difficulties with the linker. But I wouldn't recommend it for proper work.
(I also once had to include a header.dat into a C++ program, because Rational Rose didn't allow headers to be part of the issued software and we needed that particular source file on the running system (for arcane reasons).)
I am using glib in my application, and I see there are convenience wrappers in glib for C's remove, unlink and rmdir. But these only work on a single file or directory at a time.
As far as I can see, neither the C standard nor glib include any sort of recursive directory walk functionality. Nor do I see any specific way to delete an entire directory tree at once, as with rm -rf.
For what I'm doing this I'm not worried about any complications like permissions, symlinks back up the tree (infinite recursion), or anything that would rule out a very naive
implementation... so I am not averse to writing my own function for it.
However, I'm curious if this functionality is out there somewhere in the standard libraries gtk or glib (or in some other easily reused C library) already and I just haven't stumbled on it. Googling this topic generates a lot of false leads.
Otherwise my plan is to use this type of algorithm:
dir_walk(char* path, void* callback(char*) {
if(is_dir(path) && has_entries(path)) {
entries = get_entries(path);
for(entry in intries) { dir_walk(entry, callback); }
}
else { callback(path) }
}
dir_walk("/home/user/trash", remove);
Obviously I would build in some error handling and the like to abort the process as soon as a fatal error is encountered.
Have you looked at <dirent.h>? AFAIK this belongs to the POSIX specification, which should be part of the standard library of most, if not all C compilers. See e.g. this <dirent.h> reference (Single UNIX specification Version 2 by the Open Group).
P.S., before someone comments on this: No, this does not offer recursive directory traversal. But then I think this is best implemented by the developer; requirements can differ quite a lot, so one-size-fits-all recursive traversal function would have to be very powerful. (E.g.: Are symlinks followed up? Should recursion depth be limited? etc.)
You can use GFileEnumerator if you want to do it with glib.
Several platforms include ftw and nftw: "(new) file tree walk". Checking the man page on an imac shows that these are legacy, and new users should prefer fts. Portability may be an issue with either of these choices.
Standard C libraries are meant to provide primitive functionality. What you are talking about is composite behavior. You can easily implement it using the low level features present in your API of choice -- take a look at this tutorial.
Note that the "convenience wrappers" you mention for remove(), unlink() and rmdir(), assuming you mean the ones declared in <glib/gstdio.h>, are not really "convenience wrappers". What is the convenience in prefixing totally standard functions with a "g_"? (And note that I say this even if I who introduced them in the first place.)
The only reason these wrappers exist is for file name issues on Windows, where these wrappers actually consist of real code; they take file name arguments in Unicode, encoded in UTF-8. The corresponding "unwrapped" Microsoft C library functions take file names in system codepage.
If you aren't specifically writing code intended to be portable to Windows, there is no reason to use the g_remove() etc wrappers.
Shouldn't be hard, right? Right?
I am currently trawling the OpenAFS codebase to find the header definition of pioctl. I've thrown everything I've got at it: checked ctags, grepped the source code for pioctl, etc. The closest I've got to a lead is the fact that there's a file pioctl_nt.h that contains the definition, except it's not actually what I want because none of the userspace code directly includes it, and it's Windows specific.
Now, I'm not expecting you to go and download the OpenAFS codebase and find the header file for me. I am curious, though: what are your techniques for finding the header file you need when everything else fails? What are the worst case scenarios that could cause a grep for pioctl in the codebase to not actually come up with anything that looks like a function definition?
I should also note that I have access to two independent userspace programs that have done it properly, so in theory I could do an O(n) search for the function. But none of the header files pop out to me, and n is large...
Edit: The immediate issue has been resolved: pioctl() is defined implicitly, as shown by this:
AFS.xs:2796: error: implicit declaration of function ‘pioctl’
If grep -r and ctags are failing, then it's probably being defined as the result of some nasty macro(s). You can try making the simplest possible file that calls pioctl() and compiles successfully, and then preprocessing it to see what happens:
gcc -E test.c -o test.i
grep pioctl -C10 test.i
There are compiler options to show the preprocessor output. Try those? In a horrible pinch where my head was completely empty of any possible definition the -E option (in most c compilers) does nothing but spew out the the preprocessed code.
Per requested information: Normally I just capture a compile of the file in question as it is output on the screen do a quick copy and paste and put the -E right after the compiler invocation. The result will spew preprocessor output to the screen so redirect it to a file. Look through that file as all of the macros and silly things are already taken care of.
Worst case scenarios:
K&R style prototypes
Macros are hiding the definition
Implicit Declaration (per your answer)
Have you considered using cscope (available from SourceForge)?
I use it on some fairly significant code sets (25,000+ files, ranging up to about 20,000 lines in a file) with good success. It takes a while to derive the file list (5-10 minutes) and longer (20-30 minutes) to build the cross-reference on an ancient Sun E450, but I find the results useful.
On an almost equally ancient Mac (dual 1GHz PPC 32-bit processors), cscope run on the OpenAFS (1.5.59) source code comes up with quite a lot of places where the function is declared, sometimes inline in code, sometimes in headers. It took a few minutes to scan the 4949 files, generating a 58 MB cscope.out file.
openafs-1.5.59/src/sys/sys_prototypes.h
openafs-1.5.59/src/aklog/aklog_main.c (along with comment "Why doesn't AFS provide these prototypes?")
openafs-1.5.59/src/sys/pioctl_nt.h
openafs-1.5.59/src/auth/ktc.c includes a define for PIOCTL
openafs-1.5.59/src/sys/pioctl_nt.c provides an implementation of it
openafs-1.5.59/src/sys/rmtsysc.c provides an implementation of it (and sometimes afs_pioctl() instead)
The rest of the 184 instances found seem to be uses of the function, or documentation references, or release notes, change logs, and the like.
The current working theory that we've decided on, after poking at the preprocessor and not finding anything either, is that OpenAFS is letting the compiler infer the prototype of the function, since it returns an integer and takes pointer, integer, pointer, integer as its parameters. I'll be dealing with this by merely defining it myself.
Edit: Excellent! I've found the smoking gun:
AFS.xs:2796: error: implicit declaration of function ‘pioctl’
While the original general question has been answered, if anyone arrives at this page wondering where to find a header file that defines pioctl:
In current releases of OpenAFS (1.6.7), a protoype for pioctl is defined in sys_prototypes.h. But that the time that this question was originally asked, that file did not exist, and there was no prototype for pioctl visible from outside the OpenAFS code tree.
However, most users of pioctl probably want, or are at least okay with using, lpioctl ("local" pioctl), which always issues a syscall on the local machine. There is a prototype for this in afssyscalls.h (and these days, also sys_prototypes.h).
The easiest option these days, though, is just to use libkopenafs. For that, include kopenafs.h, use the function k_pioctl, and link against -lkopenafs. That tends to be a much more convenient interface than trying to link with OpenAFS libsys and other stuff.
Doesn't it usually say in the man page synopsis?