I was inspecting the code of a linux application and i saw the #include in one of the code files. I tried looking it up on opengroup.org but i couldn't find it there, this is what the sys directory looks like: http://www.opengroup.org/onlinepubs/000095399/basedefs/sys/ . I guess it's not standard header file, but i checked it in my /usr/include/sys and it was there.
What does it do and what it is used for ? If you can provide me with some manual for it, i would be grateful. Thanks.
Used in conjunction with ptrace(2) (see PTRACE_PEEKUSER): http://linux.die.net/man/2/ptrace
The comment at the top of the header pretty much says it all:
#ifndef _SYS_USER_H
#define _SYS_USER_H 1
/* The whole purpose of this file is for GDB and GDB only. Don't read
too much into it. Don't use it for anything other than GDB unless
you know what you are doing. */
GNU specific extensions are usually pretty easy to identify (e.g. _GNU_SOURCE). However, debugging and instrumentation has to work even if those extensions aren't turned on. For instance, people want to use GDB on code that does not #define _GNU_SOURCE.
In that case, stuff that is not defined in ISO C (and not required by POSIX) is usually clearly labeled as such.
You'll also find all kinds of strange looking symbols in programs that include the Valgrind headers.
Related
Specifically, my issue is that I have CUDA code that needs <curand_kernel.h> to run. This isn't included by default in NVRTC. Presumably then when creating the program context (i.e. the call to nvrtcCreateProgram), I have to send in the name of the file (curand_kernel.h) and also the source code of curand_kernel.h? I feel like I shouldn't have to do that.
It's hard to tell; I haven't managed to find an example from NVIDIA of someone needing standard CUDA files like this as a source, so I really don't understand what the syntax is. Some issues: curand_kernel.h also has includes... Do I have to do the same for each of these? I am not even sure the NVRTC compiler will even run correctly on curand_kernel.h, because there are some language features it doesn't support, aren't there?
Next: if you've sent in the source code of a header file to nvrtcCreateProgram, do I still have to #include it in the code to be executed / will it cause an error if I do so?
A link to example code that does this or something like it would be appreciated much more than a straightforward answer; I really haven't managed to find any.
You have to send the "filename" and the source of each header separately.
When the preprocessor does its thing, it'll use any #include filenames as a key to find the source for the header, based on the collection that you provide.
I suspect that, in this case, the compiler (driver) doesn't have file system access, so you have to give it the source in much the same way that you would for shader includes in OpenGL.
So:
Include your header's name when calling nvrtcCreateProgram. The compiler will, internally, generate the equivalent of a std::map<string,string> containing the source of each header indexed by the given name.
In your kernel source, use #include "foo.cuh" as usual.
The compiler will use foo.cuh as an index or key into its internal map (created when you called nvrtcCreateProgram), and will retrieve the header source from that collection
Compilation proceeds as normal.
One of the reasons that nvrtc provides only a "subset" of features is that the compiler plays in a somewhat sandboxed environment, without necessarily having all of the supporting tools and utilities lying around that you have with offline compilation. So, you have to manually handle a lot of the stuff that the normal nvcc + (gcc | MSVC| clang) combination provides.
A possible, but non-ideal, solution would be to preprocess the file that you need in your IDE, save the result and then #include that. However, I bet there is a better way to do that. if you just want curand, consider diving into the library and extracting the part you need (blech) or using another GPU-friendly rand implementation. On older CUDA versions, I just generated a big array of random floats on the host, uploaded it to the GPU, and sampled it in the kernels.
This related link may be helpful.
You do not need to load curand_kernel.h yourself and add it to the include "aliases" mechanism.
Instead, you can simply add the CUDA include directory to your (set of) include paths, e.g. by adding --include-path=/usr/local/cuda/include to your NVRTC compiler options.
(I do this in my GPU-kernel-runner test harness, by default, to be on the safe side.)
I wonder what is inside stdio.h and conio.h etc.
I want to know how printf and scanf are are defined.
Is there a way I can open stdio.h and see what is written inside?
Depending on your implementation, you should be able to open any .h file in your favorite editor and read it directly; they're (usually) just plain text files.
However, stdio.h will only give you the declarations for printf and scanf; it won't contain the source code for them. Most compilers don't ship the source code for standard library functions; instead, they ship precompiled libraries which are linked with your code when you build the executable.
If you're willing to spend some money, P.J Plauger's The Standard C Library is a good resource that shows an implementation of the standard library functions.
When the preprocessor includes a header file into a source file, that inclusion is very much literal. That means that the header files are normal text files with source in them, and must be readable by the compiler (and therefore by you). You just have to find where they are, and you can open them like any other text file.
However, you won't find out how functions are defined, just how they are declared. And some structures are supposed to be "black boxes", whose data members should be considered private. Usually the source for the standard C library is available or downloadable, so try and find that too. It all depends on what compiler you're using.
You might also want to check out a reference site such as this one. There you can find pretty detailed information about e.g. printf.
Those headers generally chain include more machine/OS specific headers.
If you are on Linux/OS X then you can get some more info with
man stdio
Also check out http://www.cplusplus.com/reference/cstdio/ https://en.wikipedia.org/wiki/Conio.h
Most compilers allow you to read the results after the preprocessor (the compilation step that processes the #include directives) has been run. With gcc for instance, use the -E command-line option.
You can always rely on the Internet's supply of Unix-style manual pages, by searching for "man something" you can look for the relevant manual section for something.
For instance, there are pages for both printf() and scanf().
You can easily see there that the declarations aren't very special, and quite obvious from the usage. It's just int printf(const char *format, ...); for instance.
the content of some headers is defined by the C-Standard.
other headers are defined by the library that provides it.
Some headers are defined from the system for that you are writing the code (may fall into the second case since the OS provides the libs)
depending on that you may look into c language reference or you may look into the libraries manual or in the OS's API reference.
But one thin is for sure. if you can include a header (and the compiler does not complain that he could not find it) than you also can look into it. just look into the standard include directories of the compiler or the additional include directories that are specified in project file ore Makefile to find the files on your file system.
But usually the better way is to look in the Documentation because the header itself may be difficult to read because of many #ifdefs and further includes
The most fundamental way to find out what's inside those headers is to read them. Of course, you must locate them first. To this end you can use this short shell code:
gcc -E -M - << EOF
#include <stdio.h>
EOF
This will provide you with a complete list of all the headers directly or indirectly included by #include <stdio.h>. Of course, if you are only interested in the 'stdio.h' header itself, you can just do
locate stdio.h
but this will usually list quite a few false positives.
i noticed that mingw adds alot of code before calling main(), i assumed its for parsing command line parameters since one of those functions is called __getmainargs(), and also lots of strings are added to the final executable, such as mingwm.dll and some error strings (incase the app crashed) says mingw runtime error or something like that.
my question is: is there a way to remove all this stuff? i dont need all these things, i tried tcc (tiny c compiler) it did the job. but not cross platform like gcc (solaris/mac)
any ideas?
thanks.
Yes, you really do need all those things. They're the startup and teardown code for the C environment that your code runs in.
Other than non-hosted environments such as low-level embedded solutions, you'll find pretty much all C environments have something like that. Things like /lib/crt0.o under some UNIX-like operating systems or crt0.obj under Windows.
They are vital to successful running of your code. You can freely omit library functions that you don't use (printf, abs and so on) but the startup code is needed.
Some of the things that it may perform are initialisation of atexit structures, argument parsing, initialisation of structures for the C runtime library, initialisation of C/C++ pre-main values and so forth.
It's highly OS-specific and, if there are things you don't want to do, you'll probably have to get the source code for it and take them out, in essence providing your own cut-down replacement for the object file.
You can safely assume that your toolchain does not include code that is not needed and could safely be left out.
Make sure you compiled without debug information, and run strip on the resulting executable. Anything more intrusive than that requires intimate knowledge of your toolchain, and can result in rather strange behaviour that will be hard to debug - i.e., if you have to ask how it could be done, you shouldn't try to do it.
Under the /usr/include directory in Linux i entered the command:
find -type f -name unistd.h which gave the following output:
./unistd.h
./linux/unistd.h
./asm-generic/unistd.h
./bits/unistd.h
./asm/unistd.h
./sys/unistd.h
my question is, what is the purpose of each unistd.h, since there is only one definiton of that file in the single unix specification ?
Thanks in advance.
linux/unistd.h actually points to asm/unistd.h, which in turn points to either asm/unistd_32.h or asm/unistd_64.h, which is where system call numbers are defined and presented to user space depending on the system's architecture. These come from the kernel.
bits/unistd.h is a collection of macros that augment unistd.h (mostly stuff to help prevent buffer overflows), which is conditionally included via:
/* Define some macros helping to catch buffer overflows. */
#if __USE_FORTIFY_LEVEL > 0 && defined __extern_always_inline
# include <bits/unistd.h>
#endif
In essence, the only POSIX required header is in fact, just unistd.h, the rest are either extensions, or definitions from the kernel. So, just including unistd.h is all you have to worry about doing, everything you need will be pulled in depending on your architecture and whatever build options you've selected.
It's a common technique in C and C++ - you have a single file with the "standard" name in the "standard" place, in this case ./unistd.h, and then have that file include one or more implementation specific files, depending on preprocessor macros. If you look at almost any "standard" C or C++ header files, you will see it including other files not mentioned in any standard.
Basically think of /usr/include/unistd.h as a smart symbolic link. It will point to a correct implementation depending on what your operating conditions are.
That said, it makes difficult sometimes to figure out what that correct implementation is.
Shouldn't be hard, right? Right?
I am currently trawling the OpenAFS codebase to find the header definition of pioctl. I've thrown everything I've got at it: checked ctags, grepped the source code for pioctl, etc. The closest I've got to a lead is the fact that there's a file pioctl_nt.h that contains the definition, except it's not actually what I want because none of the userspace code directly includes it, and it's Windows specific.
Now, I'm not expecting you to go and download the OpenAFS codebase and find the header file for me. I am curious, though: what are your techniques for finding the header file you need when everything else fails? What are the worst case scenarios that could cause a grep for pioctl in the codebase to not actually come up with anything that looks like a function definition?
I should also note that I have access to two independent userspace programs that have done it properly, so in theory I could do an O(n) search for the function. But none of the header files pop out to me, and n is large...
Edit: The immediate issue has been resolved: pioctl() is defined implicitly, as shown by this:
AFS.xs:2796: error: implicit declaration of function ‘pioctl’
If grep -r and ctags are failing, then it's probably being defined as the result of some nasty macro(s). You can try making the simplest possible file that calls pioctl() and compiles successfully, and then preprocessing it to see what happens:
gcc -E test.c -o test.i
grep pioctl -C10 test.i
There are compiler options to show the preprocessor output. Try those? In a horrible pinch where my head was completely empty of any possible definition the -E option (in most c compilers) does nothing but spew out the the preprocessed code.
Per requested information: Normally I just capture a compile of the file in question as it is output on the screen do a quick copy and paste and put the -E right after the compiler invocation. The result will spew preprocessor output to the screen so redirect it to a file. Look through that file as all of the macros and silly things are already taken care of.
Worst case scenarios:
K&R style prototypes
Macros are hiding the definition
Implicit Declaration (per your answer)
Have you considered using cscope (available from SourceForge)?
I use it on some fairly significant code sets (25,000+ files, ranging up to about 20,000 lines in a file) with good success. It takes a while to derive the file list (5-10 minutes) and longer (20-30 minutes) to build the cross-reference on an ancient Sun E450, but I find the results useful.
On an almost equally ancient Mac (dual 1GHz PPC 32-bit processors), cscope run on the OpenAFS (1.5.59) source code comes up with quite a lot of places where the function is declared, sometimes inline in code, sometimes in headers. It took a few minutes to scan the 4949 files, generating a 58 MB cscope.out file.
openafs-1.5.59/src/sys/sys_prototypes.h
openafs-1.5.59/src/aklog/aklog_main.c (along with comment "Why doesn't AFS provide these prototypes?")
openafs-1.5.59/src/sys/pioctl_nt.h
openafs-1.5.59/src/auth/ktc.c includes a define for PIOCTL
openafs-1.5.59/src/sys/pioctl_nt.c provides an implementation of it
openafs-1.5.59/src/sys/rmtsysc.c provides an implementation of it (and sometimes afs_pioctl() instead)
The rest of the 184 instances found seem to be uses of the function, or documentation references, or release notes, change logs, and the like.
The current working theory that we've decided on, after poking at the preprocessor and not finding anything either, is that OpenAFS is letting the compiler infer the prototype of the function, since it returns an integer and takes pointer, integer, pointer, integer as its parameters. I'll be dealing with this by merely defining it myself.
Edit: Excellent! I've found the smoking gun:
AFS.xs:2796: error: implicit declaration of function ‘pioctl’
While the original general question has been answered, if anyone arrives at this page wondering where to find a header file that defines pioctl:
In current releases of OpenAFS (1.6.7), a protoype for pioctl is defined in sys_prototypes.h. But that the time that this question was originally asked, that file did not exist, and there was no prototype for pioctl visible from outside the OpenAFS code tree.
However, most users of pioctl probably want, or are at least okay with using, lpioctl ("local" pioctl), which always issues a syscall on the local machine. There is a prototype for this in afssyscalls.h (and these days, also sys_prototypes.h).
The easiest option these days, though, is just to use libkopenafs. For that, include kopenafs.h, use the function k_pioctl, and link against -lkopenafs. That tends to be a much more convenient interface than trying to link with OpenAFS libsys and other stuff.
Doesn't it usually say in the man page synopsis?