I am new to embedded systems and want to learn more,
I am currently optimizing a software with regards on the footprint for an ARM embedded system, and are wondering, the header files that you include in your source files. Where are they put?
Right now I am just using a software (OVP) to simulate the ARM hardware platform but in real hardware, you have to put the header files somewhere right? Like in gcc have the standard library on the hd. Do we have to insert this library in the embedded machine as well? Space is limited! And is there any way to minimize the size of the library? Thanks!
Example
#include <stdio.h>
#include <stdlib.h>
I am using the cross compiler arm-elf-gcc
Best Regards
Mr Gigu
You appear to possess a few fundamental misunderstandings about compiled executable code. The following applies to embedded and desktop systems.
Header files are no more than sourcefiles like any other. The difference is that they are inserted into the compilation unit by the pre-processor rather than compiled directly. Also in most cases they contain declarative statements only, and do not generally contribute to the generated code in the sense of executable instructions or stored data.
At runtime none of your source code is required to exist on the target; it is the work of the compiler to generate native executable machine code from your source. It is this machine code that is stored and runs on the target.
A header file is not the same thing as a library. It is merely (generally) the declaration of library content (function prototypes and other symbol declarations such as constants, data, macros, enumerations). The library takes the form of pre-compiled/assembled object code stored in a combined archive. It is the job of the linker to combine the required library code with the object code generated from compilation of your own source. It is this linked executable that is stored and executed on the target, not the original source code.
An exception regarding header files containing declarative code only is when they contain in-line code or executable code in a macro. However such code only occupies space in your application if explicitly called by the application.
When library code is linked, only those library object code components required to resolve references in the application code are linked, not the entire library (unless the entire library is composed of only a single object file).
The library does indeed have to get included in the image that is burned into the embedded system's memory. Usually you tell the linker to strip out unused functions, which goes a long way towards conserving memory. But this memory is the memory your program takes up in flash or whatever you use for non-volatile code storage. It doesn't say anything about how much RAM your program takes at runtime. You can also tell your compiler to optimize for space, and also use different runtime libraries - the ones provided by the vendor are often not as fast or small as they could be.
Related
I've been programming in C for a while and i wondered why is important to separate this processes (Compile and Linking)?
Can someone explain please?
It is useful in order to decrease rebuilt time. If you change just one source file - it is often no need to recompile whole project but only one or few files.
Because, compilation is responsible for transforming the source code of every single source code file to a corresponding object code. That's it. So the compiler doesn't have to care about your external symbols (like the libraries and extern variables).
Linking is responsible for finding those references and then producing a single binary as if your project was written as a single source code file. (I also recommend that you should refer to wikipedia linking page to know the differnce between static and dynamic linking)
If you happen to use the tool Make, you will see that it doesn't recompile every file whenever you invoke make, it finds what files have been modified since the last build and then recompiles them only. Then the linking process is invoked. That is a big time saver when you deal with big projects (e.g. linux kernel).
It's probably less important these days than it was once.
But there was a time when compiling a project could take literally days - we used to do a "complete build" over a weekend back in the 1980s. Just parsing the source code of a single file was a fairly big deal requiring significant amounts of time and memory, so languages were designed so that their modules (source files) could be processed in isolation.
The result was "object files" - .obj (DOS/Windows/VMS) and .o (unix) files - which contain the relocatable code, the static data, and the lists of exports (objects we've defined) and the imports (objects we need). The linking stage glues all this together into an executable, or into an archive (.lib, .a, .so, .dll files etc) for further inclusion.
Making the expensive compilation task operate in isolation led the way to sophisticated incremental build tools like make, which produced a significant increase in programmer productivity - still critical for large C projects, like the Linux kernel.
It also, usefully, means that any language that can be compiled into an object file can be linked together. So, with a little effort, one can link C to Fortran to COBOL to C++ and so on.
Many languages developed since those days have pushed the boundaries of what can be stored in object files. The C++ template system requires special handling, and overloaded methods don't quite fit either as plain .o files don't support multiple functions with the same name (see C++ name mangling). Java et al uses a completely different approach, with custom code file formats and a "native code" invocation mechanism that glues onto DLLs and shared object files.
In practice, it is not important at all. Especially for simpler programs, both steps are performed with one program call such as
gcc f1.c f2.c f3.c f4.c -o program
which creates the executable program from these source files.
But the fact remains that these are separate processes and in some cases it is worth noticing this.
I've worked on systems where it takes two days to compile them. You don't want to make a small change then have to wait 2 days to test.
First I want to state for the record that this question is related to school/homework.
Let’s say computers CP1 and CP2 both share the same operating system and machine language. If a C program is compiled on CP1, in order to move it to CP2, is it necessary to transfer the source code and recompile on CP2, or simply transfer the object files.
My gut answer is that the object files should suffice. The C code is translated into assembly by the compiler and assembled into machine code by the assembler. Because the architecture shares the same machine code and operating system, I don't see a problem.
But the more I think about it, the more confused I’m starting to get.
My questions are:
a) Since its referring to object files and not executables, I’m assuming there has been no linking. Would there be any problems that surface when linking on CP2?
b) Would it matter if the code used C11 standard on CP1 but the only compiler on CP2 was C99? I'm assuming this is irrelevant once the code has been compiled/assembled.
c) The question doesn't specify shared/dynamic linked libraries. So this would only really work if the program had no dependencies on .dll/.so/ .dylib files, or else these would be required on CP2 as well.
I feel like there are so many gotchas, and considering how vague the question is I now feel that it would be safer to simply recompile.
Halp!
The answer is, it depends. When you compile a C program and move the object files to link on a different computer, it should work. But because of factors such as endianness or name mangling, your program might not work as intended, and even might crash when you try to run it.
C11 is not supported by a C99 compiler, but it does not matter if the source has been compiled and assembled.
As long as the source is compiled with the libraries on one machine, you don't need the libraries to link or run the file(s) on the other computer (static libraries only, dynamic libraries will have to be on the computer you run the application on). This said, you should make the program independent so you don't run into the same problems as before where the program doesn't work as intended or crashes.
You could get a compiler that supports EABI so you don't run into these problems. Compilers that support the EABI create object code that is compatible with code generated by other such compilers, thus allowing developers to link libraries generated with one compiler with object code generated with a different compiler.
I have tried to do this before, but not a whole lot, and not recently. Therefore, my information may not be 100% accurate.
a) I've already heard the term "object files" being used to refer to linked binaries - even though it's kinda inaccurate. So maybe they mean "binaries". I'd say linking on a different machine could be problematic if it has a different compiler -
unless object file formats are standardized, which I'm not sure about.
b) Using different standards or even compilers doesn't matter for binary code - if it's linked statically. If it relies on functions from a dynamic lib, there could be problems. Which answers c) as well: Yes, this will be a problem. The program won't start if it doesn't have all required dynamic libs in the correct version. Depends on linking mode (static vs. dynamic), again.
Q: Let’s say computers CP1 and CP2 both share the same operating system and machine language.
A: Then you can run the same .exe's on both computers
Q: If a C program is compiled on CP1, in order to move it to CP2, is it necessary to transfer the source code
A: No. You only need the source code if you want to recompile. You only need to recompile if it's a different, incompatible CPU and/or OS.
"Object files" are generally not needed at all for program execution:
http://en.wikipedia.org/wiki/Object_files
An object file is a file containing relocatable format machine code
that is usually not directly executable. Object files are produced by
an assembler, compiler, or other language translator, and used as
input to the linker.
An "executable program" might need one or more "shared libraries" (aka .dll's). In which case the same restrictions apply: the shared libraries, if not already resident, must be copied along with the .exe, and must also be compatible with the CPU and OS.
Finally, "scripts" do not need to be recompiled. You may copy the script freely from computer to computer. But each computer must have an "interpreter" to run the script: a Perl script needs a Perl interpreter, a Python script a python interpreter, and so on.
Recently I tried to write a simple compiler on the linux platform by myself.
When it comes to the backend of the compiler, I decided to generate ELF-formatted binaries without using a third-party library, such as libelf.
Instead I want to try to write machine code directly into the file coresponding to the ELF ABI just by using the write() function and controlling all details of the ELF file.
The advantage of this approach is that I can control everything for my compiler.
But I am hesitating. Is that way feasible, considering how detailed the ELF ABI is?
I hope for any suggestions and pointers to good available resources available.
How easy/feasible this is depends on what features you want to support. If you want to use dynamic linking, you have to deal with the symbol table, relocations, etc. And of course if you want to be able to link with existing libraries, even static ones, you'll have to support whatever they need. But if your goal is just to make standalone static ELF binaries, it's really very easy. All you need is a main ELF header (100% boilerplate) and 2 PT_LOAD program headers: one to load your program's code segment, the other to load its data segment. In theory they could be combined, but security-hardened kernels do not allow a given page to be both writable and executable, so it would be smart to separate them.
Some suggested reading:
http://www.linuxjournal.com/article/1059
What actually is a C runtime library and what is it used for? I was searching, Googling like a devil, but I couldn't find anything better than Microsoft's: "The Microsoft run-time library provides routines for programming for the Microsoft Windows operating system. These routines automate many common programming tasks that are not provided by the C and C++ languages."
OK, I get that, but for example, what is in libcmt.lib? What does it do? I thought that the C standard library was a part of C compiler. So is libcmt.lib Windows' implementation of C standard library functions to work under win32?
Yes, libcmt is (one of several) implementations of the C standard library provided with Microsoft's compiler. They provide both "debug" and "release" versions of three basic types of libraries: single-threaded (always statically linked), multi-threaded statically linked, and multi-threaded dynamically linked (though, depending on the compiler version you're using, some of those may not be present).
So, in the name "libcmt", "libc" is the (more or less) traditional name for the C library. The "mt" means "multi-threaded". A "debug" version would have a "d" added to the end, giving "libcmtd".
As far as what functions it includes, the C standard (part 7, if you happen to care) defines a set of functions a conforming (hosted) implementation must supply. Most vendors (including Microsoft) add various other functions themselves (for compatibility, to provide capabilities the standard functions don't address, etc.) In most cases, it will also contain quite a few "internal" functions that are used by the compiler but not normally by the end user.
The runtime library is basically a collection of the implementations of those functions in one big file (or a few big files--e.g., on UNIX the floating point functions are traditionally stored separately from the rest). That big file is typically something on the same general order as a zip file, but without any compression, so it's basically just some little files collected together and stored together into one bigger file. The archive will usually contain at least some indexing to make it relatively fast/easy to find and extract the data from the internal files. At least at times, Microsoft has used a library format with an "extended" index the linker can use to find which functions are implemented in which of the sub-files, so it can find and link in the parts it needs faster (but that's purely an optimization, not a requirement).
If you want to get a complete list of the functions in "libcmt" (to use your example) you could open one of the Visual Studio command prompts (under "Visual Studio Tools", normally), switch to the directory where your libraries were installed, and type something like: lib -list libcmt.lib and it'll generate a (long) list of the names of all the object files in that library. Those don't always correspond directly to the names of the functions, but will generally give an idea. If you want to look at a particular object file, you can use lib -extract to extract one of those object files, then use dumpbin /symbols <object file name> to find what function(s) is/are in that particular object file.
At first, we should understand what a Runtime Library is; and think what it could mean by "Microsoft C Runtime Library".
see: http://en.wikipedia.org/wiki/Runtime_library
I have posted most of the article here because it might get updated.
When the source code of a computer program is translated into the respective target language by a compiler, it would cause an extreme enlargement of program code if each command in the program and every call to a built-in function would cause the in-place generation of the complete respective program code in the target language every time. Instead the compiler often uses compiler-specific auxiliary functions in the runtime library that are mostly not accessible to application programmers. Depending on the compiler manufacturer, the runtime library will sometimes also contain the standard library of the respective compiler or be contained in it.
Also some functions that can be performed only (or are more efficient or accurate) at runtime are implemented in the runtime library, e.g. some logic errors, array bounds checking, dynamic type checking, exception handling and possibly debugging functionality. For this reason, some programming bugs are not discovered until the program is tested in a "live" environment with real data, despite sophisticated compile-time checking and pre-release testing. In this case, the end user may encounter a runtime error message.
Usually the runtime library realizes many functions by accessing the operating system. Many programming languages have built-in functions that do not necessarily have to be realized in the compiler, but can be implemented in the runtime library. So the border between runtime library and standard library is up to the compiler manufacturer. Therefore a runtime library is always compiler-specific and platform-specific.
The concept of a runtime library should not be confused with an ordinary program library like that created by an application programmer or delivered by a third party or a dynamic library, meaning a program library linked at run time. For example, the programming language C requires only a minimal runtime library (commonly called crt0) but defines a large standard library (called C standard library) that each implementation has to deliver.
I just asked this myself and was hurting my brain for some hours. Still did not find anything that really makes a point. Everybody that does write something to a topic is not able to actually "teach". If you want to teach someone, take the most basic language a person understands, so he does not need to care about other topics when handling a topic. So I came to a conclusion for myself that seems to fit well in all this chaos.
In the programming language C, every program starts with the main() function.
Other languages might define other functions where the program starts. But a processor does not know the main(). A processor knows only predefined commands, represented by combinations of 0 and 1.
In microprocessor programming, not having an underlying operating system (Microsoft Windows, Linux, MacOS,..), you need to tell the processor explicitly where to start by setting the ProgramCounter (PC) that iterates and jumps (loops, function calls) within the commands known to the processor. You need to know how big the RAM is, you need to set the position of the program stack (local variables), as well as the position of the heap (dynamic variables) and the location of global variables (I guess it was called SSA?) within the RAM.
A single processor can only execute one program at a time.
That's where the operating system comes in. The operating system itself is a program that runs on the processor. A program that allows the execution of custom code. Runs multiple programs at a time by switching between the execution codes of the programs (which are loaded into the RAM). But the operating system IS A PROGRAM, each program is written differently. Simply putting the code of your custom program into RAM will not run it, the operating system does not know it. You need to call functions on the operating system that registers your program, tell the operating system how much memory the program needs, where the entry point into the program is located (the main() function in case of C). And this is what I guess is located within the Runtime Library, and explains why you need a special library for each operating system, cause these are just programs themselves and have different functions to do these things.
This also explains why it is NOT dynamically linked at runtime as .dll files are, even if it is called a RUNTIME Library. The Runtime Library needs to be linked statically, because it is needed at startup of your program. The Runtime Library injects/connects your custom program into/to another program (the operating system) at RUNTIME. This really causes some brain f...
Conclusion:
RUNTIME Library is a fail in naming. There might not have been a .dll (linking at runtime) in the early times and the issue of understanding the difference simply did not exist. But even if this is true, the name is badly chosen.
Better names for the Runtime Library could be: StartupLibrary/OSEntryLibrary/SystemConnectLibrary/OSConnectLibrary
Hope I got it right, up for correction/expansion.
cheers.
C is a language and in its definition, there do not need to be any functions available to you. No IO, no math routines and so on. By convention, there are a set of routines available to you that you can link into your executable, but you don't need to use them. This is, however, such a common thing to do that most linkers don't ask you to link to the C runtime libraries anymore.
There are times when you don't want them - for example, in working with embedded systems, it might be impractical to have malloc, for example. I used to work on embedding PostScript into printers and we had our own set of runtime libraries that were much happier on embedded systems, so we didn't bother with the "standard".
The runtime library is that library that is automatically compiled in for any C program you run. The version of the library you would use depends on your compiler, platform, debugging options, and multithreading options.
A good description of the different choices for runtime libraries:
http://www.davidlenihan.com/2008/01/choosing_the_correct_cc_runtim.html
It includes those functions you don't normally think of as needing a library to call:
malloc
enum, struct
abs, min
assert
Microsoft has a nice list of their runtime library functions:
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/crt-alphabetical-function-reference?view=msvc-170
The exact list of functions would vary depending on compiler, so for iOS you would get other functions like dispatch_async() or NSLog().
If you use a tool like Dependency Walker on an executable compiled from C or C++ , you will see that one of the the DLLs it is dependent on is MSVCRT.DLL. This is the Microsoft C Runtime Library. If you further examine MSVCRT.DLL with DW, you will see that this is where all the functions like printf(), puts(0, gets(), atoi() etc. live.
i think Microsoft's definition really mean:
The Microsoft implementation of
standard C run-time library provides...
There are three forms of the C Run-time library provided with the Win32 SDK:
* LIBC.LIB is a statically linked library for single-threaded programs.
* LIBCMT.LIB is a statically linked library that supports multithreaded programs.
* CRTDLL.LIB is an import library for CRTDLL.DLL that also supports multithreaded programs. CRTDLL.DLL itself is part of Windows NT.
Microsoft Visual C++ 32-bit edition contains these three forms as well, however, the CRT in a DLL is named MSVCRT.LIB. The DLL is redistributable. Its name depends on the version of VC++ (ie MSVCRT10.DLL or MSVCRT20.DLL). Note however, that MSVCRT10.DLL is not supported on Win32s, while CRTDLL.LIB is supported on Win32s. MSVCRT20.DLL comes in two versions: one for Windows NT and the other for Win32s.
see: http://support.microsoft.com/?scid=kb%3Ben-us%3B94248&x=12&y=9
please, could someone explain to me a few basic things about working with languages like C? Especially on Windows?
If I want to use some other library, what do I need from the library? Header files .h and ..?
What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
To use another library, what is preferred way? LoadLibrary() or #include <>?
There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
How do I create one big .exe? Is this called "static linking"?
How to include some random file into .exe? Say a program icon or start-up song?
How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Well, so many question... Thanks for every info!
1: If I want to use some other library, what do I need from the library? Header files .h and ..?
... and, usually a *.lib file which you pass as an argument to your linker.
2: What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
This might be useful: Static libraries, dynamic libraries, DLLs, entry points, headers … how to get out of this alive?
3: Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
Yes, it matters. For interop between compilers, the normal way is to use a C-style (not C++-style) API, with well-defined parameter-passing conventions (e.g. __stdcall), or to use 'COM' interfaces.
4: To use another library, what is preferred way? LoadLibrary() or #include <>?
#include is for the compiler (e.g. so that it can compile calls to the library); and LoadLibrary (or, using a *.lib file) is for the run-time linker/loader (so that it can substitute the actual address of those library methods into your code): i.e. you need both.
5: There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
If it's only source then you can compile that source (once) into a library, and then (when you build your project) link to that library (without recompiling the library).
6: How do I create one big .exe? Is this called "static linking"?
Yes, compile everything and pass it all to the linker.
7: How to include some random file into .exe? Say a program icon or start-up song?
Define that in a Windows-specific 'resource file', which is compiled by the 'resource compiler'.
8: How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
Yes.
9: If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
I don't understand your question/example.
10: What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Yes.
If I want to use some other library, what do I need from the library? Header files .h and ..?
You need header .h or .hpp for C,C++ although some languages don't require header files. You'll also need .a, .so, .dll, .lib, .jar etc files. These files contain the machine code that you linker can link into your program. Goes without saying that the format of library is must be understood by you linker.
What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
dll and .a are library files, that contain code components that you can link into your own program. a .exe is your final program into which .a or .dll has already been linked.
Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
Yes, it is important that the library that you are using is compatible with your platform. Typically Unix libraries will not run on windows and vice versa, if you are using JAVA you are better off since a .jar files will usually work on any platform with JAVA enabled (though versions matter )
To use another library, what is preferred way? LoadLibrary() or #include <>?
include is not a way to use a library its just a preprocessor directive telling you preprocessor to include a external source file in your current source file. This file can be any file not just .h although usually it would be .h or a .hpp
You'll be better off my leaving the decision about when to load a library to you runtime environment or your linker, unless you know for sure that loading a library at a particular point of time is going to add some value to your code. The performance cost and exact method of doing this is platform dependent.
There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
If you have source code you'll need to recompile it every time you make a change to it.
however if you have not changed the source of library in anyway there is no need to recompile it. The build tool like Make are intelligent enough to take this decision for you.
How do I create one big .exe? Is this called "static linking"?
Creating a static .exe is dependent on the build tool you are using.
with gcc this would usually mean that you have to you -static option
gcc -static -o my.exe my.c
How to include some random file into .exe? Say a program icon or start-up song?
Nothing in programming is random. If it were we would be in trouble. Again the way you can play a song or display an icon is dependent on the platform you are using on some platforms it may even be impossible to do so.
How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
You'll need a header file with all your function prototypes and you can split you program into several .c files that contain one or more functions. You main files will include the header file. All source files need to be compiled individually and then linked into one executable. Typically you'll get a .o for every .c and then you link all the .o together to get a .exe
If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
Yes one library may require another library however its not advisable to package different libraries together, you may be violating the IPR and also for the fact that each library is usually a well define unit with a specific purpose and combining them into one usually doesn't make much sense.
What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Again depends on the platform, usually on most OS the memory will be recovered after the program dies but on certain platforms like an embedded system it may be permanently lost.
It always a good idea to clean up the resources your program has used.
In all seriousness, the place to go to learn how to run your local environment is the documentation for your local environment. After all we on not even know exactly what your environment is, much less have it in front of us.
But here are some answers:
1. You need the headers, and a linkable object of some kind. Or you need the source so that you can build these.
3. It matters that the library is in a format that your linker understands. In c++ and perhaps other languages, it also needs to understand the name mangling that was used.
6. Forcing all the library code to be included in the executable is, indeed, called "static linking".
7. There is at least one StackOverflow question on "resource compilers".