I'm compiling a Linux toolchain based on Newlib for a toy project.
Newlib's official page reports:
Newlib is a C library intended for use on embedded systems.
but without providing any particular reason.
What I'm trying to figure out is:
Why newlib consider itself embedded "only"?
What are the downsides to use it on desktop or server environments?
Unlike Glibc, which has very specific code to make sure that it is replaceable with later API compatible versions, the Newlib isn't so; nor does it support dynamic linking anyway. And it doesn't make much sense to statically link in the C library in every possible executable in a desktop environment. Therefore, Newlib is mostly suitable for embedded targets with small number of statically linked executables.
Newlib also fulfils only the parts of the C standard library and a a minimal part of the POSIX C library extensions. Specifically it doesn't concern itself with networking at all. It is somewhat debatable if anyone in their right mind, would want to build a desktop system without any networking at all, in 2017.
Another thing to note is the non-technical aspect of licensing. Glibc uses the LGPL license, which does allow linking against proprietary programs, provided that (note that IANAL) the user is able to replace the LGPL-licenced library with another one. In practice this means that either the library is dynamically linked in, or, in case of a statically linked library, the user is provided with object files that they can use to link against the replacement library to produce an executable. This means that the license itself might not be suitable for small embedded systems with proprietary software. Newlib doesn't contain any LGPL code, unless it is built for Linux targets.
Related
My host application took over the ownership of e.g. a FILE object which came from a dynamic library. Can I call fclose() on this object safely even though my host application and the dynamic library are compiled with different versions of clang / gcc?
Background
On Windows (with different VS runtimes) it would be illegal and I have to first extract the fclose() function from the runtime library which is used by the dynamic library since all runtimes have their own pools and internal structures for file or memory objects.
An illustration for the situation in Windows would look like this:
Does this restriction apply for Linux and macOS as well?
The issue is not whether your application and the dynamic libraries were compiled with different versions of clang and/or gcc. The issue is whether, ultimately, there's one underlying C library that manipulates one kind of FILE * object and has one, compatible implementation of fclose().
Under MacOS and Linux, at least, the answer to all these questions is likely to be "yes". In my experience it's hard to get two different, incompatible C libraries into the mix; you'd have to really work at it.
Addendum: I suppose I should admit, however, that my experience may be getting dated. In my experience, on any Unix-like system, there's exactly one C library, generally /lib/libc.{a,so}. But I gather that "modern" compilers are tending to access their own compiler- and version-specific libraries off in special places, meaning that the scenario you're worried about could be a problem. To me, it seems, this way lies madness, but then again, it seems that more and more of the world seems to be embracing dependency hell, rather than trying to eliminate it.
It is not generally safe to use a library designed for one compiler with code compiled by a different compiler. A compiler may generate code that implements the nominal functions in the standard library using internal routines or interfaces, and those routines or interfaces may be different or missing in the library designed for another compiler.
Nor is it safe to take any pointer to some internal data structure from one library and use it with another library.
If the sources are just compiled with different versions of one compiler (e.g., clang 73 and clang 89), not different compilers (e.g., Apple clang versus GCC), the compiler might offer some guarantee about library compatibility. You would have to check its documentation. Or, if the compiler is intended to use the library provided with the operating system, that could work. Again, you would have to check its documentation.
On Linux, if both your code and the other library dynamically link to the same library (such as libc.so.6), both will get the same version and implementation of that library at runtime. You can check which libraries a given dynamic library links to with ldd.
If you were linking to a library that statically linked in a supporting library, you would need to be careful to pass any structures to or from it against the same version of the library. But this is more likely to come up in practice with libc++ and libstdc++ than with libc.
So, don't statically link your library to another and then pass a data structure that requires client code to separately link to the same library.
I am learning network programming through the sample source codes from this link http://cs.baylor.edu/~donahoo/practical/CSockets/textcode.html. During the compilation, just wondering why in Solaris environment, i have to manually link socket and nsl library in the make file but when in the linux machine, i dont need to do that ?
Documentation used: http://developers.sun.com/solaris/articles/solaris_linux_app.html
This is because linux's libc, the glibc (-lc, which is linked by default to all programs) includes socket part of POSIX; and nis/nis+ dynamic libraries in linux are loaded dynamically by libc too.
But in Solaris, there are a lot of libraries with basic functionality, which are not in libc.
(libc, libucb, libmalloc, libsocket, libxnet, etc). I think, it was a design solution to allow user link only parts of API he needs.
In linux there are some basic libraries outside libc too: libaio, librt, libm.
With separate library it is easier to update only some parts of system; and it is possible to have several implementations (e.g. to provide greater compatibility/workarounds with older versions of UNIX) of some libraries coexisting in same system.
This question is discussed a lot, e.g. http://web.archiveorange.com/archive/v/KcxCHdLNpD6NANxmAt3b http://mail.opensolaris.org/pipermail/opensolaris-code/2007-January/010316.html
are seriously considering folding libnsl and libsocket into libc.
It would be nice to move ONLY the current POSIX-based and other
standards-based functionality (Unix98 etc.) libnsl+libsocket functions
to libc and keep all the compatibilty-wrapper stuff in libnsl/libsocket
to avoid that libc gets bloated with 20years of Unix
backwards-compatibility workarounds
Because in Linux, the entire networking API is implemented in libc.so which is linked into every C program by default, while in Solaris, its implemented in separate libraries.
As per my understanding, C libraries must be distributed along with compilers. For example, GCC must be distributing it's own C library and Forte must be distributing it's own C library. Is my understanding correct?
But, can a user library compiled with GCC work with Forte C library? If both the C libraries are present in a system, which one will get invoked during run time?
Also, if an application is linking to multiple libraries some compiled with GCC and some with Forte, will libraries compiled with GCC automatically link to the GCC C library and will it behave likewise for Forte.
GCC comes with libgcc which includes helper functions to do things like long division (or even simpler things like multiplication on CPUs with no multiply instruction). It does not require a specific libc implementation. FreeBSD uses a BSD derived one, glibc is very popular on Linux and there are special ones for embedded systems like avr-libc.
Systems can have many libraries installed (libc and other) and the rules for selecting them vary by OS. If you link statically it's entirely determined at compile time. If you link dynamically there are versioning and path rules which come into play. Generally you cannot mix and match at runtime because of bits of the library (from headers) that got compiled into the executable.
The compile products of two compilers should be compatible if they both follow the ABI for the platform. That's the purpose of defining specific register and calling conventions.
As far as Solaris is concerned, you assumption is incorrect. Being the interface between the kernel and the userland, the standard C library is provided with the operating system. That means whatever C compiler you use (Forte/studio or gcc), the same libc is always used. In any case, the rare ports of the Gnu standard C library (glibc) to Solaris are quite limited and probably lacking too much features to be usable. http://csclub.uwaterloo.ca/~dtbartle/opensolaris/
None of the other answers (yet) mentions an important feature that promotes interworking between compilers and libraries - the ABI or Application Binary Interface. On Unix-like machines, there is a well documented ABI, and the C compilers on the system all follow the ABI. This allows a great deal of mix'n'match. Normally, you use the system-provided C library, but you can use a replacement version provided with a compiler, or created separately. And normally, you can use a library compiled by one compiler with programs compiled by other compilers.
Sometimes, one compiler uses a runtime support library for some operations - perhaps 64-bit arithmetic routines on a 32-bit machine. If you use a library built with this compiler as part of a program built with another compiler, you may need to link this library. However, I've not seen that as a problem for a long time - with pure C.
Linking C++ is a different matter. There isn't the same degree of interworking between different C++ compilers - they disagree on details of class layout (vtables, etc) and on how exception handling is done, and so on. You have to work harder to create libraries built with one C++ compiler that can be used by others.
Only few things of the C library are mandatory in the sense that they are not needed for a freestanding environment. It only has to provide what is necessary for the headers
<float.h>, <iso646.h>, <limits.h>, <stdarg.h>, <stdbool.h>, <stddef.h>, and <stdint.h>
These usually don't implement a lot of functions that must be provided.
The other type of environments are called "hosted" environments. As the name indicated they suppose that there is some entity that "hosts" the running program, usually the OS. So usually the C library is provided by that "hosting environment", but as Ben said, on different systems there may even be alternative implementations.
Forte? That's really old.
The preferred compilers and developer tools for Solaris are all contained in Oracle Solaris Studio.
C/C++/Fortran with a debugger, performance analyzer, and IDE based on NetBeans, and lots of libraries.
http://www.oracle.com/technetwork/server-storage/solarisstudio/index.html
It's (still) free, too.
I think there a is a bit of confusion about terms: a library is NOT DLL's or .so: in the real sense of programming languages, Libraries are compiled code the LINKER will merge with our binary (.o). So the linker (or the compiler via some directives...) can manage them, but OS can't, simply is NOT a concept related to OS.
We are used to think OSes are written in C and we can rebuild the OS using gcc/libraries or similar, but C is NOT linux / unix.
We can also have an OS written in Pascal (Mac OS was in this manner many years ago..) AND use libraries with our favorite C compiler, OR have an OS written in ASM (even if not all, as in first Windows version), but we must have C libraries to build an exe.
I'm a bit naive when it comes to application development in C. I've been writing a lot of code for a programming language I'm working on and I want to include stuff from ICU (for internationalization and unicode support).
The problem is, I'm just not sure if there are any conventions for including a third party library. for something like readline where lots of systems are probably going to have it installed already, it's safe to just link to it (I think). But what about if I wanted to include a version of the library in my own code? Is this common or am I thinking about this all wrong?
If your code requires 3rd party libraries, you need to check for them before you build. On Linux, at least with open-source, the canonical way to do this is to use Autotools to write a configure script that looks for both the presence of libraries and how to use them. Thankfully this is pretty automated and there are tons of examples. Basically you write a configure.ac (and/or a Makefile.am) which are the source files for autoconf and automake respectively. They're transformed into configure and Makefile.in, and ./configure conditionally builds the Makefile with any configure-time options you specify.
Note that this is really only for Linux. I guess the canonical way to do it on Windows is with a project file for an IDE...
If it is a .lib and it has no runtime linked libraries it gets complied into you code. If you need to link to dynamic libraries you will have to assure they are there provide a installer or point the user to where they can obtain them.
If you are talking about shipping your software off to end users and are worried about dependencies - you have to provide them correct packages/installers that include the dependencies needed to run your software, or otherwise make sure the user can get them (subject to local laws, export laws, etc, etc, etc, but that's all about licensing).
You could build your software and statically link in ICU and whatever else you use, or you can ship your software and the ICU shared libraries.
It depends on the OS you're targeting. For Linux and Unix system, you will typically see dynamic linking, so the application will use the library that is already installed on the system. If you do this, that means it's up to the user to obtain the library if they don't already have it. Package managers in Linux will do this for you if you package your application in the distro's package format.
On Windows you typically see static linking, which means the application bundles the library and it will use that specific version. many different applications may use the same library but include their own version. So you can have many copies of the library floating around on your system.
The problem with shipping a copy of the library with your code is that you don't get the benefit of the library's maintainers' bug fixes for free. Obscure, small, and unsupported libraries are generally worth linking statically. Otherwise I'd just add the dependency and ensure that whatever packages you ship indicate it appropriately.
I've been tasked with adding streams support (C89/C90) to the libraries for my company's legacy embedded C compiler. Our target hardware typically has 1MB or less of code space and does not have an operating system.
We have a lot of stream-like implementations throughout the codebase that I can use as a starting point. For example, a console that works over a TCP sockets or serial port, a web server that reads from FAT on SD card or in-memory file, and even a firmware updater that reads from many sources.
Before I go and re-invent the wheel, I'm wondering if there are existing implementations that I could either port or use as a starting point for my work. Even though we provide full source code to our customers, GPL-licensed code isn't an option since our customers don't want to release source code to their products.
Can anyone recommend a book (annotated Unix source, CompSci text) or public domain/BSD-licensed source? I'd prefer to look at an older OS targeted to a single device, as current operating systems contain a tangle of macros and layers of typedefs that make following even a simple struct definition difficult.
Take a look at P.J. Plauger's book The Standard C Library, which describes in detail one possible implementation of the complete C89 standard library.
You should be able to pull most of what you need from the source code for the GNU C standard library. It is licensed with the Lesser GPL, which means you can link to the library without affecting the license of your software (or forcing your customers to release their code). Porting this to your platform (thus keeping the LGPL-ed code in its own library) may be easier than implementing your own from scratch.
Several different projects have taken GNU GLIBC and optimized it for embedded systems. You may want to look at:
Embedded GLIBC (LGPL)
uLIBC (LGPL)
Newlib (multiple free licenses)
In particular, EGLIBC and uLIBC were designed to run properly on embedded systems that lack a MMU.
You can also have a look at BSD's implementation of libc
Alternatively there is STLSoft, who provides several libraries (including the C standard lib) under a BSD license. I can't attest to their quality since I haven't used their code myself, but it might be worth looking at if you can't work LGPL-ed code into your project.
Wouldn't *BSD (Net|Open|Free)'s libc be suitable? At least as a starting point.
Try looking at http://www.minix3.org/
Check your development tools. Some development tools come with their on source for their software libraries.
I took the source for the Compiler's printf and adapted for a debug port on an embedded system. There is less work when you have a foundation to build from.