I understand that the C language is an ISO standard, and I can see from Wikipedia that the standard includes 29 header files, and that conforming to these header files, a C application is theoretically 'portable'.
In practice, however, I recently tried doing a tutorial on a simple C HTTP server that uses header files that aren't part of the C standard. So in this case, the simplest of applications that I can think of - a C application comprising a single int main(void) function, and that is less than 100 lines, with the aim of listening on a network interface goes beyond the C standard?
In this case what is the relationship between the C language as a specification and (assuming I'm writing an application for Linux) the POSIX specification as a language?
As far as I can tell, "man7.org" provides a list of the C header files that define the API of all Unix/Linux systems (I'm assuming this is the same as 'POSIX') systems, as well as a list of system calls for the Linux platform.
This includes 82 header files, of which the 29 C standard library headers are a subset, and some 10 000 system calls (at least I assume that everything in this list that is NOT a head file is a system call).
I would assume that any reasonably functional program written in C would go beyond the standard library and make use of OS specific header files. Would it not be more accurate to say that programming an application to run on Linux would actually be "POSIX programming"?
I guess it would also be possible to stick to the standard library, and define custom header files for portable logic implementation across POSIX & non-POSIX systems (including platform-specific assembly routines). Is this ever done?
POSIX is not a specification for a language, it is a specification for an operating system, just one part of which is the wider C library specification and additional restrictions on to how the C language itself needs to be implemented on such operating systems.
There are many popular cross-platform libraries. One popular library that concerns the areas that the POSIX C specification is mostly concerned with is the Apache Portable Runtime:
The mission of the Apache Portable Runtime (APR) project is to create and maintain software libraries that provide a predictable and consistent interface to underlying platform-specific implementations. The primary goal is to provide an API to which software developers may code and be assured of predictable if not identical behaviour regardless of the platform on which their software is built, relieving them of the need to code special-case conditions to work around or take advantage of platform-specific deficiencies or features.
APR includes things like the sockets and threads and processes and can be used to compile the same application for various operating systems - many unix-like ones and Windows - with minimal changes.
POSIX is not a standard but a family of standards specifying an entire operating system.
The POSIX C standard is a superset of the standard C library, the relationship between these two is well described in this other question
Related
The C Standard Library is independent of any operating system and system.
So, why use the input/output functions from the standard library?
Unix-specific POSIX system calls exist. Windows-specific input/output system calls exist.
Don't standard library functions eventually call system calls internally? Is this just for portability?
The API presented by the C standard library is uniform between operating systems, well, uniform provided all the "unspecified" parts of C roughly align (like the size of int).
The implementation of the C standard library is not independent of the operating system. Basically the implementation consists of the compiled source code the provides the API, and that compiled code matches the CPU / machine instruction sets, and possibly other items specific to the hardware bus width, supporting chip sets and other actual hardware details.
So, programming against the C API helps your program be "more portable" but that doesn't mean that any specific implementation of the C API is portable. Finally, there are lots of small details that aren't specified in detail, or are allowed to vary between platforms (like byte order, size of int, and so on). So even a program written against the standard C API might not work correctly on another machine, unless you write code that accommodates and reacts to the parts of the C API that might differ between platforms.
POSIX is basically a standard that eventually became incorporated into most C development environments. It is designed to provide a single API to program against for multiple UNIX platforms for items that lie outside of the core C language. There are POSIX implementations for Windows too, but Microsoft's historical offerings are notorious for not actually working correctly.
Yes, these APIs (if available) are implemented with code that eventually performs operating specific calls, and is presented in "machine code" that is very specific to the CPU instruction set. There are dozens of CPUs out there, and each major platform has its own matching compiler and matching C API libraries, if the C language is available.
The C language and API is there for portability, but portability isn't it's primary reason for existing (and there are lots of small corner cases where the same code isn't portable across all platforms unless it is written a certain way.) The primary reason it's there is not portability, it is because if the language features weren't consistently available across all platforms, then you wouldn't have "one C language" that could be used on multiple machines, you would have "many C-like languages, where each supported item would have to be checked" meaning you might know C on your development platform, but not know C on another platform.
As for the libraries, there are many libraries that might be absent in a typical machine, and when developing, you generally have to use a dependency checker to ensure the library is present (and sometimes the correct functions are available in the library) before successful use of the machine for development. Autoconf, for example, has m4 macros that can be configured to check if a library is present before compiling the programs.
One of the Units of Functionality that POSIX states an OS needs to provide to be POSIX compliant is POSIX_C_LANG_SUPPORT. Basically this is the whole C Standard Library with some more things.
My question is simple: developers of POSIX compliant OSes usually just download an open source version of C Standard Library (e.g. glib or uClibc) and adapt it to fit POSIX or they implement everything from scratch? Is there any advantage in rewriting the C Library instead of just picking one of the very known implementations and adjust it to my needs?
Really, it is done in the inverse way.
We have different Unix versions: two main families: SystemV and BSD, different manufacturers, and so there was a need to standardize. US government wanted also standardized programs, so POSIX (version 1) was created, by standardizing OS interfaces (a step further than just C standard).
Windows NT is also POSIX (version 1) compatible, just because government wanted standardized tools. So POSIX was designed very very broad.
Then with time, there were need to standardize some more Unix (and similar) systems. Not as just one system, one API, but as common API, and so programs (e.g. GUI libraries or databases) could eventually use extension, but also make sure that program that follow the standard works on compatible system.
This was SUS (Single Unix Specification). This required a UNIX like system (unlike POSIX 1).
Then POSIX became not so important: application that in theory could work on all POSIX systems didn't really work on POSIX Windows.
So the new version of POSIX merged old POSIX plus SUS plus new useful function missing in SUS.
Now Linux is important, so Linux implementations (e.g. glibc) is taken into account when updating POSIX. You will see in the mailing list, that POSIX is defined by "vendors" of different Unix and similar systems.
So, it is not that operating systems extend POSIX, it is just that POSIX takes the most useful and standard options from different OS. It creates new interfaces just when existing interfaces are so incompatible, that by standardizing, it will break existing programs.
For the "second" question: when you develop a new operating system, you choose what way to go. Usually it is just derivation and fork (and distributions): again from the two Unix families, of just deriving Linuxes from RedHat or Debian). Sometime system is build from scratch, because of the design. Kernel provides most of system calls, so e.g. glibc needs a lot of systemcall (given by kernel) implemented in a similar way as POSIX. Glibc is not complete. Note: early Linux distributions used other libraries. GLibc was also written from scratch.
Well, we are all dwarves standing on the shoulders of giants.
Writing a new OS is a huge undertaking, so the wise one will re-use whatever (design, libraries, compilers, other software) they can. It's still in all likelyhood far too much work, so why make it even harder by rewriting everything from scratch?
I've heard while looking at different C implementations that any system that hopes to implement C must minimally include certain libraries, stdarg.h etc. My question is why this is, it can't be that the C library is not Turing complete without some headers, and since the headers have been written it must be true that I could write them myself. Why, then, is it not permissible to have a C implementation consisting of just a compiler+linker toolchain? (of course, in this case interacting with the OS would require inline assembly or linked assembly code as well as knowledge of the system's syscalls etc., but that doesn't mean that C can't be written, does it?)
You confuse a property of the programming language, i.e. the language itself with additional features mandated by the standard.
"Turing complete" is just about the language itself; basically if you can use it to solve a certain class of problems (for a more exact definition, please see Wikipedia for a starter(!) ). That is quite an abstract concept and does not include any libraries. Basically, if you use such libraries, you just have to be able to write those libraries in the language itself. This is true for the C language.
About the libraries required: Your premise is wrong. C very well allows to omit the libraries themselves. That is the difference between a hosted (full libraries) and a freestanding (few target-specific headers, but no generated code). See 4p6.
The few headers are normally part of the compiler itself. They basically provide some typedefs and #defined constants, e.g. the range of the integer types (limits.h) and types of guaranteed minimum width (stdint.h, often also fixed-width types). stddef.h e.g. provides size_t and NULL.
While you do not need to use those headers, they already allow writing portable code for the program logic. Just see them as part of the language itself, tailored to the target.
The gcc C compiler, for instance actually is a freestanding implementation: It only provides the required headers, but not the standard library. Instead, it relies on the system library, which is e.g. glibc on Linux.
Note: Generally it is a bad idea to re-invent the wheel. So if you are on a hosted environment (i.e. full-grown OS), you should use the features available. Otherwise you might run into trouble, as these e.g. mightr provide additional functions not directly seen by your code. E.g. debugging or system/user-wide configuration like localisation support. Also debugging support might depend on you using the standard library, e.g. valgrind. Replacing memory allocation with your own code at least makes this much more difficult.
Not to mention maintainability. Not just others will understand your code easier if you use the standard names&semantics, but also yourself - just wait some years and try understanding your old code.
OTOH, if you are on a bare-metal embedded system, there is actually little use of most features the standard library. Including e.g. printf or scnaf just bloats your firmware, often without any actual use. For such systems, there are stripped-down libraries (e.g. newlib) which may be not completely compliant or allow to omit certain costly features, e.g. floating point conversion or the math lib. Still you only should use them iff you really need many of their features. And sometimes there is a middle way, but that requires some knowledge about the dependencies of the library.
Two reasons: compatibility and system interaction.
If you don't implement the whole C standard library, then code other people write won't work. Even the most basic C program uses library calls.
#include <stdio.h>
int main() {
printf("Hello world!\n");
return 0;
}
Without an agreed upon and fully implemented stdio.h that program will not run because the compiler doesn't know what printf() means.
Then there's system interaction. C has been called "portable assembly". This is because different computing environments do things differently, but C takes are of that for you (well, some of it). You can't write a portable stdio.h in assembly without losing your mind. But its more than that. Each C header file protects you from something that each environment does (or used to) do very differently.
stdlib.h shields you from differing memory models and process control.
stdio.h shields you from differing IO systems.
math.h shields you from differing floating point implementations.
limits.h shields you from differing data sizes.
locale.h shields you from differing locales.
And so on...
The C libraries provide a standard API that each environment can write to. When C is ported to a new environment, that environment is responsible for implementing those libraries according to the particulars of that system. You don't have to do that.
Nowadays we live in a much more homogeneous environment than when C was developed, but most of the C standard library still protects you from basic differences in how operating systems and hardware do things.
It didn't always used to be this way. An example that comes to mind is the hell of running a game on DOS. There was no standard interface to the sound and video card (if you had them). Each program had to ship drivers for each sound and video card they supported. If yours wasn't in there, sorry. If their driver was buggy, sorry.
Programming without the C standard library is kind of like that, but far far worse.
What is the difference between the C programming language and C programming under Linux?
Are the syntax same in both them?
Or is the difference only when you execute the program?
The C language is governed by the ISO approved C standard and it does not take in to account the underlying platform on which you use C. So from the perspective of the language standard there is no difference, and a standard compliant program shall work correctly on both.
However in practical usage one needs to do platform specific things for ex: IPC mechanisms, multithreading, file access and so on which are specific to the platform, such functionality will vary from platform to platform because each will provide functionality specific to itself. Note that such functionality is not covered by the C language standard, so using it makes the program non portable across other platforms.
Linux is a platform that can be used for the development of programs and applications using languages such as C. The only thing is that its supposed to be is its simplicity and one's liking to a particular operating system. Otherwiswe there is no difference in the syntax. It is absolutely same.
There are languages and there are platforms. Popular languages are typically governed by standards (e.g., ANSI). C is a programming language.
Linux, Windows, Android, etc, are platforms (or, specifically, operating systems). Each platform offers a set of libraries (API calls) that you can access to do different things on that platform. System/library calls for file system access, networking, specific windowing/GUI system, etc, can be different on different platforms. So knowing how to "write C on Linux" means you know C and you know a lot of Linux platform calls. Even different windowing systems under Linux can have different API calls.
There are also standards across platforms, such as POSIX, which work to make the library calls the same across different platforms. Although this doesn't deal with most of the disparity between GUI APIs.
The C language programming syntax is defined under the ISO C standard. The resulting execution depends on the compiler used to turn code into an executable program and the machine on which the compile runs (or at least the target architecture it runs for). The results from that compilation will depend on the use of the programming syntax (the code) against the interpretation of that code from the compiler. If the programmer restricts his programming habits to writing conformant C code excluding implementation-defined behavior or undefined behavior, it's resulting executable will behave identically on any platform.
Then you think of it as if there was roughly three "layers" of C implementation you could make: kernel programming, system programming and userspace programming.
Kernel programming is hardware-level programming and usually leverage implementation-defined behavior to interface the hardware world to the software world. They provide a C interface to system programmers. They are different from machine to machine and the architercture resulting from these implementation defines the difference between various OS (ex: window vs linux vs OsX vs MIT exokernel, etc).
System programmers leverage the kernel's (the system's) API to build C standard library (they define the implementation of higher level C standard functionnalities). Ex: glibc and the gnu c compiler (gcc) should be iso C conformant to unambiguous section of the C standard and defines the implementation of implementation-define AND undefined behavior. That layer of implementation is hardware independant (to some extend) since the kernel level constitute an hardware abstraction. But they handle resource from that abstraction layer (ex: RAM or writting to a file on the hard drive or sending a stream of data on an internet socket).
Userspace programmers code the programs that uses the standard API and the compilers to build "usable" pieces of software such as gnome-terminal or i3 windows tiling manager (I can't find an example a C code "user-friendly" running under windows from the top of my head...). Unless these software implementation resort to implementation-define code or undefined behavior code, it should be platform independent.
The answer is simple: There is no difference!
However each operating system has its own API. This API does not depend on the programming language.
Example: The "MessageBox()" function exists in Windows only, not in Linux. It is a Windows-Specific function (available in any programming language under Windows).
There are also some library functions that are named differently in Linux and in Windows.
One example would be the "stricmp()" function (Windows) that is named "strcasecmp()" under Linux. However this is not an issue of the C programming language but of the libraries (.H files and .SO files).
Different operating systems will have different APIs (Application programming interfaces) which can be libraries built for building application software for your specific OS. GNU/Linux has libraries specific to it such as sys/socket.h, linux.h, sys/types.h, etc.
Is there a C library available for operations such as file operations, getting system information and the like which is generic, which can be used when compiled in different platforms and which behaves in a similar way?
Edit: Something like Java or .NET platform abstracting the hardware.
Have you tried the standard library? It should be implemented on any system that has an ISO compliant C runtime.
Yes; the ISO Standard C library. It may not cover all the functionality you want, but that is exactly because it is generic, and as such is also lowest common denominator. It only supports features that can reasonably be expected to exist on most hardware, including embedded systems.
The way to approach this is perhaps to specify the range of target platforms you need to support, and then the application domains (e.g. GUI, networking, multi-threading, image processing, file handling etc.), and then select the individual cross-platform libraries that suit your needs. There is probably no one library to fulfil all your needs, and in some cases no common library at all.
That said, you will always be better served in this respect by embracing C++ where you can use any C library as well as C++ libraries. Not only is the C++ standard library larger, but libraries such as Boost, wxWidgets, ACE cover a broader domain spectrum too. Another approach is to use a cross-platform language such as Java, which solves the problem by abstracting the hardware to a virtual machine. Similarly .NET/Mono and C# may provide a solution for suitably limited set of target platforms.
Added following comment:
Hardware abstraction in a real-machine targeted language (as opposed to a VM language such as Java or CLR based languages) is provided by the operating system, so what you perhaps need is a common operating system API. POSIX is probably the closest you will get to that, being supported on Linux, Unix, OSX (which is Unix), QNX, VxWorks, BeOS and many others; but not importantly Windows. One way of using POSIX on Windows is to use Cygwin. Another is to use a VM to host a POSIX OS such as Linux.
For anything not found in the standard library, GLib is a good first place to look along with other libraries built to interact with it. It offers for example threads, mutexes and IPC that you won't be able to write portably using plain standard libraries, and you can use many more GNU libraries that follow the same conventions up to a full GUI in GTK+. GLib supports the usual popular operating systems.
If the C standard library is not sufficient for your needs, a minimal hyperportable subset of POSIX might be the target you want to code to. For example, the main non-POSIX operating system Windows still has a number of functions with the same names as POSIX functions which behave reasonably closely - open, read, write, etc.
Documenting what exactly a "hyperportable subset of POSIX" includes, and which non-POSIX operating systems would conform to such a subset, is a moderately difficult task, which would be quite useful in and of itself for the sake of avoiding the plague of "MyCompanyName portable runtime" products which appear again and again every few years and unnecessarily bloat popular software like Firefox and Apache.
If you need facilities beyond what the Standard C library provides, take a look at the Apache Portable Runtime (APR). You should also review whether POSIX provides the functionality you are after, though that comes with its own bag of worms.
If you want to get into graphics and the like, then you are into a different world - GTK, Glib and Qt spring to mind, though I've not used any of them.