I am quite new to the FILE family of functions that the standard C library provides.
I recently stumbled across fopen() and the similar functions after researching how stdout, stdin and stderr work alongside functions like printf().
I was wondering, what is needed to use fopen() on an embedded system (which doesn't necessarily have operating system support). After reading more about it, is seems like a cool thing to do on more powerful embedded systems to hook into say, a UART/SPI interface, so that calling printf() would print data out of the UART. Simarly, you could read data from a UART buffer by calling scanf().
This would also increase portability! (code written for say, Linux, would be easier to port if printf() was supported). You could also print debug data to a file if it was running in a production environment, and read from it later.
Can you just use fopen() on a bare-bones embedded system? If so who/where/when is the "FILE" then created (as far as I now, fopen() does not malloc() space for the file, nor do you specify how much)? Or do you need a operating system with FAT file support. If so, would something like http://ultra-embedded.com/?fat_filelib work? Would using FreeRTOS help at all?
Check the documentation for your toolchain's C library - it should have something to say about re-targeting the library.
For example if you are using Newlib you must re-implement some or all of the [syscalls stubs][3] to suit your target. The low level open() syscall in this case will allow fopen() to work as necessary. At its simplest, you might implement open() to support higher-level stdio access to serial ports, but if you are expecting standard file-system access, then you will still need an underlying file-system to map it too.
Another example of re-targeting the Keil/ARM standard library can be found here.
Yes, it's often possible to use fopen() and similar routines in code for embedded systems. The way it often works is that the vendor supplies a C compiler and associated libraries
targeted for their system, which implement some supported subset of the language in a way that's appropriate for that system (e.g. an implementation of printf() that outputs via a UART, or fopen() that uses RAM to simulate some sort of filesystem).
On the Keil compiler, the stdio library is designed to allow the user to define the __FILE structure in any desired fashion. A function like fprintf will perform a sequence of calls to fputc, which will receive a copy of the pointer passed to fprintf. One may define something like fopen to "create" a __FILE and populate its members via any desired means (if there will never be more than one file open at a time, one could simply fill in the fields of a static instance and return that). Variables __stdin, __stdout, and __stderror may likewise be defined as desired (stdin is defined to point to __stdin, and likewise with stdout and stderror).
"Can you just use fopen() on a bare-bones embedded system?"
It depends. Depends on the configuration of your embedded system, the types of memories interfaced, on what memory do you want to implement the file system, the file system library code size (ROM & RAM requirements).
FILE manipulation functions can be used independent of any OS. But a proper file system must be used and FAT is not the only file system (JFFS2, YAFS,...some other proprietary file system)
The file system is generally (but not always) implemented on Flash memories (Nand Flash, Nor Flash). USB device is also a flash (Nand flash). The Nand Flash & Nor Flash may have Parallel interface, I2C interface or SPI interface.
Related
I ask this, because I am getting very conflicting definitions of System calls.
One one hand, I have seen the definition that they are an API the OS provides that a user program can call. Since this API is a high level interface, it has to be implemented in a high level language like C.
On the other hand, I have seen that the actual OS syscalls are machine instructions, for which you have to set certain registers to call (according to some compliance standard set by the OS). But this looks nothing like the UNIX APIs like open(), write() and read(), so what is going on here.
I have also read that these high level interfaces are implemented in the C libraries which do the actual assembly code syscalls. In that case, why do we say the OS provides this interface when it is actually provided by the C language. What if I want to perform a UNIX syscall directly to the OS without having to use C?
There are two open functions - one, the syscall open exposed by the operating system (e.g. Linux), and two, the C-library function open, exposed by the C standard library (e.g. glibc).
You can see two different man pages for these functions - run man 2 open to see the man page regarding the syscall, and man 3 open to see the man page regarding the C standard function.
Functions you mentioned like open, write, and read can be confusing - because they exist both as syscalls and as C standard functions. But they are separate entities entirely - in fact, glibc's open function doesn't even use the open syscall - it uses the openat syscall.
On Windows, where the syscall open doesn't even exist - the C standard library function open does still exist, and uses WinAPI's CreateFile behind the scenes.
What if I want to perform a UNIX syscall directly to the OS without
having to use C?
This is possible - indeed, glibc has to do it to implement C standard library functions. But it's tricky, and involves implementing wrappers for the syscalls and sometimes even handcrafting assembly.
If you want to see things for yourself, you can look at how glibc implements open:
int
__libc_open (const char *file, int oflag, ...)
{
int mode = 0;
if (__OPEN_NEEDS_MODE (oflag))
{
va_list arg;
va_start (arg, oflag);
mode = va_arg (arg, int);
va_end (arg);
}
return SYSCALL_CANCEL (openat, AT_FDCWD, file, oflag, mode);
}
...
weak_alias (__libc_open, open)
notice that the function ends with a call to the macro SYSCALL_CANCEL, which will end up calling the OS-exposed openat syscall.
Are OS libraries written in assembly or in C
That is a question that can not really be answered as it depends. Technically there are no limitations on the implementation (i.e. it can be written in any language, though C is probably the most common followed by assembly).
The important part here is the ABI. This defines how OS calls can be made.
You can make system calls in assembly (if you know the ABI you can manually write all the code to comply), the C compiler knows the ABI and will automatically generate all the code required to make a call.
Most languages though allow you to make system calls, they will either know the ABI or have a wrapper API that translates the calls from a language call to the appropriate ABI for that OS.
I ask this, because I am getting very conflicting definitions of System calls.
The definitions will depend on the context. You will have to give examples of what the definitions are AND in what context they are being used.
One one hand, I have seen the definition that they are an API the OS provides that a user program can call.
Sure this is one way to look at it.
More strictly I would ays the OS provides a set of interfaces that can be used to perform privileged tasks. Now those interfaces can be exposed via an API provided by a particular environment that makes them easier to use.
Since this API is a high level interface, it has to be implemented in a high level language like C.
Sort of true.
An environment can expose an API does not mean that it needs a high level language (and C is not a high level language, it is one step above assembly, it is considered a low level language). And just because it is exposed by the language does not mean it is implemented in that language.
On the other hand, I have seen that the actual OS syscalls are machine instructions, for which you have to set certain registers to call (according to some compliance standard set by the OS).
OK. Here we have moved from System Calls to syscalls. We should be very careful on how we use these terms to make sure we are not conflating different terms.
I would (and this is a bit abstract still) think about the computer as several levels of abstraction:
Hardware
------ --------------
syscalls
OS --------------
System Calls (read/write etc..)
------ --------------
Language Interface (read/write etc..)
You can poke the hardware directly if you want (if you know how), but it is better if you can make syscalls (if you know how), but it better to use the OS System Calls which use a well defined ABI, but it better to use the language interface (what you would call the API) to call the underlying System Calls.
But this looks nothing like the UNIX APIs like open(), write() and read(), so what is going on here.
Here the UNIX OS provides the open/close/read interface.
The C libraries provides a very thin API wrapper interface above the the OS System Calls. The C compiler will then generate the correct instructions to call the System Calls using the correct ABI, which in turn will call the next layer down in the OS to use the syscalls.
I have also read that these high level interfaces are implemented in the C libraries which do the actual assembly code syscalls.
The high level interface can be written in any language. But the C one is so easy to use that most other languages don't bother doing it themselves but simply call via the C interface.
It's VERRRY rare to ever directly write something in assembly. By writing in C you can compile it for many different CPU architectures whereas by writing in assembly you are basically stuck with one specific architecture. Most operating systems are written in C. We say the OS provides the interface because you are interacting with the operating system which happens to be written in C.
I have read that C language does not include instructions for input and for output and that printf, scanf, getchar, putchar are actually functions.
Which are the primitive C language instructions to obtain the function printf , then?
Thank you.
If you want to use printf, you have to #include <stdio.h>. That file declares the function.
If you where thinking about how printf is implemented: printf might internally call any other functions and probably goes down to putc (also part of the C runtime) to write out the characters one-by-one. Eventually one of the functions needs to really write the character to the console. How this is done depends on the operating system. On Linux for example printf might internally call the Linux write function. On Windows printf might internally call WriteConsole.
The function printf is documented here; in fact, it is not part of the C language itself. The language itself does not provide a means for input and output. The function printf is defined in a library, which can be accessed using the compiler directive #include <stdio.h>.
No programming language provides true "primitives" for I/O. Any I/O "primitives" rely on lower abstraction levels, in this language or another.
I/O, at the lowest level, needs to access hardware. You might be looking at BIOS interrupts, hardware I/O ports, memory-mapped device controlers, or something else entirely, depending on the actual hardware your program is running on.
Because it would be a real burden to cater for all these possibilities in the implementation of the programming language, a hardware abstraction layer is employed. Individual I/O controllers are accessed by hardware drivers, which in turn are controlled by the operating system, which is providing I/O services to the application developer through a defined, abstract API. These may be accessed directly (e.g. by user-space assembly), or wrapped further (e.g. by the implementation of a programming language's interpreter, or standard library functions).
Whether you are looking at "commands" like (bash) echo or (Python) print, or library functions like (Java) System.out.println() or (C) printf() or (C++) std::cout, is just a syntactic detail: Any I/O is going through several layers of abstraction, because it is easier, and because it protects you from all kinds of erroneous or malicious program behaviour.
Those are the "primitives" of the respective language. If you're digging down further, you are leaving the realm of the language, and enter the realm of its implementation.
I once worked on a C library implementation myself. Ownership of the project has passed on since, but basically it worked like this:
printf() was implemented by means of vfprintf() (as was, eventually, every function of the *printf() family).
vfprintf() used a couple of internal helpers to do the fancy formatting, writing into a buffer.
If the buffer needed to be flushed, it was passed to an internal writef() function.
this writef() function needed to be implemented differently for each target system. On POSIX, it would call write(). On Win32, it would call WriteFile(). And so on.
In the C language, when printing something on the screen, we usually use printf, puts and so on. Which are all defined in the or other header documents.
Is there any way to print something on screen without using such functions? That is to say, how is printf realised?
Eventually the C function printf will result in a sys_write system call, directly or by going through write (see man 2 write). The actual implementation depends on the compiler and the standard libraries.
Printing to screen requires access to framebuffer (hardware) and userspace programs are not allowed to have a direct access to it. So what they do is make a system call and kernel performs the required function for them. printf -> write system call -> kernel writes the data into framebuffer and then control is given back to user program.
Even if you don't want to use printf or puts (they are implemented in hosted libc) still you have to use write system call to tell the kernel on which device you want to write the buffer.
The standard headers are not, necessarily, a library containing functions written in C code.
They are functions with C "interfase", however it's very probably that they contain explicit machine code, adapted, in each case, to the target system.
The standard headers provide, in this way, ways of doing special process that it would not be possible to achieve in strict C code.
In the specific case of printf(), the situation is even more clear, because if none header is #include-d, then there is not any mechanism through the use of the C syntax only that performs an Input/Output operation.
library ncurses can help you, but if you want to use a low level function use write() and if you want to do kernel programming you have to use printk()
I'm using MPLAB C32. I want to map my peripherals in streams, in order to do something like:
FILE* uart2 = fopen("UART2", 9600, param2, param3);
or just
FILE* uart2 = fopen("UART");
and then use fprintf to write on it:
fprintf(uart2, fmt, params);
What's the usual way of creating a FILE? Without having a filesystem, I just want to map peripherals on it.
You have to write drivers for each peripheral. Also you have to redefine your FILE* so you can have enough information to call appropriate driver. And also you need to redefine fopen and fprintf functions too. But this is pointless. I don't recommend this kind of practice. There's already built library PIC32 Peripheral Library is out. I recommend to use it.
According to the MPLAB C Compiler User's Guide, Document DS51686B, (http://ww1.microchip.com/downloads/en/DeviceDoc/51686B.pdf), the library has support for wiring up devices to the stdio facilities. An as luck would have it, UART 2 is set to stdin/stdout/stderr by default, but only output will work since _mon_getc isn't implemented by default. If you define your own, that should enable input from UART 2 via stdin.
2.3 Standard I/O:
The standard input/output library functions support two modes of
operation, Simple and Full. The Simple mode supports I/O via a two
function interface on a single character device used for stdout, stdin
and stderr. The Full mode supports the complete set of standard I/O
functions. The library will use Full mode if the application calls
fopen, otherwise Simple mode is used.
Simple mode performs I/O using four functions, _mon_puts, _mon_write,
_mon_getc and _mon_putc, to perform the raw device I/O. The default
implementation of _mon_getc always returns failure (i.e., by default,
character input is not available). The default implementation of
_mon_putc writes a character to UART2. It is assumed that the
application has performed any necessary initialization of the UART.
The default implementations of _mon_puts and _mon_write both simply
call _mon_putc iteratively. All four functions are defined as weak
functions, and so may be overridden by the user application if
different functionality is desired. See the “32-Bit Language Tools
Libraries” for detailed information on these functions.
If you need more control than that, a description of how to customize the runtime to 'connect' your devices to the stdio facilities of the compiler's runtime library is documented in the MPLAB C32 "32-Bit Language Tools Libraries" document DS51685 (http://ww1.microchip.com/downloads/en/DeviceDoc/MPLAB%20C32%20Libraries.pdf).
It looks like most of the functions required to support 'full mode' are documented in "2.18 MISCELLANEOUS FUNCTIONS"
I wrote a custom (VERY basic "Hello world!") bootloader in Assembler and I would like to execute a C program in that. Would the C program work, or fail due to a lost stdio.h file? And how could I bundle the C program along with the bootloader into a single .bin file to dd to a flash drive/CD?
I'm not sure what you mean by "lost stdio.h", but many C runtime functions, including those prototyped in stdio.h, are implemented using system calls. Without an OS running, those system calls won't work.
It is possible to write C code that runs without an OS, for example most common bootloaders have just a tiny amount of assembler and mostly C code. The trick is to avoid using runtime libraries. Alternatives to syscalls, for e.g. display, are BIOS calls and hardware-specific I/O.
To take just one example, in addition to dynamic allocation, fopen in read mode needs the following low-level operations:
Reading a block of data from storage
Reading the file system metadata (often, superblock and root directory)
Processing file system metadata to find out where the file content is stored
Creating a FILE object that contains enough information for fread and fgetc to find the data on disk
You don't have an OS to help with any of that, your C code will need to implement a driver (possibly calling the BIOS) for block read, and implement the behavior of the other steps.