I am not actually sure where the best place to post this is as it is a combination of newlib code, FreeRTOS and a custom implementation. Application is embedded ARM using GCC (arm-eabi...), newlib from standard GCC for ARM installation, FreeRTOS on embedded target (STM32).
The root issue is that calling fprint(stderr, "error\n") fails the first time it is called. Calling printf before solves the issue.
Fails
int err = fprint(stderr, "error\n"); // err = -1, failed!
OK
int err = printf("hi\n"); // err = 3, OK
err = fprint(stderr, "error\n"); // err = 6, OK
I have found the cause of this issue, its a bit long winded to explain, but it comes down to threads reent structure being not fully initialised until the first call of a std out function that implicitly uses stdout, stderr or stdin. e.g. printf, puts etc.
I will try explain what is going on:
When a thread is created, FreeRTOS initiallises the threads reent struct using the default initialiser. This initialises the std file pointers to point to the FILE structures in the reent struct itself.
FreeRTOS tasks.c initialises the tasks reent struct:
#if ( configUSE_NEWLIB_REENTRANT == 1 )
{
/* Initialise this task's Newlib reent structure. */
_REENT_INIT_PTR( ( &( pxNewTCB->xNewLib_reent ) ) );
}
#endif
_REENT_INIT_PTR sets the std file pointers in the reent struct to the file descriptors in the reent struct itself (https://github.com/eblot/newlib/blob/master/newlib/libc/include/sys/reent.h):
#define _REENT_INIT_PTR(var) \
{ memset((var), 0, sizeof(*(var))); \
(var)->_stdin = &(var)->__sf[0]; \
(var)->_stdout = &(var)->__sf[1]; \
(var)->_stderr = &(var)->__sf[2]; \
The file descriptors in the reent struct are by default zeroed (first line memset) so invalid.
Calling printf causes a call to initialise the reent struct. This is done by calling __sinit (via _REENT_SMALL_CHECK_INIT) if the structure has not been initialised before (https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/printf.c).
int
_DEFUN(printf, (fmt),
const char *fmt _DOTS)
{
int ret;
va_list ap;
struct _reent *ptr = _REENT;
_REENT_SMALL_CHECK_INIT (ptr);
va_start (ap, fmt);
ret = _vfprintf_r (ptr, _stdout_r (ptr), fmt, ap);
va_end (ap);
return ret;
}
(https://github.com/eblot/newlib/blob/master/newlib/libc/include/sys/reent.h)
# define _REENT_SMALL_CHECK_INIT(ptr) \
do \
{ \
if ((ptr) && !(ptr)->__sdidinit) \
__sinit (ptr); \
} \
while (0)
__sinit does a whole lot of work including initialising the global reeent structure (if it has not been initialised), and copying the global reent structures stdio file pointers to the tasks local stdio file points, thereby making them valid. __sinit definiton is in https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/findfp.c.
Note that fprintf does not call __sinit, so in this case, the stderr file pointer is used uninitialised if fprintf is called before printf (https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/fprintf.c).
int
_DEFUN(fprintf, (fp, fmt),
FILE *fp _AND
const char *fmt _DOTS)
{
int ret;
va_list ap;
va_start (ap, fmt);
ret = _vfprintf_r (_REENT, fp, fmt, ap);
va_end (ap);
return ret;
}
So, while I am not claiming that anything is broken, I am not sure how the tasks local reent structure is meant to get initialised before any calls to fprintf. A simple printf at the start of the thread would resolve this. But it would mean I need to do that at the start of every task that might use fprintf. By default assert calls fprintf, so I would need to have a printf at the start of any thread that might use assert (actually how this issue was found).
I am interested to hear any feedback or advice on this one. I already have a workaround for this application (custom assert function), but I would like to understand and learn a bit more about what is going on here.
Related
I'm implementing my own fast_malloc() to replace malloc(). I need debugging prints inside it. Are there any print calls which are guaranteed to NOT ever call malloc(), or do I need to create my own, safe versions?
Previously, I had accidentally caused infinite recursion by having my malloc() call printf(), which then calls malloc(), which then calls printf()...forever.
If I need to create my own safe versions which use a fixed-size static array under the hood as the buffer to format into, that's all I need to know. I can do that.
How about puts() or putc()? They should be safe, no?
I'm on Linux Ubuntu 20.04. Ideally, whatever I do will be cross-platform-compliant, but I suppose I can customize if I need to for low-level system calls.
Related:
Related, but not a duplicate since it is specific to snprintf(): snprintf that calls malloc, or snprintf that does not call malloc
Does fprintf use malloc() under the hood?
Are there any print calls which are guaranteed to NOT ever call malloc()
Usually you can call fprintf(stderr, ...). This is because stderr is unbuffered by default.
However, this may not work from within malloc early in the process lifetime, before the rest of libc has initialized itself.
Your best bet is to use write(2).
Here is a safe_printf() function which never calls malloc()!
I ended up writing a safe_printf() function which never calls malloc(), in case anyone ever needs it. I've tested it, and it works fine inside malloc(). It uses the write() system call (not std C) to write the chars to stdout.
Here it is:
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdarg.h>
#ifdef DEBUG
/// Debug printf function.
/// See: https://stackoverflow.com/a/1941336/4561887
#define DEBUG_PRINTF(...) safe_printf("DEBUG: "__VA_ARGS__)
#define DEBUG_ASSERT(test_condition) assert(test_condition)
#else
#define DEBUG_PRINTF(...) \
do \
{ \
} while (0)
#define DEBUG_ASSERT(test_condition) \
do \
{ \
} while (0)
#endif
/// \brief A **safe** version of `printf()`, meaning it NEVER calls `malloc()`, unlike
/// `printf()`, which may.
/// \details This is achieved by using a fixed-size buffer to format strings into. You may see
/// a much simpler implementation from the Hacettepe University in Turkey, here, as a
/// basic example:
/// https://web.cs.hacettepe.edu.tr/~bbm341/codes/ecf/signals/safe_printf.c
#define SAFE_PRINTF_BUF_SIZE 2048
__attribute__((__format__(printf, 1, 2)))
int safe_printf(const char *format, ...)
{
static char buf[SAFE_PRINTF_BUF_SIZE];
// number of chars successfully written to `buf` and which we need to print
int chars_to_print = 0;
va_list args;
va_start(args, format);
int char_count_or_error = vsnprintf(buf, sizeof(buf), format, args);
va_end(args);
DEBUG_ASSERT(char_count_or_error >= 0); // if fails: ENCODING ERROR
DEBUG_ASSERT(char_count_or_error < (int)sizeof(buf)); // if fails: BUFFER IS TOO SMALL
if (char_count_or_error < 0)
{
// ERROR: ENCODING ERROR
return char_count_or_error;
}
else if (char_count_or_error >= (int)sizeof(buf))
{
// ERROR: BUFFER IS TOO SMALL to write the full character sequence!
chars_to_print = sizeof(buf) - 1;
char_count_or_error = -char_count_or_error; // make negative to show it was an error
}
else // char_count_or_error >= 0 && char_count_or_error < sizeof(buf)
{
// No error
chars_to_print = char_count_or_error;
}
ssize_t num_bytes_written = write(STDOUT_FILENO, buf, chars_to_print);
DEBUG_ASSERT(num_bytes_written >= 0); // If fails: failure to write
// Return BUFFER IS TOO SMALL error
if (char_count_or_error < 0)
{
return char_count_or_error;
}
return num_bytes_written;
}
References:
The comment by #kaylum:
write(STDOUT_FILENO, str, strlen(str))
The answer by #Employed Russian
A much simpler implementation from the Hacettepe University in Turkey, here, as a basic example: https://web.cs.hacettepe.edu.tr/~bbm341/codes/ecf/signals/safe_printf.c
Linux write(2) reference page: https://man7.org/linux/man-pages/man2/write.2.html
The vsnprintf() usage example here: http://www.cplusplus.com/reference/cstdio/vsnprintf/
stdarg.h reference: https://www.cplusplus.com/reference/cstdarg/
See also:
[my answer: this is where I identified the infinite recursion problem where printf() calls malloc(), which calls printf(), repeatedly; that is what motivated me to write the safe_printf() implementation above!] "Segmentation fault (core dumped)" for: "No such file or directory" for libioP.h, printf-parse.h, vfprintf-internal.c, etc
I have a question where we need to implement internal working of open system call in gemOS.The question is as follows in the main function following
int create_fd = open(filename, O_CREAT|O_RDWR, O_READ|O_WRITE);
In Visual Studio Code when I checked definition of open using ctrl + enter, it lead me to following function. Lets suppose that Open system call has O_CREAT flag present, then accordingly _syscall3 will be called.
int open(char * filename, int flags, ...){
va_list ap;
long mode;
va_start(ap, flags);
mode = va_arg(ap, long);
va_end(ap);
if((flags & O_CREAT) == O_CREAT){
return _syscall3(SYSCALL_OPEN, (u64)filename, flags, mode);
}
else{
return _syscall2(SYSCALL_OPEN, (u64)filename, flags);
}
}
When I checked definition of _syscall3 it gave me following code.I was not able to understand what is written inside asm volatile command. So can anyone explain me what is happening.
static long _syscall3(int syscall_num, u64 arg1, u64 arg2, u64 arg3){
asm volatile (
"int $0x80;"
"leaveq;"
"retq;"
:::"memory"
);
return 0; /*gcc shutup!*/
}
Also when I tried printing something before asm volatile line, the function kinda stop executing. By the way the function that we need to implement for this call is following.
extern int do_regular_file_open(struct exec_context *ctx, char* filename, u64 flags, u64 mode){
/**
* TODO Implementation of file open,
* You should be creating file(use the alloc_file function to creat file),
* To create or Get inode use File system function calls,
* Handle mode and flags
* Validate file existence, Max File count is 16, Max Size is 4KB, etc
* Incase of Error return valid Error code
* */
return 100;
}
I was trying to map Open function written in main file to this do_regular_file_open, but due to asm volatile I wasn't able to understand whats happening.
This question already has answers here:
How are variable arguments implemented in gcc?
(3 answers)
Closed 7 years ago.
I'm reading Johnson M. Hart Windows System programming. He has a method that makes use of the va_start standard c method. In the below sample can someone explain why the Handle hOut is being passed to va_start?
BOOL PrintStrings (HANDLE hOut, ...)
/* Write the messages to the output handle. Frequently hOut
will be standard out or error, but this is not required.
Use WriteConsole (to handle Unicode) first, as the
output will normally be the console. If that fails, use WriteFile.
hOut: Handle for output file.
... : Variable argument list containing TCHAR strings.
The list must be terminated with NULL. */
{
DWORD msgLen, count;
LPCTSTR pMsg;
va_list pMsgList; /* Current message string. */
va_start (pMsgList, hOut); /* Start processing msgs. */
while ((pMsg = va_arg (pMsgList, LPCTSTR)) != NULL) {
msgLen = lstrlen (pMsg);
if (!WriteConsole (hOut, pMsg, msgLen, &count, NULL)
&& !WriteFile (hOut, pMsg, msgLen * sizeof (TCHAR), &count, NULL)) {
va_end (pMsgList);
return FALSE;
}
}
va_end (pMsgList);
return TRUE;
}
va_start is a macro that pulls of variadic parameters. It works by being given the parameter immediately before the first variadic parameter. The point is that va_start needs to know where the variadic parameters can be found. They are found immediately after the last named parameters. Hence the use of hOut.
Some details on how variadic parameters are typically implemented:
How are variable arguments implemented in gcc?
http://blog.aaronballman.com/2012/06/how-variable-argument-lists-work-in-c/
Write a program in C using only low-level I/O..
The program must have complete error trapping. In particular the
program should use perror() to report system errors...
In my program...
test("checked argument count");
if((input_file1 = open(argv[1], O_RDONLY)) < 0)
{
test("couldn't open file1");
perror(argv[1]);
close(input_file1);
exit(1);
}
test("opened file1");
Would this be considered "complete error trapping" if I implement such code for every read/write attempt?
Note: test() is just for debugging and will be deleted later:
void test(const char * message)
{
printf("\ttesting: %s \n", message);
}
You shouldn't close the file descriptor that you failed to open.
Other than that, yes, you've done sufficient error checking on the open() call. Now repeat for the other open() calls, and the read() calls, and the write() calls, and presumably the close() calls that are part of the main-line processing — close() calls in the error paths are a best-effort and don't need to be error checked in the same way.
Your error reporting is not very helpful, though. You say 'file1' but that isn't the name of the file. Using perror() isn't going to help much there, either; I never use it because it doesn't give me enough control over the message format. You are passing the file name as the string; that's considerably better than people often do, but you can't also express which operation the program was attempting that failed. I'd use fprintf(stderr, ...) in conjunction with errno and strerror(). Be careful not to clobber errno by calling a function that itself sets errno (is your test() function safe?). If you aren't sure, capture errno and (if necessary) reset it to the captured value:
int errnum = errno;
test("couldn't open file", argv[1]);
errno = errnum;
perror(argv[1]);
exit(1);
The revised test() function might be:
#include <stdarg.h>
extern void test(char const *fmt, ...);
void test(char const *fmt, ...)
{
va_list args;
va_start(args, fmt);
vfprintf(stderr, fmt, args);
va_end(args);
putc('\n', stderr);
}
That's the core of it; you'd need to adapt that to work with your current internals of the test() function. The declaration of test() with the ellipsis does not require the <stdarg.h> header; the implementation of test() does require the header.
I'm writing an assignment which involves adding some functionality to PostgreSQL on a Solaris box. As part of the assignment, we need to print some information on the client side (i.e.: using elog.)
PostgreSQL already has lots of helper methods which print out the required information, however, the helper methods are packed with hundreds of printf calls, and the elog method only works with c-style strings.
Is there I way that I could temporarily redirect printf calls to a buffer so I could easily send it over elog to the client?
If that's not possible, what would be the simplest way to modify the helper methods to end up with a buffer as output?
If you define your own version of printf and link to it prior to the libc version, your version will supersede the standard version.
You should also be able to supersede the standard version by using LD_PRELOAD to load a library that has printf defined.
To write your own printf, you will want to use stdarg functionality:
int printf(const char *fmt, ...)
{
int rv;
va_list ap;
va_start(ap, fmt);
if (redirect_printf)
{
#ifdef HAVE_VLOG
// If you have a vlog function that takes a va_list
vlog(fmt, ap);
rv = ...;
#else
char buffer[LARGESIZE];
rv = vsnprintf(buffer, sizeof(buffer), fmt, ap);
log(buffer);
#endif;
}
else
{
rv = vprintf(fmt, ap);
}
return rv;
}
This simple version will truncate data when the final formatted output is greater than LARGESIZE. If you don't want that, you can also call vsnprintf first with a NULL buffer to get the final size, do a dynamic allocation and then a second call to vsprintf to format the buffer.
You're wrong — elog supports format strings just like printf. Here's an example from Postgres source code:
elog(DEBUG4, "TZ \"%s\" gets max score %d", tzname, i);
So all you need is to add elog where there is printf using the same parameters.
The simplest way is to modify the helper methods to call sprintf(). Whether or not you can hack that in easily, I don't know. Maybe
#define printf(...) sprintf(buffer, __VA_ARGS__)
Will do it for you. You'll still need to define buffer for each helper function, and get its contents returned to whoever cares about them.
If you can tolerate the use of a temporary file you could redirect standard out with the freopen() call:-
newstdout = freopen("/tmp/log", "w", stdout);
This will force all the printf's to be written to /tmp/log instead of the console output. At some convenient point later in your program you could then open the same file for reading:-
readfd = fopen("/tmp/log", "r");
and forward the contents that have been added using something like this:-
void forward_to_elog(void)
{
int bytesread;
char buf[100];
memset(buf,0,100);
do {
memset(buf,0,100);
bytesread = fread(buf, sizeof(buf)-1, 1, readfd);
/* call elog(buf) */ ;
} while(bytesread);
}
If you keep the file open you can call forward_to_elog() multiple times to incrementally forward the contents that have been added.
The tmpnam() function can be used to get a name for the temporary file if you don't want to have to statically code one.