This question already has answers here:
How are variable arguments implemented in gcc?
(3 answers)
Closed 7 years ago.
I'm reading Johnson M. Hart Windows System programming. He has a method that makes use of the va_start standard c method. In the below sample can someone explain why the Handle hOut is being passed to va_start?
BOOL PrintStrings (HANDLE hOut, ...)
/* Write the messages to the output handle. Frequently hOut
will be standard out or error, but this is not required.
Use WriteConsole (to handle Unicode) first, as the
output will normally be the console. If that fails, use WriteFile.
hOut: Handle for output file.
... : Variable argument list containing TCHAR strings.
The list must be terminated with NULL. */
{
DWORD msgLen, count;
LPCTSTR pMsg;
va_list pMsgList; /* Current message string. */
va_start (pMsgList, hOut); /* Start processing msgs. */
while ((pMsg = va_arg (pMsgList, LPCTSTR)) != NULL) {
msgLen = lstrlen (pMsg);
if (!WriteConsole (hOut, pMsg, msgLen, &count, NULL)
&& !WriteFile (hOut, pMsg, msgLen * sizeof (TCHAR), &count, NULL)) {
va_end (pMsgList);
return FALSE;
}
}
va_end (pMsgList);
return TRUE;
}
va_start is a macro that pulls of variadic parameters. It works by being given the parameter immediately before the first variadic parameter. The point is that va_start needs to know where the variadic parameters can be found. They are found immediately after the last named parameters. Hence the use of hOut.
Some details on how variadic parameters are typically implemented:
How are variable arguments implemented in gcc?
http://blog.aaronballman.com/2012/06/how-variable-argument-lists-work-in-c/
Related
I am not actually sure where the best place to post this is as it is a combination of newlib code, FreeRTOS and a custom implementation. Application is embedded ARM using GCC (arm-eabi...), newlib from standard GCC for ARM installation, FreeRTOS on embedded target (STM32).
The root issue is that calling fprint(stderr, "error\n") fails the first time it is called. Calling printf before solves the issue.
Fails
int err = fprint(stderr, "error\n"); // err = -1, failed!
OK
int err = printf("hi\n"); // err = 3, OK
err = fprint(stderr, "error\n"); // err = 6, OK
I have found the cause of this issue, its a bit long winded to explain, but it comes down to threads reent structure being not fully initialised until the first call of a std out function that implicitly uses stdout, stderr or stdin. e.g. printf, puts etc.
I will try explain what is going on:
When a thread is created, FreeRTOS initiallises the threads reent struct using the default initialiser. This initialises the std file pointers to point to the FILE structures in the reent struct itself.
FreeRTOS tasks.c initialises the tasks reent struct:
#if ( configUSE_NEWLIB_REENTRANT == 1 )
{
/* Initialise this task's Newlib reent structure. */
_REENT_INIT_PTR( ( &( pxNewTCB->xNewLib_reent ) ) );
}
#endif
_REENT_INIT_PTR sets the std file pointers in the reent struct to the file descriptors in the reent struct itself (https://github.com/eblot/newlib/blob/master/newlib/libc/include/sys/reent.h):
#define _REENT_INIT_PTR(var) \
{ memset((var), 0, sizeof(*(var))); \
(var)->_stdin = &(var)->__sf[0]; \
(var)->_stdout = &(var)->__sf[1]; \
(var)->_stderr = &(var)->__sf[2]; \
The file descriptors in the reent struct are by default zeroed (first line memset) so invalid.
Calling printf causes a call to initialise the reent struct. This is done by calling __sinit (via _REENT_SMALL_CHECK_INIT) if the structure has not been initialised before (https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/printf.c).
int
_DEFUN(printf, (fmt),
const char *fmt _DOTS)
{
int ret;
va_list ap;
struct _reent *ptr = _REENT;
_REENT_SMALL_CHECK_INIT (ptr);
va_start (ap, fmt);
ret = _vfprintf_r (ptr, _stdout_r (ptr), fmt, ap);
va_end (ap);
return ret;
}
(https://github.com/eblot/newlib/blob/master/newlib/libc/include/sys/reent.h)
# define _REENT_SMALL_CHECK_INIT(ptr) \
do \
{ \
if ((ptr) && !(ptr)->__sdidinit) \
__sinit (ptr); \
} \
while (0)
__sinit does a whole lot of work including initialising the global reeent structure (if it has not been initialised), and copying the global reent structures stdio file pointers to the tasks local stdio file points, thereby making them valid. __sinit definiton is in https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/findfp.c.
Note that fprintf does not call __sinit, so in this case, the stderr file pointer is used uninitialised if fprintf is called before printf (https://github.com/eblot/newlib/blob/master/newlib/libc/stdio/fprintf.c).
int
_DEFUN(fprintf, (fp, fmt),
FILE *fp _AND
const char *fmt _DOTS)
{
int ret;
va_list ap;
va_start (ap, fmt);
ret = _vfprintf_r (_REENT, fp, fmt, ap);
va_end (ap);
return ret;
}
So, while I am not claiming that anything is broken, I am not sure how the tasks local reent structure is meant to get initialised before any calls to fprintf. A simple printf at the start of the thread would resolve this. But it would mean I need to do that at the start of every task that might use fprintf. By default assert calls fprintf, so I would need to have a printf at the start of any thread that might use assert (actually how this issue was found).
I am interested to hear any feedback or advice on this one. I already have a workaround for this application (custom assert function), but I would like to understand and learn a bit more about what is going on here.
I'm implementing my own fast_malloc() to replace malloc(). I need debugging prints inside it. Are there any print calls which are guaranteed to NOT ever call malloc(), or do I need to create my own, safe versions?
Previously, I had accidentally caused infinite recursion by having my malloc() call printf(), which then calls malloc(), which then calls printf()...forever.
If I need to create my own safe versions which use a fixed-size static array under the hood as the buffer to format into, that's all I need to know. I can do that.
How about puts() or putc()? They should be safe, no?
I'm on Linux Ubuntu 20.04. Ideally, whatever I do will be cross-platform-compliant, but I suppose I can customize if I need to for low-level system calls.
Related:
Related, but not a duplicate since it is specific to snprintf(): snprintf that calls malloc, or snprintf that does not call malloc
Does fprintf use malloc() under the hood?
Are there any print calls which are guaranteed to NOT ever call malloc()
Usually you can call fprintf(stderr, ...). This is because stderr is unbuffered by default.
However, this may not work from within malloc early in the process lifetime, before the rest of libc has initialized itself.
Your best bet is to use write(2).
Here is a safe_printf() function which never calls malloc()!
I ended up writing a safe_printf() function which never calls malloc(), in case anyone ever needs it. I've tested it, and it works fine inside malloc(). It uses the write() system call (not std C) to write the chars to stdout.
Here it is:
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdarg.h>
#ifdef DEBUG
/// Debug printf function.
/// See: https://stackoverflow.com/a/1941336/4561887
#define DEBUG_PRINTF(...) safe_printf("DEBUG: "__VA_ARGS__)
#define DEBUG_ASSERT(test_condition) assert(test_condition)
#else
#define DEBUG_PRINTF(...) \
do \
{ \
} while (0)
#define DEBUG_ASSERT(test_condition) \
do \
{ \
} while (0)
#endif
/// \brief A **safe** version of `printf()`, meaning it NEVER calls `malloc()`, unlike
/// `printf()`, which may.
/// \details This is achieved by using a fixed-size buffer to format strings into. You may see
/// a much simpler implementation from the Hacettepe University in Turkey, here, as a
/// basic example:
/// https://web.cs.hacettepe.edu.tr/~bbm341/codes/ecf/signals/safe_printf.c
#define SAFE_PRINTF_BUF_SIZE 2048
__attribute__((__format__(printf, 1, 2)))
int safe_printf(const char *format, ...)
{
static char buf[SAFE_PRINTF_BUF_SIZE];
// number of chars successfully written to `buf` and which we need to print
int chars_to_print = 0;
va_list args;
va_start(args, format);
int char_count_or_error = vsnprintf(buf, sizeof(buf), format, args);
va_end(args);
DEBUG_ASSERT(char_count_or_error >= 0); // if fails: ENCODING ERROR
DEBUG_ASSERT(char_count_or_error < (int)sizeof(buf)); // if fails: BUFFER IS TOO SMALL
if (char_count_or_error < 0)
{
// ERROR: ENCODING ERROR
return char_count_or_error;
}
else if (char_count_or_error >= (int)sizeof(buf))
{
// ERROR: BUFFER IS TOO SMALL to write the full character sequence!
chars_to_print = sizeof(buf) - 1;
char_count_or_error = -char_count_or_error; // make negative to show it was an error
}
else // char_count_or_error >= 0 && char_count_or_error < sizeof(buf)
{
// No error
chars_to_print = char_count_or_error;
}
ssize_t num_bytes_written = write(STDOUT_FILENO, buf, chars_to_print);
DEBUG_ASSERT(num_bytes_written >= 0); // If fails: failure to write
// Return BUFFER IS TOO SMALL error
if (char_count_or_error < 0)
{
return char_count_or_error;
}
return num_bytes_written;
}
References:
The comment by #kaylum:
write(STDOUT_FILENO, str, strlen(str))
The answer by #Employed Russian
A much simpler implementation from the Hacettepe University in Turkey, here, as a basic example: https://web.cs.hacettepe.edu.tr/~bbm341/codes/ecf/signals/safe_printf.c
Linux write(2) reference page: https://man7.org/linux/man-pages/man2/write.2.html
The vsnprintf() usage example here: http://www.cplusplus.com/reference/cstdio/vsnprintf/
stdarg.h reference: https://www.cplusplus.com/reference/cstdarg/
See also:
[my answer: this is where I identified the infinite recursion problem where printf() calls malloc(), which calls printf(), repeatedly; that is what motivated me to write the safe_printf() implementation above!] "Segmentation fault (core dumped)" for: "No such file or directory" for libioP.h, printf-parse.h, vfprintf-internal.c, etc
I used the code below that uses the "{}" but seems its not working as expected when using C.
int basic_regx(char *format, char *name)
{
regex_t regex;
char array[100];
if( regcomp(®ex, format, 0) )
return;
if( !regexec(®ex, name, 0, NULL, 0) )
printf ("Succeeded\n");
else
printf ("Not Succeeded\n");
regfree(®ex);
}
If I call function with follwoing:
Success - basic_regx("^[0-9]$","0");
Not Success -- basic_regx("^[0-9]{1,4}$","0");
Success - basic_regx("^[0-9]{1,4}$","0{1,4}");
Thats mean the {} is not taken as expected by the reg implementation.
regcomp and regexec use POSIX regular expressions; in other words, it only supports features described in POSIX. If you want Perl-style regular expressions, which support this expression, you may need an external library such as PCRE.
However, with POSIX regular expressions, you can do the equivalent:
basic_regx("^[0-9]\\{1,4\\}$","0");
I have one problem with variable length argument debug log print function. I will just simulate the code here.
void SecondPrint(int level, const char* format,...)
{
//Printing the log here
va_list arg;
va_start(arg, format);
vprintf(format, arg);
va_end(arg);
}
void FirstPrint(int level, const char* format,...)
{
SecondPrint(level,format);
}
void main()
{
int level = 100;
FirstPrintf("Log level = %d message = %s \n",level,"Error message");
}
"SecondPrint" is supposed to print "100 Error message" as expected, But its not printing like that its printing " Error message".
I am not getting whats wrong with this one. I am suspecting the way of calling "SecondPrint" from "FirstPrint" function. FirstPrint is receiving remaining arguments through ... but its invoking SecondPrint with level and format arguments.
I can't use SecondPrint from main directly. I have to use FirstPrint and FirstPrint has to call SecondPrint to print the log. So how can I achieve this one.. I thought to use VA_ARGS but it is for only macro definitions not in function definition.
And one more thing I can't do like *#define FirstPrint(a,b,...) SecondPrint(a,b,...)*
any help is highly appreciated thanks in advance.
C varargs are not designed to be passed more than one level; the kind of necessary stack manipulation required is too deep for the language. Typically, in a case like this, you would have a version of SecondPrint analagous to vprintf -- SecondPrintV or similar, and you would have FirstPrint invoke SecondPrintV after extracting the varargs, rather than invoking SecondPrint directly (and, for consistency, usually have SecondPrint invoke SecondPrintV internally).
You need to pass a va_list to SecondPrint, so it can access the arguments to FirstPrint
void SecondPrint(int level, const char* format, va_list args)
{
//Printing the log here
vprintf(format, arg);
}
void FirstPrint(int level, const char* format,...)
{
va_list args;
va_start(args, format);
SecondPrint(level, format, args);
va_end(args);
}
void main()
{
int level = 100;
FirstPrintf("Log level = %d message = %s \n",level,"Error message");
}
It may be possible to do it in specific platform-specific ways, or (since the arguments are exactly the same in the same order) use assembly to "replace" the current function frame with the function to call. GCC also has an extension to do it with the __builtin_apply_args() and __builtin_apply() functions (although you must somehow figure out how to compute the size of the stack argument data).
I'm writing an assignment which involves adding some functionality to PostgreSQL on a Solaris box. As part of the assignment, we need to print some information on the client side (i.e.: using elog.)
PostgreSQL already has lots of helper methods which print out the required information, however, the helper methods are packed with hundreds of printf calls, and the elog method only works with c-style strings.
Is there I way that I could temporarily redirect printf calls to a buffer so I could easily send it over elog to the client?
If that's not possible, what would be the simplest way to modify the helper methods to end up with a buffer as output?
If you define your own version of printf and link to it prior to the libc version, your version will supersede the standard version.
You should also be able to supersede the standard version by using LD_PRELOAD to load a library that has printf defined.
To write your own printf, you will want to use stdarg functionality:
int printf(const char *fmt, ...)
{
int rv;
va_list ap;
va_start(ap, fmt);
if (redirect_printf)
{
#ifdef HAVE_VLOG
// If you have a vlog function that takes a va_list
vlog(fmt, ap);
rv = ...;
#else
char buffer[LARGESIZE];
rv = vsnprintf(buffer, sizeof(buffer), fmt, ap);
log(buffer);
#endif;
}
else
{
rv = vprintf(fmt, ap);
}
return rv;
}
This simple version will truncate data when the final formatted output is greater than LARGESIZE. If you don't want that, you can also call vsnprintf first with a NULL buffer to get the final size, do a dynamic allocation and then a second call to vsprintf to format the buffer.
You're wrong — elog supports format strings just like printf. Here's an example from Postgres source code:
elog(DEBUG4, "TZ \"%s\" gets max score %d", tzname, i);
So all you need is to add elog where there is printf using the same parameters.
The simplest way is to modify the helper methods to call sprintf(). Whether or not you can hack that in easily, I don't know. Maybe
#define printf(...) sprintf(buffer, __VA_ARGS__)
Will do it for you. You'll still need to define buffer for each helper function, and get its contents returned to whoever cares about them.
If you can tolerate the use of a temporary file you could redirect standard out with the freopen() call:-
newstdout = freopen("/tmp/log", "w", stdout);
This will force all the printf's to be written to /tmp/log instead of the console output. At some convenient point later in your program you could then open the same file for reading:-
readfd = fopen("/tmp/log", "r");
and forward the contents that have been added using something like this:-
void forward_to_elog(void)
{
int bytesread;
char buf[100];
memset(buf,0,100);
do {
memset(buf,0,100);
bytesread = fread(buf, sizeof(buf)-1, 1, readfd);
/* call elog(buf) */ ;
} while(bytesread);
}
If you keep the file open you can call forward_to_elog() multiple times to incrementally forward the contents that have been added.
The tmpnam() function can be used to get a name for the temporary file if you don't want to have to statically code one.