I'm trying to write a portable program that deals with ustar archives. For device files, these archives store the major and minor device numbers. However, the struct stat as laid out in POSIX only contains a single st_rdev member of type dev_t described with “Device ID (if file is character or block special).”
How can I convert between a pair of major and minor device numbers and a single st_rdev member as returned by stat() in a portable manner?
While all POSIX programming interfaces use the device number (of type dev_t) as is, FUZxxl pointed out in a comment to this answer that the common UStar file format -- most common tar archive format -- does split the device number into major and minor. (They are typically encoded as seven octal digits each, so for compatibility reasons one should limit to 21-bit unsigned major and 21-bit unsigned minor. This also means that mapping the device number to just the major or just the minor is not a reliable approach.)
The following include file expanding on Jonathon Reinhart's answer, after digging on the web for the various systems man pages and documentation (for makedev(), major(), and minor()), plus comments to this question.
#if defined(custom_makedev) && defined(custom_major) && defined(custom_minor)
/* Already defined */
#else
#undef custom_makedev
#undef custom_major
#undef custom_minor
#if defined(__linux__) || defined(__GLIBC__)
/* Linux, Android, and other systems using GNU C library */
#ifndef _BSD_SOURCE
#define _BSD_SOURCE 1
#endif
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#elif defined(_WIN32)
/* 32- and 64-bit Windows. VERIFY: These are just a guess! */
#define custom_makedev(dmajor, dminor) ((((unsigned int)dmajor << 8) & 0xFF00U) | ((unsigned int)dminor & 0xFFFF00FFU))
#define custom_major(devnum) (((unsigned int)devnum & 0xFF00U) >> 8)
#define custom_minor(devnum) ((unsigned int)devnum & 0xFFFF00FFU)
#elif defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__) || defined(__DragonFly__)
/* FreeBSD, OpenBSD, NetBSD, and DragonFlyBSD */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#elif defined(__APPLE__) && defined(__MACH__)
/* Mac OS X */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#elif defined(_AIX) || defined (__osf__)
/* AIX, OSF/1, Tru64 Unix */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#elif defined(hpux)
/* HP-UX */
#include <sys/sysmacros.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#elif defined(sun)
/* Solaris */
#include <sys/types.h>
#include <sys/mkdev.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#else
/* Unknown OS. Try a the BSD approach. */
#ifndef _BSD_SOURCE
#define _BSD_SOURCE 1
#endif
#include <sys/types.h>
#if defined(makedev) && defined(major) && defined(minor)
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum) major(devnum)
#define custom_minor(devnum) minor(devnum)
#endif
#endif
#if !defined(custom_makedev) || !defined(custom_major) || !defined(custom_minor)
#error Unknown OS: please add definitions for custom_makedev(), custom_major(), and custom_minor(), for device number major/minor handling.
#endif
#endif
One could glean additional definitions from existing UStar-format -capable archivers. Compatibility with existing implementations on each OS/architecture is, in my opinion, the most important thing here.
The above should cover all systems using GNU C library, Linux (including Android), FreeBSD, OpenBSD, NetBSD, DragonFlyBSD, Mac OS X, AIX, Tru64, HP-UX, and Solaris, plus any that define the macros when <sys/types.h> is included. Of the Windows part, I'm not sure.
As I understand it, Windows uses device 0 for all normal files, and a HANDLE (a void pointer type) for devices. I am not at all sure whether the above logic is sane on Windows, but many older systems put the 8 least significant bits of the device number into minor, and the next 8 bits into major, and the convention seems to be that any leftover bits would be put (without shifting) into minor, too. Examining existing UStar-format tar archives with references to devices would be useful, but I personally do not use Windows at all.
If a system is not detected, and the system does not use the BSD-style inclusion for defining the macros, the above will error out stopping the compilation. (I would personally add compile-time machinery that could help finding the correct header definitions, using e.g. find, xargs, and grep, in case that happens, with a suggestion to send the addition upstream, too. touch empty.h ; cpp -dM empty.h ; rm -f empty.h should show all predefined macros, to help with identifying the OS and/or C library.)
Originally, POSIX stated that dev_t must be an arithmetic type (thus, theoretically, it might have been some variant of float or double on some systems), but IEEE Std 1003.1, 2013 Edition says it must be an integer type. I would wager that means no known POSIX-y system ever used a floating-point dev_t type. It would seem that Windows uses a void pointer, or HANDLE type, but Windows is not POSIX-compliant anyway.
Use the major() and minor() macros after defining BSD_SOURCE.
The makedev(), major(), and minor() functions are not specified in
POSIX.1, but are present on many other systems.
http://man7.org/linux/man-pages/man3/major.3.html
I have a program based on an antique version of ls for Minix, but much mangled modified by me since then. It has the following code to detect the major and minor macros — and some comments about (now) antique systems where it has worked in the past. It assumes a sufficiently recent version of GCC is available to support #pragma GCC diagnostic ignored etc. You have to be trying pretty hard (e.g. clang -Weverything) to get the -Wunused-macros option in effect unless you include it explicitly.
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunused-macros"
/* Defines to ensure major and minor macros are available */
#define _DARWIN_C_SOURCE /* In <sys/types.h> on MacOS X */
#define _BSD_SOURCE /* In <sys/sysmacros.h> via <sys/types.h> on Linux (Ubuntu 12.0.4) */
#define __EXTENSIONS__ /* Maybe beneficial on Solaris */
#pragma GCC diagnostic pop
/* From Solaris 2.6 sys/sysmacros.h
**
** WARNING: The device number macros defined here should not be used by
** device drivers or user software. [...] Application software should make
** use of the library routines available in makedev(3). [...] Macro
** routines bmajor(), major(), minor(), emajor(), eminor(), and makedev()
** will be removed or their definitions changed at the next major release
** following SVR4.
**
** #define O_BITSMAJOR 7 -- # of SVR3 major device bits
** #define O_BITSMINOR 8 -- # of SVR3 minor device bits
** #define O_MAXMAJ 0x7f -- SVR3 max major value
** #define O_MAXMIN 0xff -- SVR3 max major value
**
** #define L_BITSMAJOR 14 -- # of SVR4 major device bits
** #define L_BITSMINOR 18 -- # of SVR4 minor device bits
** #define L_MAXMAJ 0x3fff -- SVR4 max major value
** #define L_MAXMIN 0x3ffff -- MAX minor for 3b2 software drivers.
** -- For 3b2 hardware devices the minor is restricted to 256 (0-255)
*/
/* AC_HEADER_MAJOR:
** - defines MAJOR_IN_MKDEV if found in sys/mkdev.h
** - defines MAJOR_IN_SYSMACROS if found in sys/macros.h
** - otherwise, hope they are in sys/types.h
*/
#if defined MAJOR_IN_MKDEV
#include <sys/mkdev.h>
#elif defined MAJOR_IN_SYSMACROS
#include <sys/sysmacros.h>
#elif defined(MAJOR_MINOR_MACROS_IN_SYS_TYPES_H)
/* MacOS X 10.2 - for example */
/* MacOS X 10.5 requires -D_DARWIN_C_SOURCE or -U_POSIX_C_SOURCE - see above */
#elif defined(USE_CLASSIC_MAJOR_MINOR_MACROS)
#define major(x) ((x>>8) & 0x7F)
#define minor(x) (x & 0xFF)
#else
/* Hope the macros are in <sys/types.h> or otherwise magically visible */
#endif
#define MAJOR(x) ((long)major(x))
#define MINOR(x) ((long)minor(x))
You will justifiably not be all that keen on the 'hope the macros are … magically visible' part of the code.
The reference to AC_HEADER_MAJOR is to the macro in the autoconf that deduces this information. It would be relevant if you have a config.h file generated by autoconf.
POSIX
Note that the POSIX pax command defines the ustar format and specifies that it includes devmajor and devminor in the information, but adds:
… Represent character special files and block special files respectively. In this case the devmajor and devminor fields shall contain information defining the device, the format of which is unspecified by this volume of POSIX.1-2008. Implementations may map the device specifications to their own local specification or may ignore the entry.
This means that there isn't a fully portable way to represent the numbers. This is not wholly unreasonable (but it is a nuisance); the meanings of the major and minor device numbers varies across platforms and is unspecified too. Any attempt to create block or character devices via ustar format will only work reasonably reliably if the source and target machines are running the same (version of the same) operating system — though usually they're portable across versions of the same operating system.
Related
I'm working on building a custom version of openwrt with a build tool and keep running into a error I cant seem to fix.
heres the code block its dating back to.
#include <signal.h>
#if ! HAVE_STACK_T && ! defined stack_t
typedef struct sigaltstack stack_t;
#endif
#ifndef SIGSTKSZ
# define SIGSTKSZ 16384
#elif HAVE_LIBSIGSEGV && SIGSTKSZ < 16384
/* libsigsegv 2.6 through 2.8 have a bug where some architectures use
more than the Linux default of an 8k alternate stack when deciding
if a fault was caused by stack overflow. */
# undef SIGSTKSZ
# define SIGSTKSZ 16384
#endif
heres the out put error
In file included from /usr/include/signal.h:328,
from ./signal.h:52,
from c-stack.c:49:
c-stack.c:55:26: error: missing binary operator before token "("
55 | #elif HAVE_LIBSIGSEGV && SIGSTKSZ < 16384
| ^~~~~~~~
First of all please note that this is not standard C but POSIX extensions. POSIX has the nasty habit of poisoning standard libraries with non-standard, non-conforming extensions.
This means that if you compile with -std=c17 or equivalent instead of -std=gnu17 (default), gcc will strip off all non-standard, non-conforming POSIX junk. And then those things will not be found at all even if you include signal.h, which in turn can give you very confusing compiler errors.
That being said, I think you perhaps meant to do:
#elif defined(HAVE_LIBSIGSEGV) && SIGSTKSZ < 16384
As for how you can find out what SIGSTKSZ is without digging through some library header, here's a simple trick that gives you the exact macro definition:
#include <stdio.h>
#include <signal.h>
#define STR(s) #s
#define WHAT_ARE_YOU(x) puts(STR(x))
int main (void)
{
WHAT_ARE_YOU(SIGSTKSZ);
}
When compiling as GNU C on some random Linux machine, I get 8192.
You could also do gcc -E to view the preprocessor output (which may be a bit confusing to read since it expands all headers). Doing that for the above example gives puts("8192");.
My machine has CRTSCTS #defined inside a #ifdef __USE_MISC which stops it being available to C programs which I compile.
/usr/include/x86_64-linux-gnu/bits/termios.h:
...
#define B4000000 0010017
#define __MAX_BAUD B4000000
#ifdef __USE_MISC
# define CIBAUD 002003600000 /* input baud rate (not used) */
# define CMSPAR 010000000000 /* mark or space (stick) parity */
# define CRTSCTS 020000000000 /* flow control */
#endif
/* c_lflag bits */
#define ISIG 0000001
#define ICANON 0000002
...
How do I get access to CRTSCTS without just hacking the value 020000000000 into my program?
I already #include <termios.h> and many other headers.
I am also using:
#define __USE_POSIX199309
#define _POSIX_C_SOURCE 199309L
#include <time.h> /* nanosleep needs lines above */
The __USE_MISC macro is an internal definition that, though I suppose you could define this yourself, it's better to find the proper feature-test macro that enables it.
On my CentOS 7 system, /usr/include/features.h has a whole range of these kinds of macros, and it appears that __USE_MISC is enabled by _BSD_SOURCE or _SVID_SOURCE; it's not clear which one is compatible with other things you're going to need.
... // features.h
#if defined _BSD_SOURCE || defined _SVID_SOURCE
# define __USE_MISC 1
#endif
Try #define _BSD_SOURCE at the top of your program - before all the includes - and see how it goes.
Ref: http://man7.org/linux/man-pages/man7/feature_test_macros.7.html
I get the error
‘CHAR_WIDTH’ undeclared
when I try to compile this simple program:
#include <stdio.h>
#include <limits.h>
int main()
{
printf("CHAR_BIT = %d\n", CHAR_BIT);
printf("CHAR_WIDTH = %d\n", CHAR_WIDTH);
return (0);
}
with
gcc ./show_char_width.c -o show_char_width
and gcc: GNU C17 (Ubuntu 8.3.0-6ubuntu1) version 8.3.0 (x86_64-linux-gnu) compiled by GNU C version 8.3.0, GMP version 6.1.2, MPFR version 4.0.2, MPC version 1.1.0, isl version isl-0.20-GMP,
kernel: 5.0.0-37-generic.
As stated here CHAR_WIDTH should be defined in limits.h which is included in my program. So why I get this error?
With the -v option I found that the library will be searched in those directories:
#include "..." search starts here:
#include <...> search starts here:
/usr/lib/gcc/x86_64-linux-gnu/8/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/8/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
/usr/lib/gcc/x86_64-linux-gnu/8/include-fixed contain a limits.h that include syslimits.h from the same dir which in turn include the next limits.h, that from my understanding should be located in the /usr/include directory.
The CHAR_WIDTH macro is indeed defined in those files but under some conditions that exceed my actual knowledge.
The conditions I found untill now are:
/* The integer width macros are not defined by GCC's <limits.h> before
GCC 7, or if _GNU_SOURCE rather than
__STDC_WANT_IEC_60559_BFP_EXT__ is used to enable this feature. */
#if __GLIBC_USE (IEC_60559_BFP_EXT)
# ifndef CHAR_WIDTH
# define CHAR_WIDTH 8
# endif
and :
#ifdef __STDC_WANT_IEC_60559_BFP_EXT__
/* TS 18661-1 widths of integer types. */
# undef CHAR_WIDTH
# define CHAR_WIDTH __SCHAR_WIDTH__
That's why I need your help.
Note: I get the same error with all other macros described in A.5.1 notably: SCHAR_WIDTH, INT_WIDTH, LONG_WIDTH, etc.
CHAR_WIDTH isn't standard, nor are any other *_WIDTH macros, but the width of a character type must be the same as CHAR_BIT anyway*.
As for *_WIDTH macros in general, they aren't strictly needed as they are compile-time-computable from the maximum value of the corresponding unsigned type, i.e., you can have a #define WIDTH_FROM_UNSIGNED_MAX(UnsignedMax) that expands to an integer constant expression that's also usable in preprocessor conditionals (#if), though implementations are a little bit obscure (see Is there any way to compute the width of an integer type at compile-time?), e.g.:
#define WIDTH_FROM_UNSIGNED_MAX(UnsignedMax) POW2_MINUS1_BITS_2039(UnsignedMax)
/* Number of bits in inttype_MAX, or in any (1<<k)-1 where 0 <= k < 2040 */
#define POW2_MINUS1_BITS_2039(X) ((X)/((X)%255+1) / 255%255*8 + 7-86/((X)%255+12))
//compile-time test it, assuming uint{8,16,32,64}_t exist
#include <inttypes.h>
#if WIDTH_FROM_UNSIGNED_MAX(UINT8_MAX) != 8
#error
#endif
#if WIDTH_FROM_UNSIGNED_MAX(UINT16_MAX) != 16
#error
#endif
#if WIDTH_FROM_UNSIGNED_MAX(UINT32_MAX) != 32
#error
#endif
#if WIDTH_FROM_UNSIGNED_MAX(UINT64_MAX) != 64
#error
#endif
Some people just do CHAR_BIT * sizeof(integer_type), but that isn't strictly portable, because it assumes integer_type doesn't have padding bits (it usually doesn't but theoretically it can have them), and can't use it in #if conditionals either.
*Unfortunately, to glean this info, you need jump all over the standard:
The width of an integer type is (slightly indirectly) defined as the number of its nonpadding bits (6.2.6.2p6).
6.2.6.2p2 says signed char doesn't have any padding bits. Because of how integers can be represented in C (6.2.6.2p2), that implies unsigned char can't have any padding bits either, and since char must have the same limits as either signed char or unsigned char (5.2.4.2.1p2) while having the same sizeof value (namely 1, 6.5.3.4p4), a plain char can't have any padding bits either, and so CHAR_BIT == width of (char|signed char|unsigned char).
Most POSIX-compatible systems provide a function to get or set one of high-resolution timers:
int clock_gettime(clockid_t clock_id, struct timespec *tp);
Documentation for each system usually lists several symbolic names as possible clock_id values, but actual numeric values are never mentioned. It turns out that not only the numeric values are different across various systems, but symbolic names are also not the same for the same meaning. Even more so, not all the clocks actually supported by a system are defined in time.h (bits/time.h) — some are only defined in, let's say, linux/time.h.
That is, in a Linux system we may have:
#define CLOCK_REALTIME 0
#define CLOCK_MONOTONIC 1
#define CLOCK_PROCESS_CPUTIME_ID 2
#define CLOCK_THREAD_CPUTIME_ID 3
#define CLOCK_MONOTONIC_RAW 4
#define CLOCK_REALTIME_COARSE 5
#define CLOCK_MONOTONIC_COARSE 6
#define CLOCK_BOOTTIME 7
#define CLOCK_REALTIME_ALARM 8
#define CLOCK_BOOTTIME_ALARM 9
#define CLOCK_SGI_CYCLE 10 // In linux/time.h only.
#define CLOCK_TAI 11 // In linux/time.h only.
In Cygwin environment (not a verbatim excerpt):
#define CLOCK_REALTIME 1 // Means CLOCK_MONOTONIC?
#define CLOCK_MONOTONIC 4 // Means CLOCK_MONOTONIC_RAW?
#define CLOCK_PROCESS_CPUTIME_ID 2
#define CLOCK_THREAD_CPUTIME_ID 3
In FreeBSD:
#define CLOCK_REALTIME 0
#define CLOCK_VIRTUAL 1
#define CLOCK_PROF 2
#define CLOCK_MONOTONIC 4
#define CLOCK_UPTIME 5 // Synonymous to CLOCK_BOOTTIME?
#define CLOCK_UPTIME_PRECISE 7
#define CLOCK_UPTIME_FAST 8
#define CLOCK_REALTIME_PRECISE 9 // Same as CLOCK_REALTIME?
#define CLOCK_REALTIME_FAST 10 // Synonymous to CLOCK_REALTIME_COARSE?
#define CLOCK_MONOTONIC_PRECISE 11 // Same as CLOCK_MONOTONIC?
#define CLOCK_MONOTONIC_FAST 12 // Synonymous to CLOCK_MONOTONIC_COARSE?
#define CLOCK_SECOND 13
#define CLOCK_THREAD_CPUTIME_ID 14
#define CLOCK_PROCESS_CPUTIME_ID 15
In AIX:
#define CLOCK_REALTIME ...
#define CLOCK_MONOTONIC ...
#define CLOCK_PROCESS_CPUTIME_ID ...
#define CLOCK_THREAD_CPUTIME_ID ...
In SunOS:
#define CLOCK_REALTIME ...
#define CLOCK_HIGHRES ... // Synonymous to CLOCK_MONOTONIC_RAW?
In QNX:
#define CLOCK_REALTIME ...
#define CLOCK_SOFTTIME ...
#define CLOCK_MONOTONIC ...
And so on.
This makes me wonder how to use clock_gettime() with the first argument other than CLOCK_REALTIME in a portable way. For example, if I want to use CLOCK_MONOTONIC_COARSE or CLOCK_BOOTTIME, how do I know that BSD calls them CLOCK_MONOTONIC_FAST and CLOCK_UPTIME, respectively, instead?
Is it wise to make assumptions based on numeric values of the first 4 symbolic names? Such as:
#define POSIX_CLOCK_REALTIME 0
#define POSIX_CLOCK_MONOTONIC 1
#define POSIX_CLOCK_PROCESS_CPUTIME_ID 2
#define POSIX_CLOCK_THREAD_CPUTIME_ID 3
#define POSIX_CLOCK_MONOTONIC_RAW 4
#if CLOCK_REALTIME == POSIX_CLOCK_MONOTONIC
#warning This platform has monotonic realtime clock.
#end if
#if CLOCK_MONOTONIC == POSIX_CLOCK_MONOTONIC_RAW
#warning This platform has undisciplined monotonic clock.
#end if
If a system actually supports CLOCK_TAI but does not define it in time.h, how can I check and use that, taking into account that the same numeric value 11 may stand for CLOCK_MONOTONIC_PRECISE or whatever in other systems?
If I was trying to write a maximally-portable program, I would limit myself to the symbolic values defined in the relevant Standard. (From what you say, it sounds like CLOCK_REALTIME -- and perhaps only that value -- is standard.)
If some systems implement useful extensions that I wanted to use, I would test for them by saying
#ifdef CLOCK_MONOTONIC_COARSE
... my code calling clock_gettime(CLOCK_MONOTONIC_COARSE, &ts) ...
#endif
And if it's really true that there are systems out there where the same symbolic values end up meaning distinctly different things, I would throw up my hands and declare that to be a portability nightmare, the kind that I, at least, try to stay far away from.
In answer to your other questions, (1) no, it's not wise to make assumptions about the symbolic values, and (2) there's no good way to use a clock that somehow does exist but is not defined in time.h (though this situation would hopefully be quite rare).
Also, are you assuming that just because the numeric value 1 is used for CLOCK_MONOTONIC on Linux but for CLOCK_REALTIME under Cygwin means that those clocks might somehow be the same? That's probably not the case. The numbers are arbitrary and you shouldn't care about them; that's why the symbolic constants exist!
I expected the code to output suse.sys but it actually prints win.sys. Why is this the case?
#define SYS SUSE
#if SYS == WIN
#define HDR "win.sys"
#elif SYS == SUSE
#define HDR "suse.sys"
#else
#define HDR "default.sys"
#endif
#include HDR
#include <stdio.h>
int main()
{
char *name = HDR;
printf("%s\n", name);
return 0;
}
This is similar to the example in the C Programming language second edition. The .sys files don't contain anything, they have no real use.
The preprocessor comparison with == works on integer values, not strings or names of macros. You should be able to fix this by first defining the macros SUSE and WIN with integer values, e.g.,
#define SUSE 1
#define WIN 2
#define SYS SUSE
After this both SYS and SUSE resolve to the integer 1, and the comparison should work.
However, I would suggest a more conventional approach of defining different macros altogether for the systems, e.g.:
#define SYS_SUSE
//#define SYS_WIN
#if defined(SYS_SUSE)
#define HDR "suse.sys"
#elif defined(SYS_WIN)
#define HDR "win.sys"
#else
#define HDR "default.sys"
#endif
This approach has the advantage of being able to more conveniently specify the system on command-line, makefiles, etc. without depending on the numeric constants being defined in every context:
cc -DSYS_WIN -c foo.c