Ubuntu 18.04
I'm trying to use statx syscall introduced in the Linux Kernel 4.11. There is a manual entry:
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h> /* Definition of AT_* constants */
int statx(int dirfd, const char *pathname, int flags,
unsigned int mask, struct statx *statxbuf);
So I tried to write an example by myself:
const char *dir_path = NULL;
const char *file_path = NULL;
//read from command line arguments
int dir_fd = open(dir_path, O_DIRECTORY);
struct statx st; //<--------------------------- compile error
statx(dir_fd, file_path, 0, &statx);
But it simply does not compile. The error is the sizeof(statx) is unknown. And actually it is not defined in sys/stat.h, but in linux/stat.h which is not included by sys/stat.h. But after including linux/stat.h the problem is there is no definition for
int statx(int dirfd, const char *pathname, int flags,
unsigned int mask, struct statx *statxbuf);
I expected that since
$ uname -r
4.15.0-39-generic
and 4.15.0-39-generic newer than 4.11 I can use it.
What's wrong?
Currently as the glibc does not provide a wrapper for the statx call, you have to use your kernels definitions. So either copy the statx structure definition from your kernel or just use it from the API the linux kernel provides. The struct statx is currently defined in linux/stat.h.
linux provides a example call to statx available here.
#update library support was added in glibc 2.28
Related
If dlsym is available in dynamic linking setup, I can get access to the original impl pointers using dlsym with RTLD_NEXT and use them in my overrides, e.g. as follows:
// paste these in main.c
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <dlfcn.h>
int open(const char *path, int flags)
{
fprintf(stderr, "log_file_access_preload: open(\"%s\", %d)\n", path, flags);
typedef int (*orig_open_func_type)(const char *pathname, int flags);
orig_open_func_type orig_func = (orig_open_func_type)dlsym(RTLD_NEXT, "open");
return orig_func(path, flags);
}
FILE* fopen(const char *path, const char *mode)
{
fprintf(stderr, "log_file_access_preload: fopen(\"%s\", \"%s\")\n", path, mode);
typedef FILE* (*orig_fopen_func_type)(const char *path, const char *mode);
orig_fopen_func_type orig_func = (orig_fopen_func_type)dlsym(RTLD_NEXT, "fopen");
return orig_func(path, mode);
}
Is there a way to do static linking in such a way that doesn't hide the original libc/POSIX symbols and so that I can use them in my overrides? Should I create my own copy of musl *.a files with renamed original symbols? Should it work? Is there another way?
Usecase: implement redirection of file read/access functions for a custom LaTeX program (compilation process is controlled by me, statically built with musl) to read files from ISO or TAR archive (that contains a prepared TeX Directory Structure) without extraction to disk
Currently I'm running Ubuntu 16.04 with linux kernel version to be 4.16. I wrote a dummy program that changes its scheduler to SCHED_DEADLINE. But when I tried to compile it, it cannot find definition of structs and macros needed for SCHED_DEADLINE. Most of the code snippet was taken from here (page 24). Below is the test program:
#define _GNU_SOURCE
#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sched.h>
int main(int argc, char* argv[]) {
struct sched_attr attr;
attr.size = sizeof(attr);
attr.sched_policy = SCHED_DEADLINE;
attr.sched_runtime = 30000000;
attr.sched_period = 100000000;
attr.sched_deadline = attr.sched_period;
if (sched_setattr(gettid(), &attr, 0))
perror("sched_setattr()");
return 0;
}
Here's the output of the compilation:
sched_deadline.c: In function ‘main’:
sched_deadline.c:11:20: error: storage size of ‘attr’ isn’t known
struct sched_attr attr;
^
sched_deadline.c:12:21: error: invalid application of ‘sizeof’ to incomplete type ‘struct attr’
attr.size = sizeof(struct attr);
^
sched_deadline.c:13:22: error: ‘SCHED_DEADLINE’ undeclared (first use in this function)
attr.sched_policy = SCHED_DEADLINE;
My gcc version:
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
However, the sample code posted in the official website works for me, but the sample code manually defines all the needed macros and system calls in the program. My goal was to compile the application without adding those definitions, which should already be included in the newest kernel version. I have seen various places saying that SCHED_DEADLINE is officially supported after Linux 3.14.10, and upgrading the kernel would automatically solve this issue.
Things I've tried:
Recompiling 4.16 kernel. Previously I thought I need to turn on a switch in config file, but I was not able to find it.
Look into /usr/include/linux/sched.h. Clearly the macros are defined in this header file, but somehow my compiler cannot find it.
I also looked into other posts in the community, but all those questions are for older linux (pre 3.14.10).
You need to include #include <linux/sched.h>
But for the definition of sched_setattr() and gettid(), see the link posted by #CraigEstey
The reason about that, it that glibc will not add function wrappers of linux specific syscall.
For example for gettid(), in the manual we can read this:
Note: There is no glibc wrapper for this system call; see NOTES.
Glibc does not provide a wrapper for this system call; call it using
syscall(2).
The thread ID returned by this call is not the same thing as a POSIX thread ID
Have a look at this article: https://lwn.net/Articles/711058/
#define _GNU_SOURCE
#include <stdint.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <sched.h>
#include <linux/sched.h>
#include <sys/types.h>
struct sched_attr {
uint32_t size;
uint32_t sched_policy;
uint64_t sched_flags;
/* SCHED_NORMAL, SCHED_BATCH */
int32_t sched_nice;
/* SCHED_FIFO, SCHED_RR */
uint32_t sched_priority;
/* SCHED_DEADLINE (nsec) */
uint64_t sched_runtime;
uint64_t sched_deadline;
uint64_t sched_period;
};
int sched_setattr(pid_t pid, const struct sched_attr *attr, unsigned int flags)
{
return syscall(__NR_sched_setattr, pid, attr, flags);
}
int main(int argc, char* argv[]) {
struct sched_attr attr = {
.size = sizeof(attr),
.sched_policy = SCHED_DEADLINE,
.sched_runtime = 30000000,
.sched_period = 100000000,
.sched_deadline = 100000000
};
pid_t tid = syscall(SYS_gettid);
if (sched_setattr(tid, &attr, 0))
perror("sched_setattr()");
return 0;
}
Or a more shorter code, without the redefinition of struct sched_attr
#define _GNU_SOURCE
#include <stdint.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <linux/sched/types.h>
#include <linux/sched.h>
#include <sys/types.h>
int sched_setattr(pid_t pid, const struct sched_attr *attr, unsigned int flags)
{
return syscall(__NR_sched_setattr, pid, attr, flags);
}
int main(int argc, char* argv[]) {
struct sched_attr attr = {
.size = sizeof(attr),
.sched_policy = SCHED_DEADLINE,
.sched_runtime = 30000000,
.sched_period = 100000000,
.sched_deadline = 100000000
};
pid_t tid = syscall(SYS_gettid);
if (sched_setattr(tid, &attr, 0))
perror("sched_setattr()");
return 0;
}
But this needs to be executed as root, otherwise I got sched_setattr(): Operation not permitted
Or the application needs to have the right linux capabilities.
I want to write a program using the new SCHED_DEADLINE scheduling policy available since Linux 3.14.
I start out with a simple program trying to use the sched_setattr function.
#include <sched.h>
int main(void)
{
// struct sched_attr attr;
// attr.size = sizeof(struct sched_attr);
// attr.sched_policy = SCHED_DEADLINE;
sched_setattr(0, (void*)0, 0);
return 0;
}
However when compiling I get the following error:
$gcc dead.c
dead.c: In function ‘main’:
dead.c:8:2: warning: implicit declaration of function ‘sched_setattr’ [-Wimplicit-function-declaration]
sched_setattr(0, (void*)0, 0);
^~~~~~~~~~~~~
/tmp/ccGxWxZE.o: In function `main':
dead.c:(.text+0x19): undefined reference to `sched_setattr'
collect2: error: ld returned 1 exit status
My system is running Ubuntu 16.10 Yakkety, with kernel 4.8.0-59-generic. The sched.h file included is found in /usr/include/sched.h and is provided by the package libc6-dev. This headerfile does not contain the function sched_setattr and friends that I am trying to use.
However the kernel (and kernel headers) I have installed comes with a sched.h header file containing the definitions I need. It is located at /usr/src/linux-headers-4.8.0-58/include/linux/sched.h, on my system.
So I naively think lets just build against the newer linux headers instead of the libc6-dev provided headers. My program will only run on this or newer kernels, but that is just fine.
I modify the first line to be: #include <linux/sched.h> and execute:
gcc -I/usr/src/linux-headers-$(uname -r)/include -I/usr/src/linux-headers-$(unam -r)/arch/x86/include dead.c
Now I am getting page after page of errors and warning. This does not seem the way to go.
What is the correct way to build a userspace program against a newer Linux headers than those that are provided by libc?
And subsequently how do I build the program above?
sched_setattr() is a syscall and doesn't seem to have one-to-one libc wrapper. You could do the wrapper yourself, something like this:
#define _GNU_SOURCE
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <unistd.h>
#include <linux/sched.h>
#include <sys/syscall.h>
#include <sys/types.h>
struct sched_attr {
uint32_t size; /* Size of this structure */
uint32_t sched_policy; /* Policy (SCHED_*) */
uint64_t sched_flags; /* Flags */
int32_t sched_nice; /* Nice value (SCHED_OTHER, SCHED_BATCH) */
uint32_t sched_priority; /* Static priority (SCHED_FIFO, SCHED_RR) */
/* Remaining fields are for SCHED_DEADLINE */
uint64_t sched_runtime;
uint64_t sched_deadline;
uint64_t sched_period;
};
static int sched_setattr (pid_t pid, const struct sched_attr *attr, unsigned int flags)
{
return syscall (SYS_sched_setattr, pid, attr, flags);
}
int main (int argc, char *argv[])
{
struct sched_attr attr;
int res;
memset (&attr, 0, sizeof (struct sched_attr));
attr.size = sizeof (struct sched_attr);
res = sched_setattr (getpid (), &attr, 0);
if (res < 0) {
perror ("sched_setattr");
return 1;
}
return 0;
}
Looking at the errors reported when trying to include kernel header files required to get the definition of struct sched_attr and reading the comments found by Googling "kernel headers in user space", I really can't suggest trying to include kernel header files just for this.
I have successfuly intercepted calls to read(),write(),open(),unlink(),rename(), creat() but somehow with exactly the same semantics intercepting stat() is not taking place. I have changed the execution environmnet using LD_PRELOAD.
Am I missing something?
The code is quite huge, which part of it will be most helpful to post so you can help?
Thanks.
Edit: I kept the interposed stat() wrapper simple to check if it works.
int stat(const char *path,struct stat *buff)
{
printf("client invoke: stat %s",path);
return 1;
}
Compile a function that calls stat(); see what reference(s) are generated (nm -g stat.o). Then you'll have a better idea of which function(s) to interpose. Hint: it probably isn't called stat().
If you are compiling with 64 bit file offsets, then stat() is either a macro or a redirected function declaration that resolves to stat64(), so you will have to interpose on that function too.
Well it was not very simple when running in linux. Gnu libc does some tricks. You need to intercept the __xstat and if you want to call the original save the call.
Here is how I got it to work
gcc -fPIC -shared -o stat.so stat.c -ldl
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
static int (*old_xstat)(int ver, const char *path, struct stat *buf) = NULL;
static int (*old_xstat64)(int ver, const char *path, struct stat64 *buf) = NULL;
int __xstat(int ver, const char *path, struct stat *buf)
{
if ( old_xstat == NULL ) {
old_xstat = dlsym(RTLD_NEXT, "__xstat");
}
printf("xstat %s\n",path);
return old_xstat(ver,path, buf);
}
int __xstat64(int ver, const char *path, struct stat64 *buf)
{
if ( old_xstat64 == NULL ) {
old_xstat64 = dlsym(RTLD_NEXT, "__xstat64");
}
printf("xstat64 %s\n",path);
return old_xstat64(ver,path, buf);
}
I'm trying to use the nftw to process some files under a directory
#include <ftw.h>
#include <stdio.h>
int wrapper(const char * fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf) {
printf("File %d\n", ftwbuf->base);
return(0);
}
int main(int argc, char ** argv) {
const char *name;
int flags = 0;
name = argv[1];
nftw(name, wrapper, 20, flags);
return 0;
}
When I'm compiling (gcc kconfig_parser.c -o parser) , I've got this warning and this error..
kconfig_parser.c:5: warning: ‘struct FTW’ declared inside parameter list
kconfig_parser.c:5: warning: its scope is only this definition or declaration, which is probably not what you want
kconfig_parser.c: In function ‘wrapper’:
kconfig_parser.c:6: error: dereferencing pointer to incomplete type
I've checked the definition of the struct and the prototype of the callback, and some examples, it should be fine... What am I doing wrong ? (I've removed almost everything of my code to clear it)...
thanks
Linux, for some reason, still uses SUSv1 for this API, where nftw() is still considered an extension.
From the Linux manual page, the include has to be:
#define _XOPEN_SOURCE 500
#include <ftw.h>
Hmm. Your code works for me. Check your include paths, maybe? Though this is a system header, so it should be pretty hard to miss this. Or were you accidentally compiling a version that didn't have the #include <ftw.h> line?
$ gcc -o ftw ftw.c
$ ./ftw my-directory
File 10
File 11
File 16
File 16
File 16
File 16
File 16
... etc ...
edit: The test above was done on Mac OS X. In a (now deleted) comment the OP mentioned he was on Debian, for which the man page mentions that #define _XOPEN_SOURCE 500 is necessary, as Juliano points out.
On a CentOs versions the header file didn't use "#define _XOPEN_SOURCE 500"
i had to do this below,
#define __USE_XOPEN_EXTENDED 1
#include <ftw.h>
On Ubuntu 18.04, this is what seems to work now (similar to CentOS as mentioned by JohnMeg).
#define __USE_XOPEN_EXTENDED 1
#include <ftw.h>