LKM can create dynamically entries inside /proc/sys, but sysctl (not the Linux command but C's sysctl) accepts as first argument an array of ints with predefined values representing entries inside /proc/sys. My question is: can I read a dynamically created entry with sysctl or do I need to use fopen, read, etc...?
You need to use the file system interface: fopen, fread, etc (or open, read, if you prefer).
And about the C function called sysctl, don't use it:
Use of this system call has long been discouraged, and it is so unloved that it is likely to disappear in a future kernel version. Since Linux 2.6.24, uses of this system call result in warnings in the kernel log. Remove it from your programs now; use the /proc/sys interface instead.
Related
I have third-party library with a function that does some computation on the specified data, and writes the results to a file specified by file name:
int manipulateAndWrite(const char *filename,
const FOO_DATA *data);
I cannot change this function, or reimplement the computation in my own function, because I do not have the source.
To get the results, I currently need to read them from the file. I would prefer to avoid the write to and read from the file, and obtain the results into a memory buffer instead.
Can I pass a filepath that indicates writing to memory instead of a
filesystem?
Yes, you have several options, although only the first suggestion below is supported by POSIX. The rest of them are OS-specific, and may not be portable across all POSIX systems, although I do believe they work on all POSIXy systems.
You can use a named pipe (FIFO), and have a helper thread read from it concurrently to the writer function.
Because there is no file per se, the overhead is just the syscalls (write and read); basically just the overhead of interprocess communication, nothing to worry about. To conserve resources, do create the helper thread with a small stack (using pthread_attr_ etc.), as the default stack size tends to be huge (on the order of several megabytes; 2*PTHREAD_STACK_SIZE should be plenty for helper threads.)
You should ensure the named pipe is in a safe directory, accessible only to the user running the process, for example.
In many POSIXy systems, you can create a pipe or a socket pair, and access it via /dev/fd/N, where N is the descriptor number in decimal. (In Linux, /proc/self/fd/N also works.) This is not mandated by POSIX, so may not be available on all systems, but most do support it.
This way, there is no actual file per se, and the function writes to the pipe or socket. If the data written by the function is at most PIPE_BUF bytes, you can simply read the data from the pipe afterwards; otherwise, you do need to create a helper thread to read from the pipe or socket concurrently to the function, or the write will block.
In this case, too, the overhead is minimal.
On ELF-based POSIXy systems (basically all), you can interpose the open(), write(), and close() syscalls or C library functions.
(In Linux, there are two basic approaches, one using the linker --wrap, and one using dlsym(). Both work fine for this particular case. This ability to interpose functions is based on how ELF binaries are linked at run time, and is not directly related to POSIX.)
You first set up the interposing functions, so that open() detects if the filename matches your special "in-memory" file, and returns a dedicated descriptor number for it. (You may also need to interpose other functions, like ftruncate() or lseek(), depending on what the function actually does; in Linux, you can run a binary under ptrace to examine what syscalls it actually uses.)
When write() is called with the dedicated descriptor number, you simply memcpy() it to a memory buffer. You'll need to use global variables to describe the allocated size, size used, and the pointer to the memory buffer, and probably be prepared to resize/grow the buffer if necessary.
When close() is called with the dedicated descriptor number, you know the memory buffer is complete, and the contents ready for processing.
You can use a temporary file on a RAM filesystem. While the data is technically written to a file and read back from it, the operations involve RAM only.
You should arrange for a default path to one to be set at compile time, and for individual users to be able to override that for their personal needs, for example via an environment variable (YOURAPP_TMPDIR?).
There is no need for the application to try and look for a RAM-based filesystem: choices like this are, and should be, up to the user. The application should not even care what kind of filesystem the file is on, and should just use the specified directory.
You could not use that library function. Take a look at this on how to write to in-memory files:
Is it possible to create a C FILE object to read/write in memory
I was looking to implement the behavior of Linux command cp -Rf <src_dir>/* <dst_dir> that copies everything inside the 'src_dir' into 'dst_dir' recursively
I looked online for help and got a few solutions but they did either of the following that I do not want to do:
using rename(src_dir, dst_dir), which essentially 'moves' the contents and not copies.
I need to keep the contents of the 'src_dir' intact.
Opening to read each file int the 'src_dir'.
I would like to do this w/o opening the files and reading the content.
Can the above be achieved with C without using system("cp -Rf <src_dir>/* <dst_dir>")?
(Edited the original question after realising from a few initial comments that the original lang was confusing)
I was looking to implement the behavior of linux system commands
1) cp that copies everything inside the 'src_dir' into 'dst_dir' recursively,
Can the above be achieved using C (and no linux system calls)?
Of course not; remember that a system call is the main way an application can interact with the Linux kernel: look on syscalls(2) for an exhaustive list of system calls (you are likely to need stat(2)...). Do not confuse system calls with calls to the C library system(3) function (which uses fork(2), execve(2), waitpid(2) etc....)
However, you might be interested in nftw(3) (which of course is implemented above several system calls). Look also into opendir(3) & readdir(3)
You cannot change anything on the file systems without going thru system calls.
addenda
(the question has changed to)
Can the above be achieved with C without using system("cp -Rf /* ")?
Yes; you could use nftw(3) first to scan the file tree (or play appropriately with readdir(3) etc...), then detect the directories to be made (using mkdir(2)...); you'll make appropriately the directories and you'll explicitly copy file contents (e.g. using stdio(3) functions or open(2), read(2), write(2), close(2) system calls....). Notice that open(2) (called by fopen(3)...) is required to read or write any content from a file (even using mmap(2)...) Of course, to copy a file, you need to open it, open its fresh copy, read it and write to the copy in a loop, and close both source and destination files. But you should close (both source & destination files) once the copy is done. BTW, a given process could have several hundreds (or thousands of) open file descriptors (see setrlimit(2)...) and opendir(3) also consume one.
(your number of opened file descriptors (most of them being directories, with opendir(3)) is bounded by the file tree depth so is practically likely to be small, less than a hundred; and using clever caching with nftw(3) you could even avoid that)
BTW, cp is part of GNU coreutils which is free software, so you can study its source code. And you might also use strace(1) to understand what cp is doing.
Read Advanced Linux Programming and also Operating System : Three Easy Pieces (both are freely downloadable, as chapters in PDF)
There is no way to copy the content of one file with a single system call. You need a loop. But look also into the Linux specific copy_file_range(2) which I don't recommend, better loop on POSIX read(2) and write(2)
I am working on an embedded system with no filesystem and I need to execute programs that take input data from files specified via command like arguments or directly from stdin.
I know it is possible to bake-in the file data with the binary using the method from this answer: C/C++ with GCC: Statically add resource files to executable/library but currently I would need to rewrite all the programs to access the data in a new way.
Is it possible to bake-in a text file, for example, and access it using a fake file pointer to stdin when running the program?
If your system is an OS-less bare-metal system, then your C library will have "retargetting" stubs or hooks that you need to implement to hook the library into the platform. This will typically include low-level I/O functions such as open(), read(), write(), seek() etc. You can implement these as you wish to implement the basic stdin, stdout, stderr streams (in POSIX and most other implementations they will have fixed file descriptors 0, 1 and 2 respectively, and do not need to be explicitly opened), file I/O and in this case for managing an arbitrary memory block.
open() for example will be passed a file or device name (the string may be interpreted any way you wish), and will return a file descriptor. You might perhaps recognise "cfgdata:" as a device name to access your "memory file", and you would return a unique descriptor that is then passed into read(). You use the descriptor to reference data for managing the stream; probably little more that an index that is incremented by the number if characters read. The same index may be set directly by the seek() implementation.
Once you have implemented these functions, the higher level stdio functions or even C++ iostreams will work normally for the devices or filesystems you have supported in your low level implementation.
As commented, you could use the POSIX fmemopen function. You'll need a libc providing it, e.g. musl-libc or possibly glibc. BTW for benchmarking purposes you might install some tiny Linux-like OS on your hardware, e.g. uclinux
If I have some sort of library that has a function like so void foo(FILE* fp), and I have a char* array of data, is there any way for me to pass a pointer to the array or something similar to the foo function?.
What I'm doing now is using system() to write to a temporary file, and I don't really like doing that.
No, the stdio library has no facility for defining your own buffering scheme.
Given POSIX and virtual memory, you can use mmap to open a memory region with a backing file. The OS may opt not to write the file to disk. Details vary by OS, but this can approach the ideal solution.
Or, as Barmar suggests, use pipe to send the data to the main thread from an auxiliary thread within your process.
As for the simple solution… why use system() when you can use mktemp(), fopen(), fwrite()?
I think you want fmemopen. Assuming you're running on Linux (or something else posixy; if you need Windows support, will have to check), it should be available.
I need to write a C program that accepts three command line arguments:
input file one
input file two
name of output file
The program needs to read the data in from files 1 and 2 and concatenate the first file followed by the second file, resulting in the third file.
This seems like it should be pretty easy, but one of the stipulations of the assignment is to only use low-level I/O.
What exactly does that mean (low-level I/O)?
To answer the only question (what is low-level I/O) it probably means operating system native input/output functions.
In POSIX this would be e.g. open(), close(), read() and write().
On Windows e.g. CreateFile(), CloseHandle(), ReadFile() and WriteFile().
Low level basically stands for OS level. This can be done by using System calls.
Application developers often do not have direct access to the system calls, but can access them through an application programming interface (API). The functions that are included in the API invoke the actual system calls. By using the API, certain benefits can be gained:
Portability: as long a system supports an API, any program using that API can compile and run.
Ease of Use: using the API can be significantly easier then using the actual system call.
For more information on system calls have a look here ,here and here.
For your program have a look here.