Is there anything like a string file in stdio/string/stdlib ? I mean a special way to fopen a FILE stream, which actually directs the writes to an internal buffer and takes care of buffer allocation/reallocation ? After fclose, the text should be available as null-terminated char[] or similar.
I need to interface to legacy code that receives a FILE* as an argument and writes to it, and I'd prefer to avoid writing to a temporary disk file.
Other forms of storage could do instead of char[] (f.i. string), but a FILE* pointer must be available.
I am looking for an alternative to creating a temporary disk file.
fmemopen & open_memstream are in the POSIX 2008 standard, probably inspired by GNU libc string streams, and give in-memory FILE* streams.
See also this question quite similar to yours, and also that answer.
BTW, many operating systems have RAM based or virtual memory based filesystems (à la tmpfs)
If you are coding in C++11 (not in C) and perhaps for some earlier C++ standard you can of course use std::stringstream-s
So you could use open_memstream on Posix, and some other solution on Windows (just with #if _POSIX_C_SOURCE > 200809L per feature_test_macros(7) ...)
The C standard does not provide (yet) any in-memory FILE streams, so if you need them you have to code or use platform-specific functions.
Create the temporary file using CreateFile(... FILE_ATTRIBUTE_TEMPORARY, FILE_FLAG_DELETE_ON_CLOSE ...) and then convert the HANDLE to FILE*.
You said you didn't like a write to a temporary file, so these flags to CreateFile are a strong hint to Windows to keep the file in cache if possible. And if Windows would run of of RAM, even a char[] can end up in a swap file anyway.
Related
I'm using a system function which writes the output information into a stream of file pointer.
func(FILE *__fp)
I need to use this information in my program rather than printing this out to a file. For that I thought of creating a tmpfile() and writing to it then reading back from it. But is there a better way to get this information?
There are OS-specific solutions to writing to a memory buffer instead of a file, like for example the POSIX fmemopen or open_memstream (both which should be very useful considering your linux tag).
You can also change the internal buffer to your own with setvbuf.
On an unrelated note: Symbols starting with a leading underscore and followed by another underscore (like for example your __fp argument) are reserved. Such symbols may only be used by "the implementation", i.e. the compiler and library.
I am quite new to the FILE family of functions that the standard C library provides.
I recently stumbled across fopen() and the similar functions after researching how stdout, stdin and stderr work alongside functions like printf().
I was wondering, what is needed to use fopen() on an embedded system (which doesn't necessarily have operating system support). After reading more about it, is seems like a cool thing to do on more powerful embedded systems to hook into say, a UART/SPI interface, so that calling printf() would print data out of the UART. Simarly, you could read data from a UART buffer by calling scanf().
This would also increase portability! (code written for say, Linux, would be easier to port if printf() was supported). You could also print debug data to a file if it was running in a production environment, and read from it later.
Can you just use fopen() on a bare-bones embedded system? If so who/where/when is the "FILE" then created (as far as I now, fopen() does not malloc() space for the file, nor do you specify how much)? Or do you need a operating system with FAT file support. If so, would something like http://ultra-embedded.com/?fat_filelib work? Would using FreeRTOS help at all?
Check the documentation for your toolchain's C library - it should have something to say about re-targeting the library.
For example if you are using Newlib you must re-implement some or all of the [syscalls stubs][3] to suit your target. The low level open() syscall in this case will allow fopen() to work as necessary. At its simplest, you might implement open() to support higher-level stdio access to serial ports, but if you are expecting standard file-system access, then you will still need an underlying file-system to map it too.
Another example of re-targeting the Keil/ARM standard library can be found here.
Yes, it's often possible to use fopen() and similar routines in code for embedded systems. The way it often works is that the vendor supplies a C compiler and associated libraries
targeted for their system, which implement some supported subset of the language in a way that's appropriate for that system (e.g. an implementation of printf() that outputs via a UART, or fopen() that uses RAM to simulate some sort of filesystem).
On the Keil compiler, the stdio library is designed to allow the user to define the __FILE structure in any desired fashion. A function like fprintf will perform a sequence of calls to fputc, which will receive a copy of the pointer passed to fprintf. One may define something like fopen to "create" a __FILE and populate its members via any desired means (if there will never be more than one file open at a time, one could simply fill in the fields of a static instance and return that). Variables __stdin, __stdout, and __stderror may likewise be defined as desired (stdin is defined to point to __stdin, and likewise with stdout and stderror).
"Can you just use fopen() on a bare-bones embedded system?"
It depends. Depends on the configuration of your embedded system, the types of memories interfaced, on what memory do you want to implement the file system, the file system library code size (ROM & RAM requirements).
FILE manipulation functions can be used independent of any OS. But a proper file system must be used and FAT is not the only file system (JFFS2, YAFS,...some other proprietary file system)
The file system is generally (but not always) implemented on Flash memories (Nand Flash, Nor Flash). USB device is also a flash (Nand flash). The Nand Flash & Nor Flash may have Parallel interface, I2C interface or SPI interface.
Platform: Debian Wheezy 3.2.0-4-686-pae
Complier: GCC (Debian 4.7.2-5) 4.7.2 (Code::Blocks)
I want to move a file from one location to another. Nothing complex like moving to different drives or to different file systems. I know the "standard" way to do this would be simply copying the file and then removing the original. But I want some way of preserving the file's ownership, mode, last access/modification, etc. . I am assuming that I will have to copy the file and then edit the new file's ownership, mode, etc. afterwards but I have no idea how to do this.
The usual way to move a file in C is to use rename(2), which sometimes fail.
If you cannot use the rename(2) syscall (e.g. because source and target are on different filesystems), you have to query the size, permission and other metadata of the source file with stat(2); copy the data looping on read(2), write(2) (using a buffer of several kilobytes), open(2), close(2) and the metadata using chmod(2), chown(2), utime(2). You might also care about copying attributes using getxattr(2), setxattr(2), listxattr(2). You could also in some cases use sendfile(2), as commented by David C. Rankin.
And if the source and target are on different filesystems, there is no way to make the move atomic and avoid race conditions (So using rename(2) is preferable when possible, because it is atomic according to its man page). The source file can always be modified (by another process) during the move operations...
So a practical way to move files is to first try doing a rename(2), and if that fails with EXDEV (when oldpath and newpath are not on the same mounted filesystem), then you need to copy bytes and metadata. Several libraries provide functions doing that, e.g. Qt QFile::rename.
Read Advanced Linux Programming - and see syscalls(2) - for more (and also try to strace some mv command to understand what it is doing). That book is freely and legally downloadable (so you could find several copies on the Web).
The /bin/mv command (see mv(1)) is part of GNU coreutils which is free software. You could either study its source code, or use strace(1) to understand what that command does (in terms of syscalls(2)). In some open source Unix shells like sash or busybox, mv might be a shell builtin. See also path_resolution(7) and glob(7).
There are subtle corner cases (imagine another process or pthread doing some file operations on the same filesystem, directory, or files). Read some operating system textbook for more.
Using a mix of snprintf(3), system(3), mv(1) could be tricky if the file name contains weird characters such as tab or or newlines, or starts with an initial -. See errno(3).
If the original and new location for the file are on the same filesystem then a "move" is conceptually identical to a "rename."
#include <stdio.h>
int rename (const char *oldname, const char *newname)
Is there any way to make pointers read a file as a block of memory in C?
Can access of a file can be faster...?
Treating a file as memory (and letting the OS do the file IO for you) is termed 'memory mapping'.
On POSIX (e.g. Linux), the mmap() function does this.
On Windows, the OpenFileMapping() function and friends do this. Microsoft have excellent description of how this works, why to use it, and particulars on their platform here.
I am designing C library which does some mathematical calculations. I need to specify serialization interface to be able to save and then load some data. The question is, is it correct (from binary compatibility point of view) to use FILE* pointer in the public API of library?
Target platfoms are:
Linux x86, x86_64 with gcc >= 3.4.6
Windows x86, x86_64 >= WinXP with VS >= 2008sp1
I need to be as much binary compatible as it possible, so at the moment my variant is the following:
void SMModuleSave(SMModule* module, FILE* dest);
SMModule* SMModuleLoad(FILE* src);
So I am curious if it is correct to use FILE* or better switch to wchar*/char* ?
I don't agree with ThiefMaster: there's no benefit in going native (ie using file descriptors of type int on linux and handles of type void * on windows) when there's an equivalent portable solution.
I'd probably go with FILE * instead of opening the files by name from within the library: It might be more of a hassle for library users, but it's also more flexible as most libc implementations provide various ways for file opening (fopen(), _wfopen(), _fdopen(), fdopen(), fmemopen(),...) and you don't have to maintain seperate wide-char APIs yourself.
I'd use neither but let the user pass a file descriptor as an int.
Then you can fdopen() it in your code to get a FILE*.
However, when using windows it might not be the best solution even though it does have some helper functions to get a numeric file descriptor.
However, passing a FILE* or a const char* should be fine, too. I'd prefer passing a filename as it's less code to write if a library takes care of opening/closing a file.
Yes it is correct, from a stable binary interface perspective, to use FILE * here. I think perhaps you're confusing this with using FILE as opposed to a pointer to it. Notice that your standard library's fopen, fgets, etc. functions all use (both as arguments and return values) the FILE * type as part of their public interfaces.
A FILE * is a standard ANSI/ISO C89 and C99 (even K&R) type. It is a portability dream and I'd prefer it over anything else. You're safe to go with it. It won't get any better than that.