I have a C function that reads a stream of characters from a FILE*.
How might I create a FILE* from a string in this situation?
Edit:
I think my original post may have been misleading. I want to create a FILE* from a literal string value, so that the resulting FILE* would behave as though there really was a file somewhere that contains the string without actually creating a file.
The following is what I would like to do:
void parse(FILE* f, Element* result);
int main(int argc, char** argv){
FILE* f = mysteryFunc("hello world!");
Element result;
parse(f,&result);
}
Standard C provides no such facility, but POSIX defines the fmemopen() function that does exactly what you want.
Unfortunately, C's standard library doesn't provide this functionality; but there are a few ways to get around it:
Create a temporary file, write your string to it, then open it for reading. If you've got POSIX, gettempnam will choose a unique name for you
The other option (again for POSIX only) is to fork a new process, whose job will be to write the string to a pipe, while you fdopen the other end to obtain a FILE* for your function.
As #KeithThompson pointed out, fmemopen does exactily what you want, so if you have POSIX, use that. On any other platform, (unless you can find the platform-equivalent), you'll need a temporary file.
Last time I had this kind of problem I actually created a pipe, launched a thread, and used the thread to write the data into the pipe... you would have to look into operating system calls, though.
There are probably other ways, like creating a memory mapped file, but I was looking for something that just worked without a lot of work and research.
EDIT: you can, of course, change the problem to "how do I find a nice temporary filename". Then you could write the data to a file, and read it back in :-)
pid_t pid;
int pipeIDs[2];
if (pipe (pipeIDs)) {
fprintf (stderr, "ERROR, cannot create pipe.\n");
return EXIT_FAILURE;
}
pid = fork ();
if (pid == (pid_t) 0) {
/* Write to PIPE in this THREAD */
FILE * file = fdopen( pipe[1], 'w');
fprintf( file, "Hello world");
return EXIT_SUCCESS;
} else if (pid < (pid_t) 0) {
fprintf (stderr, "ERROR, cannot create thread.\n");
return EXIT_FAILURE;
}
FILE* myFile = fdopen(pipe[0], 'r');
// DONE! You can read the string from myFile
.... .....
Maybe you can change the code a little bit to receive a custom handle.
void parse(my_handle *h, Element *result)
{
// read from handle and process
// call h->read instead of fread
}
and defines the handle like this:
struct my_handle
{
// wrapper for fread or something
int (*read)(struct my_handle *h, char *buffer, int readsize);
// maybe some more methods you need
};
implement your FILE* wrapper
struct my_file_handle
{
struct my_handle base;
FILE *fp;
};
int read_from_file(struct my_handle *h, char *buffer, int readsize)
{
return fread(buffer, 1, readsize, ((my_file_handle*)h)->fp);
}
// function to init the FILE* wrapper
void init_my_file_handle(struct my_file_handle *h, FILE *fp)
{
h->base.read = read_from_file;
h->fp = fp;
}
Now, implement your string reader
struct my_string_handle
{
struct my_handle base;
// string buffer, size, and current position
const char *buffer;
int size;
int position;
};
// string reader
int read_from_string(struct my_handle *h, char *buffer, int readsize)
{
// implement it yourself. It's easy.
}
// create string reader handle
void init_my_string_handle(struct my_string_handle *h, const char *str, int strsize)
{
// i think you know how to init it now.
}
//////////////////////////////////////////////////
And now, you can simply send a handle to your parse function. The function doesn't care where the data comes from, it can even read data from network!
This is an old question, but deserves a better answer.
C has always had the ability to read and write strings using the formatted I/O functions. You just need to keep track of where you are in the string!
Reading a string
To read a string you need the %n format string specifier, which returns the number of bytes read each time we use sscanf(). Here is a simple example with a loop:
#include <stdio.h>
int main(void)
{
const char * s = "2 3 5 7";
int n = 0;
int value;
while (sscanf( s+=n, "%d%n", &value, &n ) == 1)
{
printf( "value = %d\n", value );
}
}
Another way to have written that loop would be:
for (int value, n; sscanf( s, "%d%n", &value, &n ) == 1; s += n)
Whichever floats your boat best.
The loop is not important.
What is important is that we increment the value of s after every read.
Notice how we don’t bother to remember original value of s in this example? If it matters, use a temporary, as we do in our next example.
It is also important that we stop reading when sscanf fails. This is the normal usage for the scanf family of functions.
Writing a string
In this case sprintf() helps us by directly returning the number of bytes written. Here’s a simple example of building a string using several formatted outputs:
#include <stdio.h>
int main(void)
{
char s[100] = {0};
char * p = s;
p += sprintf( p, "%d %s", 3, "three" );
p += sprintf( p, "; " );
p += sprintf( p, "%.2f %s", 3.141592, "pi" );
*p = '\0'; // don’t forget it!
printf( "s = \"%s\"\n", s );
printf( "number of bytes written = %zu = %zu\n", p-s, strlen(s) );
}
The important points:
This time we do not want to clobber s (and in this particular example couldn’t even if we wanted to), so we use a helper p.
We cannot forget to manually add that null-terminator. (Which should make sense, since we are manually building the string.)
BUFFER OVERFLOW IS POSSIBLE
That last point is significant, and a usual concern when building strings in C. As always, whether using strcat() or sprintf(), always make sure you have enough room to append everything you intend to write to your string!
Don’t use %n when writing
We could have used the %n specifier as well, but then we hit a cross-platform issue with MSVC: Microsoft targets %n and the printf() family of functions as a security issue. Whether or not you accept Microsoft’s reasoning you must live with the way things are.
If you are undeterred, you can add a little platform-specific code and use it anyway:
#ifdef _WIN32
_set_printf_count_output( 1 );
#endif
int n;
printf( "Hello%n world!", &n );
True FILE * I/O
Notice that we aren’t touching actual FILE * I/O functions, like fgetc()? If you need that, then you need an actual file.
As mentioned above, use tmpfile() to open a temporary read/write file and use the usual FILE * I/O functions on it. Our read-a-string example could be re-written as:
#include <stdio.h>
int main(void)
{
FILE * f = tmpfile();
if (!f) return 1;
fprintf( f, "2 3 5 7" );
rewind( f );
int value;
while (fscanf( f, "%d", &value ) == 1)
{
printf( "value = %d\n", value );
}
fclose( f );
}
This works just fine. Remember that tmpfile() might not give you an actual file on disk with a filename. You don’t need that anyway. In other words, it may very well be an in-memory buffer provided by the OS... which is kind of what this thread is about anyway, right?
Hopefully these options will give a deeper insight into the C standard I/O functions and their use. Next time you need to read or build a formatted string in parts, you will have a better grasp of the tools already provided for you.
Related
i need to create a variadic function (stdarg library) which will loop through all the files i pass it and will count words similiar to a word i pass as a parameter,
#include <stdio.h>
#include <stdarg.h>
void countWords(char* name, FILE* file, ...){
va_list params;
FILE* currentFile;
FILE* f;
int words = 0;
va_start(params, file);
currentFile = file;
while (currentFile != NULL)
{
f = fopen(currentFile, "r+"); //which file should i open every time? this doesnt compile
// comparing words in each file code
currentFile = va_arg(params, FILE*);
}
va_end(params);
}
i cant read the file (no metter what i try it doesnt compile),
and how can i loop through each file comparing my word? i would really appreciate guidance
thank you!
If you are passing the file name or more precisely the path to the file, then this
FILE *currentFile;
currentFile = va_arg(params, FILE *);
should be
char *currentFile;
currentFile = va_arg(params, char *);
If you pass FILE pointers, you should not open them because if the rest of the program is correct then they shall be already opened inside the funcion, otherwise it doesn't make any sense to pass FILE *'s.
So the function should probably be
#include <stdio.h>
#include <stdarg.h>
void countWords(char *word, char *filename, ...)
{
va_list params;
FILE *file;
int words;
words = 0;
va_start(params, file);
while (filename != NULL)
{
file = fopen(filename, "r+");
// comparing words in each file code
filename = va_arg(params, char *);
}
va_end(params);
}
You would call it like this
countWords("example", "/path/to/file/1", ..., "/path/to/file/n", NULL);
and you should be careful with string literals probably use the const qualifier in this situation, because even if the parameters are not string literals it wouldn't make sense to modify them inside countWords() so to prevent accidentally modifying them const could help, although you can always modify them anyway. Even if modifying a string literal invokes undefined behavior you cannot completely forbid your program from doing so.
i need to create a variadic function
Unless this is a homework assignment about variadic functions, you don't need a variadic function at all. Variadic functions are not type safe and type safety is a desirable property.
Others have already pointed this out: You need a function that takes a filename and counts the occurrences of a word in a single file
size_t occurrences(const char *word, const char *filename) { ... }
You can then easily loop over an array of files, e.g.:
size_t count = 0;
for (int i = 1, i < argc; i++) {
size += occurrence("pink", argv[i]);
}
If you like, you can wrap this in a separate function which takes an array of file names with a file count. This is easily done, because whether you take your files from a command line or from somewhere else, you already have them stored in some kind of array.
Now consider the variadic variant, which makes sense only if you know the files you want to process at compile time, e.g.:
size_t n = var_occurrences("banana",
"alpha.txt", "beta.txt", "gamma.txt", NULL);
The same can be achieved with the non-variadic approach:
const char *w = "banana";
size_t n = occurrences(w, "alpha.txt")
+ occurrences(w, "beta.txt")
+ occurrences(w, "gamma.txt");
This is a bit more typing, but everything else is much more straightforward. If you must use variadic functions, go ahead and look at iharob's answer. But variadic functions are not a good solution to your task.
Switching to C from Java, and I'm having some troubles grasping memory management
Say I have a function *check_malloc that behaves as such:
// Checks if malloc() succeeds.
void *check_malloc(size_t amount){
void *tpt;
/* Allocates a memory block in amount bytes. */
tpt = malloc( amount );
/* Checks if it was successful. */
if ( tpt == NULL ){
fprintf(stderr, "No memory of %lu bytes\n", amount);
exit(1);
}
return tpt;
}
I also have the following variables to work with:
FILE *f = fopen("abc.txt", "r"); // Pointer to a file with "mynameisbob" on the first line and
// "123456789" on the second line
char *pname; // Pointer to a string for storing the name
}
My goal is to use *check_malloc to dynamically allocate memory so that the String pointed to by *pname is just the correct size for storing "mynamisbob", which is the only thing on the first line of the text file.
Here is my (failed) attempt:
int main(int argc, char *argv[]){
FILE *f = fopen("abc.txt", "r"); // A file with "mynameisbob" on the first line and
// "123456789" on the second line
char *pname; // Pointer to a string for storing the name
char currentline[150]; // Char array for storing current line of file
while(!feof(f)){
fgets(currentline,100,f);
pname = ¤tline;
}
But I know this probably isn't the way to go about this, because I need to use my nice check_malloc* function.
Additionally, in my actual text file there is a "<" symbol before the name on the first line.But I just want the *pname to point to a String saying "mynameisbob" without the "<" symbol. This isn't that important now, it just is reinforcement to me that I know I can't just set the pointer to point straight to currentline.
Can anyone help me fix my thinking on this one? Thanks a lot.
In C you need to copy chars, not the "strings" (which are just pointers). Check out strcpy() and strlen(). Use strlen() to determine how long the line actually is which fgets has read, then use your malloc() to allocate exactly that (plus 1 for the 0). Then copy the chars over with strcpy().
There are several problems in your code, see my comments in this example:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// Checks if malloc() succeeds.
void *check_malloc (size_t amount) {
void *tpt;
/* Allocates a memory block in amount bytes. */
tpt = malloc( amount );
/* Checks if it was successful. */
if (tpt == NULL) {
fprintf (stderr, "No memory of %lu bytes\n", amount);
exit (EXIT_FAILURE);
}
return tpt;
}
// To avoid subtle errors I have defined buffer size here
#define BUFFER_SIZE 150
// I have used the (void) version of main () here, while not strictly neccessary, you where not using argc and argv anyway, so the can be left out in this case
int main (void) {
// It might be a good idea to make the filename a char[] as well, but I leave that as an exercise to the reader.
FILE *f = fopen("abc.txt", "r"); // A file with "mynameisbob" on the first line and
// "123456789" on the second line
// You have to check whether the file was *actually openend*
if (f == NULL) {
fprintf (stderr, "Could not open file abc.txt\n"); // '"...%s\n", filename);' might better.
exit (EXIT_FAILURE);
}
char *pname; // Pointer to a string for storing the name
char currentline[BUFFER_SIZE]; // Char array for storing current line of file
while(!feof (f)) {
char *res = fgets (currentline, BUFFER_SIZE, f);
// fgets returns NULL when EOF was encountered before the next '\n'
if (res) {
size_t read = strlen (res);
// The line might have been empty
if (read) {
// Better use "sizeof *varname", while char is always 1 byte it is a good practice
pname = check_malloc ((read + 1) * sizeof *pname); // + 1 because we have to provide an extra char für '\0'
strncpy (pname, currentline, read); // You have to use strcpy or strncpy to copy the contents of the string rather than just assigning the pointer
// What was allocated must be freed again
free (pname);
}
}
}
fclose(f); // Always close everything you open!
return EXIT_SUCCESS;
}
Actually you really don't have to use pname in this simple case, because currentline already contains the line, but since you're trying to learn about memory management this should give you a general idea of how things work.
In your code you had this line:
pname = ¤tline;
There are two problems here:
As already mentioned in my code assigning currentline to pname only copies the pointer not the contents.
The correct assignment would be pname = currentline (without the address operator &), because currentline is also a pointer under the hood (it behaves like char *currentline even though it's statically allocated).
I don't know the title correctly addresses my problem or not. So, I will just go with it.
Here is the problem, I have to input a char array of a file path (in Windows) containing lots of backslashes in it, eg. "C:\myfile.txt" and return an unsigned char array of C-style file paths, eg. "C:\myfile.txt".
I tried to write a function.
unsigned char* parse_file_path(char *path);
{
unsigned char p[60];
int i,j;
int len = strlen(path);
for(i=0,j=0; i<len; i++, j++)
{
char ch = path[i];
if(ch==27)
{
p[j++]='\\';
p[j]='\\';
}
else
p[j] = path[i];
}
p[j]='\0';
return p;
}
The weird thing (for me) I am encountering is, here path contains only one backslash '\'. In order to get one backslash, I have to put '\' in path. This is not possible, cause path cannot contain '\'. When I call it like this parse_file_path("t\es\t \it), it returns
t←s it. But parse_file_path("t\\es\\t \\it") returns t\es\t \it.
How can I accomplish my task? Thanks in advance.
If I can just mention another problem with your code.
You are returning a local variable (your unsigned char p). This is undefined behavior. Consider declaring a char* p that you assign memory to dynamically using malloc and then returning p as you do. E.g. something like:
char* p = malloc(60);
A common practice is to use sizeof when allocating memory with malloc but here I've passed 60 directly as the C standard guarantees that a char will be 1 byte on all platforms.
But you have to free the memory assigned with malloc.
Or alternatively, you can change the function to take a buffer as an input argument that it then writes to. That way you can pass a normal array where you would call this function.
Regarding your slashes issue, here:
p[j++]='\\';
p[j]='\\';
Position j in p will be changed to \\, then j will be incremented and at the very next line you do the same for the succeeding char position. Are you sure you want the two assignments?
By the way if you are inputting the path from the command line, the escaping will be taken care of for you. E.g. consider the following code:
#include <stdio.h>
#include <string.h> /* for strlen */
#include <stdlib.h> /* for exit */
int main()
{
char path[60];
fgets(path, 60, stdin); /* get a maximum of 60 characters from the standard input and store them in path */
path[strlen(path) - 1] = '\0'; /* replace newline character with null terminator */
FILE* handle = fopen(path, "r");
if (!handle)
{
printf("There was a problem opening the file\n");
exit(1); /* file doesn't exist, let's quite with a status code of 1 */
}
printf("Should be good!\n");
/* work with the file */
fclose(handle);
return 0; /* all cool */
}
And then you run it and input something like:
C:\cygwin\home\myaccount\main.c
It should print 'Should be good!' (provided the file does exist, you can also test with 'C:\').
At least on Windows 7 with cygwin this is what I get. No need for any escapes as this is handled for you.
I'm trying to save a struct with a char* string into a file.
struct d_object {
int flags;
int time;
int offset;
char *filename;
};
The problem is that when doing that I will obviously only save the address of that pointer rather than the string. So what I've done is simply use a character array and but I'm forced to set the maximum size of the string. This works fine, however I was wondering if there is anyway of storing the struct with a char* (that I malloc at some point) in a file and then retrieve it. I can save the string and the struct separate and then retrieve them but it's quite a mess. It would be preferable if I could load and save the entire struct (the one above) into the file. Thanks!
The code with the char array is below:
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
struct d_object {
int flags;
int time;
int offset;
char filename[255];
};
int main(int argc, char **argv) {
struct d_object fcb;
fcb.flags=5;
fcb.time=100000;
fcb.offset=220;
strncpy(fcb.filename,"myfile",255);
int fd=open("testfile",O_RDWR);
write(fd,&fcb,sizeof(fcb));
close(fd);
int fd2 = open("testfile",O_RDONLY);
struct d_object new_fcb;
read(fd2,&new_fcb,sizeof(new_fcb));
printf("read from file testfile: %s\n",new_fcb.filename);
return 0;
}
P.S.: I'm not using the STREAM functions simply because this is actually meant to be run on an embedded OS that doesn't have them. I've just adapted the code for *BSD/Linux so it makes more sense when asking the question.
I understand that portability is not an issue, since you are working for an embedded system. In other case, you should use something like XML.
You can transform back your code to:
struct d_object {
int flags;
int time;
int offset;
char * filename;
};
And then save each piece of data individually:
write( fd, &record.flags, sizeof( int ) );
write( fd, &record.time, sizeof( int ) );
write( fd, &record.offset, sizeof( int ) );
int filename_length = strlen( filename );
write( fd, &filename_length, sizeof( int ) );
write( fd, record.filename, filename_length );
For reading, you'll have to read each item separatedly, and then the filename:
int filename_length;
read( fd, &emptyRecord.flags, sizeof( int ) );
read( fd, &emptyRecord.time, sizeof( int ) );
read( fd, &emptyRecord.offset, sizeof( int ) );
read( filename_length, sizeof( int ), 1, file );
emptyRecord.filename = (char *) malloc( sizeof( char ) * ( filename_length +1) );
read( fd, emptyRecord.filename, filename_length );
*( emptyRecord.filename + filename_length ) = 0;
Serialization is never pretty. How about storing the length of the string in the pointer, and letting the string follow the struct in the file? Something like this (warning, brain-compiled code):
void write_object(struct d_object *s, int fd) {
struct d_object copy = *s;
copy.filename = (char*)strlen(s->filename);
write(fd, ©, sizeof(copy));
write(fd, s->filename, (size_t)copy.filename);
}
void read_object(struct d_object *s, int fd) {
read(fd, s, sizeof(struct d_object));
char *filename = malloc(((size_t)s->filename) + 1);
read(fd, filename, (size_t)s->filename);
filename[(size_t)s->filename] = '\0';
s->filename = filename;
}
Now that I know the nature of your problem, why not try flexible arrays? Instead of using char *filename;, use char filename[1] and malloc(sizeof struct d_object + filename_len) to allocate your structs. Add a size member, and you can easily write the object to disk with a single call to write and load it from disk with 2 calls (first to read the size element, second to read the whole object after allocating it).
Note that the "official" way to do flexible arrays in C99 is [] rather than [1], but [1] is guaranteed to work as a consequence of other requirements in the standard and works on C89 too. [1] will waste a few bytes though, so if your compiler supports [] you might want to use it.
Your problem here is really a symptom of a much larger issue: you shouldn't be reading/writing binary data structures between memory and files. Not only is there no clear way to read/write structures with pointers to other data (this applies not only to strings but to nested data, linked lists, etc.) but the format of the data on disk will depend on your host machine and C implementation and will not be portable to other environments.
Instead, you should design a format for your data on disk and write functions to save and load the data, or use an existing format and find library code for using it. This is usually referred to as "serialization".
If your data is hierarchical, JSON, XML, or EBML may be appropriate. If it's fairly simple, flat text files or a homebrew binary format (written byte-by-byte so it's portable) may be appropriate.
Since you seem to be unfamiliar with these issues, it might be worthwhile to write some code for loading/saving a few simple binary file formats (like .tga or .wav) as an exercise before you try to design something for your own data.
Nope, there's no magic built directly into the language to do this for you.
What you need to do is to write a couple of functions to convert your data into writable form, and to read it back in from file. This is called "serialization" / "deserialization" in languages that make more of a fuss about it.
Your particular structure, you could do something like write the binary stuff to the file straight from the struct, and then follow it up with the contents of the character buffer. You could make things easier for yourself come read time if you precede the character data with an int specifying its length.
When you read that stuff back in, you'll want to malloc/calloc yourself a chunk of memory to hold the char data in; if you stored the size you'll know just how big to make that malloc. Read the binary data into the struct, read the char data into the malloc'd memory, store the pointer to the malloc chunk into the struct, and you've re-created the original.
No magic. Just data and a bit of elbow grease.
EDIT
While I wrote about doing this, Thomas coded an example. I think our answers complement each other very well; together they should tell you everything you need to know.
C# has a neat feature of being able to write to a memory stream using the MemoryStream object.
I'm looking for similar functionality from C using a FILE* pointer.
I want to be able to sprintf() but with the added functionality of having C remember "where I was" in the buffer I'm writing to.
If you are using the GNU C Library, you can use fmemopen(). There may be other non-portable extensions for other environments, but there's no portable way using FILE*s.
You could also, however, wrap snprintf, if you don't insist on actually using FILE*s. For example, glib (note: not the same as the GNU C Library, and portable) has a g_string_append_printf that does what you want.
There's also an ugly hack which works with plain ISO-C: You can use fopen() to open a null file (/dev/null on *nix, NUL on Windows) and set the array as the file's buffer via
setvbuf(file, buffer, _IOFBF, buffer_size)
This should work fine as long as fflush() isn't called anywhere in the code. Also, the programmer has to explicitly take care of the string delimiter.
I don't really see a need to do this, though: As snprintf() returns the number of characters written, it's trivial to keep track of a buffer's position.
One can even write a function to automagically resize the buffer on overflow: bufprintf.c
The function's prototype is
int bufprintf(char **buffer, size_t *size, size_t *offset,
const char *format, ...);
An example program could look like this:
#include <stdio.h>
extern int bufprintf(char **buffer, size_t *size, size_t *offset,
const char *format, ...);
int main(void)
{
size_t size = 0; // must be set!
size_t offset;
char * buffer;
for(int i = 0; i < 100; ++i)
bufprintf(&buffer, &size, &offset, "we rock %i\n", i);
puts(buffer);
printf("size:\t%u\noffset:\t%u\n", (unsigned)size, (unsigned)offset);
}
sprintf returns the number of characters that were printed into the string. You can use that value to increment the pointer of your buffer.
buffer += sprintf(buffer, "%d", i);
Make sure that you keep around a copy of the original pointer, as that is what you will be using when passing the buffer somewhere else.
Why not use mmap? You can map a file to memory and use a FILE* pointer.
Update: It does not work. Sorry.
MemoryStream is not a feature of C#, the language. It's simply a class in the BCL. You could write it yourself in C#.
And so it is in C - you'd write some functions that work similarly in use to fopen, fprintf, fwrite, fclose, etc., but give them names like mopen, mwrite, etc. (presumably) and write them so they operate on a memory buffer.
How about just manually managing your pointers. I don't have a compiler in front me but hopefully the code below gets the idea across:
char huge_buffer[REALLY_BIG_SIZE];
char *write_pos = huge_buffer;
//...
void fprintf_to_mem(char **mem_ptr, const char *fmt, ...)
{
va_list args;
int num_written;
va_start(args, mem_ptr);
num_written = vsprintf(*write_pos, fmt, args);
*write_pos += num_written;
va_end(args);
}
//...
fprintf_to_mem(&write_pos, "Testing %d %d %d", 1, 2, 3);
fprintf_to_mem(&write_pos, "Hello world!\r\n");
I would expect that to output "Testing 1 2 3Hello world!\r\n" to huge_buffer.