I am pretty new to C and memory allocation in general. Basically what I am trying to do is copy the contents of an input file of unknown size and reverse it's contents using recursion. I feel that I am very close, but I keep getting a segmentation fault when I try to put in the contents of what I presume to be the reversed contents of the file (I presume because I think I am doing it right....)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int recursive_back(char **lines, int lineNumber, FILE *input) {
char *input_line = malloc(sizeof(char) * 1000);
lines = realloc(lines, (lineNumber) * 1000 * sizeof(char));
if(fgets(input_line, 201, input) == NULL) {
*(lines + lineNumber) = input_line;
return 1;
}
else {
printf("%d\n", lineNumber);
return (1+recursive_back(lines, ++lineNumber, input));
}
}
void backward (FILE *input, FILE *output, int debugflag ) {
int i;
char **lines; //store lines in here
lines = malloc(1000 * sizeof(char *) ); //1000 lines
if(lines == NULL) { //if malloc failed
fprintf(stderr, "malloc of lines failed\n");
exit(1);
}
int finalLineCount, lineCount;
finalLineCount = recursive_back(lines, 0, input);
printf("test %d\n", finalLineCount);
for(i = finalLineCount; i > 0; i--) {
fputs(*(lines+i), output); //segfault here
}
}
I am using a simple input file to test the code. My input file is 6 lines long that says "This is a test input file". The actual input files are being opened in another function and passed over to the backward function. I have verified that the other functions in my program work since I have been playing around with different options. These two functions are the only functions that I am having trouble with. What am I doing wrong?
Your problem is here:
lines = realloc(lines, (lineNumber) * 1000 * sizeof(char));
exactly as #ooga said. There are at least three separate things wrong with it:
You are reallocating the memory block pointed to by recursive_back()'s local variable lines, and storing the new address (supposing that the reallocation succeeds) back into that local variable. The new location is not necessarily the same as the old, but the only pointer to it is a local variable that goes out of scope at the end of recursive_back(). The caller's corresponding variable is not changed (including when the caller is recursive_back() itself), and therefore can no longer be relied upon to be a valid pointer after recursive_back() returns.
You allocate space using the wrong type. lines has type char **, so the object it points to has type char *, but you are reserving space based on the size of char instead.
You are not reserving enough space, at least on the first call, when lineNumber is zero. On that call, when the space requested is exactly zero bytes, the effect of the realloc() is to free the memory pointed to by lines. On subsequent calls, the space allocated is always one line's worth less than you think you are allocating.
It looks like the realloc() is altogether unnecessary if you can rely on the input to have at most 1000 lines, so you should consider just removing it. If you genuinely do need to be able to reallocate in a way that the caller will see, then the caller needs to pass a pointer to its variable, so that recursive_back() can modify it via that pointer.
Related
First of all, I'm quite new with C, and I know this is a very repeated question, however, I could not find anything that could help me with my problem.
Here is my code:
It takes a text file and stores each line in an array.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main()
{
FILE *file;
file = fopen("test.txt", "r");
char buffer[600];
char *lines[10000];
int i = 0;
while(fgets(buffer, sizeof(buffer), file))
{
lines[i] = malloc(sizeof(buffer));
strcpy(lines[i], buffer);
i++;
free(lines[i]);
}
fclose(file);
return 1;
}
This works fine for small text files.
However it doesn't with large ones (even setting buffer and lines with much bigger numbers). Actually, if I increment buffer[] and *lines[] like 1000000 bytes, it doesn't give anything (if I understood well, it gives undefined behaviour). And I need to get this work with a 100.000 lines file with variable length lines,
So, how could I declare a very large array so I can pass each line? Since, as I exposed, it doesn't work with a large file.
Any help is appreciated!
char *lines[10000]; is just an array of pointers to the lines, not the array (memory) that is going to store the actual lines.
malloc is allocating a chunk of memory for each line, you are suppose to call free only when you are done using this chunk.
If you remove the free your solution would work, but you need to remember to free at some other point.
And I need to get this work with a 100.000 lines file with variable length lines,
So, how could I declare a very large array so I can pass each line?
This line
char *lines[10000];
gives you a variable with automatic storage duration - often called a local variable.
On most systems such a variable are located on a stack and most systems have a fixed limit for the size of the stack and thereby also a limit for the size of such a local variable.
So if you change the code to
char *lines[1000000];
to be able to handle larger files, it is likely that the variable use too much memory on the stack, i.e. you have a stack overflow.
A simple solution is to allocate the variable dynamically. Like:
char **lines = malloc(1000000 * sizeof *lines);
This will allocate 1000000 char-pointers and you can use lines as if it's an array - for instance like:
lines[i] = malloc(sizeof(buffer));
For something like this I'll also recommend that you take a look at realloc so that you can adjust the size of memory as needed.
Besides that your use of free seems strange and it's for sure wrong as you increment i between the malloc and the free.
You can allocate any space just as big as you need. So you will get rid of the fixed and limited numbers.
I have "massaged" your example in this way. The only thing I didn't is a first round through the file to obtain the longest line. So I kept the fixed buffer length.
Allocate only as many pointer to the lines as you need. For this you define a pointer to pointers to char.
Allocate only as many characters for each line as you need. This is done most conveniently with the function strdup(). If your library doesn't have it (it is not standard) you can replace it with the right combination of strlen(), malloc(), and strcpy(). How to do this is left as an exercise for you. ;-)
Handle allocation errors, especially if you plan to read huge files.
Free the allocated memories blocks, the sequence for the lines is not important. But lines has to be kept until all lines[*] are freed.
This is the code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main()
{
FILE *file;
file = fopen("test.txt", "r");
char buffer[600];
char **lines = NULL;
int i = 0;
while (fgets(buffer, sizeof(buffer), file))
{
lines = realloc(lines, (i + 1) * sizeof (char*));
if (lines == NULL)
{
// any error handling you like...
return EXIT_FAILURE;
}
lines[i] = strdup(buffer);
if (lines[i] == NULL)
{
// any error handling you like...
return EXIT_FAILURE;
}
i++;
}
fclose(file);
// work with the lines
for (int j = 0; j < i; ++j)
{
free(lines[j]);
}
free(lines);
return EXIT_SUCCESS;
}
Some notes:
Because of the realloc() on each line the run time of you program will scale bad for files with a giant number of lines. To improve this you might like to use some better algorithm, for example by allocating in steps of growing numbers. But this is a completely different issue.
You don't need to free allocated memory yourself at all if you need the memory until the end of the program. The C runtime will then free the memory automatically.
I am trying to write a function to convert a text file into a CSV file.
The input file has 3 lines with space-delimited entries. I have to find a way to read a line into a string and transform the three lines from the input file to three columns in a CSV file.
The files look like this :
Jake Ali Maria
24 23 43
Montreal Johannesburg Sydney
And I have to transform it into something like this:
Jake, 24, Montreal
...etc
I figured I could create a char **line variable that would hold three references to three separate char arrays, one for each of the three lines of the input file. I.e., my goal is to have *(line+i) store the i+1'th line of the file.
I wanted to avoid hardcoding char array sizes, such as
char line1 [999];
fgets(line1, 999, file);
so I wrote a while loop to fgets pieces of a line into a small buffer array of predetermined size, and then strcat and realloc memory as necessary to store the line as a string, with *(line+i) as as pointer to the string, where i is 0 for the first line, 1 for the second, etc.
Here is the problematic code:
#include <stdio.h>
#include<stdlib.h>
#include<string.h>
#define CHUNK 10
char** getLines (const char * filename){
FILE *file = fopen(filename, "rt");
char **lines = (char ** ) calloc(3, sizeof(char*));
char buffer[CHUNK];
for(int i = 0; i < 3; i++){
int lineLength = 0;
int bufferLength = 0;
*(lines+i) = NULL;
do{
fgets(buffer, CHUNK, file);
buffLength = strlen(buffer);
lineLength += buffLength;
*(lines+i) = (char*) realloc(*(lines+i), (lineLength +1)*sizeof(char));
strcat(*(lines+i), buffer);
}while(bufferLength ==CHUNK-1);
}
puts(*(lines+0));
puts(*(lines+1));
puts(*(lines+2));
fclose(file);
}
void load_and_convert(const char* filename){
char ** lines = getLines(filename);
}
int main(){
const char* filename = "demo.txt";
load_and_convert(filename);
}
This works as expected only for i=0. However, going through this with GDB, I see that I get a realloc(): invalid pointer error. The buffer loads fine, and it only crashes when I call 'realloc' in the for loop for i=1, when I get to the second line.
I managed to store the strings like I wanted in a small example I did to try to see what was going on, but the inputs were all on the same line. Maybe this has to do with fgets reading from a new line?
I would really appreciate some help with this, I've been stuck all day.
Thanks a lot!
***edit
I tried as suggested to use calloc instead of malloc to initialize the variable **lines, but I still have the same issue.I have added the modifications to the original code I uploaded.
***edit
After deleting the file and recompiling, the above now seems to work. Thank you to everyone for helping me out!
You allocate line (which is a misnomer since it's not a single line), which is a pointer to three char*s. You never initialize the contents of line (that is, you never make any of those three char*s point anywhere). Consequently, when you do realloc(*(line + i), ...), the first argument is uninitialized garbage.
To use realloc to do an initial memory allocation, its first argument must be a null pointer. You should explicitly initialize each element of line to NULL first.
Additionally, *(line+i) = (char *)realloc(*(line+i), ...) is still bad because if realloc fails to allocate memory, it will return a null pointer, clobber *(line + i), and leak the old pointer. You instead should split it into separate steps:
char* p = realloc(line[i], ...);
if (p == null) {
// Handle failure somehow.
exit(1);
}
line[i] = p;
A few more notes:
In C, you should avoid casting the result of malloc/realloc/calloc. It's not necessary since C allows implicit conversion from void* to other pointer types, and the explicit could mask an error where you accidentally omit #include <stdlib.h>.
sizeof(char) is, by definition, 1 byte.
When you're allocating memory, it's safer to get into a habit of using T* p = malloc(n * sizeof *p); instead of T* p = malloc(n * sizeof (T));. That way if the type of p ever changes, you won't silently be allocating the wrong amount of memory if you neglect to update the malloc (or realloc or calloc) call.
Here, you have to zero your array of pointers (for example by using calloc()),
char **line = (char**)malloc(sizeof(char*)*3); //allocate space for three char* pointers
otherwise the reallocs
*(line+i) = (char *)realloc(*(line+i), (inputLength+1)*sizeof(char)); //+1 for the empty character
use an uninitialized pointer, leading to undefined behaviour.
That it works with i=0 is pure coindicence and is a typical pitfall when encountering UB.
Furthermore, when using strcat(), you have to make sure that the first parameter is already a zero-terminated string! This is not the case here, since at the first iteration, realloc(NULL, ...); leaves you with an uninitialized buffer. This can lead to strcpy() writing past the end of your allocated buffer and lead to heap corruption. A possible fix is to use strcpy() instead of strcat() (this should even be more efficient here):
do{
fgets(buffer, CHUNK, file);
buffLength = strlen(buffer);
lines[i] = realloc(lines[i], (lineLength + buffLength + 1));
strcpy(lines[i]+lineLength, buffer);
lineLength += buffLength;
}while(bufferLength ==CHUNK-1);
The check bufferLength == CHUNK-1 will not do what you want if the line (including the newline) is exactly CHUNK-1 bytes long. A better check might be while (buffer[buffLength-1] != '\n').
Btw. line[i] is by far better readable than *(line+i) (which is semantically identical).
Switching to C from Java, and I'm having some troubles grasping memory management
Say I have a function *check_malloc that behaves as such:
// Checks if malloc() succeeds.
void *check_malloc(size_t amount){
void *tpt;
/* Allocates a memory block in amount bytes. */
tpt = malloc( amount );
/* Checks if it was successful. */
if ( tpt == NULL ){
fprintf(stderr, "No memory of %lu bytes\n", amount);
exit(1);
}
return tpt;
}
I also have the following variables to work with:
FILE *f = fopen("abc.txt", "r"); // Pointer to a file with "mynameisbob" on the first line and
// "123456789" on the second line
char *pname; // Pointer to a string for storing the name
}
My goal is to use *check_malloc to dynamically allocate memory so that the String pointed to by *pname is just the correct size for storing "mynamisbob", which is the only thing on the first line of the text file.
Here is my (failed) attempt:
int main(int argc, char *argv[]){
FILE *f = fopen("abc.txt", "r"); // A file with "mynameisbob" on the first line and
// "123456789" on the second line
char *pname; // Pointer to a string for storing the name
char currentline[150]; // Char array for storing current line of file
while(!feof(f)){
fgets(currentline,100,f);
pname = ¤tline;
}
But I know this probably isn't the way to go about this, because I need to use my nice check_malloc* function.
Additionally, in my actual text file there is a "<" symbol before the name on the first line.But I just want the *pname to point to a String saying "mynameisbob" without the "<" symbol. This isn't that important now, it just is reinforcement to me that I know I can't just set the pointer to point straight to currentline.
Can anyone help me fix my thinking on this one? Thanks a lot.
In C you need to copy chars, not the "strings" (which are just pointers). Check out strcpy() and strlen(). Use strlen() to determine how long the line actually is which fgets has read, then use your malloc() to allocate exactly that (plus 1 for the 0). Then copy the chars over with strcpy().
There are several problems in your code, see my comments in this example:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// Checks if malloc() succeeds.
void *check_malloc (size_t amount) {
void *tpt;
/* Allocates a memory block in amount bytes. */
tpt = malloc( amount );
/* Checks if it was successful. */
if (tpt == NULL) {
fprintf (stderr, "No memory of %lu bytes\n", amount);
exit (EXIT_FAILURE);
}
return tpt;
}
// To avoid subtle errors I have defined buffer size here
#define BUFFER_SIZE 150
// I have used the (void) version of main () here, while not strictly neccessary, you where not using argc and argv anyway, so the can be left out in this case
int main (void) {
// It might be a good idea to make the filename a char[] as well, but I leave that as an exercise to the reader.
FILE *f = fopen("abc.txt", "r"); // A file with "mynameisbob" on the first line and
// "123456789" on the second line
// You have to check whether the file was *actually openend*
if (f == NULL) {
fprintf (stderr, "Could not open file abc.txt\n"); // '"...%s\n", filename);' might better.
exit (EXIT_FAILURE);
}
char *pname; // Pointer to a string for storing the name
char currentline[BUFFER_SIZE]; // Char array for storing current line of file
while(!feof (f)) {
char *res = fgets (currentline, BUFFER_SIZE, f);
// fgets returns NULL when EOF was encountered before the next '\n'
if (res) {
size_t read = strlen (res);
// The line might have been empty
if (read) {
// Better use "sizeof *varname", while char is always 1 byte it is a good practice
pname = check_malloc ((read + 1) * sizeof *pname); // + 1 because we have to provide an extra char für '\0'
strncpy (pname, currentline, read); // You have to use strcpy or strncpy to copy the contents of the string rather than just assigning the pointer
// What was allocated must be freed again
free (pname);
}
}
}
fclose(f); // Always close everything you open!
return EXIT_SUCCESS;
}
Actually you really don't have to use pname in this simple case, because currentline already contains the line, but since you're trying to learn about memory management this should give you a general idea of how things work.
In your code you had this line:
pname = ¤tline;
There are two problems here:
As already mentioned in my code assigning currentline to pname only copies the pointer not the contents.
The correct assignment would be pname = currentline (without the address operator &), because currentline is also a pointer under the hood (it behaves like char *currentline even though it's statically allocated).
So, for some reason, I need to make a external file (.DAT) to store data by appending the new one to the end of old data.
#include <stdio.h>
#include <stdlib.h>
int main () {
typedef struct {
char *Name;
int Index;
} DataFile;
static FILE *file;
size_t result;
DataFile *DataTable;
file = fopen("database.DAT","ab");
DataTable = (DataFile *) malloc (sizeof(DataFile));
DataTable[0].Name = "somefile.txt";
DataTable[0].Index = 7;
printf("%s %d \n",DataTable[0].Name,DataTable[0].Index);
result = fwrite(DataTable,sizeof(DataFile),1,file);
fclose(file);
free(DataTable);
return 0;
}
After running code above, I then check if the data stored correctly. So, I make this code below.
#include <stdio.h>
#include <stdlib.h>
int main () {
typedef struct {
char *Name;
int Index;
} DataFile;
static FILE *file;
size_t result;
long size;
int i;
DataFile *DataTable;
file = fopen("database.DAT","rb");
if (file == NULL) printf("Error1");
// Determine the size of file
fseek(file,0,SEEK_END);
size = ftell(file);
rewind(file);
DataTable = (DataFile *) malloc ((size/sizeof(DataFile)) * sizeof(DataFile));
if (DataTable == NULL) printf("Error2");
result = fread(DataTable,sizeof(DataFile),size/sizeof(DataFile),file);
fclose(file);
for (i=0; i<result; i++) {
printf("%s %d \n",DataTable[i].Name,DataTable[i].Index);
}
free(DataTable);
return 0;
}
However, it gives output
somefile.txt 7
from the first code block and
Error1 7
from the second code block.
I notice that the problem is not because the failure either when opening .DAT file or when allocating memory for DataTable. Also, it works for int type (Index) but not for char* type (Name) when reading from .DAT file. I have no idea what to do to solve this char*-type-reading problem (and where 'error1' comes from). (not even google gives me answer.)
Your structure DataFile stores one pointer and one integer. When you write it to the file, you write some program specific pointer to a string, and an integer.
When reading from it, you just refill your structure with the pointer and the integer, wich means that DataFile.Name will be a pointer to a probably-not-initialized memory segment. But since you created your file pointing to the first hard-coded string ("filename.txt"), some undefined but understandable behaviour happens, and your pointer in this case points to the first hard-coded string you wrote in you second program (which in your case is Error1)
What you really want to do is write the real string in your file.
A simple solution, if you want to the keep the hole writing structure thing is to create an array instead of a pointer
typedef struct {
char Name[512];
int Index;
} DataFile;
then initialize your data with
strncpy(DataTable[0].Name, "somefile.txt", sizeof(DataTable[0].Name) - 1); // just to make sure you dont overflow your array size
DataTable[0].Name[sizeof(DataTable[0].Name) - 1] = '\0';
and retreview your data the way you did.
A char* is only a pointer, i.e. the address of the character array containing your strings. You don't write the strings themselves to the file. After reading the file, as the same strings aren't in your memory at the same addresses any more, the application will fail.
You'll have to come up with a way to save the strings themselves to file as well. Probably by first writing their length, and then writing their content. Upon reading, you can use the length information to allocate memory dynamically, then read into that memory.
In your writing code you haven't allocated storage for char *Name. When you perform the DataTable[0].Name = "somefile.txt" instruction you're not actually copying the "somefile.txt" into memory pointed by Name, it's actually assigning a Name a value pointing to a constant characters string (moreover, it will become dangling pointer since the string is an rvalue, i.e. doesn't have a memory to be addressed via). Same goes for your file reading code.
You need to:
Allocate storage for your Name.
Copy the string using memcpy or similar into the allocated storage.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Pointer to local variable
Can a local variable's memory be accessed outside its scope?
gcc 4.4.4 c89
In main I call a function to pass a line of text to a function. I want to perform some operation on it. However, that would mean that line is of no use. So in my get_string function I copy the contents and return the result. The only problem, is that the memory to that result would be lost and pointing to something unexpected.
I am just wondering how can I pass the result back, without and still keep the ordinal line of data?
Many thanks for any advice,
code snippet from main:
if(fgets(line_data, (size_t)STRING_SIZE, fp) == NULL) {
fprintf(stderr, "WARNING: Text error reading file line number [ %d ]\n", i);
}
if(get_string(line_data) != NULL) {
if(strcmp(get_string(line_data), "END") == 0)
break;
}
else {
fprintf(stderr, "WARNING: Cannot get name of student at line [ %d ]\n", i);
}
/* Fill student info */
strncpy(stud[i].name, line_data, (size_t)STRING_SIZE);
Call this function
char* get_string(char *line_data)
{
char *quote = NULL;
char result[STRING_SIZE] = {0};
strncpy(result, line_data, (size_t)STRING_SIZE);
/* Find last occurance */
if((quote = strrchr(result, '"')) == NULL) {
fprintf(stderr, "Text file incorrectly formatted for this student\n");
return NULL;
}
/* Insert nul in place of the quote */
*quote = '\0';
/* Overwite the first quote by shifting 1 place */
memmove(result - 1, result, strlen(result) + 1);
return result;
}
Just return strdup(result).
It will allocate and copy your string.
However, you have to free the result after using it in the outer function.
You also could take a buffer in input (with its size), and fill it with what you want.
For your direct question - either use malloc(3) and tell the user of the function to de-allocate the return pointer (this is sort of prone to memory leaks since it's so easy to ignore return value in C), or provide the second parameter as a receive buffer:
char* get_string( const char* line_data, char* receive_buf, size_t buf_size );
The third parameter is for the function to know how large the receive buffer is.
Now to your code - the line memmove(result - 1, result, strlen(result) + 1); corrupts your stack.
You want to malloc the memory for result:
char *result; result = malloc(STRING_SIZE);
As you have it, the memory for result exists on the stack and thus only during the time that execution is inside get_string()
You'll also need to free result before returning NULL to prevent a memory leak.
As a rule of thumb, you should never return a pointer to a function's local variable. You know why: once a function returns, the memory allocated for its variables can be reused for something else. The idea to return a pointer to the result buffer is inherently bad.
You should think whether you really need to keep a copy of the quoted string. What if you tested the "END" string before calling get_string? If you need to quote and output data later, it is done easily. Say:
printf("\"%s\"", student_record);
So get_string could actually work in the buffer in place and return the error code (0 for success). Since you know the final result is a smaller nul terminated string, you wouldn't even need a length parameter.
int get_string(char* student_record);
If you really need to keep a copy of the quoted string, then you need to pass another buffer. I'd still return an int to indicate success (0) or failure (say -1).
int get_string( const char* line_data, char* student_record, size_t buf_size );
I personally prefer letting the caller allocate its own buffer. It leaves it a chance to use a fixed length buffer (simpler memory management). Ex:
char student_record[512];
...
if (!get_string(student_record)) {
// error
}