I have an existing file and I'd like to append data to it and make sure it can never (or almost never) get corrupted, even if something fails during writing of the appended data.
One method for ensuring files won't get corrupted it to write the data to a temp file, and then rename/mv the temp file to the original file.
But doing so with append is more tricky.
I have the whole file content in memory (it's not a huge file), so I have two options in mind:
Copy the original file to a temp file, append the data to the temp file and then mv/rename the temp file to the original file
Write the whole content of the file (including the data I want to append) to a temp file and then mv/rename the temp file to the original file
The downside of both options is that they're slower than just append the data to the original file. Are there better ways to do this?
If not, which option is faster?
I need this to work on Windows, Linux and MacOS.
I'm not sure if the programming language I'm using is relevant, but I'm using Rust to write the data.
Related
I created a file with fopen and I deleted the first value from the file and I want to take all of the values that are in the file and move them to the start of the file.
Example:
File: [Info,data,string]
Wanted: [data,string,]
What is happening: [,data,string]
Any help will be great.
Then you have better to copy all data after the first byte into another file, then move that file into the original. There's no system call to delete some part of the file, and close the hole around the deleted data. The file internal structure is not suited to do that, and requires to copy/shift the whole file data to the proper place.
I have many files that are extracted into .txt with a batch file. But they don't have the headers. I've read that a possible solution from here that is to add to a .txt with the headers the exported rows.
With this:
echo. >> titles.txt
type data.txt >> titles.txt
This takes a lot of time and is not efficient, since it is adding the big file to the file with the text.
Another possible solution is to add to the SQL query the titles hardcoded, but this will change the type of the columns (is they are numeric they will be changed to varchar).
Is there a way to insert in the first row of the data txt the headers and not doing vice-versa?
I might be wrong, but as far as I am informed (and as far as I know from earlier experiments in doing as described): No, it is not possible! The mentioned Tasks are acting on the file sequentially. You can either open a file for reading, writing or appending. If you open the titles.txt file for writing, it is overwritten - and with this empty. If you open it for appending, it can only append to the end of the file - so you can only write the data after the Header... the only way it might work - but which is pretty nasty - is to append the title to the end of the file and during later processing (e.g. xls or whatever) Resort the rows and put the last one to the beginning. But as mentioned: nasty and not really the way to go.
If the number of files to process is a bigger problem than any individual file size, switching from bcp to sqlcmd might help.
I have opened a file using a and r+ but when I use fseek and ftell the file pointer is always 0.
My file looks like this:
1 -3
2 -8
And I want to add another line between the two but it is added in the end after the last line.
Someone in another forum said that when you open the file in append the pointer is always zero and you have to open it in r+ and if that doesn't work "you have to read the complete data and then insert the data in the variables and write it back." but I don't understand what they mean by that.
Can anyone help with inserting numbers in the middle of a file?
Thanks!
Would something like this work?
To transfer the data?
rewind(fp);
fscanf(fp,"%d",&ch);
fprintf(fp1,"%d",ch);
fseek(fp,1,0);
fscanf(fp,"%d",&ch);
fprintf(fp1,"%d",ch);
Like others already said, there's no easy way to insert data in the middle of a file. If you really want to do this, you can implement the following steps:
Create a second file
Copy all data before the place you want to insert to the second file
Insert the line you want to the second file
Copy the remaining data to the second file
Delete the original file
Rename the second file
Other approach is using binary files instead of text files. Although binary files are a bit harder to learn, once you understand how they work you'll see that working with them is much like working with arrays. To perform this task, for example, you'd not even need to use an auxiliary file.
There is no open mode that will allow you to "insert" data into a file at a random point. The only place you can add data without overwriting existing data is the end of the file (what you get opening with mode "a").
If you want to insert at a random position, you need to do it yourself.
One of the easier ways is to re-write the file completely (transfer the start of the old file to a new file, add your data to the new file, transfer the rest of the old file, and rename/overwrite at the end).
The hard way: you need to "shift" all the data from your insertion point to the end-of-file manually. That's not trivial to get right.
There isn't an easy way to insert data in the middle of the file. A file is basically an array of characters. To add a character in the middle, you need to copy everything following your insertion point down one location. With a file you need to read the data that follows and write it after your addition.
Generally, when you want to do something like this you create a new file. You copy the old file into it up to the point where you want to insert, then you write the data you want to insert, then you copy the rest of the old file. Finally, you rename the new file to the old file.
How can I go back to the very beginning of a csv file and add rows?
(I'm printing to a CSV file from C using fprintf(). At the end of printing thousands of rows (5 columns) of data, I would like to go back to the top of the file and insert some dynamic header data (based on how things went printing everything). )
Thank You.
Due to the way files are structured, this is more or less impossible. In order to accomplish what you want:
write csv data to file1
write header to file2
copy contents of file1 to file2
delete file1
Or you can hold the csv data in ram and write it to file after you're finished processing and know the header.
Another option is to set aside a certain number of bytes for the header, which will work much faster for large files at minimal space cost. Since the space is allocated in the file at the start of the write, there aren't any issues going back and filling it in. Reopen the file as random access ("r+"), which points to the top of the file by default, write header, and close.
The simplest way would be to simply store the entire contents of the file in memory until you are finished, write out the header, and then write out the rest of the file.
If memory is an issue and you can't safely store the entire file in memory, or just don't want to, then you could write out the bulk of the CSV data to a temporary file, then when you are finished, write the header out to the primary file, and copy the data from the temporary file to the primary file in a loop.
If you wanted to be fancy, after writing the main CSV data out to the primary file, you could loop through the file from the beginning, read into memory the data that you're about to overwrite with the header, then write the header over top of that data, and so forth, read each chunk into memory, overwrite it with the previous one until you reach the end and append the final chunk. In this way you "insert" data at the beginning, my moving the rest of the file down. I really wouldn't recommend this as it will mostly just add complexity without much benefit, unless there is a specific reason you can't do something simpler like using a temporary file.
I think that is not possible. Probably the easiest way would be to write the output to a temporary file, then create the data you need as the dynamic header, write them to the target file and append the previously created temporary file.
write enough blank spaces in the first line
write data
seek(0)
write header - last column will be padded with spaces
When you open a .txt file with fopen
Is there any way to delete some strings in a file without rewriting.
For example this is the txt file that i will open with fopen() ;
-------------
1 some string
2 SOME string
3 some STRING
-------------
i want to delete the line which's first character is 2 and change it into
-------------
1 some string
3 some STRING
-------------
My solution is;
First read all data and keep them in string variables. Then fopen the same file with w mode. And write the data again except line 2. (But this is not logical i am searching for an easier way in C ...)
(i hope my english wasn't problem)
The easiest way might be to memory-map the whole file using mmap. With mmap you get access to the file as a long memory buffer that you can modify with changes being reflected on disk. Then you can find the offset of that line and move the whole tail of the file that many bytes back to overwrite the line.
you should not overwrite the file, better is to open another (temp)-file, write contents inside and then delete old file and rename the file. So it is safer if problems occur.
I think the easiest way is to
read whole file
modify contents in memory
write back to a temp file
delete original file
rename temp file to original file
Sounds not too illogical to me..
For sequential files, no matter what technique you use to delete line 2, you still have to write the file back to disk.