Ok I use method from here: How to Read only the last line of a text file in Lua?
The problem is that sometimes line can be bigger.
The question is how can i find first word "foo" from the end of file and then use everything after it?
The problem is that sometimes line can be bigger.
Then you just need to seek further back from the end.
The question is how can i find first word "foo" from the end of file and then use everything after it?
Grab a big enough chunk of the file to be sure you've got the last foo, the use .*foo to skip everything up to and including the last "foo" (.* is greedy).
local f = io.open('filename', 'r')
f:seek('end', -1024)
local text = f:read('*a')
local after = string.match(text, ".*foo(.*)")
f:close()
If the file is not too big and you're ready to take the easy way out this might help:
fh=io.open('myfile.txt','rb')
str=fh:read'*a'
pat='foo'
afterFoo=str:match('.*'..pat..'(.*)$')
fh:close()
If you need a more complex, but faster (in run time on large files) solution, my guess would be that you' read in the file in chunks, reverse each of them, and look for your pattern in reverse. Don't forget to look for your pattern across the borders (the chunks must overlap at least the length of the pattern you're seeking in the general case).
For more explanation about the block reading, see my post here.
Related
I'm currently writing out a program in Ruby, which I'm fairly new at, and it requires multiple text files to be pushed into an array line by line.
I am currently unable to actually test my code since I'm at work and this is for personal use, but I'm seeking advice to see if my code is correct. I knows how to read a file and push it to the array. If possible can someone check it over and advise if I have the correct idea? I'm self taught regarding Ruby and have no-one to check my work.
I understand if this isn't the right place for trying to get this sort of advice and it's deleted/locked. Apologies if so.
contentsArray = []
Dir.glob('filepath').each do |filename|
next if File.directory?(filename)
r = File.open("#{path}#{filename}")
r.each_line { |line| contentsArray.push line}
end
I'm hoping this snippet will take the lines from multiple files in the same directory and stick them in the array so I can later splice what's in there.
Thank you for the question.
First let's assume that 'filepath' is something like the target pattern you want to glob in Dir.glob('filepath') (I used Dir.glob('src/*.h').each do |filename| in my test).
After that, File.open("#{path}#{filename}") prepends another path to the already complete path you'll have in filename.
And lastly, although this is probably not the problem, the code opens the file and never closes it. The IO object provides a readlines method that takes care of opening and closing the file for you.
Here's some working code that you can adapt:
contentsArray = []
Dir.glob('filepath').each do |filename|
next if File.directory?(filename)
lines = IO.readlines(filename)
contentsArray.concat(lines)
end
puts "#{contentsArray.length} LINES"
Here are references to the Ruby doc's for the IO::readlines and Array::concat methods used:
https://ruby-doc.org/core-2.5.5/IO.html#method-i-readlines
https://ruby-doc.org/core-2.5.5/Array.html#method-i-concat
As an alternative to using the goto (next) the code could conditionally execute on files, like this:
if File.file?(filename)
lines = IO.readlines(filename)
contentsArray.concat(lines)
end
I am reading info (numbers) from a txt file and after that I am adding to those numbers, others I had in another file, with the same structure.
At the start of each line in the file is a number, that identifies a specific product. That code will allow me to search for the same product in the other file. In my program I have to add the other "variables" from one file to the other, and then replace it, in the same place in one of those files.
I didn't open any of those files with a or a+, I did it with r and r+ because i want to replace the information in the lines that may be in the middle of the file, and not in the end of it.
The program compiles, and runs, but when it comes to replace the info in the file, it just doesn't do anything.
How should I resolve the problem?
A program can replace (overwrite) text in the middle of the file. But the question is whether or not this should be performed.
In order to insert larger text or smaller text (and close up the gap), a new text file must be written. This is assuming the file is not fixed width. The fundamental rule is to copy all original text before the insertion to a new file. Write the new text. Finally write the remaining original text. This is a lot of work and will slow down even the simplest programs.
I suggest you design your data layout before you go any further. Also consider using a database, see my post: At what point is it worth using a database?
Your objective is to design the data to minimize duplication and data fetching.
my file pointer is pointing to end of a line. I want to remove all contents of that line, how do I do that?
I might need to move the file pointer to start of the line and then delete the contents.
You can only delete from the end of a file. To delete data from the middle of a file, you generally need to copy the subsequent data to cover up the gap (or, more easily as a rule, make a new copy of the file, skipping over the part you want to delete).
If you need to do things like this very often, you'll probably want to create some sort of indexed file so you can just delete from the index -- or, of course, use a database library to handle it for you.
You can't "delete" anything from a file. In C language files are accessed through streams, and streams don't support such operation as "delete a line" or "delete" anything at all. You can delete the entire file, but that's apparently not what you need.
Within the C language approach to working with files, all you can do is copy your original file to another file, skipping the line in question. The second file will look like the original one with the line deleted. After doing that you can destroy the original file and use the new one in its place.
There's a chance you might mean something else by your "delete" (what does your "delete" mean, BTW?). You might want to overwrite the contents of the line with space characters, for one example. If so, just move the current file pointer to the beginning of the line and write the appropriate number of space characters to the file.
You have to shift all of the content beyond the line back to the location where the line to be deleted begins.
If you're working in an environment that supports it, you could mmap(2) the file, work with the whole thing in memory and use memmove(3) to make the shifts.
How can I delete few characters from a file using C program?
I could not find any predefined functions for it.
To understand the purpose, I am trying to send a file through a socket, if N bytes are sent successfully, I want to delete those bytes from the file. At the end, the file will be empty.
Any other way to do this efficiently?
Thanks
Pradeep
If they're at the end, truncate the file at the appropriate length. If they're not then you'll need to rewrite the file.
Your way is pretty inefficient for large files, since you would have to copy "the rest of the file" some bytes further to the beginning, which costs much. I would rather record the "current sending position" somewhere outside of the file and update that information. That way, you don't have to copy the rest of the file so often.
There is no straightforward way to delete bytes from the beginning of a file. You will have to start from where you want to trim the file, and read from there to the end of the file, writing to the start of the file.
It might make more sense to just track how many bytes you have already written to the file in some other file.
you should use an index which points to the beginning of the data you haven't sent yet.
It is not necessary to delete what you have sent, just pass them, when you send the whole file delete it.
If the char's are one after the other than why dont you give a try to fseek();
I've got a service which runs all the time and also keeps a log file. It basically adds new lines to the log file every few seconds. I'm written a small file which reads these lines and then parses them to various actions. The question I have is how can I delete the lines which I have already parsed from the log file without disrupting the writing of the log file by the service?
Usually when I need to delete a line in a file then I open the original one and a temporary one and then I just write all the lines to the temp file except the original which I want to delete. Obviously this method will not word here.
So how do I go about deleting them ?
In most commonly used file systems you can't delete a line from the beginning of a file without rewriting the entire file. I'd suggest instead of one large file, use lots of small files and rotate them for example once per day. The old files are deleted when you no longer need them.
Can't be done, unfortunately, without rewriting the file, either in-place or as a separate file.
One thing you may want to look at is to maintain a pointer in another file, specifying the position of the first unprocessed line.
Then your process simply opens the file and seeks to that location, processes some lines, then updates the pointer.
You'll still need to roll over the files at some point lest they continue to grow forever.
I'm not sure, but I'm thinking in this way:
New Line is a char, so you must delete chars for that line + New Line char
By the way, "moving" all characters back (to overwrite the old line), is like copying each character in a different position, and removing them from their old position
So no, I don't think you can just delete a line, you should rewrite all the file.
You can't, that just isn't how files work.
It sounds like you need some sort of message logging service / library that your program could connect to in order to log messages, which could then hide the underlying details of file opening / closing etc.
If each log line has a unique identifier (or even just line number), you could simply store in your log-parsing the identifier until which you got parsing. That way you don't have to change anything in the log file.
If the log file then starts to get too big, you could switch to a new one each day (for example).