I want to create a log file in Scheme, but every time I add a new entry, I want it to be at the beginning of the file, so when I read X number of logs from the file again, it reads the X newest entries from new to old.
Example:
22/02/14 13:50 Newest log entry
22/02/14 13:45 Older log entry
22/02/14 13:40 Oldest log entry
Does anyone know how to do this using the 'open-input-file' and 'open-output-file' procedures?
The functionality you are requesting you need to write the whole logfile every time you need to write a new entry because you will overwrite the previous first entry with the next. Usually programs don't keep the commited parts of a logfile so this introduces more memory usage and your program must know when the log is being rotated to clear the buffer.
The standard way is to append a new entry, which leaves the previous log entries where the last log write put them.
As a compromise you might look for a program that displays a log file in the reverse order and prehaps tails it like that too. It's easy to implement so I guess it would exist already. Writing such an app if it doesn't exist would be trivial.
Related
I'm using text files as a database for saving users' information for a game which i made using swi-prolog. The information is saved like this:user(Name,Password,Age,Points). What i want to do is to change a user's Points without having to rewrite the entire db. In other words, I am looking for something that will work like retractall(user(Name,_,_,_)), but with the text file. I know how to find the specific user using read/2, and how to assert a new fact using write/2, but i don't know how to delete one specific line in the text file.
Thank you for helping.
Take a look at SWI-Prolog's library(persistency). It removes a fact by adding a line that the fact is removed. If the file gets too big with add/remove lines, it provides db_sync/1 to write a clean file. OS file system operations do not allow to remove part of a file (except from truncating the end). The normal way to do this is to write a new file and, if successful, rename this to the existing one, so nothing is lost if you crash while writing the new file.
I'm writing a program/utility in C to find (and then move to a new directory) the files in the current directory that have been modified after the last time the utility was run.
What I'm trying to find out is if there is a way to find the last time this utility ran. Or alternatively, a way to store the time in the program (so as to compare the last stored time against the current time, and then update the "last time" variable to current time).
As I type this it occurs to me that I could write the time to a file (overwriting the single entry as the utility is run) and retrieve the value from the file in the program, although I don't know if this would be the best approach.
you can make a class contains info and serialize it to a text file , it's more easy to access and can store multiple values,
then to store new values first delete file and then create file again.
another approach could be a register key containing information.
hope it would be useful ;)
You can use the last access time from the filesystem (In GNU/linux you can use ls -lu to see last access time).
This is not a portable solution because it depends on filesystem and filesystem settings (see JoachimPileborg edit below)
Moreover look at this question to get last acces time in C (use atime instead of mtime).
I have a program which logs its activity.
I want to implement a log file mechanism to keep the log file under a certain size, lets say 10 MB.
The log file itself just holds commands the program executed; those commands are variable length.
Right now, the program runs on a windows environment, but I'm likely to port it to UNIX soon.
I've came up with two methods for managing the log files:
1. Keep multiple files of lower size, and if the new command exceeds the current file length, truncate the oldest file to zero size, and start writing there.
2. Keep a header in the file, which holds metadata regarding the first command in the file, and the next place to write to in the file. Also I think, each command should hold metadata about it's length this way.
My questions are as follows:
In terms of efficiency which of these methods would you use, and why?
Is there a unix command / function to this easily?
Thanks a lot for your help,
Nihil.
On UNIX/Linux platforms there's a logrotate program that manages logfiles. Details can be found for example here:
http://linuxcommand.org/man_pages/logrotate8.html
I'm intending to create a programme that can permanently follow a large dynamic set of log files to copy their entries over to a database for easier near-realtime statistics. The log files are written by diverse daemons and applications, but the format of them is known so they can be parsed. Some of the daemons write logs into one file per day, like Apache's cronolog that creates files like access.20100928. Those files appear with each new day and may disappear when they're gzipped away the next day.
The target platform is an Ubuntu Server, 64 bit.
What would be the best approach to efficiently reading those log files?
I could think of scripting languages like PHP that either open the files theirselves and read new data or use system tools like tail -f to follow the logs, or other runtimes like Mono. Bash shell scripts probably aren't so well suited for parsing the log lines and inserting them to a database server (MySQL), not to mention an easy configuration of my app.
If my programme will read the log files, I'd think it should stat() the file once in a second or so to get its size and open the file when it's grown. After reading the file (which should hopefully only return complete lines) it could call tell() to get the current position and next time directly seek() to the saved position to continue reading. (These are C function names, but actually I wouldn't want to do that in C. And Mono/.NET or PHP offer similar functions as well.)
Is that constant stat()ing of the files and subsequent opening and closing a problem? How would tail -f do that? Can I keep the files open and be notified about new data with something like select()? Or does it always return at the end of the file?
In case I'm blocked in some kind of select() or external tail, I'd need to interrupt that every 1, 2 minutes to scan for new or deleted files that shall (no longer) be followed. Resuming with tail -f then is probably not very reliable. That should work better with my own saved file positions.
Could I use some kind of inotify (file system notification) for that?
If you want to know how tail -f works, why not look at the source? In a nutshell, you don't need to periodically interrupt or constantly stat() to scan for changes to files or directories. That's what inotify does.
I've got a service which runs all the time and also keeps a log file. It basically adds new lines to the log file every few seconds. I'm written a small file which reads these lines and then parses them to various actions. The question I have is how can I delete the lines which I have already parsed from the log file without disrupting the writing of the log file by the service?
Usually when I need to delete a line in a file then I open the original one and a temporary one and then I just write all the lines to the temp file except the original which I want to delete. Obviously this method will not word here.
So how do I go about deleting them ?
In most commonly used file systems you can't delete a line from the beginning of a file without rewriting the entire file. I'd suggest instead of one large file, use lots of small files and rotate them for example once per day. The old files are deleted when you no longer need them.
Can't be done, unfortunately, without rewriting the file, either in-place or as a separate file.
One thing you may want to look at is to maintain a pointer in another file, specifying the position of the first unprocessed line.
Then your process simply opens the file and seeks to that location, processes some lines, then updates the pointer.
You'll still need to roll over the files at some point lest they continue to grow forever.
I'm not sure, but I'm thinking in this way:
New Line is a char, so you must delete chars for that line + New Line char
By the way, "moving" all characters back (to overwrite the old line), is like copying each character in a different position, and removing them from their old position
So no, I don't think you can just delete a line, you should rewrite all the file.
You can't, that just isn't how files work.
It sounds like you need some sort of message logging service / library that your program could connect to in order to log messages, which could then hide the underlying details of file opening / closing etc.
If each log line has a unique identifier (or even just line number), you could simply store in your log-parsing the identifier until which you got parsing. That way you don't have to change anything in the log file.
If the log file then starts to get too big, you could switch to a new one each day (for example).