monitoring file changes in racket (like tail -f) - file

I would like to implement a "tail -f" like behavior in Racket. That is, I would like to read from a file, and when I hit the end, be able making something like a "blocking" (read-line file), that shall return when some other process appends a line to file.
I tried synchronizing with (read-line-evt file) but, if I am at the end of file, instead of blocking until other data is available, it returns immediately.
Is there a way to do it?

I don't think that you have any way to avoid polling the file.
Note that all of Racket's input functions consider eof a value that should be returned when it reaches the end of the input stream -- so all of the events immediately return that when the end is reached. At least I don't see anything that looks like a "wait until some input is ready, not eof".
In any case, you also have the ffi, if you know about some system call that triggers a callback instead of polling the file. AFAICT, the linux source code for tail uses inotify, so you might be able to use an old package that interfaces that from racket called mzfam. (But it's pretty old and might need some update work.)

I don't know when Racket added file system change events, but I suspect it was since this question was asked many years ago. Now you can wait on such an event and see if you can read another line (It's not fine grained enough to tell specifically that more data was appended to the file, just that something changed about it.)
An example of a basic tail -f like program to demonstrate file-system-change-evt:
;;; tail.rkt
#lang racket/base
(require racket/list racket/port)
;;; Some utility functions and macros
;; Like take but return the list if it's less than n elements long
;; instead of raising an error
(define (take* list n)
(with-handlers ([exn:fail:contract? (lambda (e) list)])
(take list n)))
;; Repeat body forever until a break is received
(define-syntax-rule (forever body ...)
(with-handlers ([exn:break? (lambda (e) (void))])
(let loop ()
body ...
(loop))))
;; Display the last N lines of a file. Could be more efficient, but
;; this part's not the point...
(define (display-last-lines port n)
(for-each displayln
(reverse
(for/fold ([lines '()])
([line (in-lines port)])
(take* (cons line lines) n)))))
;; Wait for the file's status to change and try to read lines when it does.
(define (follow-tail file)
(call-with-input-file file
(lambda (port)
(display-last-lines port 10)
(forever
(sync (filesystem-change-evt file))
(for ([line (in-lines port)])
(displayln line))))))
(module+ main
(unless (= (vector-length (current-command-line-arguments)) 1)
(displayln "Usage: racket tail.rkt FILENAME" (current-error-port))
(exit 1))
(follow-tail (string->path (vector-ref (current-command-line-arguments) 0))))
After being inspired by this question and Eli's mention of inotify in his answer, and seeing that there still wasn't a Racket package to provide access to it (I think the standard file system change code uses it internally, but it's not exposed at any low level to users), I wrote it myself. A version of the core tail function from above using it:
(require inotify)
(define (follow-tail file)
(call-with-input-file file
(lambda (port)
(display-last-lines port 10)
(call-with-inotify-instance
`((,file (IN_MODIFY)))
(lambda (inotify wds)
(forever
(sync inotify)
(for ([line (in-lines port)])
(displayln line))))))))

Related

lisp package not available at read time when loading is deferred

I made a small repro for this issue using only lisp.
file1.lisp :-
(defpackage :my_package
(:use :cl))
(in-package :my_package)
(defun subscribe (x) (print x)(terpri))
(export '(subscribe))
(in-package :cl-user)
file2.lisp :-
(defun loader ()
(load "file1.lisp")
(my_package:subscribe "hello"))
(loader)
Now, running this gives the same error :-
sbcl --load file2.lisp
This is SBCL 2.1.11.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
debugger invoked on a SB-C::INPUT-ERROR-IN-LOAD in thread
#<THREAD "main thread" RUNNING {1001834103}>:
READ error during LOAD:
Package MY_PACKAGE does not exist.
Line: 3, Column: 22, File-Position: 59
Stream: #<SB-INT:FORM-TRACKING-STREAM for "file /home/omkarwagh/swarajya/scratch/file2.lisp" {10009BF613}>
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT ] Abort loading file "/home/omkarwagh/swarajya/scratch/file2.lisp".
1: [CONTINUE] Ignore runtime option --load "file2.lisp".
2: Skip rest of --eval and --load options.
3: Skip to toplevel READ/EVAL/PRINT loop.
4: [EXIT ] Exit SBCL (calling #'EXIT, killing the process).
(SB-C:COMPILER-ERROR SB-C::INPUT-ERROR-IN-LOAD :CONDITION #<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Package ~A does not exist." {10018375A3}> :STREAM #<SB-INT:FORM-TRACKING-STREAM for "file /home/omkarwagh/swarajya/scratch/file2.lisp" {10009BF613}>)
4
I guess that the problem is that we don't know the package at read time because the relevant file has not been loaded yet.
So the question is, what is the recommended practice in this case?
Original question
I have a roswell script that I've attached below. I have an asdf system called "my_system" which has only one module with one file that contains only one package called "my_package".
Somehow, the asdf system is being loaded but when I try to actually use any of the functions in it, then it fails with an error saying that "my_package" is not found
#!/bin/sh
#|-*- mode:lisp -*-|#
#|
exec ros -Q -- $0 $(readlink -f $(dirname $(readlink -f $0)))/asdf.conf "$#"
|#
(progn ;;init forms
(ros:ensure-asdf)
#+quicklisp(ql:quickload '() :silent t)
)
(defpackage :ros.script.test.3880638094
(:use :cl))
(in-package :ros.script.test.3880638094)
(require "asdf")
;(asdf:load-system "my_system")
;(print #'my_package:subscribe)
(defun main (mydir &rest argv)
(declare (ignorable argv))
(asdf:initialize-source-registry `(:source-registry :inherit-configuration (:include ,mydir)))
(asdf:load-system "my_system")
(sleep 0.2)
(print "here3")(terpri)
(print (list-all-packages))
(do-external-symbols (s (find-package :my_package)) (print s))
(print #'my_package::subscribe)(terpri)
)
;;; vim: set ft=lisp lisp:
The error is :-
Unhandled SB-C::INPUT-ERROR-IN-LOAD in thread #<SB-THREAD:THREAD "main thread" RUNNING
{1004460113}>:
READ error during LOAD:
Package MY_PACKAGE does not exist.
However, if I comment out this line :-
(print #'my_package::subscribe)(terpri)
I do in fact, see the package in the list of packages as well as the symbol in question :-
#<PACKAGE "MY_PACKAGE">
MY_PACKAGE:SUBSCRIBE
As you said, the error is signalled because the package does not exist at the read time (when the form is read as text and translated to symbols).
The most straightforward option to change the code above this is to create the symbol when function is executed, e.g., (funcall (intern "SUBSCRIBE" 'my-package)). Variant of this (using read-from-string instead of intern) can be seen in the Swank or Slynk loader.
If you use asdf (which is implied in your question), you should probably use uiop to do this for you - see uiop:symbol-call. For example, documented way to define test-op is with
:perform (test-op (o c) (symbol-call :fiveam '#:run! :foobar)))
However, most libraries structure files in such a way that this issue does not arise - the file1.lisp would be loaded before file2.lisp (e.g., using asdf system definition).

using sbcl schedule-timer with sly

I want to use a timer to periodically print info to the repl (and do some other things).
SBCL 2.0.1
Emacs 26.1
sly 1.0.0-beta-3
(I can't create a sly tag...)
in sly-repl:
(defvar *timer* (make-timer (lambda ()
(write-line ".")
(force-output))))
(schedule-timer *timer* 2 :repeat-interval 1)`
and works fine - after 2 seconds, "." is printed
in timer-test.lisp:
(defvar *timer* (make-timer (lambda ()
(write-line ".")
(force-output))))
(schedule-timer *timer* 2 :repeat-interval 1)`
compiles fine
when I execute the schedule-timer form, the REPL tells me:
Timer #<TIMER {1002C5EB33}> failed to interrupt thread #<SB-THREAD:THREAD "slynk-worker" FINISHED values: T {1002C3DFA3}>.
I don't know enough about threads in SBCL to sort this out. Is there a simple way to get a scheduled timer (in a file) to output to the REPL?
with thanks for your help.
Here is a version of your code which tries to use some SWANK functionality to find out what the REPL thread is and schedule the timer in that thread. Note that this won't work with SLY as far as I can tell because SLY has different package names at least. However it may provide a clue and it's too long to be a comment.
Note also that I don't know if this is the right way of finding the REPL thread, or if the REPL thread actually persists for any length of time.
If repl-thread-maybe fails to find a candidate thread, it returns t, which causes make-timer to run the timer in its own thread. This is safe I think, but will mean that any output from the timer (assuming that's what you are after) goes somewhere other than the REPL.
Anyway, here it is for what it's worth
;;;;
;;;
(in-package :cl-user)
#-(and SBCL SWANK)
(eval-when (:compile-toplevel :load-toplevel :execute)
(error "Not SBCL / not SWANK"))
(defun repl-thread-maybe (&key (repl-thread-name "repl-thread")
(fallback-value t))
;; I have no idea if this is the right way to find the REPL thread,
;; but it kind of works.
(or (find-if (lambda (thread)
(string-equal (swank/backend:thread-name thread)
repl-thread-name))
(swank/backend:all-threads))
fallback-value))
(defparameter *timer* (make-timer (lambda ()
(write-line ".")
(force-output))
:thread (repl-thread-maybe)))
(schedule-timer *timer* 2 :repeat-interval 1)

embedded perl in C, perlapio - interoperability with STDIO

I just realized, that the PerlIO layer seems to do something more than just (more or less) easily wrap the stdio.h-functions.
If I try to use a file-descriptor resolved via PerlIO_stdout() and PerlIO_fileno() with functions from stdio.h, this fails.
For example:
PerlIO* perlStdErr = PerlIO_stderr();
fdStdErrOriginal = PerlIO_fileno(perlStdErr);
relocatedStdErr = dup(fdStdOutOriginal);
_write(relocatedStdErr, "something", 8); //<-- this fails
I've tried this with VC10. The embedded perl program is executed from a different context - so it's not possible to use PerlIO from the context where the write to the relocatedStdErr is performed.
For the curious: I need to execute a perl script and forward the output of the script's stdout/stderr to a log whilst keeping the ability to write on stdout for myself. Moreover this should work platform independent (linux, windows console application, win32 desktop application). Just to forward the stdout/stderr doesn't work in Win32 desktop applications since there is none ;) - you need to use the perl's stdout/stderr.
Needed solution: Be able to write on a filehandle (or descriptor) derived from perlio NOT using the PerlIO stack.
EDIT - my solution:
As Story Teller was pointing to PerlIO_findFILE, this did the trick.
So here an excerpt of the code - see the comments inside for descriptions:
FILE* stdErrFILE = PerlIO_findFILE(PerlIO_stderr()); //convert to Perl's stderr to stdio FILE handle
fdStdErrOriginal = _fileno(stdErrFILE); //get descriptor using MSVC
if (fdStdErrOriginal >= 0)
{
relocatedStdErr = _dup(fdStdErrOriginal); //relocate stdErr for external writing using MSVC
if (relocatedStdErr >= 0)
{
if (pipe(fdPipeStdErr) == 0) //create pipe for forwarding stdErr - USE PERL's IO since win32subsystem(non-console) "_pipe" doesn't work
{
if (dup2(fdPipeStdErr[1], fdStdErrOriginal) >= 0) //hang pipe on stdErr - USE PERL's IO (since it's created by perl)
{
close(fdPipeStdErr[1]); //close the now duplicated writer on stdErr for further usage - USE PERL's IO (since it's created by perl)
//"StreamForwarder" creates a thread that catches/reads the pipe's input and forwards it to the processStdErrOutput function (using the PerlIO)
stdErrForwarder = new StreamForwarder(fdPipeStdErr[0], &processStdErrOutput, PerlIO_stderr());
return relocatedStdErr; //return the relocated stdErr to be able to '_write' onto it
}
}
}
}
...
...
_write(relocatedStdErr, "Hello Stackoverflow!", 20); //that works :)
One interesting thing that I actually don't understand is, that the perl documentation says that is't necessary to #define PERLIO_NOT_STDIO 0 to be able to use PerlIO_findFILE(). But for me, that works fine without it and further I like to use PerlIO and the stdio together anyway. That's a point I didn't figured out what is going on.

Emacs executes `c-mode-hook` twice

Here's my emacs config
(require 'cc-mode)
(defun ffy-c-mode-hook ()
"This is settings for the C/C++ mode"
(message "ffy-c-mode-hook executed")
(electric-pair-mode +1)
(electric-indent-local-mode +1)
(if electric-indent-mode
(let ((binding (key-binding (kbd "RET"))))
(local-set-key (kbd "RET") (key-binding (kbd "C-j")))
(local-set-key (kbd "C-j") binding)))
(c-toggle-hungry-state +1)
(c-set-style "gnu")
(setq c-basic-offset 4))
(add-hook 'c-mode-hook #'ffy-c-mode-hook)
Apparently c-mode-hook is executed twice, because when I open a C file I see:
user-error: Beginning of history; no preceding item
ffy-c-mode-hook executed [2 times]
Is it a feature or a bug ? No other hooks are executed multiple times AFAIK. Plus it seems that now I can't toggle features in the hook.
I looked at the variable c-mode-hook and it its value is (ffy-c-mode-hook)
The versions of Emacs and CC-mode are
GNU Emacs 24.5.1 (i686-pc-mingw32) of 2015-04-11 on LEG570
Using CC Mode version 5.32.5
Stacktrace on ffy-c-mode-hook
ffy-c-mode-hook()
run-hooks(change-major-mode-after-body-hook prog-mode-hook c-mode-common-hook c-mode-hook c-mode-hook)
apply(run-hooks (change-major-mode-after-body-hook prog-mode-hook c-mode-common-hook c-mode-hook c-mode-hook))
run-mode-hooks(c-mode-hook)
c-mode()
set-auto-mode-0(c-mode nil)
set-auto-mode()
Following my comment about the bug report http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16759 I don't think that language hooks are guaranteed to be executed once for a number of language modes. I suspect the issue is due to using define-derived-mode which is a lisp macro for defining a mode that already includes calls to hooks, this means the hooks called in the mode will be an additional execution.
You might want to try the initialization hook. The manual says:
Variable: c-initialization-hook
Hook run only once per Emacs session, when CC Mode is initialized. This is a good place to change
key bindings (or add new ones) in any of the CC Mode key maps. See
Sample Init File.
The sample it gives is here: https://www.gnu.org/software/emacs/manual/html_node/ccmode/Sample-Init-File.html#Sample-Init-File

Emacs lisp - problems with filename-string on win7

On Win7 with Emacs24 I encounter problems when Emacs programs want to open (pdf) files. The problems remain when I activate or deactivate openwith-mode. I either get a 'wrong-type-argument arrayp nil' message inside Emacs or Acrobat Reader is started but gives an error message 'can't open/find that file'.
I tried to debug it and always ended up in files.el.
It seems that the name of the pdf-file to be opened is constructed by concatenating the absolute filename and the file extension .pdf. However, the filename-string given to AcroRd32 appears to look like this:
AcroRd32 "c:\\absolute\file\name".pdf
This doesn't work on the command line either. I have to change it (manually) to
AcroRd32 "c:\\absolute\file\name.pdf"
or to
AcroRd32 c:\\absolute\file\name.pdf
to make it work.
I don't know if this is considered a bug, or if it is a problem only for me. I tried to change the elisp code to something like
(format "%s" (concat absolute-filename file-extension))
to get rid of those double-quotes, but to no avail. And anyway, I don't feel comfortable to mess around in a basic library like files.el, and its really hard to edebug that library since its invoked permanently.
Maybe somebody encountered the same problem and found a solution?
[I use GNU Emacs 24.0.91.1 (i386-mingw-nt6.1.7601) of 2011-11-22 on MARVIN.]
PS 1 Test Case 1
I get the following error message when I do M-x toggle-debug-on-error and then try to open a pdf file in dired:
Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
file-truename(nil)
find-file-noselect-1(#<buffer test.pdf<4>> "~/.emacs.d/org/projects/sandbox/test.pdf" nil nil "~/.emacs.d/org/projects/sandbox/test.pdf" ((2816 7 . 27468) (16087 . 35227)))
find-file-noselect("c:/Users/tj2/.emacs.d/org/projects/sandbox/test.pdf" nil nil nil)
find-file("c:/Users/tj2/.emacs.d/org/projects/sandbox/test.pdf")
dired-find-file()
call-interactively(dired-find-file nil nil)
and the following message:
Openwith mode enabled
find-file-noselect-1: Wrong type argument: arrayp, nil
Does it matter that my .emacs.d is really a windows symlink (mklink) to a Dropboxfolder?
PS 2 Test Case 2
here is the message I get in the maven-compile buffer, when doing C-c C-s (LilyPond-command-view) in a ,ly buffer:
-*- mode: compilation; default-directory: "~/.emacs.d/org/projects/sandbox/" -*-
Compilation started at Tue Dec 20 09:16:28
AcroRd32 "c:/Users/tj2/.emacs.d/org/projects/sandbox/2test".pdf
Compilation exited abnormally with code 1 at Tue Dec 20 09:16:35
In the message buffer I find
Compilation exited abnormally with code 1
Error during redisplay: (invalid-regexp "Unmatched ( or \\(")
This error doesn't trigger the debugger, although I did M-x toggle-debug-on-error.
Sounds like a bug, to me. Consider reporting it: M-x report-emacs-bug.
Dunno why Michael H. sent you to a Sunrise Commander page with a tip about OpenWith. Perhaps I'm missing something in your question where you indicate that you use one of those packages?
I would suggest reporting an Emacs bug. And if you want to see more about opening Windows apps associated with file types etc. then I'd suggest consulting this page.
This seems to be a problem with openwith.el, so I don't think you'll get much help with an Emacs bug report since openwith.el is not part of Emacs.
I've found a similar error (I'm on Linux) and decided that it would be better to use a "cleaner" alternative that doesn't tweak find-file-noselect (see that page on emacs.sxe for why). The OpenWith wiki page pointed me to a less-popular little bit of glue code,
https://www.emacswiki.org/emacs/download/run-assoc.el (code)
https://www.emacswiki.org/emacs/RunAssoc (docs)
By default this package's run-associated-program is not well-integrated with the usual Emacs workflow, but here is how you can integrate it with helm-find-files (also documented at the above link).
(require 'run-assoc)
(setq associated-program-alist
'(("evince" "\\.pdf$")
("play" "\\.mp3$")))
(defun helm-find-files-maybe-run-assoc (orig-fun &rest args)
(let ((sel (helm-get-selection)))
(if (string-match (mapconcat
(lambda (x) (second x))
associated-program-alist "\\|")
(helm-get-selection))
(run-associated-program sel)
(apply orig-fun args))))
(advice-add 'helm-execute-selection-action
:around #'helm-find-files-maybe-run-assoc)

Resources