Using inotify to keep track of all files in a system - database

Question:
Can inotify be used to reliably record files in a [linux] system?
Details:
I am attempting to use inotifywait to track users movements (currently using bash, but it has been suggested that I migrate to a scripting language). Ultimately I want to add new files to a database upon creation (create, moved_from), update existing rows in a database upon file modification (modify, attrib, move_to), and finally remove a row upon file deletion (delete). I am, however, running into many problems as even an action as seemingly simple as save, generates many inotifywait messages. Observe the following commands and their output (note, the use of /home/user/ is purely for example purposes):
Examples:
Example 1: Listen for file creation:
$ inotifywait -mr /home/user/ -e create --format %w:%f:%e:%T --timefmt %T
Touch:
$touch test.txt
/home/user/:test.txt:CREATE:21:35:30
Open a new file with vim then issue :w command:
$vim test2.txt
/home/user/:test2.txt:CREATE:21:35:30
Open an existing file with vim then issue :w command:
$vim test2.txt
/home/user/:4913:CREATE:21:35:30
/home/user/:test2.txt:CREATE:21:35:30
Open a new file with gedit then click save:
$gedit test3.txt
/home/user/:test3.txt~:CREATE:21:35:30
Open an existing file with gedit then click save:
$gedit test3.txt
/home/user/:.goutputstream-HN3ZDW:CREATE:21:35:30
/home/user/:test3.txt~:CREATE:21:35:30
Note that not only are two new files displayed as having ben created (4913 and .goutputstream-HN3ZDW), but also that the only file being created is test3.txt~ and not test3.txt, even though the file test3.txt is created when checked with the ls command. For completeness, here is the above example, but with a few more options.
Example 1: Listen for file creation, modification, deltion, and movement:
$ inotifywait -mr /home/user/ -e create -e modify -e delete -e moved_to -e moved_from --format %w:%f:%e:%T --timefmt %T
Touch:
$touch test.txt
/home/user/:test.txt:CREATE:21:35:30
Open a new file with vim then issue :w command:
$vim test2.txt
/home/user/:test2.txt:CREATE:22:12:32
Open an existing file with vim then issue :w command:
$vim test2.txt
/home/user/:4913:CREATE:22:04:35
/home/user/:4913:DELETE:22:04:35
/home/user/:test2.txt:MOVED_FROM:22:04:35
/home/user/:test2.txt~:MOVED_TO:22:04:35
/home/user/:test2.txt:CREATE:22:04:35
/home/user/:test2.txt~:DELETE:22:04:35
Open a new file with gedit then click save:
$gedit test3.txt
/home/user/:test3.txt~:CREATE:21:35:30
Open an existing file with gedit then click save:
$gedit test3.txt
/home/user/:.goutputstream-0WQ2DW:CREATE:22:06:34
/home/user/:test3.txt~:CREATE:22:06:34
/home/user/:.goutputstream-0WQ2DW:MOVED_FROM:22:06:34
/home/user/:test3.txt:MOVED_TO:22:06:34
Basically my question is "is it possible to use inotify to update a file in a database"? For example, if a user edits a file and saves it, I want it to be reflected in the database as an update to that file, and not a brand new file replacing a completely different file. Any help would be greatly appreciated, even if it's a suggestion pointing me in a different direction.

inotify tells you what happens like it happens.
Gedit, like most editors, saves by first writing a temporary file then moving that file into place. This avoids overwriting the file with a half-written version in case the editor or the whole system crashes while the file is being written. Vim takes a different approach (this can be configured, I won't go into details here — see e.g. why inode value changes when we edit in “vi” editor?): it first creates a temporary backup file, then writes the new file.
If you want these to be recorded as a single editing event, you'll have to perform some pattern recognition on the even log. A create-write-move sequence that replaces an existing file and a create-move-create delete sequence like vim's would be the archetypal patterns. Note that the pattern might be interleaved with other events.
I have a suspicion that there's a better way to do what you want to do, but I don't understand what you're trying to do. If you're trying to log user actions, you have already found a way, but there are simpler ways: loggedfs or the audit subsystem. If you want to keep a backup of all file versions, either hook up the editor to a version control system (this lets users control what gets backed up) or use a versioning filesystem such as copyfs. You can even store the files in the database directly, by using a filesystem like mysqlfs or postgresqlfs (admittedly neither project looks maintained).

Related

Can't open file for writing when vim recover file with swp file

I want to recover the /home/lufei/market/resources/views/user.blade.php, here is the place of swp file.
> lufei#localhost:~/market$ ls resources/views/.user.blade.php.swp
> resources/views/.user.blade.php.swp
I use vim -r resources/views/.user.blade.php.swp to recover the file, I get this,
E306: Cannot open resources/views/.user.blade.php.swp Press ENTER or
type command to continue
When I use sudo vim -r resources/views/.user.blade.php.swp I can open the file, but when I use :wq in the vim I get
"/root/market/resources/views/user.blade.php"
"/root/market/resources/views/user.blade.php" E212: Can't open file
for writing Press ENTER or type command to continue
From the info above I think vim want to save the file to the root directory. From the vi - getting an error E212: Can't open file for writing, I guess if I create path /root/market/resources/views/ may work. But I want to directly save the file to the right place /home/lufei/market/resources/views/user.blade.php. So I use su && cd /home/lufei/market then vim -r resources/views/.user.blade.php.swp. But when I :wq in vim , I also get
"~/market/resources/views/user.blade.php"
"~/market/resources/views/user.blade.php" E212: Can't open file for writing
Press ENTER or type command to continue
From above trying, no matter I use normal user or the root user, when I recover the file, vim will try to save the file to the wrong place which path is not exist.
Can I directly recover the file to the right place, without create a temp path and then copy the recoveried file back to the right path.
Make sure you have enough access rights to open the file as lufei, and then start your recovery:
$ chmod u=rw resources/views/.user.blade.php.swp
$ vim -r resources/views/.user.blade.php.swp
This may be caused by the edit session crush like the items 2 in command sudo vim market/resources/views/user.blade.php. It make the swp file make the wong original file path. Note I really not have the /root/market/resources/views/user.blade.php file even the /root/market/ path. This is the first I create the file user.blade.php and without save it, and then shutdown the computer.
E325: ATTENTION
Found a swap file by the name "resources/views/.user.blade.php.swp"
owned by: root dated: Thu Nov 9 17:33:23 2017
file name: ~root/market/resources/views/user.blade.php
modified: YES
user name: root host name: localhost
process ID: 128985
While opening file "resources/views/user.blade.php"
(1) Another program may be editing the same file. If this is the case,
be careful not to end up with two different instances of the same
file when making changes. Quit, or continue with caution.
(2) An edit session for this file crashed.
If this is the case, use ":recover" or "vim -r resources/views/user.blade.php"
to recover the changes (see ":help recovery").
If you did this already, delete the swap file "resources/views/.user.blade.php.swp"
to avoid this message.
Swap file "resources/views/.user.blade.php.swp" already exists!
[O]pen Read-Only, (E)dit anyway, (R)ecover, (D)elete it, (Q)uit, (A)bort:
Solution:
sudo vim market/resources/views/user.blade.php, if vim show above info, just press R and then save the file. If it not show above info, use :rec(over), then save the file.
use vim open any file env create a new file, with sudo vim aaa.txt, then use :rec(over) user.blade.php then save it. This will save the file to user.blade.php not the aaa.txt. Make sure the aaa.txt and .user.blade.php in same directory.

Why when I create a Makefile with Notepad its saved .txt

When I create Makefile with Notepad it's saved with .txt How can I get rid of it ? also when I try to run the program on cygwin it says *** No targets specified and no makefile found. Stop.
P.S I am new to Makefile and Linux environments.
Windows will 'helpfully' add the .txt to any file that you create with notepad. You have several options:
(1) just rename the file from makefile.txt to makefile and things should work
(2) use the -f command line argument with make, i.e make -f makefile.txt to get make to use the correct file.
(3) seeing that you have cygwin install use emacs, or vi, or gedit as your editor.
(4) consider using notepad++ (or something similar) if you must use windows.
(5) or (and this would be my default suggestion), ditch Windows and just install Linux.
At the bottom of the save dialog box, select
Save as type: All Files (*.*)
from the drop-down menu instead of the default
Save as type: Text Document (*.txt)
Then, you could use any extension that you desire.
Make sure you select the correct file type (any) and not text file (*.txt) when saving the file (the latter adds .txt automatically if you don't do it). Btw you can also rename the file afterwards, but you have to show the file extensions in explorer to be able to change it.

How to view difference between local hijacked file and the server copy of the file in clearcase

Presently I follow this process:
Right click the root of the project and click on Find Modified Files.
Then in the ClearCase Snapshot View Update window, the modified (hijacked) files are shown.
By right click on the file name and click on Compare with Old Version, difference between local hijacked file and the server copy are displayed.
However, the above approach takes very long time (as step 1 itself takes 10 minutes). If I already know the hijacked file path, then is there a faster way to do the same?
This 2007 thread summarizes the options you have; the command line with cleartool 9.0.1.3 also offers a diff, even though the thread claims that it doesn't. See bottom of the post.
convert the hijack to checked out (then you can diff)
OR
rename the hijacked file to something else (like filename.hijacked), reload filename (cleartool update filname) update the file, then run cleartool diff filename filename.hijacked.
OR
run the diff against the dynamic view version (from command-prompt, cd to the hijacked file location, then do the cleartool diff filename M:\view\VOB\path\to\file)
I used to do the third option, as it does not require to touch the hijacked file.
With cleartool, compare an extended path name specifying a version, e.g. /main/LATEST, to an simple path. (A command line can be opened conveniently from the context menu of a folder in ClearCase Explorer.)
cleartool diff file.c##/main/LATEST file.c

How to touch (create a blank file) in Stata?

I'm aware of the package touch, which does exactly this. However after installing and using the package, Stata 12 SE (Mac 10.9.5) says:
The touch command has been deprecated.
Is there an alternative? I need this to initialize a blank result file, so that I can append my regression result in a loop without caring whether I'm in the first iteration or not.
NEW ANSWER
Use one of the following in your do file to create the blank xls file "newfile.xls":
shell echo /dev/null > newfile.xls
which will overwrite existing files of the same name. To create a new blank file of that name:
shell touch newfile.xls
would also work. However if a file of the same name exists, touch would only change the modification and access times.
OLD ANSWER
The following will save an empty Stata data file "newdata.dta". Substitute your own name.
save newdata, emptyok
With Mac, you can just use the native touch: !touch filename.xls,
or
!> empty_results.txt, if you want to over-write an existing file or create it if it does not exist.

How can I load a Maya .MA file from MEL when it has an unresolved reference?

I am trying to use a MEL script to load ANIMATION.MA file that references CHARACTER_RIG.MA. The CHARACTER_RIG.MA and ANIMATION.MA files are produced by someone else and supplied to me. The ANIMATION.MA is looking for N:/Project/Maya//char/character/CHARACTER_RIG.MA
If I open ANIMATION.MA from Maya, or use the equivalent MEL command I always get prompted with:
"Reference File Not Found"
Reference File Not Found: N:/Project/Maya//char/character/CHARACTER_RIG.MA.
[Abort File Read] [Skip] [Browse...] [Retry]
If I tap browse, and select the CHARACTER_RIG.MA then it opens perfectly. I can see it created a reference in the Reference Editor that has the Unresolved Path (N:/...) , the Resolved Path (/my/path) and the namespace and the namespaceRN.
My question is, how do I do the equivalent of the "Browse..." from MEL? I tried pre-creating a reference, but it doesn't let me set the unresolved path, so when I load the ANIMATION.MA it keeps prompting in MAYA.
file -f -options "v=0" -typ "mayaAscii" -o "/Source/project/assets/anims/ANIMATION.MA"
If you know the directory where the file is, then you can use the dirmap command. The command dirmap allows you to remap directory structures if your disk configuration changes. So in this case it would look like:
dirmap -en true;
dirmap -m "N:/Project/Maya//char/character" "/my/path";
Possibly more manageable if you have lots of mappings to do especially when moving form a windows machine to a *nix one. However it is much more useful to define your project structure because then things just work when you move, tough this may not be the best of choice for shared assets.
I ended up finding several solutions:
Rename the RIG.MA file to match the filename in ANIM.MA (they were different in my case) and put it in one of the search or project folders that MAYA uses and it will automatically find it.
or
Programatically through code (or manually) edit the ANIM.MA file to remap the file/folder of the RIG.MA to where you want to load it from. Note: You
also need to remap any other files, such as textures. I did this with
perl -pi -e 's/\Qold-path\E/\Qnew-path\E/g' ANIMATION.MA
HTH someone else.
Quick and easy, File, Project, Set and select the folder where meshes or whatever it is.

Resources