I want to create a temporary copy of a DLL file in the temp folder, and have it deleted on application exit. Due to unrelated reasons too long to explain, I cannot simply remove the file at the end of the function/script that creates it.
I tried using CreateFile() with FILE_FLAG_DELETE_ON_CLOSE, but when I try to copy the original file to this file, I get ERROR_SHARING_VIOLATION.
Here's my code:
BOOL CopySuccess = 0;
if ((_waccess(TempFilePath, 0)) == -1) {
printf("Temp copy \'%ls\' not found, creating copy now\n", TempFilePath);
CreateFileW(TempFilePath, (GENERIC_READ | GENERIC_WRITE), (FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE), NULL, CREATE_NEW, FILE_FLAG_DELETE_ON_CLOSE, NULL);
CopySuccess = CopyFileExW(OriFilePath, TempFilePath, NULL, NULL, FALSE, NULL);
if (!CopySuccess)
{
ErrorExit(TEXT("Copy dll to temp file failed"));
}
}
AFAIK, I used the correct flags in the CreateFile() call to enable shared access to the file.
What am I doing wrong/what is an alternative approach?
I need the logic to maintain this structure. Without going into details, for reasons that escape my control, this script will be run around 10 times per second, so I need a way to copy the file just once, then have it deleted once the application exits, due to error, ctrl-c event, or normal exit.
As a reply to the comments:
I tried writing the contents of the original file to the tempfile created with CreateFile(). This didn't work because the handle returned by CreateFile() is not valid to use as a library (library handles are of type HMODULE). Closing the handle then re-opening it not a possibility, as closing all handles to the file causes it to be deleted as per the FILE_FLAG_DELETE_ON_CLOSE flag.
I figured the issue would be on CopyFile()'s side. I didn't think of writing my own function, so instead I tackled the problem the following way:
There's one specific var that increases by a fixed amount in every iteration of the main script, so I wrote an if statement that would check:
If the copy of the dll already existed
If the current value of the variable was below the 2nd iteration value
If both conditions were met, the already existing copies of the dlls would be deleted. Similarly, the dlls are only attempted to be copied in the 1st iteration of the main script.
This is not an actual answer to the question, but a way to circumvent it. I'll give a try to writing my own version of CopyFile(). If I succeed and it behaves as I intend it to behave, I'll post the code and an explanation as an answer here. Thanks all!
Related
I have a Fortran code that updates in real time the content of a text file by adding some new real time measuraments at the very bottom of it. This text file is used (reading only) both by a fluid dynamics code (that continuously runs in real time) and by another executable built from matlab code (that performs plotting). I want to add a line in the Fortran code that says: update the text file ONLY IF it is not opened by any other program. I tried using INQUIRE:
do
INQUIRE(FILE = filename,OPENED = ISopen)
if (.not.ISopen) then
ADD NEW MEASUREMENTS HERE
exit
endif
endif
enddo
and before running this fortran program I opened the file with textpad. However, the variable ISopen is false. So I guess maybe INQUIRE only works for testing if file is opened within the fortran program itself. In fact if I add at the beginning of the above snippet of code:
OPEN (33,FILE = filename)
then ISopen is true. I then created an executable from a fortran code containing only:
OPEN (33,FILE = filename)
pause
CLOSE(33)
and I run it and let it in a paused status. I then run the first code I posted above and ISopen is still false. Any idea how to test if a file is open by any other program from within Fortran? My operative system is windows 7.
thanks
At the end I solved in this way. I could not find a way to know if the file is opened, and even if there was a way it could still happen that some other program opens it right between the time I check if its open and then I modify it. Therefore, I just create a temporary copy of the file to modify, I modify this temporary copy and then I move the file back by overwriting the original one. The latter operation is performed only if the file is not locked (i.e. no other program opened the file to read the data), so I keep trying to copy it until it succeeds. I tested in many situations and it works. The code is:
USE IFPORT
IMPLICIT NONE
character*256 :: DOScall
logical :: keepTRYING
integer :: resul
DOScall = 'copy D:\myfile.txt D:\myfile_TMP.txt' !create temporary copy
resul = SYSTEM(DOScall)
open(15,file ='D:\myfile_TMP.txt',form = 'formatted')
!.... perform here some writing operations on myfile_TMP........
close(15)
do
resul = SYSTEM('move D:\myfile_TMP.txt D:\myfile.txt')
if (resul==0) then
exit
else
pause(10)
endif
enddo
Note that works perfectly fine for multiple programs performing reading and one program writing in the file (as in my case where I have 2 programs reading and only one writing). If multiple programs write the same file I guess some other parallel techniques have to be used.
I do not think it is possible to check if a file is opened by an external process. Some programming languages allow you to check if the file is locked or not, and that is still far from telling you if the file is opened or not, both program must acquire and release system lock for it to really work. To my knowledge, the standard fortran does not have that feature directly, but you can use the semaphore from C with the interoperability stuff.
Most user application (editors mostly) however, before updating a file, usually check if the content on the disk has changed since they capture a copy, and alert the user. They also do that if they lost and acquire the focus. If you restrict your goal to updating only if the content has not changed since you opened it, you can do the same or simply open-add and close any time you want to add a new entry. A good editor will notify the user on the other side that the content had been change by another process.
An alternative is to simulate a lock yourself and buffer the data in fortran. By buffering I mean, collect some new data (let say 100, 1000 or whatever number that is convenient) and send them to the file at once. For each update, you open, update and close the file.
Your lock can be made of two simple files (emptys one for example), one created by the reader (matlab) and the other created by writer (fortran program). Let name them reading.ongoing for the reader and writing.ongoing for the writer.
On the fortran side, you do the following anytime you have collected enough data to write:
check for the existence of reading.ongoing (using inquire function), proceed only if it does not exist
create writing.ongoing
check for the existence of reading.ongoing again, if it exists, delete writing.ongoing and go back to step 1. If it does not exist, proceed forward.
open, write the data and close the data file
delete writing.ongoing
On matlab side, do similar thing, inverting the role of reading.ongoing and writing.ongoing.
In an exceptional race condition you could be blocked because they are all trying at the same time. In that case, you could modify the step 1. of matlab to force it to wait for few millisecond before proceeding. This will get you on the road as long as none of the program get killed between step 1 and 5.
You can also use semaphore with the interoperability stuff in fortran. See the following post for example. You can also similar think on the side of matlab, I do not have any example. That will lower your headhache and let the system manage the lock for you.
In my sceniro I am checking the file in a particluar folder and checking whether it exists or not. In case it exists i am doing some operations on it.
But my issue is that these files are getting generated by other process [which usally takes 20-30 min to generate the file], so for example at any instance when I checked the file in the folder and I find it. Now, I need to make sure that it is not in process right now so that I can continue my operation on that it. Below is the code snippet which I am using:
EDIT
My question is this that how can I identify that file is not in use (or is getting generated) by other process.
$Filesitems = Get-ChildItem -Path "D:\SomeFolder\Sub_Folder"
foreach ($objItem in $Filesitems)
{
if ($objItem.Name.Contains("Process"))
{
Write-Host $objItem.Name
}
}
I would probably try to use a 'lock'-file to avoid this behavior.
Ensure whenever the process works on the file, the process also generates an (empty) file.lock file and removes it when the process is finished working on the file.
In this scenario you can check whether a lockfile exists, and so you know that another process is still working on that particular file
For a particular piece of homework, I'm implementing a basic data storage system using sequential files under standard C, which cannot load more than 1 record at a time. So, the basic part is creating a new file where the results of whatever we do with the original records are stored. The previous file's renamed, and a new one under the working name is created. The code's compiled with MinGW 5.1.6 on Windows 7.
Problem is, this particular version of the code (I've got nearly-identical versions of this floating around my functions) doesn't always remove the old file, so the rename fails and hence the stored data gets wiped by the fopen().
FILE *archivo, *antiguo;
remove("IndiceNecesidades.old"); // This randomly fails to work in time.
rename("IndiceNecesidades.dat", "IndiceNecesidades.old"); // So rename() fails.
antiguo = fopen("IndiceNecesidades.old", "rb");
// But apparently it still gets deleted, since this turns out null (and I never find the .old in my working folder after the program's done).
archivo = fopen("IndiceNecesidades.dat", "wb"); // And here the data gets wiped.
Basically, anytime the .old previously exists, there's a chance it's not removed in time for the rename() to take effect successfully. No possible name conflicts both internally and externally.
The weird thing's that it's only with this particular file. Identical snippets except with the name changed to Necesidades.dat (which happen in 3 different functions) work perfectly fine.
// I'm yet to see this snippet fail.
FILE *antiguo, *archivo;
remove("Necesidades.old");
rename("Necesidades.dat", "Necesidades.old");
antiguo = fopen("Necesidades.old", "rb");
archivo = fopen("Necesidades.dat", "wb");
Any ideas on why would this happen, and/or how can I ensure the remove() command has taken effect by the time rename() is executed? (I thought of just using a while loop to force call remove() again so long as fopen() returns a non-null pointer, but that sounds like begging for a crash due to overflowing the OS with delete requests or something.)
So suddenly, after reading Scott's mention of permissions, I thought about "Permission Denied" and applied some Google. Turned out it's a pretty common, if obscure, error.
caf was right, it was in another piece of code. Namely, I had forgotten to fclose that same file in the function meant to show the contents. Since I wasn't tracking that particular detail, it appeared to be random.
Disclaimer: Weekly math assigments make for very little sleeptime. ¬¬
That sounds quite strange, and even more so when you say that the same code works OK with a different filename - I would strongly suspect a bug elsewhere in your code. However, you should be able to work around it by renaming the file you want to remove:
rename("IndiceNecesidades.old", "IndiceNecesidades.older");
remove("IndiceNecesidades.older");
rename("IndiceNecesidades.dat", "IndiceNecesidades.old");
It would probably be a good idea to check the remove() function for errors. man remove says that the function returns 0 on success and -1 on failure, setting errno to record the error. Try replacing the call with
if (remove("IndiceNecesidades.old") != 0){
perror("remove(\"IndiceNecesidades.old\") failed");
}
which should give an error message saying what failed.
Further, it doesn't appear that the remove is neccessary
man rename()
The rename() system call causes the
link named old to be renamed as new.
If new exists, it is first removed.
Both old and new must be of the same
type (that is, both must be either
directories or non-directories) and
must reside on the same file system.
The rename() system call guarantees
that an instance of new will always
exist, even if the system should crash
in the middle of the operation.
If the final component of old is a
symbolic link, the symbolic link is
renamed, not the file or directory to
which it points.
EPERM will be returned if:
[EPERM] The directory
containing old is marked sticky, and
neither the containing directory nor
old are owned by the effective user
ID.
[EPERM] The new file
exists, the directory containing new
is marked sticky, and neither the
containing directory nor new are owned
by the effec-
tive user ID.
so the next step would be to check you have permissions on the containing directory
I want to append data to a file in /tmp.
If the file doesn't exist I want to create it
I don't care if someone else owns the file. The data is not secret.
I do not want someone to be able to race-condition this into writing somewhere else, or to another file.
What is the best way to do this?
Here's my thought:
fd = open("/tmp/some-benchmark-data.txt", O_APPEND | O_CREAT | O_NOFOLLOW | O_WRONLY, 0644);
fstat(fd, &st);
if (st.st_nlink != 1) {
HARD LINK ATTACK!
}
Problem with this: Someone can link the file to some short-lived file of mine, so that /tmp/some-benchmark-data.txt is the same as /tmp/tmpfileXXXXXX which another script of mine is using (and opened properly using O_EXCL and all that). My benchmark data is then appended to this /tmp/tmpfileXXXXXX file, while it's still being used.
If my other script happened to open its tempfile, then delete it, then use it; then the contents of that file would be corrupted by my benchmark data. This other script would then have to delete its file between the open() and the fstat() of the above code.
So in other words:
This script Dr.Evil My other script or program
open(fn2, O_EXCL | O_CREAT | O_RDWR)
link(fn1,fn2)
open(fn1, ...)
unlink(fn2)
fstat(..)=>link is 1
write(...)
close(...)
write(...)
seek(0, ...)
read(...) => (maybe) WRONG DATA!
And therefore the above solution does not work. There are quite possibly other attacks.
What's the right way? Besides not using a world-writable directory.
Edit:
In order to protect against the result that the evil user creates the file with his/her ownership and permissions, or just wrong permissions (by hard linking your file and then removing the original, or hardlinking a short-lived file of yours) I can check the ownership and permission bits after the nlink check.
There would be no security issue, but would also prevent surprises. Worst case is that I get some of my own data (from another file) at the beginning of the file copied from some other file of mine.
Edit 2:
I think it's almost impossible to protect against someone hard-linking the name to a file that's opened, deleted and then used. Examples of this is EXE packers, which sometimes even execute the deleted file via /proc/pid/fd-num. Racing with this would cause the execution of the packed program to fail. lsof could probably find if someone else has the inode opened, but it seems to be more trouble than it's worth.
Whatever you do, you'll generally get a race condition where someone else creates a link, then removes it by the time your fstat() system call executes.
You've not said exactly what you're trying to prevent. There are certainly kernel patches which prevent making (hard or symbolic) links to files you don't own in world-writable directories (or sticky directories).
Putting it in a non world-writable directory seems to be the right thing to do.
SELinux, which seems to be the standard enhanced security linux, may be able to configure policy to forbid users to do bad things which break your app.
In general, if you're running as root, don't create files in /tmp. Another possibility is to use setfsuid() to set your filesystem uid to someone else, then if the file isn't writable by that user, the operation will simply fail.
Short of what you just illustrated, the only other thing I've tried ended up almost equally racey and more expensive, establishing inotify watches on /tmp prior to creating the file, which allows for catching the event of a hardlink in some instances.
However, its still really racey and inefficient, as you would also need to complete a breadth first search of /tmp, at least up to the level that you want to create the file.
There (to my knowledge) is no "sure" way to avoid this kind of race, other than not using word writeable directories. What are the consequences of someone intercepting your i/o via hard link .. would they get anything useful or just make your application exhibit undefined behavior?
In what situations does Windows allow you to overwrite an open file? Is that ever allowed? This includes renaming a different file to the same name as an open file.
If you look at the documentation for CreateFile(), there is this dwShareMode parameter. This can determine what another process can do with that file while it's open.
Specifying FILE_SHARE_READ lets another process open the file for reading. There's FILE_SHARE_WRITE, which means that another process can write to it. There's also FILE_SHARE_DELETE, which allows delete and (IIRC) also rename.
If someone opened the file without FILE_SHARE_WRITE and you open the file for write access, you'll get ERROR_SHARING_VIOLATION. Otherwise you should be able to write to it.
Unfortunately if one process comes along and doesn't set sharing flags to allow something you need, you're pretty much out of luck. Although you might be able to use MoveFileEx() with the option MOVEFILE_DELAY_UNTIL_REBOOT. But I'm not sure if that works; I don't know much about that call except that it exists. :-)