I have written a WinSCP script which downloads files from a remote server to my local directory. My local directory changes everyday.
option batch abort
option confirm off
open sftp://sftpsite -hostkey="ssh-rsa ab:cd:....."
synchronize local ????? /Home/user/
exit
I am not sure what my local directory should be.
If you want to use today's date as a name of the target local directory, use %TIMESTAMP#yyyymmdd# syntax:
synchronize local C:\Data\%TIMESTAMP#yyyymmdd# /Home/user/
Note that if you are downloading the files to a new folder every day, it's not really synchronization. It is a normal full download. So get command might be more meaningful:
get /Home/user/* C:\Data\%TIMESTAMP#yyyymmdd#\
So.. I think i figured it out. My local directory should be something like this:
C:\Data\%datestamp%
where datestamp is something which changes everyday and is obtained by running a batch process.
option batch abort
option confirm off
open sftp://sftpsite -hostkey="ssh-rsa ab:cd:....."
synchronize local C:\Data\%datestamp% /Home/user/
exit
Related
I don't know if this is the right place to ask this question, but I am going to ask it anyway.
I have a frequent problem when I try to rename one of my folders; it says access is denied. I have full administrative rights on my computer. The problem occurs only when I try to do it via command prompt or batch files. I can manually rename the folder with no problems whatsoever. But I need to be able to rename it using the ren command. It is an irritating problem, and I have heard other people having the same problems. They usually fix it by re-taking ownership of the folder and restarting the computer. This works for me, but I hate having to do this every time. Is there a permanent solution that I can do to stop this problem? Also, when this problem occurs, it happens to all folders; I can't rename any of them using the ren command.
The answer is quite simple:
Windows does not permit deletion or renaming a directory/folder which is
the current directory of any running process (application/service) like the command process, or
any subdirectory of the directory to rename is the current directory of any running process, or
any file is opened in the directory to rename or any of its subdirectories by any process with OF_SHARE_DENY_READ, OF_SHARE_DENY_WRITE, OF_SHARE_EXCLUSIVE used on opening the file.
In other words as long as the directory itself or any file or subdirectory in this directory or its subdirectories is in use by any application, Windows denies renaming or deleting the directory.
On Unix/Linux it is possible to delete or rename a directory while it is in use by 1 or more running processes. The running *nix process has to handle the special use case that the directory or file just accessed successfully a millisecond before does suddenly not exist anymore.
My approach to solving this problem is to run the utility Handle, by Mark Russinovich of Windows Sysinternals. You can find it here.
This utility will show all the open handles on the system and which process is using them. One of them may have a path to the folder you are trying to rename.
I typically redirect the output of handle.exe to a file.
handle >handles.txt
Then I look at handles.txt in a text editor and search for the name of the folder I was trying to rename. It will appear in a section of handles belonging to the process that is keeping the handle to that folder open, and hence preventing it from being renamed. It shows the pid of the offending process, which you can decide how to deal with, in case you need to terminate it.
e.g. Trying to rename myfolder I forgot I had started a service running using pm2 (a process monitor) The output of Handle.exe included the following:
node.exe pid: 12752 DISCO\Stu
40: File (---) C:\Dev\myfolder\service
I fixed this by disabling Quick Access:
See Screenshot:
[
Well before you can run this command you have to have the privileges to do so as overwriting or modifying files from the command line requires admin rights. Once you got that done, enter the following:
ren (path name) (modified name, no quotation marks)
1 copy path of the folder. To do this, hold shift and right-click, you'll see the "copy as path option"
2 open cmd as admin. To do this, press window + X, then A.
3 in the cmd, type "cd C: then paste the path" so it will be, for example, like:
cd C:\Users\abc\Desktop\music
before you press enter, quote the last name; for example:
cd C:\Users\abc\Desktop\"music"
4 Next, type: ren *.present extension *.new extension; For example, from mp3 to exe:
ren *.mp3 *.exe
If no extension was existing, then, for example:
ren *. *.exe
This should work and eliminate the access denied nuisance.
I solved this problem by giving the user who run the batch full right on the directory. It seems the right from a group membership is not sufficient: the user was member of the local group Administrators and I received a "Access denied". After having added the user account in the directory security with full rights, it does the job.
I don't know if this is the right place to ask this question, but I am going to ask it anyway.
I have a frequent problem when I try to rename one of my folders; it says access is denied. I have full administrative rights on my computer. The problem occurs only when I try to do it via command prompt or batch files. I can manually rename the folder with no problems whatsoever. But I need to be able to rename it using the ren command. It is an irritating problem, and I have heard other people having the same problems. They usually fix it by re-taking ownership of the folder and restarting the computer. This works for me, but I hate having to do this every time. Is there a permanent solution that I can do to stop this problem? Also, when this problem occurs, it happens to all folders; I can't rename any of them using the ren command.
The answer is quite simple:
Windows does not permit deletion or renaming a directory/folder which is
the current directory of any running process (application/service) like the command process, or
any subdirectory of the directory to rename is the current directory of any running process, or
any file is opened in the directory to rename or any of its subdirectories by any process with OF_SHARE_DENY_READ, OF_SHARE_DENY_WRITE, OF_SHARE_EXCLUSIVE used on opening the file.
In other words as long as the directory itself or any file or subdirectory in this directory or its subdirectories is in use by any application, Windows denies renaming or deleting the directory.
On Unix/Linux it is possible to delete or rename a directory while it is in use by 1 or more running processes. The running *nix process has to handle the special use case that the directory or file just accessed successfully a millisecond before does suddenly not exist anymore.
My approach to solving this problem is to run the utility Handle, by Mark Russinovich of Windows Sysinternals. You can find it here.
This utility will show all the open handles on the system and which process is using them. One of them may have a path to the folder you are trying to rename.
I typically redirect the output of handle.exe to a file.
handle >handles.txt
Then I look at handles.txt in a text editor and search for the name of the folder I was trying to rename. It will appear in a section of handles belonging to the process that is keeping the handle to that folder open, and hence preventing it from being renamed. It shows the pid of the offending process, which you can decide how to deal with, in case you need to terminate it.
e.g. Trying to rename myfolder I forgot I had started a service running using pm2 (a process monitor) The output of Handle.exe included the following:
node.exe pid: 12752 DISCO\Stu
40: File (---) C:\Dev\myfolder\service
I fixed this by disabling Quick Access:
See Screenshot:
[
Well before you can run this command you have to have the privileges to do so as overwriting or modifying files from the command line requires admin rights. Once you got that done, enter the following:
ren (path name) (modified name, no quotation marks)
1 copy path of the folder. To do this, hold shift and right-click, you'll see the "copy as path option"
2 open cmd as admin. To do this, press window + X, then A.
3 in the cmd, type "cd C: then paste the path" so it will be, for example, like:
cd C:\Users\abc\Desktop\music
before you press enter, quote the last name; for example:
cd C:\Users\abc\Desktop\"music"
4 Next, type: ren *.present extension *.new extension; For example, from mp3 to exe:
ren *.mp3 *.exe
If no extension was existing, then, for example:
ren *. *.exe
This should work and eliminate the access denied nuisance.
I solved this problem by giving the user who run the batch full right on the directory. It seems the right from a group membership is not sufficient: the user was member of the local group Administrators and I received a "Access denied". After having added the user account in the directory security with full rights, it does the job.
i'm trying to move file from 1 server to an other with schedule batch file, for that i use google drive as a third part where i store my files. i have made some script
#echo "executed %date:~-10,2%%date:~-7,2%%date:~-4,4%" >> Logs.txt
copy /y "C:\backup\Portal2%date:~-10,2%%date:~-7,2%%date:~-4,4%.bak" "c:\users\administrator\google drive\"
this script move a file from a directory to an other, it works fine if i click it,but if i try to schedule it doesn't work.
I tried to change path to a random path in my computer and scheduled it, it worked perfectly.
its like my computer doesn't recognize google drive at all.
When copying over a network you need to use credentials in the scheduled task that have access to the network resource.
This is probably a bit late for you but might be helpful to others who follow in your footsteps. I had this problem and it came down to security. When you run something in batch it doesn't have the correct permissions. I can't remember the exact solution but you need to set the batch task to run with your username.
I'm using psftp.exe to download files from the server. Is there an easy way to delete these files once I have downloaded them but leave the new ones that might have appeared on the server when I was downloading to be downloaded next time?
Here's my command line:
psftp.exe domain.com -i keys\private.ppk
get *.xml
Edit: I want to download the files from a Linux box to a Windows PC.
There's no easy way to do this with psftp. You would have to parse its output to find files that were successfully downloaded.
Though you can do this easily with WinSCP. Just use get -delete *.xml command.
Full WinSCP script would be:
open sftp://domain.com/ -privatekey=keys\private.ppk -hostkey=...
get -delete *.xml
exit
See an introduction to WinSCP scripting.
See also a guide for converting PSFTP script to WinSCP.
You can also have WinSCP GUI generate script like this for you.
(I'm the author of WinSCP)
Martin's answer is good. The below is more industrial.
Moving them to a staging area before the download may be prudent.
Generally you would move/rename the files on the server as a starting point. They are going to be deleted anyway so nothing should miss them? nor would you want to fall back over a recent file.
(so restart after this point in the event of a subsequent failure)
Then perform the download.
Then perform the delete.
I would approach the issue differently. Instead of deleting the file from the server, add the downloaded filename to a local table of "Already downloaded files". Then when you scan the FTP again for new files, ignore any that are in that table.
That way the next time you run your download script you only get the new files, but the old files remain on the server.
You could have another script that runs periodically and deletes all files over a certain age.
WINSCP is alright, and Martin (the author) drops in to practically every PuTTy thread to recommend it, but it's a fully GUI-based app and not for me. If you really need everything to be done on the commandline then WINSCP is often not an option.
Long story short... we have multiple servers which we run perflog monitoring on every night. My job is to convert these logs to .csv format and send them to my e-mail.
This bit it already automated via a .sh script an ex-employee wrote.
What I want automated is to run a batch job after the perfmon logging to look at a specific folder and find the latest .blg file and run the sh script on it (the script is called upload) so that I don't have to log in to each server and do it manually.
e.g.
upload myInitials cd /cygdrive/someLocation/logs/$latestFile$.blg
myInitials and the location can be hard-coded... I just wouldn't know how to find the latest file in the folder and automate it all via a batch file.
Any pointers would be very helpful!
# Jeremy:
Sorry, I probably should have mentioned in my question that the servers are running 2003 and 2008.
I don't think it would be absolutely necessary to register a change notification on the folder - If the log runs from noon till 7 in the morning, the script will run immediately after (you can set a script to run after a perfmon log has finished in log properties) so the log will almost definitely be the latest file in the folder anyway.
Like I said, I already have a .sh file in place to convert to csv and send to my e-mail, I just need to incorporate it into a batch file so that instead of me going to each of the servers and opening up cygwin and typing upload xx /cygdrive/location/logs/xyz.blg, I can have it automated to run straight after the log has finished without me having to RDC into it.
Thanks for the input!
If you have a Shell script and you job is to call the shell script from a windows batch file then this will work.This assumes the cygwin is installed in C:
Contents of start_cyg.bat
#echo off
set PATH=%PATH%:"C:\Cygwin\bin"
rem bash --login -i
bash "/cygdrive/d/cyg.sh"
Contents of cyg.sh
#!/bin/bash
TAIL=`ls -lrt | tail -1`
echo "TAIL:$TAIL"
If you call start_cyg.bat from windows command prompt you can get the output of the cyg.sh in the console
for getting newest file in a directory, ls -1tr | tail -1 should work.
First, I don't know if it would meet your requirements, but the Windows Task Scheduler 2 in Vista+ is very robust and can trigger an event even based on log entries. However, extraction and parsing of that log entry may require some scripting, and might have concurrency issues, even if that log entry did indicate the last used process. Chances are none of this is helpful, but just throwing it out there.
Programatically, it would be simple as you can register a change notification on a folder. When a change occurs, you go find the latest file. Then launch the batch file to launch your shell script, or whatever your desired sequence may be.
I think cygwin may even support change notification events via scripting, though I'm unsure. I believe there are linux extensions for this, but I may be wrong.
If it were me, I'd just write a little C++ app to do whatever I wanted.. but for you maybe any (or more likely none) of the above helps ;o.