I am writing a script to iterate through all files in a folder using a batch script. The command I am using is
for /r F:\foo\ %%i in (bar_*.txt) do echo %%i
This works just fine. But if I substitute the %%i with %%files, I get an error which says %files was unexpected at this time. The documentation page says only single characters are allowed. What is the reason behind this? Is there any way through which I can give a more meaningful name to the variable?
Loop variables must be single character variables because of tokens=1,4,6-9 feature which would not be possible with loop variables with any name.
For example from CSV file input.csv the first data column, the fourth data column and the data columns six to nine should be written into CSV file output.csv.
#echo off
if not exist "input.csv" exit /B
if exist "output.csv" del "output.csv"
for /F "usebackq tokens=1,4,6-9 delims=," %%A in ("input.csv") do (
echo %%A,%%B,%%C,%%D,%%E,%%F>>"output.csv"
)
Note 1: This works only if there are no empty field values in each data row.
Also it would be very difficult for Windows command processor to distinguish between referencing a loop variable and an environment variable if loop variables could have any name.
You can always assign value of a loop variable to an environment variable and use on the other command lines within the loop the environment variable using delayed expansion.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
for /R F:\foo\ %%# in (bar_*.txt) do (
set "FileName=%%#"
echo Current file is: !FileName!
)
endlocal
Note 2: This works only for file names not containing ! in name or path.
Environment variables are not case-sensitive, but loop variables are case-sensitive. And avoid using letters for loop variables which could be interpreted also as modifiers for example on using %~nI, %~nxI or something similar as this could be problematic.
BTW: Why not using COPY with option /S or XCOPY or ROBOCOPY to copy recursively all files matching the wildcard pattern bar_*.txt?
Related
I want to write a batch program to copy some deeply nested folders with the same suffix in this case 100,
It only copied all the folders but only one file in the top most folder (they are alphabetically arranged in the server) *100.bat was copied.
I want to copy all files in all folders with name_of_folder100.
Thanks for your Time.
This is my attempt:
#echo off
:: variables
set hour=%time:~0,2%
if "%hour:~0,1%"==" " set hour=0%time:~1,1%
set
set drive= E:\PWD_BACKUP_%date:~10,4%_%date:~4,2%_%date:~7,2%_%hour%_%time:~3,2%
set PWD_drive_100=E:\PWD_BACKUP_\PWD_100_%date:~10,4%_%date:~4,2%_%date:~7,2%_%hour%_%time:~3,2%
set backupcmd=xcopy /s /c /d /e /h /i /r /y
%backupcmd% "\\xx.xx.xx.xx\live_projects\PWD\*100" %PWD_drive_100%
It's normal to follow the #echo off line with setlocal. This makes sure that any variables altered or created in the batch file are not maintained in the cmd session's environment when the batch ends.
Use set "var=value" for setting string values - this avoids problems caused by trailing spaces. Don't assign a terminal \, Space or " - build pathnames from the elements - counterintuitively, it is likely to make the process easier. If the syntax set var="value" is used, then the quotes become part of the value assigned.
The set drive=... statement will include the space following the = in the value assigned to drive. This is probably of no consequence in the current situation, but may be significant for other batches.
Other than the last 2 lines of code, there is a great deal of date/time-manipulation going on. We have to rely on your local knowledge here as the date/time format is user-dependent.
The overall problem is that * cannot be used for directorynames. Logically, what you appear to want to copy from your description is ...e_projects\PWD\*100\*, but that structure is not supported by xcopy.
So I'll restate the problem as "copy all directories that have a name ending 100"
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
rem The following settings for the directories and filenames are names
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
FOR /f "delims=" %%e IN (
'dir /b /s /ad "%sourcedir%\PWD\*" ^|findstr /e "100"'
) DO (
SET "dirname=%%e"
SET "dirname=!dirname:*\PWD\=!"
ECHO XCOPY "%%e\*" "%destdir%\!dirname!\"
)
GOTO :EOF
Always verify against a test directory before applying to real data.
The setlocal enabledelayedexpansion invokes delayedexpansion mode, where within a code block (parenthesised sequence of lines) !var! is the run-time value of var (as var is changed) and %var% is the value that var contained when the code block was parsed.
Documentation: Stephan's DELAYEDEXPANSION link
We then establish the source and destination directories. I've posted my test directories, you would need to edit them to suit your particular circumstances.
The dir command lists all directories (/ad) in basic form (/b) that is, name only - no size, date, etc. The /s extends the scan to subdirectories also.
The dir output is piped to findstr (the caret escapes the pipe to show that it is part of the command to be executed, not of the for) which filters the dir list for (directorynames) which /e end "100".
The for/f assigns each line of the result to %%e. The delims=" ensures the entire line is assigned to %%e(in case%%e` contains separators like Space)
%%e thus contains something like U:\your files\PWD\sub 5\subsub 5\subsubsub\subsubsubsub100, so we need to remove the part up to \PWD\ to get the subdirectory name in a format suitable for the xcopy command. Batch cannot substring a metavariable like %%e directly, so we assign %%e to a user-variable dirname (which is therefore changing within the loop, so we need to use !dirname! to acess it's run-time value) and the following set removes all characters up to and including \PWD\. (See set /? from the prompt for documentation)
Then it's just a matter of performing the xcopy. Note that I've added an echo keyword to "disarm" the xcopy so that the xcopy command is merely listed to the console instead of being executed. Once you've verified that the command is correct, remove that echo to actually execute the xcopy.
The destination of the xcopy is </kbd> which tells xcopy that the destination is a directory, and to create it if it does not exist.
Add >nul to the xcopy command to suppress the report if you want.
I have a configuration file which I need for my bash script which has a layout:
A=C:/Example1/A
B=C:/Example2/B
C=C:/Example3/C
I want to use the same configuration file for my windows batch file. I need to convert the above file into variables which I have done using:
for /f "delims=" %%x in (test.txt) do (set "%%x")
How do I go about converting this file into variables while also converting all the forward slashes into backslashes?
Thanks!
add after your for line,
for /f "delims==" %%x in (q888.txt) do call set "%%x=%%%%x:/=\%%"
or, as a replacement for your existing for,
for /f "tokens=1*delims==" %%x in (q888.txt) do set "%%x=%%y"&call set "%%x=%%%%x:/=\%%"
(I used a file called q888.txt for testing)
The first smply executes a substitution, using a parsing trick. The second combines the set and substitution into one cascaded command by tokenising on = into %%x and %%y
This could be done with the following batch code:
#echo off
if not exist "test.txt" goto :EOF
setlocal EnableDelayedExpansion
for /F "usebackq tokens=1* delims==" %%I in ("test.txt") do (
if not "%%~J" == "" (
set "Value=%%~J"
set "Value=!Value:/=\!"
set "_%~n0_%%~I=!Value!"
)
)
echo The variables set from file are:
echo/
set "_%~n0_"
echo/
pause
endlocal
The batch file first checks if the file to process exists in current directory at all. The batch file processing is exited with a jump to predefined label EOF (end of file, requires enabled extensions which are enabled by default) in case of the file test.txt does not exist at all.
Next the file is read line by line with skipping empty lines and lines starting with a semicolon by command FOR which splits each line up into two strings.
The first string left of first equal sign is assigned to loop variable I. Everything right of first equal sign is assigned next loop variable J according to ASCII table.
The IF condition in the loop checks if a value is also defined for a variable. The value is assigned to an environment variable on which a string substitution is executed using delayed expansion to replace all / by \.
Then the modified value is assigned to an environment variable with a name starting with _, the name of the batch file, one more underscore and the string assigned to loop variable I read from the file.
For demonstration the variables with their values are finally output before the local variables are discarded on execution of last command ENDLOCAL.
I strongly recommend not assigning the values read from the file directly to environment variables whose name is completely also read from the file as this makes the batch file easy to manipulate by just modifying the contents of the text file. For example path=C:\Temp in text file would otherwise result in set "Path=C:\Temp" and from this point of batch file execution the running Windows command process would not find anymore any standard executable in directories defined by default in environment variable PATH like %SystemRoot%\System32.
A second variant which incorporates answer posted by Magoo with above batch code:
#echo off
if not exist "test.txt" goto :EOF
setlocal DisableDelayedExpansion
for /F "usebackq tokens=1* delims==" %%I in ("test.txt") do if not "%%~J" == "" set "_%~n0_%%~I=%%~J" & call set "_%~n0_%%~I=%%_%~n0_%%~I:/=\%%"
echo The variables set from file are:
echo/
set "_%~n0_"
echo/
pause
endlocal
The advantage of this variant is that delayed expansion is not needed for this solution which makes it possible to correct process also lines from file containing 1 or more exclamation marks on which first variant fails. And it is also a little bit faster, not noticeable faster for a human, but nevertheless a bit faster.
In both batch code blocks _%~n0_ can be replaced by (nearly) anything including also nothing although that is not recommended. Using just an underscore would be also possible as there are no environment variables defined by default by Windows which start with an underscore.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /? ... explains %~n0 (name of argument 0 - the batch file name - without path and without file extension).
echo /?
endlocal /?
for /?
goto /?
if /?
pause /?
set /?
setlocal /?
The simplest solution is to let the ~f FOR variable modifier put the full path in canonical form (including conversion of forward slashes to back slashes). I use the DELIMS and TOKENS options to split each line into the variable name and path so that I can apply the ~f to the path. It is important to use tokens=1* instead of tokens=1,2 just in case the path includes a = character.
for /f "delims== tokens=1*" %%A in (test.txt) do (set "%%A=%%~fB")
Note, however, that this strategy only works if your "test.txt" already contains full, absolute paths. If the file contains relative paths, then the ~f modifier will add drive and or folder values from the current directory to turn the relative path into an absolute path.
I wanted to take list of files to delete from user as a argument. One line per argument.
How can store the list of files separated by new line in a variable.
I am using below command.
Set DeletionFiles=${p:DeleteFiles}"
for %%i in (%DeletionFiles%) do (
echo %%i
)
Then i wanted to iterated them on a loop.
${p:DeleteFiles} will get replaced by it's value from external app, which will contain list of files separated by new line.I can not change it.
#ECHO OFF
SETLOCAL
SET "deletionfiles="
:dloop
SET "deleteme="
SET /p "deleteme=Which file to delete ? "
IF DEFINED deleteme SET "deleteme=%deleteme:"=%"
IF DEFINED deleteme SET "deletionfiles=%deletionfiles%,"%deleteme%""&goto dloop
ECHO delete %deletionfiles:~1%
GOTO :EOF
There is no need to use a newline. Your for command (or a del command) will operate perfectly happily on a comma-(or space-)separated list.
Note that there are certain characters that batch uses for special purposes and batch string-processing may not process them in the expected manner. These characters include % ^ and &.
${p:DeleteFiles} will get replaced by it's value from external app,
which will contain list of files separated by new line.I can not
change it.
After the replacement the batch file looks like:
Set DeletionFiles=file1.jpg
file2.jpg
file3.jpg
"
This isn't a valid batch file anymore.
Furthermore it's a bad idea to modify the batch file itself, as this works only once.
You could place the ${p:DeleteFiles} into another file, like input.txt.
Your batch would look like
echo ${p:DeleteFiles} > input.txt
<external program for replacing the DeleteFiles> input.txt
for /F "tokens=*" %%A in (input.txt) do (
echo File: %%A
)
If I understand you correctly, your external program will generate a list of files. You then want to store this multi-line list to a variable. What do you want to do with the variable once you have it? I assume you want to delete the files, but your question isn't clear on that point, so I'll try to over-answer to cover it.
for /f "delims=" %%a in ('{command that generates your list}') do (
echo Doing stuff to %%a...
echo %%a>>listOfFilesToDelete.txt
set var=%%a
if "%var:~0,7%"="DoNotDelete" copy "%%a" \someArchiveFolder\
del "%%a"
)
This will read each line in your generated list as variable %%a. It will then do whatever command(s) you specify. This way, you can run a command on each of the files in the list. In the above code it's
Printing each line to the console embedded in some text
Outputting it to a file
Checking the first 7 characters of the line against a specified string and then copying it to a folder if it matches
And then deleting it
If you still need to reference each line from your generated list, you can even setup an array-like structure. (See Create list or arrays in Windows Batch)
setlocal EnableDelayedExpansion
:: Capture lines in an 'array'
set /a i=0
for /f "delims=" %%a in ('dir /b') do (
set /a i+=1
set var!i!=%%a
)
:: Loop through the 'array'
for /L %%a in (1,1,%i%) do (
echo Do more stuff with !var%%a!
)
Just like above, this will read each line in your generated list as variable %%a. It will then set a variable var!i! equal to the value of the current line. You can then reference each line as var1, var2, and so on, or, as the second section shows, you can loop through them all using for /L. You'll need to get a grasp on working with delayed expansion variables, though.
I am moving files based on their names to preset folders. I don't want to make new folders. So files should only be moved if the corresponding folder is existing already.
The file names all follow the same pattern: 1234_123456_AA_***********.(doc/pdf)
I have the following script below which works:
#echo on
for /r %%f in (*.*) do (
echo processing "%%f"
for /f "tokens=1-3 delims=_" %%a in ("%%~nxf") do (
move "%%f" C:\Users\xxxxxxxxx\Desktop\MOVEFILES\%%a_%%b_%%c\
)
)
pause
But the issue I am running into is that some of the files names have a '0' place holder in loop variable %%b, for example 1234_0123456_AA. But this file name should be interpreted like 1234_123456_AA and I want this file moved into the appropriate folder.
I have written this:
#echo on
SETLOCAL ENABLEDELAYEDEXPANSION
for /r %%f in (*.*) do (
for /f "tokens=1-3 delims=_" %%a in ("%%~nxf") do (
set z=%%b%
echo !z:~-6!
move "%%f" C:\Users\xxxxxxxxx\Desktop\MOVEFILES\%%a_%%z_%%c\
)
)
pause
I get the echo to remove the '0' place holder, but need to get that back into %%b in the file path of where the file should be moved to.
What to modify in code to get the desired behavior?
Use the following code with the corrections of JosefZ applied and some additional improvements:
#echo off
setlocal EnableDelayedExpansion
for /R %%F in (*_*_*.*) do (
for /F "tokens=1-3 delims=_" %%A in ("%%~nxF") do (
set "MiddlePart=%%B"
set "MiddlePart=!MiddlePart:~-6!"
if exist "%USERPROFILE%\Desktop\MOVEFILES\%%A_!MiddlePart!_%%C\*" (
move /Y "%%F" "%USERPROFILE%\Desktop\MOVEFILES\%%A_!MiddlePart!_%%C"
)
)
)
endlocal
pause
The improvements on code are explained below in details.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
echo /?
endlocal /?
if /?
move /?
set /?
setlocal /?
1. Search pattern
The first improvement is the pattern used in first FOR as only files should be moved with at least 2 underscores in file name. This pattern is still not the best one, but good enough for this task.
2. Loop variable
It is better to use loop variables in upper case to avoid problems with the modifiers.
For example using as loop variable %%f and using inside the loop %%~f to use the string (must not be a file or folder name) of loop variable f without surrounding quotes, command processor exits batch processing with an error message because it expects one more letter, the loop variable as %%~f is interpreted as full name of file/folder of loop variable ?.
The loop variables and the modifiers are case sensitive. Therefore %%~F is interpreted by command processor as string of loop variable F without surrounding quotes and %%~fF as file/folder name with full path and extension of the file/folder of loop variable F.
Some other characters like # can be also used as loop variable.
3. Assigning value to environment variable with quotes
On assigning a string to an environment variable, it is always good to use
set "variable=text or other variable"
Using the quotes as shown here has the advantage that not visible spaces/tabs after last double quote are ignored by command processor.
But with using just
set variable=text or other variable
everything after first equal sign up to line termination is assigned to the variable including trailing spaces and tabs added perhaps by mistake on this line in the batch file. This is nearly never good and a common source of a batch execution problem which can be easily avoided by using quotes right.
Using the quotes as show below is also not good as in this case both double quotes are part of the text assigned to the variable (plus trailing spaces/tabs). This is sometimes useful, but is most often a coding mistake.
set variable="text or other variable"
4. Delayed expansion
Referencing a variable set or modified within a block defined with (...) requires delayed expansion if the current variable value should be used and not the value assigned to the variable above the block. Therefore using %%z was wrong in original code as variable z was not defined above first FOR loop and therefore was replaced always with nothing on execution of the loops.
5. Environment variable USERPROFILE
Running in a command prompt window set results in getting displayed all predefined environment variables for the current user account. There is the variable USERNAME, but also USERPROFILE containing path to the user´s profile directory with the Desktop folder and other user account related folders. Using environment variable USERPROFILE makes the batch file less dependent on Windows version and language of Windows.
ATTENTION:
The first FOR runs because of /R recursive on current directory and all its subdirectories. As the inner FOR loop moves all found files in current directory tree to subdirectories of %USERPROFILE%\Desktop\MOVEFILES, the current directory should be never any directory of this path.
I'm creating a simple production environment for work and in doing so need to set specific environment variables for specific projects in batch file.
Here's what i want to achieve:
1) Define a single environment variable which would define a list of directories
2) Recurse down each directory and add all leaf folders to a final environment variable.
[EDIT] After looking back at what i originally posted i was able to remove some redundancy. But the "The input line is too long." error occurs when %LINE% gets too long. Using the short path expansion does help but it can still error out. I'll look at how to break the echo to a temp file next as suggested.
Here's what i currently have:
#echo off
set RECURSE_THESE_DIRS=C:\Users\eric\Autodesk
set TMP_FILE=%CD%TMP_FILE.%RANDOM%.txt
setLocal EnableDelayedExpansion
for %%i in (%RECURSE_THESE_DIRS%) do (
if exist %%~si\NUL (
for /f "tokens=*" %%G in ('dir /b /s /a:d %%i') do set LIST=!LIST!;%%G
)
)
set LIST=%LIST:~1%
rem !!! AT THE ECHO LINE BELOW IF %LIST% IS TOO LONG, THIS SCRIPT FAILS
rem !!! WITH The input line is too long. ERROR :(
echo %LIST%>%TMP_FILE%
endlocal
for /F "delims=" %%G in (%TMP_FILE%) do set FINAL_VAR=%%G
del /F %TMP_FILE%
So by setting RECURSE_THESE_DIRS to directories i wish to parse, i end up with a %FINAL_VAR% which i can use to specify paths for proprietary software i use. Or i could use this script to append to %PATH%, etc...
This works for me but i would love suggestions to improve/streamline my script?
The root of your problem is that batch is limited to fit the variable name, contents and = into 8192 bytes, hence your directory-list simply isn't going to fit into one variable.
Personally, I'd just spit out a dir/s/b/a-d list to a tempfile and process that file with a for/f "delims=" - after all, you'd be likely to need to process your ;-separated envvar similarly in whatever process you are proposing to execute.
For instance, here's a test producing the same error - not using filenames at all
#ECHO OFF
SETLOCAL
SET "var=hello!1234"
SET var=%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%
SET count=8000
:loop
SET /a count +=1
ECHO %count%
SET var=%var%x
ECHO %var%
GOTO loop
GOTO :EOF
This should fail where count=8184.
Suggestions:
Use for /d /r to handle the recursion
Maybe i'm wrong, but in your script, you traverse the directory hierarchy, adding each directory to temp file which is then readed to concatenate its lines into a variable which is then writed to temp file, to be read again in another variable. If concatenation of directories fit into a variable, why not concatenate them without any temporary file?
If concatenation is in the limit of line length, as suggested by npocmaka, and if soported by end application, user short paths. Also, instead of adding the new values in the same instruction (not line) that contains the for loop, use a call to a subrutine with the new value passed to it, and then append the value. It's slower, but command lines executed will be shorter.