comparing filenames with .bat and writing missing entries into .txt - batch-file

i want to compare some files in a folder and then write missing ones into a text file.
i've got a folder
c:\sessions\
with files in it. those files are looking like:
Blabla-1111-FOO
Blabla-1111-BAR1
Bleble-2222-FOO
Bleble-2222-BAR1
Bleble-2222-BAR2
Bleble-2222-BAR3
Bleble-2222-BAR4
Blublu-5678-FOO
Blublu-5678-BAR1
Blublu-5678-BAR2
Blublu-5678-BAR3
Blublu-5678-BAR4
Blublu-5678-BAR5
Bleble-6666-FOO
Bloblo-7777-FOO
Bloblo-8888-FOO
Bloblo-9999-FOO
i need a .bat file which writes down all files ending with -FOO where no matching -BAR exists into a noBARs.txt. So in this case the noBARs.txt should look like this:
Bleble-6666-FOO
Bloblo-7777-FOO
Bloblo-8888-FOO
Bloblo-9999-FOO
There can be multiple -BAR files that belong to one -FOO file.
Thanks in advance guys.

You could try the following:
#echo off
setlocal enabledelayedexpansion
set "source=c:\sessions"
set "output=noBARs.txt"
set "find=*foo"
set "repl=bar*"
1>"%output%" (
for %%e in ("%source%\%find%") do (
set "file=%%e"
if not exist "!file:~0,-3!%repl%" (
echo %%~ne
)
)
)
This will iterate through all the files that contain *foo within their filename. Then it replaces the last 3 characters of the filename with bar* and checks for the existence of the file. If the corresponding bar file doesn't exist it will write the original filename to the output file noBARs.txt.
I've added an alternative version of the script based on the last comment of dEEkAy in which he states that files ending with FOO followed by a numerical constant should be tested as well. I'm still quite uncertain what the exact intention of dEEkAy is but I'm assuming that the file names *FOO* should be handled the same as *FOO.
The following input:
Blabla-1111-FOO
Blabla-1111-BAR1
Bleble-2222-FOO
Bleble-2222-BAR1
Bleble-2222-BAR5
Blublu-3333-FOO6
Blublu-3333-BAR1
Blublu-5678-FOO
Blublu-5678-FOO1
Blublu-5678-BAR2
Blublu-5678-BAR4
Bleble-6666-FOO
Bloblo-7777-FOO5
Bloblo-7777-BAR8
Bloblo-9999-FOO
Blublu-9999-FOO9
Bloblo-!^^!-FOO
Will produce the following output:
Bleble-6666-FOO
Bloblo-!^^!-FOO
Bloblo-9999-FOO
Blublu-9999-FOO9
The alternative version of the script is also more robust than the original. It should now be capable of handling file names and paths that contain exclamation marks and carets appropriately. Unfortunately, it does come with a cost. Due to the call command which will be executed for each iteration of the for-loop, the script has become a lot slower. Perhaps someone with more batch-file scripting experience can come up with a better solution.
#echo off
setlocal disabledelayedexpansion
set "source=c:\sessions"
set "output=noBARs.txt"
set "find=foo"
set "repl=bar"
1>"%output%" (
for %%e in ("%source%\*%find%*") do (
set "path=%%~dpe"
set "file=%%~ne"
setlocal enabledelayedexpansion
call :ReplaceEndOfString file match "%find%" "%repl%"
if not exist "!path!!match!*" (
echo !file!
)
endlocal
)
)
exit /b
:ReplaceEndOfString (__in *source, __out *dest, __in find, __in repl) {
set "temp=!%1:*%~3=%~3!"
set "%2=!%1:%temp%=%~4!"
exit /b
}
Keep in mind that "%source%\*%find%*" will match any file that contains %find% within its file name, whereas "%source%\*%find%?" only matches file names that end with %find% including one optional character that could be anything.
I just happened to need some code similar to this and decided to take another look. The main bottleneck of the code seems to be the invocation of the call command. While I don't see a way around this (other than using macros), the execution time of the call command can be greatly improved (with a factor of 2, roughly).
Calling a function by its label seems to be a lot slower than calling the set command. I'm not entirely sure but I can remember something about calling labels being slow due to requiring a rescan of the entire script file. Carets and exclamation marks within file names also appear to remain intact because the delayed expansion phase seems to occur before the set command is called.
#echo off
setlocal DisableDelayedExpansion
set "source=c:\sessions"
set "output=nobars.txt"
set "find=foo"
set "repl=bar"
1>"%output%" (
for %%e in ("%source%\*%find%*") do (
set "path=%%~dpe"
set "file=%%~ne"
setlocal EnableDelayedExpansion
set "end=!file:*%find%=%find%!"
call set "match=%%file:!end!=%repl%%%"
if not exist "!path!!match!*" (
echo !file!
)
endlocal
)
)
exit /b

Related

Using xcopy or copy for a single file from multiple folders

So in the batch script I'm building I am taking a single file from a folder, copying it over to a destination folder, and renaming it based on the number of times that the script has been looped. Essentially I need to take a file that's named the samething from a bunch of different folders spread across multiple computers at times and copy them into a new folder to work with. I've read up on xcopy and copy as that seemed like the thing to use but I haven't been able to find anything that lets me tell it to only copy over a single named file. I've posted what I have so far for the script below with commented lines for the sections I haven't figured out:
ECHO off
SETLOCAL enabledelayedexpansion
ECHO Note: Your combined permission list cvs can be found in the desktop folder
SET /A #=-1
:start
SET /A #+=1
:again
ECHO Please input the file path to the permissionoutput.txt
SET /p permissionoutputpath=
SET "sourcefolder=%permissionoutputpath%"
SET "destinationfolder=C:\Users\kayla\Desktop\HOLDER-CombinedPermissionsLists"
IF not exist "%sourcefolder%\permissionoutput.txt" Echo file not found&goto again
copy "%sourcefolder%\permissionoutput.txt" "%destinationfolder%\permissionoutput%#%.txt"
ECHO Add another file to combine: y or n?
SET /p addanotherfile=
if %addanotherfile%==y goto :start
UPDATE: Code corrected with answer to be fully functional for use as a reference
SET /A #=-1
:start
SET /A #+=1
:again
ECHO Please input the file path to the permissionoutput.txt
SET /p permissionoutputpath=
SET "sourcefolder=%permissionoutputpath%"
SET "destinationfolder=C:\Users\kayla\Desktop\HOLDER-CombinedPermissionsLists"
IF not exist "%sourcefolder%\permissionoutput.txt" Echo file not found&goto again
copy "%sourcefolder%\permissionoutput.txt" "%destinationfolder%\permissionoutput%#%.txt"
ECHO Add another file to combine: y or n?
SET /p addanotherfile=
if /i "%addanotherfile%"=="y" goto start
# is a legitimate variable-name. It's initialised to -1 then incremented on each loop through :start so the first value it will have when it's used is 0. (If you want to start at 1 just initialise it to 0 instead)
Next - your sets - BUT spaces are significant in a string set command are would be included in the variablename/value assigned if present in the set instruction. "quoting the assignment" ensures any stray trailing spaces on the line are not included in the value assigned.
Well - next, make sure the file exists and if it doesn't, then produce a message and loop back to :again which bypasses the increment of #.
Otherwise, simply copy the file. You're aware of its sourcename, and your destinationname is constructed by including %#% to include the current value of # (all batch variables without exception are strings - the set /a instruction merely converts from string to binary to perform the required calculation, then converts the result back to a string for storage in the environment.)
Finally, interpreting the request to add another file. if /i makes the comparison case-insensitive. Since you have no direct control over the user's response, "quoting each side" ensures the if syntax isn't violated in case the user enters "yup sure 'nuff" or some other unexpected response.
The leading colon is not required in a goto. I prefer to omit it to keep conguity with the call command where no-colon means an external routine will be called and a colon means the routine is in this batch file.

Batch file - output csv to different folders

everything in my code is working fine except the last part.
I am wanting to output each text file to the folder with the same name. It is outputing the three text files into the one folder PentahoOutputs. However I am wanting to output it as the following:
folder system2.object2.assets contains file system2.object2.assets
folder system3.object3.assets contains file system3.object3.assets
folder system4.object4.assets contains file system4.object4.assets
#echo off SetLocal EnableDelayedExpansion
SET DELIMS=,
SET COMMAND=AddChange
SET EN=EN
SET ASSETS=Assets
SET DIREC = C:\Users\user.username\Documents\Training\BatchFiles\PentahoOutputs\
SET DELIMS2=.
FOR /D %%a IN (C:\Users\user.username\Documents\Training\BatchFiles\PentahoOutputs\*) DO ( SET subdirs=%%a
result=!subdirs:~71,7!
result2=!subdirs:~79,7!
set "concats=!result!!delims!!result2!!DELIMS!!COMMAND!!DELIMS!!EN!"
echo !concats!
echo !CONCATS! >>C:\Users\user.username\Documents\Training\BatchFiles\PentahoOutputs\!result!!delims2!!result2!!delims2!!assets!.CSV
)
PAUSE>NUL
edit ********** below
changing the problem code to the following puts each of the three files in each of the three folders... however i want one file in each folder
for /d %%b in (C:\Users\usre.username\Documents\Training\BatchFiles\PentahoOutputs\*) DO ( echo !CONCATS! >>%%b\!result!!delims2!!result2!!delims2!!assets!.csv )
From your posted code - given aschipfl's change as noted (although you don't attempt to use direc)
Your posted code has been mangled in an attempt, I assume, to disguise usernames. It also appears that you've cut down the actual code to show only the relevant section. This is good and understandable (but your edit has a glaring typo in the code - which is why you should cut-and-paste as far as possible.)
So - the setlocal following the #echo off must be separated by a & command-concatenator or be (my preference) on a separate line.
Within your for ... %%a ... block, you've removed the required set keyword for result*.
The fixed values you've used for substringing don't suit the changes you've made to the pathname, so the result in result is (eg) "tem3.ob"
If a value does not change within a block (like delims) then it's probably best to use %delims% - result changes, so you'd use !result! not %result%. !delims! also works, of course - but using the delayed-expansion form primes the reader to believe it's going to vary. (opinion)
'tis best with a string assignment to use set "var=value" as the quotes ensure that stray trailing spaces are not included in the value assigned. You only ever need to have that happen once...
OK - here's a revision
#echo OFF
SetLocal
SET DELIMS=,
SET COMMAND=AddChange
SET EN=EN
SET ASSETS=Assets
SET DIREC=U:\Users\user.username\Documents\Training\BatchFiles\PentahoOutputs\
SET DELIMS2=.
FOR /D %%a IN (%direc%*) DO (
FOR /f "tokens=1,2,3 delims=." %%p IN ("%%~nxa") DO IF /i "%%r"=="%assets%" (
echo %%p%delims%%%q%DELIMS%%COMMAND%%DELIMS%%EN%
echo %%p%delims%%%q%DELIMS%%COMMAND%%DELIMS%%EN% >> %%a\%%~na.CSV
)
)
GOTO :EOF
Note that I've used U: for the test directory (it's a ramdrive on my machine)
Given the outer loop, %%a is assigned the full pathname to the directory.
Since you imply that your target directorynames are system2.object2.assets then %%~nxa (the Name and eXtension of %%a) conveniently holds this string. Parsing that using delims of . and selecting the first 3 tokens would assign system2 to %%p, object2 to %%q and assets to %%r This avoids the substringing problem and permits system and object to be any length - not just 7.
The if statement ensures that the main block for for...%%p is only executed for directories found which fit ..asset (/i makes the if case-insensitive)
The required line can then be constructed from the metavariables and constants, as can the destination filename, so the enabledelayedexpansion is not required.

Modify variable within loop of batch script

I am moving files based on their names to preset folders. I don't want to make new folders. So files should only be moved if the corresponding folder is existing already.
The file names all follow the same pattern: 1234_123456_AA_***********.(doc/pdf)
I have the following script below which works:
#echo on
for /r %%f in (*.*) do (
echo processing "%%f"
for /f "tokens=1-3 delims=_" %%a in ("%%~nxf") do (
move "%%f" C:\Users\xxxxxxxxx\Desktop\MOVEFILES\%%a_%%b_%%c\
)
)
pause
But the issue I am running into is that some of the files names have a '0' place holder in loop variable %%b, for example 1234_0123456_AA. But this file name should be interpreted like 1234_123456_AA and I want this file moved into the appropriate folder.
I have written this:
#echo on
SETLOCAL ENABLEDELAYEDEXPANSION
for /r %%f in (*.*) do (
for /f "tokens=1-3 delims=_" %%a in ("%%~nxf") do (
set z=%%b%
echo !z:~-6!
move "%%f" C:\Users\xxxxxxxxx\Desktop\MOVEFILES\%%a_%%z_%%c\
)
)
pause
I get the echo to remove the '0' place holder, but need to get that back into %%b in the file path of where the file should be moved to.
What to modify in code to get the desired behavior?
Use the following code with the corrections of JosefZ applied and some additional improvements:
#echo off
setlocal EnableDelayedExpansion
for /R %%F in (*_*_*.*) do (
for /F "tokens=1-3 delims=_" %%A in ("%%~nxF") do (
set "MiddlePart=%%B"
set "MiddlePart=!MiddlePart:~-6!"
if exist "%USERPROFILE%\Desktop\MOVEFILES\%%A_!MiddlePart!_%%C\*" (
move /Y "%%F" "%USERPROFILE%\Desktop\MOVEFILES\%%A_!MiddlePart!_%%C"
)
)
)
endlocal
pause
The improvements on code are explained below in details.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
echo /?
endlocal /?
if /?
move /?
set /?
setlocal /?
1. Search pattern
The first improvement is the pattern used in first FOR as only files should be moved with at least 2 underscores in file name. This pattern is still not the best one, but good enough for this task.
2. Loop variable
It is better to use loop variables in upper case to avoid problems with the modifiers.
For example using as loop variable %%f and using inside the loop %%~f to use the string (must not be a file or folder name) of loop variable f without surrounding quotes, command processor exits batch processing with an error message because it expects one more letter, the loop variable as %%~f is interpreted as full name of file/folder of loop variable ?.
The loop variables and the modifiers are case sensitive. Therefore %%~F is interpreted by command processor as string of loop variable F without surrounding quotes and %%~fF as file/folder name with full path and extension of the file/folder of loop variable F.
Some other characters like # can be also used as loop variable.
3. Assigning value to environment variable with quotes
On assigning a string to an environment variable, it is always good to use
set "variable=text or other variable"
Using the quotes as shown here has the advantage that not visible spaces/tabs after last double quote are ignored by command processor.
But with using just
set variable=text or other variable
everything after first equal sign up to line termination is assigned to the variable including trailing spaces and tabs added perhaps by mistake on this line in the batch file. This is nearly never good and a common source of a batch execution problem which can be easily avoided by using quotes right.
Using the quotes as show below is also not good as in this case both double quotes are part of the text assigned to the variable (plus trailing spaces/tabs). This is sometimes useful, but is most often a coding mistake.
set variable="text or other variable"
4. Delayed expansion
Referencing a variable set or modified within a block defined with (...) requires delayed expansion if the current variable value should be used and not the value assigned to the variable above the block. Therefore using %%z was wrong in original code as variable z was not defined above first FOR loop and therefore was replaced always with nothing on execution of the loops.
5. Environment variable USERPROFILE
Running in a command prompt window set results in getting displayed all predefined environment variables for the current user account. There is the variable USERNAME, but also USERPROFILE containing path to the userĀ“s profile directory with the Desktop folder and other user account related folders. Using environment variable USERPROFILE makes the batch file less dependent on Windows version and language of Windows.
ATTENTION:
The first FOR runs because of /R recursive on current directory and all its subdirectories. As the inner FOR loop moves all found files in current directory tree to subdirectories of %USERPROFILE%\Desktop\MOVEFILES, the current directory should be never any directory of this path.

Using if condition in CMD

EDIT: I have solved my problem based on suggestions from npocmaka and magoo. Now that i understand it is indentation that caused this, can you guys please suggest how i make my code cleaner in future?
Here's my bat file:
#echo off
setlocal enabledelayedexpansion
rem set lang pre-requisites
rem Purpose can be 'test' or 'deploy'
SET purpose="test"
SET lang="English-India"
SET lang_code="EnIn"
rem if purpose is test then the folder will be EnInP101M2Tsub, if purpose is deploy then EnInP101M2DFull
if "%purpose%"=="test"( SET folder=="%lang_code%P101M2Tsub" )
else if "%purpose%"=="deploy"( SET folder=="%lang_code%P101M2DFull" )
rem set required paths here
SET audio="C:\Users\Administrator\..path..\%lang%\%purpose%\BGM\M2\%folder%\*.wav"
SET source="C:\Users\Administrator\Downloads\Matt_Trial\..path..\%lang%\%purpose%\BGM\M2"
SET target="C:\Users\Administrator\Downloads\Matt_Trial\..path..\%lang%\%purpose%\BGM\M3\*"
rem copying the data
xcopy %source% %target% /e /i /h
rem copying audio files into the other folders
for /d %%a in (%target%) do copy %audio% "%%a"
rem renaming M3 folders
cd %target%
for /d %%a in (*) do (
set "p=%%a"
set "fp=!p:~0,8!" & set "tp=!p:~10!"
ren %%a !fp!M3!tp!
)
** For your Understanding **
I have 2 folders in my parent folder (M1,M2,M3).
Step 1: I am copying contents in M2 to a new folder M3.
Step 2: I need to copy contents of a folder under M2 which has my audio into all the folders in M3.
Step 3: I rename the folders under M3.
I hope I made myself clear.
I have to figure out the audio path based on the set variables. I need help with the if condition part. As of now I keep getting The system cannot find the file specified. Please help!!
A few errors.
SET will INCLUDE " after the = EXCEPT if you are using `SET "var=data" which is used to ensure that trailing spaces on the line are not included in the data assigned.
Any character after the = will be included (some need to be 'escaped' if they have a special meaning to batch, so set var==data will include the second = in the data assigned.
You haven't fallen into the spaces-in-variables trap - the syntax set var =data will set a variable named varspace not var.
There must be a space (strictly a separator, I believe) between the second operand of an IF statement and ( (if used)
Similarly, the ELSE clause must be )SpaceelseSpace( - all on the same line. This sequence of characters cannot be broken.
In your IF, you are using "%purpose%" Since purpose us set to "test" then this will be evaluated as ""test"", which is likely to be confusing, hence don't include " in your data assigned unless you have a really good reason.
I don't have 50 rep yet so have to comment via answer instead.
When i run into issues with my batch files I find it helpful to REM the #ECHO OFF line.
Then create a short-cut to the batch file and change the 'Target' field to pipe the output to a text file.
So, if your batch is at C:\test\mybatch.bat then create a shortcut to the batch in C:\test\
Next right-click the shortcut and select 'Properties'.
Under the 'Shortcut' tab, add the following to the end of the text in the target field..
>>output.txt
Run the shortcut which will save all the commands and errors to the output.txt file where you can see what is causing the issue, such as a variable not being set.
This:
#echo off
setlocal enabledelayedexpansion
rem set lang pre-requisites
rem Purpose can be 'test' or 'deploy'
SET purpose="test"
SET lang="English-India"
SET lang_code="EnIn"
rem if purpose is test then the folder will be EnInP101M2Tsub, if purpose is deploy then EnInP101M2DFull
if "%purpose%"=="test"( SET folder=="%lang_code%P101M2Tsub" )
else if "%purpose%"=="deploy"( SET folder=="%lang_code%P101M2DFull" )
Rather should look like (as you have only two options..)
#echo off
setlocal enabledelayedexpansion
rem set lang pre-requisites
rem Purpose can be 'test' or 'deploy'
SET "purpose=test"
SET "lang=English-India"
SET "lang_code=EnIn"
rem if purpose is test then the folder will be EnInP101M2Tsub, if purpose is deploy then EnInP101M2DFull
if "%purpose%"=="test" (
SET "folder=%lang_code%P101M2Tsub"
)
if "%purpose%"=="deploy" (
SET "folder=%lang_code%P101M2DFull"
)
echo %folder%
endlocal
You cannot start a line with else and set does not work properly with double = . And your comparisons are not correct as you have a quotes in variable values.
You have to use round parenthesis. Remember that if .... ( and ) else ( are inextricably linked to a single line (Each one has to be in a single line).
...
if "%purpose%"=="test" (
SET "folder=%lang_code%P101M2Tsub"
) else (
if "%purpose%"=="deploy" (
SET "folder=%lang_code%P101M2DFull"
)
)
...
Note:
I found extra issues in your set assignment:
Extra equal.
Double quotes.
Suppose Lang_code is equals to -EN-
SET folder=="%lang_code%P101M2Tsub"
Echo [%folder%]
It shows
[="-EN-P101M2Tsub"]
The correct way should be:
SET "folder=%lang_code%P101M2Tsub"
Echo [%folder%]
It shows
[-EN-P101M2Tsub]

Batch - Recurse directories from a variable and expand results in another variable

I'm creating a simple production environment for work and in doing so need to set specific environment variables for specific projects in batch file.
Here's what i want to achieve:
1) Define a single environment variable which would define a list of directories
2) Recurse down each directory and add all leaf folders to a final environment variable.
[EDIT] After looking back at what i originally posted i was able to remove some redundancy. But the "The input line is too long." error occurs when %LINE% gets too long. Using the short path expansion does help but it can still error out. I'll look at how to break the echo to a temp file next as suggested.
Here's what i currently have:
#echo off
set RECURSE_THESE_DIRS=C:\Users\eric\Autodesk
set TMP_FILE=%CD%TMP_FILE.%RANDOM%.txt
setLocal EnableDelayedExpansion
for %%i in (%RECURSE_THESE_DIRS%) do (
if exist %%~si\NUL (
for /f "tokens=*" %%G in ('dir /b /s /a:d %%i') do set LIST=!LIST!;%%G
)
)
set LIST=%LIST:~1%
rem !!! AT THE ECHO LINE BELOW IF %LIST% IS TOO LONG, THIS SCRIPT FAILS
rem !!! WITH The input line is too long. ERROR :(
echo %LIST%>%TMP_FILE%
endlocal
for /F "delims=" %%G in (%TMP_FILE%) do set FINAL_VAR=%%G
del /F %TMP_FILE%
So by setting RECURSE_THESE_DIRS to directories i wish to parse, i end up with a %FINAL_VAR% which i can use to specify paths for proprietary software i use. Or i could use this script to append to %PATH%, etc...
This works for me but i would love suggestions to improve/streamline my script?
The root of your problem is that batch is limited to fit the variable name, contents and = into 8192 bytes, hence your directory-list simply isn't going to fit into one variable.
Personally, I'd just spit out a dir/s/b/a-d list to a tempfile and process that file with a for/f "delims=" - after all, you'd be likely to need to process your ;-separated envvar similarly in whatever process you are proposing to execute.
For instance, here's a test producing the same error - not using filenames at all
#ECHO OFF
SETLOCAL
SET "var=hello!1234"
SET var=%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%%var%
SET var=%var%%var%%var%%var%
SET count=8000
:loop
SET /a count +=1
ECHO %count%
SET var=%var%x
ECHO %var%
GOTO loop
GOTO :EOF
This should fail where count=8184.
Suggestions:
Use for /d /r to handle the recursion
Maybe i'm wrong, but in your script, you traverse the directory hierarchy, adding each directory to temp file which is then readed to concatenate its lines into a variable which is then writed to temp file, to be read again in another variable. If concatenation of directories fit into a variable, why not concatenate them without any temporary file?
If concatenation is in the limit of line length, as suggested by npocmaka, and if soported by end application, user short paths. Also, instead of adding the new values in the same instruction (not line) that contains the for loop, use a call to a subrutine with the new value passed to it, and then append the value. It's slower, but command lines executed will be shorter.

Resources