Parse specific strings (2 strings) from multiple text files from a folder - batch-file

I am trying to parse multiple lines which starts with:
Procedure = xxxxx::xxxx
Description = xxxxxx
Also, I want to ignore SH Library Procedure = $(Stem)_test which has same word Procedure in the .txt file.
I want to search all *.txt files and accumulate date in output file which I will use to upload in req management tool.
Here is the sample of the file:
Harness Lib Greenhills BLD Add Excluded Files = FALSE
Harness Lib Template Project File =
Harness Lib Generated Project File =
Harness Lib Generate Compiler Project File = FALSE
SH Library Procedure = $(Stem)_test
Harness Lib Source Lists Add Excluded Files = FALSE
Harness Lib Substitute Unused Source Files = FALSE
Macro Standard 1 = Set TBRUN_MACRO_STANDARD_1 in Testbed.ini
Procedure = sander_class::sander_class
Member Of = 1
Creation Date = Jun 21 2019 14:36:44
Description = This test is to verify that constructor is called. Req Tested: 67060-SWINTR-73
I have tried below code, but it does not print Procedure and in specific format.
#echo off
(
for /f "tokens=1,*" %%a in ('find "Procedure =" ^< "TEST.txt"') do (
for /f "tokens=1,*" %%d in ('find "Description =" ^< "TEST.txt"') do (
for /f "tokens=1,2 delims=#" %%c in ("%%b") do (
echo(Procedure %%~nxc
echo(
echo(Procedure %%a
echo(
echo(Description %%d
echo(
echo(Path: %%~pb
echo(
)
)
)>"output file.txt"
pause
I need output in below format for all the files (.txt) in same folder:
File Name =
Procedure = sander_class::sander_class
Description = This test is to verify that constructor is called. Req Tested: 67060-SWINTR-73

#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q57185510.txt"
SET "outfile=%destdir%\outfile.txt"
(
FOR /f "tokens=*" %%a IN (
'findstr /R /b /C:" *Procedure =" /c:" *Description =" "%filename1%" '
) DO ECHO %%a
)>"%outfile%"
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances. The listing uses a setting that suits my system.
I used a file named q57185510.txt containing your data for my testing.
Produces the file defined as %outfile%
I used the one specific file for testing; for all files then set filename1 to a filemask like *.txt.
The findstr looks in all files matching filename1 for lines that /b begin with /r regular-expressions /c:"regex-string" the "regex-string". "regex" here is a small subset of regular expressions, but the particular expressions of interest are for instance Space(any number of)ProcedureSpace= which matches the two lines in question.
The tokens=* suppresses the leading spaces, producing the required lines.

Okay, This should do it but I can't test at the moment.
#(
ECHO OFF
SETLOCAL EnableDelayedExpansion
SET "_FilePath=C:\Admin"
SET "_FileGlob=*.txt"
SET "_MatchRegex1=Procedure =.*::.*$"
SET "_MatchRegex2=Description = .*::.*$"
SET "_ExcludeRegex=^SH Library Procedure =.*$"
SET "_OutputFile=%Temp%\output file.txt"
)
FOR /F "Tokens=*" %%A IN ('
DIR /A-D /S /B "%_FilePath%\%_FileGlob%"
') DO (
SET "_FileName=%%~nxA"
SET "_CurrentFile=!_FileName: =_!"
FOR /F "Tokens=1*" %%a IN ('
FindStr /I /R
/C:"%_MatchRegex1%"
/C:"%_MatchRegex2%"
"%%A"
^| FindStr /I /R /V
/C:"%_ExcludeRegex%"
') DO (
IF NOT DEFINED !_CurrentFile! (
SET "!_CurrentFile!=Found"
ECHO.File Name: !_FileName!
)
ECHO.%%a %%b
)
)>> "_OutputFile"
PAUSE
I will have to come back through and write up exactly what is being done in more detail if needed.
Essentially I am just using a loop of a DIR command to find all of the .txt files in all directories and subdirectories under the given path.
Then I parse that pull out the file name portion and create a variable I can use to match if the file has already been parsed before so that we only print the file name once.
I don't want to print file names at all unless a match was made because I don't think you care to see long lists of files with no matches, but if you prefer that the logic to test if the file name was found previously can be removed and you can just spit out the file name in the first loop
The second loop is looping the results of running a FindStr command on the given file found in the first loop and matching the Regexes which should mean at least two entries per file more if there are more procedures.
FindStr allows sRegex Matches and /V means to exclude any string that matches and pass the rest.

Related

Batch Rename Double Letter Folder Problems

Ok so I'm super close to doing what I need to do.
I'm having an issue with my rename command and a double letter at the end of the folder. The folder names in the code have been changed for privacy, Spaces have been kept to show how the folders would be named.
The double letter is uppercase I (eye), this can't be changed.
Yes this file exists.
Example:
FolderII - error: The system cannot find the path specified.
Folder - Works
FolderI - works
for /r "C:\Folder Name" %%a in (*) do if "%%~nxa"=="FileFound" set p=%%~dpnxa
for /f "usebackq tokens=1* delims=." %%A in ("%p%") do set Build=%%B
for /f "tokens=2 delims==" %%G in ('wmic os get localdatetime /value') do set datetime=%%G
for /f "tokens=3 delims=\" %%Z in ("%p%") do set filepath=%%Z
set year=%datetime:~0,4%
set month=%datetime:~4,2%
set day=%datetime:~6,2%
set dbname=Logdb%year%.%Build%
REN "C:\Folder Name\%filepath%\%dbname%" "Logdb%month%-%day%-%year%.%Build%"
OUTPUT
EDITED!
CMD>REN "C:\Folder Name\FolderII\Logdb2020.ext" "Logdb11-23-2020.ext"
The system cannot find the file specified.
Added
CMD>REN "C:\Folder Name\Folder\Logdb2020.ext" "Logdb11-23-2020.ext"
THIS works
EDIT FOR CLARIFICATION*
I'll explain this how I intended it to work, which it does as long as the folder it's being assigned to doesn't have a II in it.
1st line: Search this particular folder for a file called "SYSCON" no extension, once found assign to p the file path of the file for 2nd line
2nd line:Open file found at 1st line and get the extension of the file listed inside the file and assign it to Build
3rd line:Get the current date to assign to the new file name in REN
4th line:Use the file path found in line 1 to get the folder name for the REN
5-7 set date variables
Line 8:Assign the new file name to variable
Line 9:Rename the old file at the location found to the new file name generated
I'm not a batch developer, I've literally written these lines as they work for me, but I'm always willing to learn how to do better, I'm a PHP programmer. This is a different project.
The folder structure is fluid for the application. The reason for the search for the initial file is to find the file in 1 of 4 folders and then get that actual folder name.
I can echo all the variables and see the correct file path, the correct file name and the correct new file name.
When it comes to rename the file in the folder with II, it fails to find the actual file to do the rename on, that's where I'm stuck.
IMAGE of Output echoed as it steps through the lines, for privacy sake I have to change the file names. Here's the CMD output for, I hope, better understanding
I'm not positive, based upon your lack of specific information, but as a best guess, I'd assume that something like this should perform the task, I think your example is trying to achieve.
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
For /F "Tokens=1-3 Delims=/ " %%G In (
'""%__AppDir__%Robocopy.exe" \: . /NJH /L | "%__AppDir__%find.exe" " 123""'
) Do Set "YYYY=%%G" & Set "MM=%%H" & Set "DD=%%I"
For /D %%G In (C:\Folder Name\*) Do For %%H In ("%%G\SYSCON"
) Do If "%%~aH" Lss "d" If "%%~aH" GEq "-" (
For /F "UseBackQ Tokens=1,* Delims=." %%I In ("%%H") Do Set "Build=%%J"
SetLocal EnableDelayedExpansion
Ren "%%G\Logdb%YYYY%.!Build!" "Logdb%MM%-%DD%-%YYYY%.!Build!"
EndLocal)
The example above expects the the string you're using for the Build variable is on the last non empty line of the target file, (ASCII text with CRLF line endings). If it is the only non empty line in that target file, then perhaps the following would be more useful:
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
For /F "Tokens=1-3 Delims=/ " %%G In (
'""%__AppDir__%Robocopy.exe" \: . /NJH /L | "%__AppDir__%find.exe" " 123""'
) Do Set "YYYY=%%G" & Set "MM=%%H" & Set "DD=%%I"
For /D %%G In (C:\Folder Name\*) Do For %%H In ("%%G\SYSCON"
) Do If "%%~aH" Lss "d" If "%%~aH" GEq "-" (
For /F "UseBackQ Tokens=1,* Delims=." %%I In ("%%H"
) Do Ren "%%G\Logdb%YYYY%.%%J" "Logdb%MM%-%DD%-%YYYY%.%%J")
It would seem that the data assigned to build contains trailing spaces and perhaps some invisible characters. The easy way would be to simply change 1* to 1,2.
Since Space is a default delimiter, %%B will be assigned the value between the first and second spaces on the line. Tough if you want spaces in the extension, but do you really want to use extensions with spaces?
The syntax SET "var=value" (where value may be empty; in which case var becomes undefined) is used to ensure that any stray trailing spaces are NOT included in the value assigned.

Batch File - Create Zip file with Command Line using the first part of the first file name

I have a folder of PDF files that have a consistent naming convention. I want to create a zip file of these PDF files but named the zip file using the portion of the file that is before the # -- all of the files are the same in the front (it is the NTID of the user that created the pdf files).
As an example these are what the files might look like in the PDF output folder (there could be 100 files all that start with the same UserID before the #:
UserID#Carlos+Alberto+Mafra-+bribery-2019-05-16
UserID#MAJELA+HOSPITALAR+LTDA-+bribery-2019-05-16
(Ideally, I would also want the current date appended to the zip file)
The zip should be called UserID-2019-05-16.zip based on the example above.
This is the code I am trying to use but not having success...
I created a batch script using others suggestions for each step. but can't get it to work end to end.
FOR %%F IN ("C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\*.pdf") DO (
set filename=%%F
goto next
)
:next
echo "%filename%"
set zipfile=%filename%
for /f "tokens=1 delims=#" %%a in ("%zipfile%") do (
)
cd "C:\Program Files\7-Zip\"
7z.exe" a "C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\" && %zipfile% && ".zip" "C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\*.pdf"
One zip file with all the PDFs that are using the first part of the string from the file names in the PDF folder.
#ECHO OFF
SETLOCAL enabledelayedexpansion
SET "sourcedir=U:\sourcedir\t w o"
SET "destdir=U:\destdir"
FOR /f "tokens=1*delims=#" %%a IN (
'dir /b /a-d "%sourcedir%\*#*-????-??-??.pdf" '
) DO (
SET "pre=%%~a"
SET "post=%%~nb"
SET "post=!post:~-10!"
IF DEFINED post ECHO "C:\Program Files\7-Zip\7z" a "%destdir%\!pre!-!post!.zip" "%sourcedir%\%%~a#%%~b"
IF NOT DEFINED post ECHO SKIP "%%~a#%%~b"
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
I used the variablenames pre and post to ensure that the names used in this process are not keywords like the more logical date.
Read a list of all filenames matching the pattern *#*-????-??-??.pdf in the source directory, tokenising on #. Assign the userid to %%a and thence to pre and the "name" part of the dregs of the actual filename to post, then select only the last 10 characters of post using delayed-expansion.
There is an opportunity here to further process post to check whether it truly fits the pattern for a date, if that is required. That routine may return post either unmolested or empty. If it's not empty then construct the required 7z command (you may wish to add -tzip) and echo this for verification - remove the echo to actuate the 7z compression. If post is emptied by a pattern-checking routine, then the filename will simply be reported as having been skipped.
If it is indeed current date you want to append to the end of the zip file, then we need to get the non locale dependent date and time. This will then copy each of the pdf files that starts with UserID to a zip with a date of the day you run the script UserID-2019-05-17 :
#echo off
set "outDir=C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\"
set "inDir=C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\"
for /f "tokens=1,2 delims==" %%i in ('wmic os get LocalDateTime /VALUE') do (
if ".%%i."==".LocalDateTime." set mydate=%%j
)
set mydate=%mydate:~0,4%-%mydate:~4,2%-%mydate:~6,2%
for %%a in (*.pdf) do for /f "delims=#" %%i in ('dir /b /a-d %%a') do (
"C:\Program Files\7-Zip\7z" a "%outDir%%%i-%mydate%.zip" "%inDir%%%~a"
)
If in fact you want to append the date of the filename instead (In other words create Zip files for each file with a different date as well as matching userid):
#echo off
setlocal enabledelayedexpansion
set "outDir=C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\"
set "inDir=C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\"
for %%a in (*.pdf) do for /f "tokens=1,* delims=#" %%i in ('dir /b /a-d %%a') do (
set fdate=%%~nj
set fdate=!fdate:~-10!
echo "C:\Program Files\7-Zip\7z" a "%outDir%%%i-!fdate!.zip "%inDir%%%~a"
)

batch file - searching by character count

I would like to create a batch file that will search in a dir for all .pdf files that have a name of 10 characters as we have many .pdf's with different characters in name so I need to sort them out and move (cut and paste) them to a 2nd directory that is prepared. Can you please help me with this batch file?
example
setdir test contain .pdfs
--+6570296402-1-982464371-120.pdf
+6581239585-1-982470028-120.pdf
5710101306.pdf
0-PZ-6562825.pdf
0-PZ-545515247-1-982466351-120.pdf
5455152471.pdf
result:
target dir - test2 - where need to be moved .pdf with 10 characters
5710101306.pdf
5455152471.pdf
etc
Thank you so much
Running from the current directory you could probably do this using Where and Move:
#Echo Off
For /F "Delims=" %%A In ('Where/F .:??????????.pdf'
) Do Move /Y %%A "Test2">Nul
(for /f "delims=" %%a in ('dir /b /a-d *.pdf') do call :select10 "%%a") >filename.txt
... more processing if required
goto :eof
:select10
set "name=%~n1"
set "name=%name:~9%"
if not defined name goto :eof
set "name=%name:~1%"
if not defined name echo %~1
goto :eof
This should solve the problem.
perform a dir list of *.pdf, selecting filenames only. Pass the filename found to subroutine :select10, in quotes in case of spaces in filename.
The subroutine set name first to the name part of the filename received, then removes the first 9 characters. If the result is an empty variable, skip to end-of-file. If not, select all but the first character. If the result is not an empty string, the name must be 11 or more characters - if it's empty, then echo the name passed in the first instance.
The parentheses around the for command will cause the echoed data to be accumulated into the file nominated.
If you want to move the file to the destination, not simply list the selections, remove the ( before the for, and the ) >filename.txt after and replace the echo with move "%1" destination\
You could also do the same without using a subroutine as:
for /f %%a in ('dir /b /a-d *.pdf') do (
set "name=%%~na"
setlocal enabledelayedexpansion
set "name=!name:~9!"
if defined name (
set "name=!name:~1!"
if not defined name move "%%a" destination\
)
endlocal
)
Using delayed expansion to process the substringing operations.

Batch file to Move files based on part of filename, to folder based on part of folder name

Found a pyhton solution here, but I need a batch file-based solution.
Have many files:
SSP4325_blah-blah-blah.xml
JKP7645_blah.xml
YTG6457-blah-blah.xml
And folder names that contain a piece of the file name:
RefID - SSP4325, JKP7645, GHT1278, YRR0023
RefID - YTG6457
I'm looking for a batch solution which would read a portion of the file name at the front (before either the first dash or underscore) and then move that file into the folder where the front of the filename exists as part of the folder name.
So in the above examples, the first two files (SSP4325 and JKP7645) were moved into the first folder because it contained it contained that text as part of the folder name.
The third file would be moved into the second folder.
I have hundreds of files and 63 folders. So I'm hoping to be able to automate.
Can't use Powershell or Python due to limitations of the environment. So hoping for a batch file approach.
Thanks. Sean.
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\*.xml" '
) DO (
FOR /f "tokens=1delims=_-" %%b IN ("%%a") DO (
FOR /f "delims=" %%d IN (
'dir /b /ad "%destdir%\*%%b*" '
) DO (
ECHO(MOVE "%%a" "%destdir%\%%d\"
)
)
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
The required MOVE commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(MOVE to MOVE to actually move the files. Append >nul to suppress report messages (eg. 1 file moved)
After establishing the directories, the outer loop puts the filename in %%a, the next loop gets the first part of that name, up to but not including the first - or _ (the delims specified) into %%b.
The inner loop finds the target directory containng %%b in the destination directory and constructs an appropriate move line.
This solution review the folders just one time and store they in an array, so this method should run faster.
#echo off
setlocal EnableDelayedExpansion
rem Process the folders
set i=0
for /D %%a in (*) do (
rem Store this folder in the next array element
set /A i+=1
set "folder[!i!]=%%a"
rem Separate folder in parts and store the number of the array element in each one
for %%b in (%%a) do set "part[%%b]=!i!"
)
rem Process the files
for %%a in (*.xml) do (
rem Get the first part of name
for /F "delims=-_" %%b in ("%%a") do (
rem If such a folder exists...
if defined part[%%b] (
rem Get the number of the corresponding array element and move the file
for %%n in (!part[%%b]!) do ECHO move "%%a" "!folder[%%n]!"
) else (
echo No folder exists for this file: "%%a"
)
)
)
This method have also several advantages: you may check if a certain folder does not exists, or get the number of files moved to each folder, etc. If you are not interested in these points, just remove the if command and make the code simpler...
An explanation of array management in Batch files is given at this answer.

Batch: search for files with certain extension, owner, fullpath and last write access and output in CSV

I'm trying to create a CSV with fullpath\filename, file owner and last write access (modification date) of all txt and html files from all hard drives of a data server.
Here's what I got so far:
set pgm=%~n0
set log=%~dpn0.log
set host=%COMPUTERNAME%
set csv=%host%.csv
set dir=D:\BME
if not exist "%csv%" type nul>"%csv%"
for /f "delims=;" %%a in ('dir /b/s %dir%\*.txt, %dir%\*.html') do (
>>%csv% echo "%%a"
)
That outputs the path + filename of all found txt and html files of a certain folder in a CSV. I tried this command to get the hard drives:
wmic logicaldisk where drivetype=3 get caption
But I can't get my head around how to store that in a variable or file and loop through it and also retrieve the owner and last modification date and put it into a new column of the csv file.
I can't get my head around how to store that in a variable
Use the following batch file.
GetDrives.cmd:
#echo off
setlocal enabledelayedexpansion
rem skip=1 to remove the header
rem findstr to remove blank lines
for /f "skip=1" %%d in ('wmic logicaldisk where drivetype^=3 get caption ^| findstr /r /v "^$"') do (
set _drive=%%d
echo !_drive!
)
endlocal
Notes:
Be careful when using drivetype=3 as I have a removable drive of type 3. In the below output C: is a fixed hard disk and F: is a removable external USB drive.
Replace echo !_drive! as appropriate with a modified version of your existing code.
Example Output:
F:\test>GetDrives
C:
F:
F:\test>
Further Reading
An A-Z Index of the Windows CMD command line - An excellent reference for all things Windows cmd line related.
enabledelayedexpansion - Delayed Expansion will cause variables to be expanded at execution time rather than at parse time.
for /f - Loop command against the results of another command.
wmic - Windows Management Instrumentation Command.
DavidPostill answered how-to store wmic logicaldisk … output in a variable;
to retrieve file last modification date: use echo "%%a","%%~ta" in your script using %~t Parameter Extension;
to retrieve file owner: echo "%%a","%%~ta","!_owner!" where _owner variable comes from getRealOwner subroutine based on modified schletti2000's answer Get ownership information from command line by using wmic.
The script:
#ECHO OFF
SETLOCAL EnableExtensions DisableDelayedExpansion
set "pgm=%~n0"
set "log=%~dpn0.log"
set "host=%COMPUTERNAME%"
set "csv=%host%.csv"
set "dir=D:\BME"
set "dirmask=%dir%\*.txt, %dir%\*.html"
rem if not exist "%csv%" type nul>"%csv%"
>"%csv%" (
for /f "delims=;" %%a in ('dir /b/s %dirmask% 2^>NUL') do (
set "_fFullPath=%%~a"
set "_fLastWrite=%%~ta"
set "_fOwner="
call :getRealOwner
SETLOCAL EnableDelayedExpansion
echo "!_fFullPath!","!_fOwner!","!_fLastWrite!"
ENDLOCAL
)
)
)
type "%csv%"
goto :continue
:getRealOwner
SET "ESCAPED=%_fFullPath:\=\\%"
SET "UNDELIMITED="
for /F "skip=2 delims=" %%g in ('
wmic path Win32_LogicalFileSecuritySetting where Path^="%ESCAPED%" ^
ASSOC /RESULTROLE:Owner /ASSOCCLASS:Win32_LogicalFileOwner ^
/RESULTCLASS:Win32_SID 2^>NUL
') do (
SET "UNDELIMITED=%%g"
call :process_wmioutput
)
if NOT defined UNDELIMITED set "_fOwner=???"
exit /B
:process_wmioutput
SET "DELIMITED=%UNDELIMITED: =•%"
FOR /F "delims=• tokens=10,12" %%G in ("%DELIMITED%") DO set "_fOwner=%%H\%%G"
exit /B
:continue
I used next settings to demonstrate various output:
set "dir=D:"
set "dirmask=%dir%\loc*.vbs %dir%\bcd*.log %dir%\act*.xsl %dir%\diag*.xml %dir%\chec*.csv"
Output - non-privileged cmd window:
==> D:\bat\SO\39034430.bat
"D:\odds and ends\tempx\links\testDJ\LocUsers.vbs","mypc\user","25.12.2014 00:13"
"D:\tempWin\ActivityLog.xsl","NT AUTHORITY\SYSTEM","24.02.2016 13:12"
"D:\tempWin\CompatTelemetryLogs\diagerr.xml","???","12.08.2015 03:17"
"D:\tempWin\CompatTelemetryLogs\diagwrn.xml","???","12.08.2015 03:17"
"D:\test\check_acl.csv","BUILTIN\Administrators","06.03.2016 14:28"
Output - privileged (run as administrator) cmd window:
=ADMIN=> D:\bat\SO\39034430.bat
"D:\odds and ends\tempx\links\testDJ\LocUsers.vbs","mypc\user","25.12.2014 00:13"
"D:\tempWin\ActivityLog.xsl","NT AUTHORITY\SYSTEM","24.02.2016 13:12"
"D:\tempWin\CompatTelemetryLogs\diagerr.xml","NT AUTHORITY\SYSTEM","12.08.2015 03:17"
"D:\tempWin\CompatTelemetryLogs\diagwrn.xml","NT AUTHORITY\SYSTEM","12.08.2015 03:17"
"D:\test\check_acl.csv","BUILTIN\Administrators","06.03.2016 14:28"

Resources