I am writing a batch program for controling my movie archive (Personel usage). This is what i am trying to do for copying folders.
:_Kopya
set "TRGT=%~1" & set "KPY-GLN[1]=%~2" & set "KPY-GLN[2]=%~3" & set "KPY-GLN[3]=%~4"
REM Checking user input and defining variables.
for /l %%s in (1,1,3) do (
if DEFINED KPY-GLN[%%s] (
for /f "tokens=1-2 delims=:" %%a in ("!KPY-GLN[%%s]!") do (
call :_Kontrol "%%a" "%%b" "" "" "aaaaa[%%s]" "bbbbb[%%s]" "" ""
if "!TEST!"=="0" goto :EOF
)
)
)
REM Copying folders.
for /l %%s in (1,1,3) do (
if NOT DEFINED bbbbb[%%s] set bbbbb[%%s]=!aaaaa[%%s]!
for /l %%a in (!aaaaa[%%s]!,1,!bbbbb[%%s]!) do (
call :_ReadLine "%MURL%" "%%a" "LINE"
if EXIST "!TRGT!\!LINE:~20!" rd /s /q !TRGT!\!LINE:~20!
robocopy /s /e "!LINE!" "!TRGT!\!LINE:~20!" >NUL 2>&1
)
)
goto :EOF
And this is the way i call,
call :_Kopya "C:\" "123:125" "124:130" "125"
Which means copy the file numbers from 123 to 125 and from 124 to 130 and 125.
It works fine but there is a problem i want to solve. When i call this function the way i show its copying file number 124 2 times and file number 125 3 times. How can i fix this issue?
PS1: %MURL% its a text file and contains local address of those files. Its something like this M:\Movies\000y.001y\The.Lord.of.the.Rings.The.Return.of.the.King.(2003){0167260}[00087]
PS2: :_ReadLine its a function that reading specific line and adding value of this line to LINE variable.
#ECHO Off
SETLOCAL
:: remove variables starting $
FOR /F "delims==" %%a In ('set $ 2^>Nul') DO SET "%%a="
:: Parameters are adirectory range*
:: where range may be a single number or start:finish
SET "directory=%~1"
:loop
SHIFT
IF "%~1"=="" GOTO :eof
FOR /f "tokens=1,2delims=:" %%a IN ("%~1") DO (
IF "%%b"=="" (CALL :kopythis %%a) ELSE (FOR /L %%c IN (%%a,1,%%b) DO CALL :kopythis %%c)
)
GOTO loop
GOTO :EOF
:kopythis
IF DEFINED $%1 GOTO :EOF
SET $%1=Y
ECHO(COPY whatever with parameters %directory% and %1
GOTO :eof
This should do what you seem to need. I'll leave you to work out the details of how to structure whatever copy mechanism you need from the parameters provided.
Note that with this approach, quoting the parameters is optional with the obvious exception of the first when it's optional if the first doesn't contain separators. It also allows any number of range parameters.
Related
#echo off
ren "(1) Rare Imports Exclusive.txt" "1RareImportsExclusive.txt"
set "replace="unid": 0,"
set "replaced="unid": 1,"
set "source=1RareImportsExclusive.txt"
set "target=1RareImportsExclusive1.txt"
setlocal enableDelayedExpansion
(
for /F "tokens=1* delims=:" %%a in ('findstr /N "^" %source%') do (
set "line=%%b"
if defined line set "line=!line:%replace%=%replaced%!"
echo(!line!
)
) > %target%
ren "1RareImportsExclusive1.txt" "(1) Rare Imports Exclusive.txt"
del "1RareImportsExclusive.txt"
endlocal
I have update my Original Post
I have over 1000 .txt files, and in the content of each file has a UNID code
with random numbers
My goal is to have 1 bat file that can ignore all random numbers and
rename the unid based on the file number
Original files
(1) Rare Imports Exclusive.txt
------- "unid": 2548,"
Outcome files
(1) Rare Imports Exclusive.txt
------- "unid": 1,"
I can only get this to work if I change all unid to 0,
and if I make 1000 .bat files
So I make as many batch files and merge them
This works for now
#ECHO Off
setlocal ENABLEDELAYEDEXPANSION
set "replacemain="unid": "
SET "sourcedir=u:\sourcedir\t w o"
SET "destdir=u:\destdir"
FOR /f "delims=" %%q IN ('dir /b /a-d "%sourcedir%\(*)*.txt"') DO (
(
FOR /f "tokens=1 delims=()" %%j IN ("%%q") DO (
rem %%j now has sequence number
for /F "tokens=1* delims=:" %%a in ('findstr /N "^" "%sourcedir%\%%q"') do (
set "line=%%b"
if defined line IF "%%b" neq "!line:unid=!" CALL :subs %%j
echo(!line!
)
)
)>"%destdir%\%%q"
)
GOTO :eof
:: substitute
:subs
SET "original=%line%"
FOR /L %%s IN (0,1,9) DO set "line=!line:%replacemain%%%s=%replacemain%!"
IF "%original%" neq "%line%" goto subs
set "line=!line:%replacemain%=%replacemain%%1!"
GOTO :eof
As I read it, this problem has nothing to do with renaming files, but changing their contents.
You would need to change the settings of sourcedir and destdir to suit your circumstances. The listing uses a setting that suits my system.
I deliberately include spaces in names to ensure that they are processed correctly.
So, assuming that the filenames in the source directory follow the pattern (?) filename.txt and the required work is to replace the string "unid": ##, with "unid": ?, (where ## is a random number and ? is the number from the filename...
Grab the filenames to %%q and derive the filenumber to %%j.
Read and regurgitate. Detect whether the line contains unid etc. and further process these using :subs.
In :subs, replace the unid,etc. string followed by a digit by the string without the digit. If any change was made, repeat. Then substitute %%j passed as %1 to the :subs routine.
--- Revision for including subdirectories.
#ECHO Off
setlocal ENABLEDELAYEDEXPANSION
:: Significant part of string
set "replacemain="unid": "
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
FOR /f "delims=" %%q IN ('dir /b /s /a-d "%sourcedir%\(*)*.txt"') DO (
rem calculate new destination directory
SET "newdest=%%~dpq"
SET "newdest=!newdest:%sourcedir%=%destdir%!"
SET "newdest=!newdest:~0,-1!"
MD "!newdest!" 2>nul
(
FOR /f "tokens=1 delims=()" %%j IN ("%%~nxq") DO (
rem %%j now has sequence number
for /F "tokens=1* delims=:" %%a in ('findstr /N "^" "%%q"') do (
set "line=%%b"
if defined line IF "%%b" neq "!line:unid=!" CALL :subs %%j
echo(!line!
)
)
)>"!newdest!\%%~nxq"
)
GOTO :eof
:: substitute
:subs
SET "original=%line%"
FOR /L %%s IN (0,1,9) DO set "line=!line:%replacemain%%%s=%replacemain%!"
IF "%original%" neq "%line%" goto subs
set "line=!line:%replacemain%=%replacemain%%1!"
GOTO :eof
You would need to change the values assigned to sourcedir and destdir to suit your circumstances. The listing uses a setting that suits my system.
I deliberately include spaces in names to ensure that the spaces are processed correctly.
First, addition of /s in the dir command to cause scanning of subdirectories. This means that %%q will contain the absolute filename of each file that fits the mask (*)*.txt.
Since the destination directory may now change, newdest is calculated by replacing the value of sourcedir with the value of destdir and removing the final \ character since %%~dpq will deliver a \-terminated string.
The directory thus calculated is then created with a md instruction, and any objection from the system (as it's likely the directory already exists) is discarded with 2>nul (send error messages nowhere).
Since %%q originally contained the filename only, we need to specifically select the name and extension of %%q (%%~nxq) when calculating %%j.
Since %%q contains the absolute filename, we remove the %sourcedir% from the file-read loop using %%a.
Finally, the destination file needs to be constructed from the calculated newdest and the name+extension of the sourcefile.
--- Re-revision
#ECHO Off
setlocal ENABLEDELAYEDEXPANSION
:: Significant part of string
set "replacemain="unid": "
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
:: for tracking directory-changes
SET "lastdir="
SET /a serial=1
FOR /f "delims=" %%q IN ('dir /b /s /a-d "%sourcedir%\(*)*.txt"') DO (
rem use following line to restart serial number for each file
REM SET /a serial=1
if "!lastdir!" neq "%%~dpq" (
rem use following line to restart serial number for each directory
REM SET /a serial=1
rem calculate new destination directory
SET "newdest=%%~dpq"
SET "lastdir=%%~dpq"
SET "newdest=!newdest:%sourcedir%=%destdir%!"
SET "newdest=!newdest:~0,-1!"
MD "!newdest!" 2>NUL
)
(
rem %%j will acquire sequence number for filename pattern (*)*.txt
REM FOR /f "tokens=1 delims=()" %%j IN ("%%~nxq") DO SET "serial=%%j"
rem use following line to select a random serial number for each file
REM CALL :randsn
for /F "tokens=1* delims=:" %%a in ('findstr /N "^" "%%q"') do (
set "line=%%b"
if defined line IF "%%b" neq "!line:unid=!" CALL :subs
echo(!line!
)
)>"!newdest!\%%~nxq"
)
GOTO :eof
:: substitute
:subs
SET "original=%line%"
FOR /L %%s IN (0,1,9) DO set "line=!line:%replacemain%%%s=%replacemain%!"
IF "%original%" neq "%line%" goto subs
set "line=!line:%replacemain%=%replacemain%%serial%!"
SET /a serial+=1
GOTO :eof
:randsn
SET /a serial=%RANDOM%
:: Ignore selections 30000..32767 for even distribution
IF %serial% geq 30000 GOTO randsn
:: Use 10000 for 4-digit maximum, 1000 for 3, 100 for 2
SET /a serial=serial %% 10000
:: Use the following line if 0 is not allowed
IF %serial%==0 GOTO randsn
GOTO :eof
This provides a choice of how the number is calculated.
To activate any particular option, change the REM in the line following the comment to (nothing). It would probably be insane to try to activate more than one option at any one time. To deselect an option, put the REM keyword back in place.
The random option will generate a random 1-4-digit number. There is absolutely no guarantee whatever that the numbers inserted in this case will be unique.
To allow the routine to operate on any file in the tree, simply change the filemask (*)*.txt to some other filemask, noting that the derivation of sequence number for filenames that do not start (number) via %%j would then become nonsense, so the setting of serial from %%j should be de-activated with a rem statement (as posted)
Not able to substring the dynamic variable inside forloop in Windows batch script.
I have the properties file in my git hub in the below format.
"collectionName=TestCollectionRun.json=test"
So I have written the below code to fetch this values.But the requirement is that I need to strip of the '.json' part from collection name.With the below code I am not able to set/echo that value.Can you please help on this!
#ECHO ON
:BEGIN
IF EXIST "test.properties" ECHO Found properties file, reading file..
SET props=test.properties
setlocal EnableDelayedExpansion
For /F "delims== tokens=1,2,3" %%G in (%props%) Do (
if "%%I" EQU "test" if "%%G" EQU "collectionName" SET collName=%%H(
SET finalCollName=%collName%:~0,-5
ECHO %finalCollName%
)
)
:END
We need the ECHO to return "TestCollectionRun".currently its not returning anything.
For /F "delims== tokens=1,2,3" %%G in (%props%) Do (
if "%%I" EQU "test" if "%%G" EQU "collectionName" SET "collName=%%~nH"&echo %%~nH
)
ECHO %CollName%
Note the second ) is now redundant. Your problem has to do with delayedexpansion which you are invoking but not using. call %%collname%% within the for loop would have shown the value after assignment if required.
This code works by interpreting %%H as a filename and assigning simply the name part of %%H (%%~nH)
Given a line content of collectionName=TestCollectionRun.json=test, here's a quick rewrite of what I think you're tring to do:
#Echo Off
Set "props=test.properties"
If Not Exist "%props%" (
Echo Properties file not found!
Echo Exiting..
Timeout /T 3 /NoBreak >NUL
Exit /B
)
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" Echo %%~nB
)
Pause
If you wanted to do something with the collection name within the loop then you would probably need to use delayed expansion:
#Echo Off
SetLocal DisableDelayedExpansion
Set "props=test.properties"
If Not Exist "%props%" (
Echo Properties file not found!
Echo Exiting..
Timeout /T 3 /NoBreak >NUL
Exit /B
)
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" (
Set "collName=%%B"
SetLocalEnableDelayedExpansion
Echo !collName!
Rem Perform substring task on the variable named collName
Set "finalCollName=!collName%:~0,-5!"
Echo !finalCollName!
EndLocal
)
)
Pause
Note, these answers will not work, as is, if your string is surrounded by doublequotes, (as in your question body), or if the line content differs (e.g. begins with spaces or tabs).
[Edit /]
Looking at your 'after the fact' question in the comments, it is clear that you do not need to substring the variable at all, so should use the first method posted:
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" (
newman run "%%B" -e "%envName%" --insecure --reporters cli,htmlextra --reporter-htmlextra-export "newman\%BUILD_NUMBER%\%%~nB.html" --disable-unicode
)
)
Pause
This assumes that both %envName% and %BUILD_NUMBER% have been previously defined correctly.
this script sorts files into known subdirectories fast, by using database files filled with scored keywords. it works for the most part but sometimes it gives me errors that I can't figure out because it acts differently when the echo is turned on, and fails. this code is very important to me so this is a huge problem for me. the echo issue has to be fixed or it's unmanageable, no other possible problems need be addressed in this question! please help, it's largely beyond my understanding.
:: sorts files from %cd% into known subdirs according to database of scored keywords. minimum score is 2.0
rem todo if echo is on it fails with certain keywords like a simple *1x0* which is super weird
rem sometimes outputs unlocated error missing operator while still doing its job correctly
rem sometimes outputs unlocated error the system cannot fing the path specified
rem keywords or subdir names cannot contain spaces
rem fails with filenames with irregular dash characters
rem Adds * to start and end of keyword. start/end wildcards should be used surgically in the keywords file instead
dir *.* /a-d >nul 2>nul || exit /b
set "tempfile=%temp%\sortables"
set "sourcedir=%~1"
setlocal enabledelayedexpansion
rem set datafile, categories according to current location
set "categories="
if /i "%cd%"=="d:\videos" (
set "datafile=videos"
set "categories=series porno"
)
if /i "%cd%"=="d:\videos\movies" (
set "datafile=movies"
set "categories=features psychedelic pornography concerts standup featurettes documentaries"
)
if /i "%cd%"=="d:\videos\movies\documentaries" (
set "datafile=docu"
set "categories=1-scarcity 2-globalists 3-disinformation 4-agendas 5-abundance"
)
if /i "%cd%"=="d:\videos\movies\features" (
set "datafile=films"
set "categories=comedy drama action thriller venture crime horror mystery fantasy science western warfare"
)
if /i "%cd%"=="d:\videos\series" (
set "datafile=series"
set "categories=comedy stories reality trippy"
)
if /i "%cd%"=="d:\videos\series\comedy" (
set "datafile=comedy"
set "categories=cartoon classic modern reality sketch standup"
)
if /i "%cd%"=="d:\videos\series\pilots" (
set "datafile=pilots"
set "categories=reality drama comedy scifi fantasy crime mystery action thriller"
)
if /i "%cd%"=="d:\videos\shorts" (
set "datafile=shorts"
set "categories=psychedelic entertaining music media useful conspiracies"
)
if /i "%cd%"=="d:\videos\shorts\media" (
set "datafile=media"
set "categories=trailers games fandom extras facts analysis features"
)
if /i "%cd%"=="d:\videos\shorts\music" (
set "datafile=music"
set "categories=bigbeat classical clubbing country electro swing reggae dub experimental geeky metal rap rock synthwave triphop xxx"
)
if not defined categories exit /b
set database=d:\system\scripts\%datafile%.txt
if not exist "%database%" echo critical error: database %datafile%.txt doesn't exist && exit /b
if defined verbal echo sorting "%cd%"
rem =============================================================================================================================
rem setup sorting categories (do not change anything lightly or without backup after this point)
rem =============================================================================================================================
set "sortingcategories="
for %%a in (%categories%) do set "sortingcategories=!sortingcategories!,%%~a"
set "sortingcategories=%sortingcategories: =_%"
rem =============================================================================================================================
rem create tempfile containing lines of: name|sortingcategory|weight
rem =============================================================================================================================
(
for /f "tokens=1,2,*delims=," %%s in (%database%) do (
set "sortingcategory=%%s"
set "sortingcategory=!sortingcategory: =_!"
for /f "delims=" %%a in (
'dir /b /a-d "%sourcedir%\*%%u*" 2^>nul'
) do (
echo %%a^|!sortingcategory!^|%%t^|%%s^|%%u
)
)
)>"%tempfile%"
type "%tempfile%" >>d:\system\scripts\sorter.log
rem =============================================================================================================================
rem reset and call processing for each file in tempfile + dummy (helps counting the last score?)
rem =============================================================================================================================
set "lastname="
echo off
for /f "tokens=1,2,3,*delims=|" %%a in ('sort "%tempfile%"') do call :resolve %%b %%c "%%a"
call :resolve dummy 0
rem declare failures
if defined verbal if not "%datafile%"=="videos" if not "%datafile%"=="music" if not "%datafile%"=="media" (
dir "%~1\*" /a-d >nul 2>nul && for /f "delims=" %%q in ('dir %1 /b /a-d') do echo unsortable in %datafile% "%%q"
)
exit /b
:resolve
IF "%~3" equ "%lastname%" GOTO accum
rem report and reset accumulators
IF NOT DEFINED lastname GOTO RESET
SET "winner=none"
SET /a maxfound=1
FOR %%v IN (%sortingcategories%) DO (
FOR /f "tokens=1,2delims=$=" %%w IN ('set $%%v') DO IF %%x gtr !maxfound! (SET "winner=%%v"&SET /a maxfound=%%x)
)
if "%winner%"=="none" goto reset
SET "winner=%winner:_= %"
SET "lastname=%lastname:&=and%"
rem this has a problem with different type of dash -
rem this once overwrote a same-name, much smaller file, wtf?
if "%winner%"=="porno" move "%sourcedir%\%lastname%" "d:\nvidia\" >nul && echo "d:\nvidia\%lastname%"
if not "%winner%"=="porno" move "%sourcedir%\%lastname%" "%sourcedir%\%winner%\" >nul && echo "%sourcedir%\%winner%\%lastname%"
if "%winner%"=="features" if exist "%sourcedir%\%lastname%" move "%sourcedir%\%lastname%" "%sourcedir%\%winner%\" >nul && echo "%sourcedir%\%winner%\%lastname%"
rem before or after successful filing we could do a surgical dupe check for only that file, rendering the old style obsolete
:RESET
FOR %%v IN (%sortingcategories%) DO SET /a $%%v=0
SET "lastname=%~3"
:accum
SET /a $%1+=%2
with echo turned on the error looks like this (note that this works fine when echo is off)
d:\VIDEOS>for /F "tokens=1,2,3,*delims=|" %a in ('sort "E:\TEMP\sortables"') do call :resolve %b %c "%a"
d:\VIDEOS>call :resolve !sortingcategory! 2 " for /F "delims=" %a in ('dir /b /a-d "\*1x0*" 2>nul') do (echo %a"
The syntax of the command is incorrect.
d:\VIDEOS>IF " for /F "delims" equ "" GOTO accum
d:\VIDEOS>
and this is the database file used (sorry for the lingo)
series,2,1x0
series,2,1x1
series,2,2x0
series,1,s0
series,1,s1
series,1,s2
series,1,s3
series,1,s4
series,1,s5
series,1,s6
series,1,s7
series,1,s8
series,1,s9
series,-1,s00
series,-1,e00
series,1,e0
series,1,e1
series,1,e2
series,1,e3
series,1,e4
series,1,e5
series,1,e6
series,1,e7
series,1,e8
series,1,e9
series,-1,extras
series,1,judge?judy?s
series,1,pilot
porno,1,amateur
porno,1,wet
porno,1,wmv
porno,1,xxx
missing,0,not appearing in this directory
it takes files from d:\videos and sorts them into d:\videos\series or d:\nvidia (hidden alternative to the other subdir d:\videos\porno)
rem =============================================================================================================================
rem create tempfile containing lines of: name|sortingcategory|weight
rem =============================================================================================================================
(
for /f "tokens=1,2,*delims=," %%s in (%database%) do (
set "sortingcategory=%%s"
set "sortingcategory=!sortingcategory: =_!"
for /f "delims=" %%a in (
'dir /b /a-d "%sourcedir%\*%%u*" 2^>nul'
) do (
echo %%a^|!sortingcategory!^|%%t^|%%s^|%%u
)
)
)>"%tempfile%"
change to
rem =============================================================================================================================
rem create tempfile containing lines of: name|sortingcategory|weight
rem =============================================================================================================================
(
for /f "usebackq tokens=1,2,* delims=," %%s in ("%database%") do (
set "sortingcategory=%%s"
set "sortingcategory=!sortingcategory: =_!"
for /f "delims=" %%a in (
'dir /b /a-d "%sourcedir%\*%%u*" 2^>nul'
) do (
>&3 echo %%a^|!sortingcategory!^|%%t^|%%s^|%%u
)
)
) 3>"%tempfile%"
Prior, echo off setting allows the echoed text
to be redirected to file from stream 1.
When you set echo on, stream 1 includes the commands
being echoed as well as the echoed text.
To avoid this merged output of commands and text,
use an auxiliary stream. CMD supports auxiliary
streams 3 through to 9.
The changed code sets the echoed text to redirect
to the handle of stream 3 (>&3).
The redirection to the file uses stream 3 (3>).
The errors posted is a delayed effect caused by
echoed commands and text written to %tempfile%
when using echo on with your posted code.
Added for loop option usebackq so the %database%
path can be double quoted.
I have a batch to check the duplicate line in TXT file (over one million line) with 13MB, that will be running over 2hr...how can I speed up that? Thank you!!
TXT file
11
22
33
44
.
.
.
44 (over one million line)
Existing Batch
setlocal
set var1=*
sort original.txt>sort.txt
for /f %%a in ('type sort.txt') do (call :run %%a)
goto :end
:run
if %1==%var1% echo %1>>duplicate.txt
set var1=%1
goto :eof
:end
This should be the fastest method using a Batch file:
#echo off
setlocal EnableDelayedExpansion
set var1=*
sort original.txt>sort.txt
(for /f %%a in (sort.txt) do (
if "%%a" == "!var1!" (
echo %%a
) else (
set "var1=%%a"
)
)) >duplicate.txt
This method use findstr command as in aschipfl's answer, but in this case each line and its duplicates are removed from the file after being revised by findstr. This method could be faster if the number of duplicates in the file is high; otherwise it will be slower because the high volume data manipulated in each turn. Just a test may confirm this point...
#echo off
setlocal EnableDelayedExpansion
del duplicate.txt 2>NUL
copy /Y original.txt input.txt > NUL
:nextTurn
for %%a in (input.txt) do if %%~Za equ 0 goto end
< input.txt (
set /P "line="
findstr /X /C:"!line!"
find /V "!line!" > output.txt
) >> duplicate.txt
move /Y output.txt input.txt > NUL
goto nextTurn
:end
#echo off
setlocal enabledelayedexpansion
set var1=*
(
for /f %%a in ('sort q42574625.txt') do (
if "%%a"=="!var1!" echo %%a
set "var1=%%a"
)
)>"u:\q42574625_2.txt"
GOTO :EOF
This may be faster - I don't have your file to test against
I used a file named q42574625.txt containing some dummy data for my testing.
It's not clear whether you want only one instance of a duplicate line or not. Your code would produce 5 "duplicate" lines if there were 6 identical lines in the source file.
Here's a version which will report each duplicated line only once:
#echo off
setlocal enabledelayedexpansion
set var1=*
set var2=*
(
for /f %%a in ('sort q42574625.txt') do (
if "%%a"=="!var1!" IF "!var2!" neq "%%a" echo %%a&SET "var2=%%a"
set "var1=%%a"
)
)>"u:\q42574625.txt"
GOTO :EOF
Supposing you provide the text file as the first command line argument, you could try the following:
#echo off
for /F "usebackq delims=" %%L in ("%~1") do (
for /F "delims=" %%K in ('
findstr /X /C:"%%L" "%~1" ^| find /C /V ""
') do (
if %%K GTR 1 echo %%L
)
)
This returns all duplicate lines, but multiple times each, namely as often as each occurs in the file.
I have a directory that contains close to a million XML files. Needless to say, it takes forever to load (20+ minutes) So, I'm writing a script to divide the files into folders with the top level being year and having months under each year. There are 4 main filenames where the date can be extracted from the 3rd token and the rest from the second token. ie:
BA1253570001_BALMIS_20130617_TRC_0_109506738E.xml
BA1254260001_ACCTV21_20140430_AMR_0_1095611492.xml
BA1736330001_SWFTOUT_20140929_LIQ_1_MTBX553494.xml
BA1739240001_FEDOUT_20140904_LIQ_1_105633316M.xml
The rest are like this:
EODMESS_20140718_MTBX473286.xml
MSGCONF_20140410_109558667V.xml
I'm sure there is an easier way to do it, but here is my code so far:
#echo on
setlocal enabledelayedexpansion
Set "starttime=%time%"
pushd C:\temp\xmls
for /f %%a in ('dir /b/o:d *.xml') do (
call :ExtractDates %%a ret
echo %%a - !ret!
for /f "tokens=1" %%b in ("!ret!") do (
for /f "tokens=1-3 delims=/" %%c in ("%%b") do (
if not exist .\%%e md .\%%e
if not exist .\%%e\%%c md .\%%e\%%c
if %%b equ %%c/%%d/%%e (
echo moving %%~nxa to .\%%e\%%c
echo move %%~nxa .\%%e\%%c
pause
)
)
)
)
echo Start time: %starttime%
echo End time: %time%
popd
exit /b
:ExtractDates
#echo on
setlocal enabledelayedexpansion
Echo Starting ExtractDates
for %%a in (BALMIS ACCTV21 FEDOUT SWFTOUT) do (
if not errorlevel 1 (set t=3) else set t=2
Call :ExtractFunc %~1 %%a !t! ret
endlocal&set "%~2=!ret!"&exit /b 0
)
exit /b
:ExtractFunc
#echo on
setlocal
Echo Starting ExtractFunc
for /f "tokens=%3 delims=_" %%a in (
'echo %~1^|Findstr "%~2"'
) do (
if not errorlevel 1 (
endlocal&set "%~4=%%a"&exit /b 0
)
)
exit /b
The problem is that the variable token isn't returning the right number and I'm not sure why. Any suggestions appreciated.
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir\t w o"
PUSHD "%sourcedir%"
FOR /f "tokens=1*delims=" %%a IN (
'dir /b /a-d "%sourcedir%\*_*_*.xml" '
) DO SET "filename=%%a"&CALL :process
POPD
GOTO :EOF
:process
FOR /f "tokens=2,3,6delims=_" %%m IN ("%filename%") DO SET "date1=%%m"&SET "date2=%%n"&SET "whichdate=%%o"
IF DEFINED whichdate SET "date1=%date2%"
IF NOT DEFINED date2 GOTO :eof
ECHO(MD .\%date1:~0,4%\%date1:~4,2%
ECHO(MOVE "%filename%" .\%date1:~0,4%\%date1:~4,2%\
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
The required MD commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(MD to MD to actually create the directories. Append 2>nul to suppress error messages (eg. when the directory already exists)
The required MOVE commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(MOVE to MOVE to actually move the files. Append >nul to suppress report messages (eg. 1 file moved)
Simply extract the two possible datestrings and use the presence of the sixth token to signal which of the two positions to select for generation of the destination directory. Skip if there's no third token (fails to fit mask specified)
Then select the required field to date1 and do some substringing.