Spinner with update for every 1000 files moved - batch-file

I have the following code for a spinner that I found somewhere long ago. I'm trying to figure out how to modify it so it displays an update for every 1000 files moved. So, it would look like this:
Moving XML Files...| 1,000 Files moved
Moving XML Files.../ 2,000 Files moved
Moving XML Files...- 3,000 Files moved
Moving XML Files...\ 4,000 Files moved
Where the spinner chars continue to move. I'll be running this on close to a million files, so I really need to have an indication of what the status is. Any help or suggestions of a better way is greatly appreciated.
CODE
#echo off
setlocal
Call :SpinnerEx
exit /b
:SpinnerEx
setlocal EnableDelayedExpansion
for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a"
FOR /L %%n in (1,1,50) DO (
call :spinner
ping localhost -n 1 > nul
)
exit /b
:spinner
set /a "spinner=(spinner + 1) %% 4"
set "spinChars=\|/-"
<nul set /p ".=Moving XML Files...!spinChars:~%spinner%,1!!CR!"
exit /b
And HERE is the code to actually do the moving provided by Magoo

Building on Magoo's script, I'd replace the
) DO SET "filename=%%a"&CALL :process
with
) DO (
SET "filename=%%a"&CALL :process
rem increment file counter
rem if total divided by 1000 has no remainder, advance the spinner
)
Something like this:
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir\t w o"
SET "spinChars=\|/-"
for /f %%a in ('"prompt $H&for %%b in (1) do rem"') do set "BS=%%a"
SET "filesmoved=0"
PUSHD "%sourcedir%"
set /P "=Moving XML Files...%spinChars:~0,1% 0 Files moved"<NUL
FOR /f "tokens=1*delims=" %%a IN (
'dir /b /a-d "%sourcedir%\*_*_*.xml" '
) DO (
SET "filename=%%a"&CALL :process
set /a filesmoved += 1, thousand = filesmoved %% 1000
setlocal enabledelayedexpansion
if !thousand! equ 0 call :spinner
endlocal
)
POPD
GOTO :EOF
:process
FOR /f "tokens=2,3,6delims=_" %%m IN ("%filename%") DO SET "date1=%%m"&SET "date2=%%n"&SET "whichdate=%%o"
IF DEFINED whichdate SET "date1=%date2%"
IF NOT DEFINED date2 GOTO :eof
ECHO(MD .\%date1:~0,4%\%date1:~4,2%
ECHO(MOVE "%filename%" .\%date1:~0,4%\%date1:~4,2%\
GOTO :EOF
:spinner
set "moved=%filesmoved%"
:spinner2
if %filesmoved% geq 4000 set /a filesmoved -= 4000 & goto :spinner2
set /a spinpos = filesmoved / 1000
for /L %%I in (1,1,50) do set /P "=%BS%"<NUL
set /P "=Moving XML Files...!spinChars:~%spinPos%,1! %moved% Files moved"<NUL
goto :EOF
The for /f... ("prompt $H...") line captures a backspace character to a variable (to %BS%). The for /L %%I in (1,1,50) line backspaces 50 times. Hopefully the rest is fairly self-explanatory.
If you'd like to test the logic without actually moving any files, here's the same script with the file iteration loop replaced with a simple for /L loop:
#ECHO OFF
SETLOCAL
SET "spinChars=\|/-"
for /f %%a in ('"prompt $H&for %%b in (1) do rem"') do set "BS=%%a"
SET "filesmoved=0"
set /P "=Moving XML Files...%spinChars:~0,1% 0 Files moved"<NUL
for /L %%I in (1,1,50000) do (
set /a filesmoved += 1, thousand = filesmoved %% 1000
setlocal enabledelayedexpansion
if !thousand! equ 0 call :spinner
endlocal
)
goto :EOF
:spinner
set "moved=%filesmoved%"
:spinner2
if %filesmoved% geq 4000 set /a filesmoved -= 4000 & goto :spinner2
set /a spinpos = filesmoved / 1000
for /L %%I in (1,1,50) do set /P "=%BS%"<NUL
set /P "=Moving XML Files...!spinChars:~%spinPos%,1! %moved% Files moved"<NUL
goto :EOF

Related

Add numbers with same text together

I would like to create a batch file in which I can see what I have collected in a game.
The game saves this information in a .txt file.
The output would look like this.
70x Silver.
Back Pearl.
41x Copper.
Amethyst.
Amethyst.
12x Silver.
Back Pearl.
21x Copper.
5x Silver.
Back Pearl.
Back Pearl.
Amethyst.
What I want to do now, is to add the items with the same name together, like this:
128x Silver.
4x Back Pearl.
62x Copper.
3x Amethyst.
There are hundreds of items with different names, not just these 4.
Would that be possible?
Any help would be appreciated. Thanks!
Another one!
#echo off
setlocal EnableDelayedExpansion
for /F "delims=" %%l in (test.txt) do for /F "tokens=1*" %%a in ("%%l") do (
set "first=%%a"
if "!first:~-1!" equ "x" (set /A "num=!first:~0,-1!") else set "num=0"
if !num! equ 0 (
set "rest=%%l"
set /A "count[!rest: =_!]+=1"
) else (
set "rest=%%b"
set /A "count[!rest: =_!]+=num"
)
)
(for /F "tokens=2* delims=[]=" %%a in ('set count[') do (
set "item=%%a"
if %%b equ 1 (
echo !item:_= !
) else (
echo %%bx !item:_= !
)
)) > summary.txt
#ECHO OFF
SETLOCAL
rem The following settings for the source directory and filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "filename1=%sourcedir%\q72672485.txt"
:: remove variables starting #
FOR /F "delims==" %%b In ('set # 2^>Nul') DO SET "%%b="
FOR /f "usebackqdelims=" %%b IN ("%filename1%") DO (
CALL :sub %%b
)
SETLOCAL ENABLEDELAYEDEXPANSION
(
FOR /F "tokens=1,2delims==" %%b In ('set # 2^>Nul') DO (
SET "line=%%cx%%b"
ECHO !line:#= !
)
)>summary.txt
endlocal
type summary.txt
GOTO :EOF
:sub
SET "quantity=%1"
SET "line=%*"
IF /i "%quantity:~-1%"=="x" (SET /a quantity=%quantity:~0,-1%&SET "line=%line:* =%") ELSE SET quantity=1
IF %quantity%==0 SET /a quantity=1&SET "line=%*"
SET /a #%line: =#%+=quantity
GOTO :eof
Different approach...
Would that be possible? - Yes.
#echo off
setlocal enabledelayedexpansion
for /f "delims=" %%a in (t.txt) do (
set " z=%%a"
set " z=!z:x =#!"
set " z=!z: =_!"
for /f "tokens=1,2 delims=#" %%b in ("!z!") do (
if "%%c" == "" (
set "x=1"
set "y=%%b
) else (
set "x=%%b"
set "y=%%c"
)
set /a #!y!+=!x!
)
)
(for /f "tokens=1,2 delims=#=" %%a in ('set #') do (
set "x=%%a"
set "y=%%bx "
echo !y:~0,4! !x:_= !
))>summary.txt
Output with your example data (I hope, alphabetic sorting is ok for you):
3x Amethyst.
4x Back Pearl.
62x Copper.
87x Silver.
(your calculation of 120 silver might be a bit optimistic with the given input data)
This is a different approach that use a "file merge" method. The overall code is somewhat simpler than other methods...
#echo off
setlocal EnableDelayedExpansion
set "lineNum=0"
(for /F "tokens=1,2* delims=:x " %%a in ('(type test.txt ^& echo 0x^) ^| findstr /N "[0-9][0-9]*x"') do (
if !lineNum! lss %%a call :locateLine %%a
set "line=%%c"
set /A "count[!line: =_!]+=%%b"
)) < test.txt
set "count[="
(for /F "tokens=2* delims=[]=" %%a in ('set count[') do (
set "item=%%a"
if %%b equ 1 (echo !item:_= !) else echo %%bx !item:_= !
)) > summary.txt
goto :EOF
:locateLine num
set /A "lineNum+=1" & set /P "line=" & if errorlevel 1 exit /B
if %lineNum% lss %1 set /A "count[%line: =_%]+=1" & goto locateLine
exit /B
Another approach (splitting the file into items with and without quantity) (also fixing the Pollux Infusion issue in my first answer):
#echo off
setlocal enabledelayedexpansion
REM process lines without quantity:
for /f "delims=" %%a in ('type test.txt^|findstr /vrc:"[0123456789][0123456789]*x "') do call :count 1 "%%a"
REM process lines with quantity:
for /f "tokens=1*delims=x " %%a in ('type test.txt^|findstr /rc:"[0123456789][0123456789]*x "') do call :count %%a "%%b"
REM reformat:
(for /f "tokens=1,2 delims=#=" %%a in ('set #') do (
set "count=%%bx "
set "line=!count:~0,5!%%a" &REM including '1x'
if %%b==1 set "line= %%a" &REM supressing '1x'
echo !line:_= !
))>summary.txt
type summary.txt
goto :eof
:count
set item=%~2
set "item=%item: =_%"
set /a #%item% +=%1
Included both with and without 1x. Remove the line, you don't want.

CMD File not deleting when the file has spaces

long time learner, first time poster. So i was tasked to find a way to be able to delete a file based off of date. the cmd/batch file should read todays date, look at the designated directory and tell you what to delete. I have borrowed a great deal from other posts here and even added a "choice" option just to be safe. The batch file will read all files in the directory and list if it should be deleted or kept. then it will ask are you sure you want to delete. if "y" is selected then it should delete the file but whenever it tries it says the file cannot be found. I know it must be because of the spaces in the file name. When i rename the files and remove the spaces it deletes them just fine. Im sorry if im all over the place, any help would be much appreciated. hereis what i have so far
#echo off
setlocal ENABLEDELAYEDEXPANSION
set day=86400
set /a year=day*365
set /a strip=day*7
set dSource=I:\Test
call :epoch %date%
set /a slice=epoch-strip
for /f "delims=" %%f in ('dir /a-d-h-s /b /s %dSource%') do (
call :epoch %%~tf
if !epoch! LEQ %slice% (echo DELETE %%f ^(%%~tf^)) ELSE echo keep %%f ^(%%~tf^)
)
echo/
if exist "%SystemRoot%\System32\choice.exe" goto UseChoice
setlocal EnableExtensions EnableDelayedExpansion
:UseSetPrompt
set "UserChoice=N"
set /P "UserChoice=Are you sure [Y/N]? "
set "UserChoice=!UserChoice: =!"
if /I "!UserChoice!" == "N" endlocal & goto :EOF
if /I not "!UserChoice!" == "Y" goto UseSetPrompt
endlocal
goto Continue
:UseChoice
%SystemRoot%\System32\choice.exe /C YN /N /M "Are you sure [Y/N]? "
if errorlevel 2 goto :EOF
for /f "delims=" %%f in ('dir /a-d-h-s /b /s %dSource%') do (
call :epoch %%~tf
if !epoch! LEQ %slice% del /f %%f ^(%%~tf^)
)
PAUSE
exit /b 0
rem Args[1]: Year-Month-Day
:epoch
setlocal ENABLEDELAYEDEXPANSION
for /f "tokens=1-6 delims=-;+^_?" %%d in ("echo %1") do set Years=%%d& set Months=%%e& set Days=%%f
if "!Months:~0,1!"=="0" set Months=!Months:~1,1!
if "!Days:~0,1!"=="0" set Days=!Days:~1,1!
set /a Days=Days*day
set /a _months=0
set i=1&& for %%m in (31 28 31 30 31 30 31 31 30 31 30 31) do if !i! LSS !Months! (set /a _months=!_months! + %%m*day&& set /a i+=1)
set /a Months=!_months!
set /a Years=(Years-1970)*year
set /a Epoch=Years+Months+Days
endlocal& set Epoch=%Epoch%
exit /b 0

I'm trying to make a batch file that moves files by random

I already have a code, but it keeps selecting the first file on the list and it's getting irritating. I have no idea what to do.
#echo off
setlocal enabledelayedexpansion
CD c:\"destination"\somefolder
set n=0
for %%f in (*.*) do (
set /a n+=1
set "file[!n!]=%%f"
)
set /a rand=(n*%random% %%4) /4
move "!file[%rand%]!" C:\destination\somefolder
pause
rand with 30bit size(value from 0 to 2^30-1 modulo n)
#echo off
setlocal EnableDelayedExpansion
pushd c:\"source"\somefolder
set "n=0"
for /f "tokens=*" %%f in ('dir /b /a-d *.*') do (
rem number files from 0 and use full filename with spaceses
set "file[!n!]=%%~ff"
set /a "n+=1"
)
popd
if %n% leq 32768 ( set /a "rand=%random%%%n%"
) else set /a "rand=((%random%<<15)+%random%)%%n%"
move "!file[%rand%]!" C:\destination\somefolder
pause
endlocal

I need a format of code that only counts odd lines

test_title.bat
:GET_DOWNLOADS
set Counter=-1
for /f "DELIMS=" %%i in ('type version.txt') do (
set /a Counter+=2
set "Line_!Counter!=%%i"
)
if exist version.txt del version.txt
exit /b
:list_files
call :GET_DOWNLOADS
For /L %%C in (1,2,%Counter%) Do (
:: removing this part makes it work fine
set line=%%C
set /a line+=1
set /a line/=2
:: alternate way doesnt work either
REM set /a line=%line% / 2
:: this part without the math part would be %%C instead of %Line%
echo %line%. !Line_%%C!
)
pause
(made an edit)
the second part isnt working for some reason
it just crashes
if i remove the line that does the math it works fine but instead display 1. 3. 5. 7.
version.txt
everything
0
minecraft
0
steam
0
obs
0
fixed test_list.bat :D
#echo off
setlocal enabledelayedexpansion
set "num=1"
set "counter=0"
for /f "DELIMS=" %%i in (version.txt) do (
set /a num+=1
if "!num!"=="2" (set /a counter+=1&set "line_!counter!=%%i"&set num=0)
)
echo.
For /L %%C in (1,1,%Counter%) Do (echo %%C. !Line_%%C!)
pause
#echo off
setlocal enabledelayedexpansion
set "num=1"
set "counter=0"
for /f "DELIMS=" %%i in (version.txt) do (
set /a num+=1
if "!num!"=="2" (set /a counter+=1&set "line_!counter!=%%i"&echo %%i&set num=0)
)
echo.
set line_1
set line_2
set line_3
pause
Would output:
everything
minecraft
steam
obs
line_1=everything
line_2=minecraft
line_3=steam

speed up folder reorg code

I have some CMD code that Rojo and Magoo helped me write that runs against some XML files in a directory. The code grabs a date and time from the files in the file name and creates a year and month folder from it and then moves the files into them. The problem that I'm having is the folder itself contains 914,000 xml files and the script just can't handle it. I need something faster or a way to multithread the script. Another option I was considering is to move a few thousand files at a time and just run it on those from a temp directory and at the very end of the script move those folders into the production location. Here is the code and another script to create the XML files to test. The date isn't validated but for this exercise, they don't need to be. This will be running on a Microsoft Server 2012 R2 VM.
running Processor Intel(R) Xeon(R) CPU E5-2650 0 # 2.00GHz, 2000 Mhz, 1 Core(s), 1 Logical Processor(s) and 4 gigs of ram. I'm also including the Powershell and VbScript tags in case someone can offer any advise for writing the code in those languages.
XML move script
#ECHO OFF
SETLOCAL
Title Reorganizing XMLs - DO NOT CLOSE THIS WINDOW!
color 0F
mode con: cols=100 lines=6
prompt $t $d$_$p$g
::Get start time
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "start=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
Echo Start time: %start%
set "sourcedir=C:\Temp\TestDummyFiles"
set "tempdir=C:\temp\xmlreorgtemp"
::call :Get1000Files %sourcedir% %tempdir% %total%
pushd %sourcedir%
SET "spinChars=\|/-"
for /f %%a in ('"prompt $H&for %%b in (1) do rem"') do set "BS=%%a"
set "spaces= "
SET /a filesMoved = 0, spinPos = 0, prev = 0
echo Moving XML Files...
setlocal enabledelayedexpansion
for /L %%I in (1,1,7) do set "BS=!BS!!BS!"
for /L %%I in (1,1,3) do set "spaces=!spaces!!spaces!"
For %%A in (%sourcedir%\*.xml) do set /a cnt+=1
echo.
Echo Total XML files: %cnt%
echo.
FOR /f "tokens=1*delims=" %%a IN ('dir /b /a-d "%sourcedir%\*.xml" ' ) DO (
set /a filesmoved += 1
call :spinner !filesmoved! "%%~nxa"
)
call :spinner %filesMoved% Done.
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "end=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
echo End time: %end%
set /A elapsed=end-start
rem Show elapsed time:
set /A hh=elapsed/(60*60*100), rest=elapsed%%(60*60*100), mm=rest/(60*100), rest%%=60*100, ss=rest/100, cc=rest%%100
if %mm% lss 10 set mm=0%mm%
if %ss% lss 10 set ss=0%ss%
if %cc% lss 10 set cc=0%cc%
echo Elapsed Time: %hh%:%mm%:%ss%
endlocal & echo;
exit /b 0
:Get1000Files
#echo off
setlocal enabledelayedexpansion
for /f %%a in ('dir "%~1" /b /a-d *.xml') do (
set /a cnt+=1 & move "%%~a" "%~2"
if !cnt! EQU 1000 exit /b
)
exit /b
:spinner <filecount> <filename>
set /a spinPos += 1, spinPos %%= 4, ten = %~1 / 10 * 10
if "%~2"=="Done." set ten=%~1
set "str=[!spinChars:~%spinPos%,1!] %ten% files moved... [%~2]"
set "str=%str:~0,79%"
call :length len "%str%"
set /a diff = 79 - len
if %diff% gtr 0 set "str=%str%!spaces:~-%diff%!"
set /P "=!BS:~-79!%str%"<NUL
if "%~2" NEQ "Done." call :process %~2
exit /b 0
:length <return_var> <string>
setlocal enabledelayedexpansion
if "%~2"=="" (set ret=0) else set ret=1
set "tmpstr=%~2"
for %%I in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if not "!tmpstr:~%%I,1!"=="" (
set /a ret += %%I
set "tmpstr=!tmpstr:~%%I!"
)
)
endlocal & set "%~1=%ret%"
exit /b 0
:process
FOR /f "tokens=2,3,6delims=_" %%m IN ("%~1") DO SET "date1=%%m"&SET "date2=%%n"&SET "whichdate=%%o"
IF DEFINED whichdate SET "date1=%date2%"
IF NOT DEFINED date2 exit /b 1
If not exist .\%date1:~0,4%\%date1:~4,2% MD .\%date1:~0,4%\%date1:~4,2%
MOVE %~1 .\%date1:~0,4%\%date1:~4,2%\ > nul
And the script to create some dummy files
#echo off
setlocal EnableDelayedExpansion
cd /d %~dp0
For /f %%a in ('copy /Z "%~dpf0" nul') Do set "CR=%%a"
set fileSize=%~Z1
set /a cnt=0
echo Creating files. Please wait.&echo.
:loop
if %cnt% GTR 5000 exit /b
set /a cnt+=1
set /p "=Creating %cnt% File(s) !CR!"<nul:
Call :random 2009 2015 yyyy
call :random 1 12 mm
call :random 1 31 dd
if %mm% LSS 10 set mm=0%mm%
if %dd% LSS 10 set dd=0%dd%
set /P "=0" > thisSize.txt < NUL
(for /L %%i in (0,1,30) do (
set /A "bit=(1<<%%i)&fileSize, fileSize&=~(1<<%%i)"
if !bit! neq 0 type thisSize.txt
if !fileSize! neq 0 type thisSize.txt >> thisSize.txt
)) > IDABCDEFG001_STUFF_%yyyy%%mm%%dd%_ABC_0_1234567890.xml
del thisSize.txt
goto :loop
exit /b
:random Min Max [RtnVar]
#echo off & setlocal
set /a rtn=%random% %% ((%~2)-(%~1)+1) + (%~1)
(endlocal
if "%~3" neq "" (set %~3=%rtn%) else echo:%rtn%
)
exit /b
The server has Powershell 4 on it.
Not powershell, but maybe this could do the work
#echo off
setlocal enableextensions disabledelayedexpansion
set "xmlFolder=C:\Temp\TestDummyFiles"
pushd "%xmlFolder%" && (
for %%x in ("*_*_*.xml") do if exist "%%x" (
for /f "tokens=2-4 delims=_" %%a in ("%%~nx") do if "%%c"=="" (set "fileDate=%%a") else (set "fileDate=%%b")
setlocal enabledelayedexpansion
for /f "tokens=1,2" %%a in ("!fileDate:~0,4! !fileDate:~4,2!") do (
endlocal
<nul set /p "=%%a\%%b : "
md ".\%%a\%%b" 2>nul
move /y "*_%%a%%b??_*.xml" ".\%%a\%%b" 2>nul | find /v ":"
)
)
popd
)
There are three reasons for your code to be slow (appart from the fact that you are handling 914000 files):
There are 914000!! files
call usage is slow. 914000 * #calls for each file = very slow
914000 status updates to console are slow
for /f
Yes, the for /f commands used in
FOR /f "tokens=1*delims=" %%a IN ('dir /b /a-d "%sourcedir%\*.xml" ' ) DO (
for /f %%a in ('dir "%~1" /b /a-d *.xml') do (
have one problem because:
The dir command has to enumerate the 914000 files
The full list needs to be loaded into memory before starting to process it
The for /f command loads data into a buffer. When the buffer is full a new bigger (4KB increase in windows 7) buffer is created and data is copied from the old buffer to the new and this process is repeated until all the data has been retrieved. Each time the buffer is resized a larger memory copy operation needs to be done so the time needed to handle all the data increases exponentially.
This means
914000 files * ( 50 chars file name + CR LF ) = 47528000 characters
47528000 characters / 4KB buffer increase = 11603 redim operations
11603 redim operations = 1103170928640 bytes moved in memory copy operations
To handle all this, the proposed code will
Use a simple for to enumerate the files. The process starts on the first file being found and more search operations are done as the files are being iterated.
Instead of processing each file, all the files matching a date are moved in only one move operation.
If you have a lot of files, then you may reorder its processing by the smallest number of groups. In your example code you create 5000 dummy files, but just in 6 years. The code below process files by year, then month:
#ECHO OFF
SETLOCAL EnableDelayedExpansion
set "sourcedir=C:\Temp\TestDummyFiles"
pushd %sourcedir%
:nextYear
for %%a in (*.xml) do set "fileName=%%a" & goto break
:break
if not defined fileName goto :EOF
FOR /f "tokens=2,3,6 delims=_" %%m IN ("%fileName%") DO SET "date1=%%m" & SET "date2=%%n" & SET "whichdate=%%o"
IF DEFINED whichdate SET "date1=%date2%"
IF NOT DEFINED date2 exit /b 1
set "YYYY=%date1:~0,4%"
set "MM=100"
for /L %%m in (1,1,12) do (
set /A MM+=1
MD "%YYYY%\!MM:~1!" 2> NUL
MOVE "*_%YYYY%!MM:~1!??_*.xml" "%YYYY%\!MM:~1!"
)
goto nextYear
In my opinion, you should started this topic with a description of the problem, like "I have 914,000 files with this format IDABCDEFG001_STUFF_yyyymmdd_ABC_0_1234567890.xml and I want to move they to folders with yyyy\mm structure". I really don't like to try to know the details of a problem reading code. I don't understand your code to get the date from the file name, so I just copied it...

Resources