I would like to create a cleaning utility for my RDX.
It would test whether the available space on the tape is less than a certain threshold and erase the oldest files until you reach this threshold.
I tinkered something from bits of code but I can not figure what's wrong :
#echo off
#setlocal enableextensions
#setlocal enabledelayedexpansion
for /f "tokens=7" %%a in ('fsutil volume diskfree e:') do set free=%%a
for /f "delims=" %%b in ('"dir e:\*.zip /A-D /OD /B"') do set file=%%b
if %free:~0,-9% lss 61 del e:\%file% else exit
Each lines seems to work separately, but not the loop.
Can you please help me?
Thanks in advance =)
Try this:
#echo off &setlocal enableextensions
for /f "tokens=7" %%a in ('fsutil volume diskfree e:') do set "free=%%a"
for /f "delims=" %%b in ('dir e:\*.zip /A-D /O-D /B') do set "file=%%b"
if %free:~0,-9% lss 61 (del "e:\%file%") else exit
In addition to all of the Peter's comments, your problem is not looping in the second FOR
Read HELP FOR and then you'd try something like this...
for /f "delims=" %%b in ('"dir e:\*.zip /A-D /OD /B"') do (
set file=%%b
if %free:~0,-9% lss 61 del e:\%file%
)
but, wait, here be more problems: %free% is not refreshed after the deletion; the comparison might be incorrect; the variable %file% does not get updated on each loop iteration... those are probably subject for other SO questions.
Not sure what you mean by "not the loop" - there are two loops, but you say each line works separately?
Worked quite happily on my machine - so what exactly do you mean?
A few things that may help:
The leading # merely means 'do not echo this command to the console before it's executed'. Once an #echo off instruction is executed, the echoing of commands stops, so the # is redundant. It is required on the #echo off command because otherwise the instruction to turn echoing off would itself be ECHOed.
Each setlocal starts a new local environment, inheriting all of the current environment variables. Using two is not wrong, it's merely redundant. setlocal enableextensions enabledelayedexpansion will suffice. You are not using delayedexpansion in this routine, so that is itself redundant. The default state of CMD.EXE is enableextensions. You have to take explicit action to DISABLE the extensions - and most batchers will assume the default condition.
On my machine, the freespace is the EIGHTH token on the line, not the 7th. This would load : into free if yours is the same.
The double-quotes surrounding the dir command are reundant. The command between the SINGLE quotes is executed. Adding the double-quotes does nothing UNLESS there is a single-quote WITHIN the command to be executed.
The IF comparison could go sadly wrong. Any %var% is resolved before the instruction is executed, so should free contain fewer than 9 characters, %free:~0:-9% would be resolved to [nothing] and the instruction would be executed as if lss 61... which would be a syntax-error. Note that in your case, free may be : and even after correction, of you have less than 1000000000 bytes free, %free:~0:-9% would be resolved to nothing and generate the error.
If there are no files matching your selection criteria found by the dir command, file will contain whatever value it had when the batch was started. In all probability, the variable file would never have been set, but suppose it was set to myveryimportantfile ? For this reason, it's best to make sure that any variables used in a batch are initialised.
So - revised:
#echo off
setlocal
FOR %%I IN (free file) DO SET "%%I="
for /f "tokens=8" %%a in ('fsutil volume diskfree e:') do set free=%%a
for /f "delims=" %%b in ('dir e:\*.zip /A-D /OD /B') do set file=%%b
IF NOT DEFINED FILE EXIT
SET "FREE=%FREE:~0,-9%"
IF NOT DEFINED FREE SET /A FREE=0
if %FREE% lss 61 del "e:\%file%"
(Case is redundant - changes in CAPS. Case of metavariables FOR %%x is significant)
Note that strictly speaking, even the SETLOCAL is redundant, but it does ensure that this batch does NOT affect the state of the environment after it has terminated.
The syntax SET "var=somethingorevennothing" is significant. A simple SET statement is aimed at STRINGS, so SPACES are significant. If a space occurs BEFORE the = then it is INCLUDED in the variable name being set. If there are spaces at the end of the line, they are also included. Quoting the SET overcomes the second problem. Vigilance overcomes the first.
SET /A ignores these spaces, but can only be used to set NUMERIC values. The numeric values are in fact stored as a STRING.
HTH
Related
I'm trying to find files that do not match (at the beginning of the filename) predefined formats contained in a .txt file.
I have the following:
#Echo off
chcp 1254>nul
setlocal DisableDelayedExpansion
for /f "usebackq tokens=1,2,3* delims=~" %%f in ("%USERPROFILE%\Desktop\xref.txt") do (
set "DIRNAME=%%f"
set "DIRNAM2=^%%f"
set "PATHNAM=%%h"
set "ALBUMNM=%%g"
SETLOCAL EnableDelayedExpansion
IF EXIST !PATHNAM!!DIRNAME! (
PushD !PATHNAM!!DIRNAME!
dir /b /a-d "*" | findstr /v /r /c:"!DIRNAM2! -*"
)
ENDLOCAL
)
pause
EXIT /b
This works great except with filenames containing bangs (exclamation points).
Here's a sampling of my .txt file (subdirectory~album name~path) which gets generated by a script:
12 Byzantine Rulers. The History of The Byzantine Empire~12 Byzantine Rulers. The History of The Byzantine Empire~g:\test\
17th Century Poetry~17th Century Poetry~g:\test\
1984 (George Orwell)~1984 (George Orwell)~g:\test\
1_2_1~1_2_1~g:\test\
21st Century American Foreign Policy~21st Century American Foreign Policy~g:\test\
99% Invisible~99% Invisible~g:\test\
Communication Matters. That’s Not What I Meant!~Communication Matters. That’s Not What I Meant!~g:\test\
There are hundreds of directories containing hundreds of files (podcasts). I'd like to fix this batch so it can also handle bangs (!).
Thx in advance.
Edit. My test data wasn't robust enough. The findstr command also doesn't work with (at least) the following characters: é’»¿ ... that is to say PushD gets me to the right directory, but FindStr doesn't do it's culling as expected.
I don't think that the issue is necessarily code page or encoding related, and less so exclamation marks, (bangs). The major issue I see is that your text file content uses smart quotes, (curly), instead of dumb quotes, (straight). Additionally you have % characters which in batch files usually require doubling. For those reasons I would first suggest that you try to replace those characters.
For example:
#Echo Off
SetLocal DisableDelayedExpansion
For /F "UseBackQ Tokens=1-3 Delims=~" %%G In ("%USERPROFILE%\Desktop\xref.txt")Do (
Set "SUBDIRN=%%G"
Set "ALBUMNM=%%H"
Set "PATHNAM=%%I"
SetLocal EnableDelayedExpansion
Set SUBDIRN=!SUBDIRN:%%=%%%%!
Set ALBUMNM=!ALBUMNM:%%=%%%%!
Set PATHNAM=!PATHNAM:%%=%%%%!
Set SUBDIRN=!SUBDIRN:’='!
Set ALBUMNM=!ALBUMNM:’='!
Set PATHNAM=!PATHNAM:’='!
Set SUBDIRN=!SUBDIRN:“="!
Set ALBUMNM=!ALBUMNM:“="!
Set PATHNAM=!PATHNAM:“="!
Set SUBDIRN=!SUBDIRN:”="!
Set ALBUMNM=!ALBUMNM:”="!
Set PATHNAM=!PATHNAM:”="!
If Exist "!PATHNAM!!SUBDIRN!\" (
PushD "!PATHNAM!!SUBDIRN!"
Dir /B/A-D|FindStr /IVRC:"^!SUBDIRN! -"
)
EndLocal
)
Pause
Exit /B
I'm not sure how a copy of this code, within the code box will handle the smart quotes, but I'm sure you'll get the idea.
I have the following code (source identified) which is great, but it fails when a title contains a percent symbol. I like the fact that it doesn't use temp files and allows me to overwrite the original file. Any thoughts on how to fix this for percent symbols?
REM Randomize the Onesies playlist
REM Source: stackoverflow.com/questions/19393155
REM DOES NOT PROPERLY CREATE PLAYLIST ENTRY FOR ALBUMS CONTAINING PERCENT SYMBOLS (but it's fast)
for /f "delims=" %%a in (G:\Playlists\Onesies.m3u8) do call set "$$%%random%%=%%a"
(for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b)>G:\Playlists\Onesies.m3u8
echo Playlist Onesies.m3u8 has been created in the folder G:\Playlists\
echo.
pause
The problematic part is call set "$$%%random%%=%%a". To explain why, let us start from the beginning.
Simply using set "$$%random%=%%a" would not work, because %random% would be expanded (read) when the entire for /F loop, or more generally spoken, the whole block of code, is parsed, so the same value would be returned for every loop iteration. The call trick, toghether with doubled percent signs around variables, introduces another parsing phase, so when the entire for /F loop is parsed, call set "$$%%random%%=%%a" is interpreted as call set "$$%random%=%a", because each %% in a batch file is interpreted as one literal %. Hence the random variable is not yet expanded.
For every loop iteration, the for /F variable becomes replaced by the current iteration value. call initiates another parsing phase as already mentioned, so %random% becomes replaced by a individual random number per loop iteration. Assuming that %a is abritrary string, the line is interpreted as call set "$$%random%=arbitrary string" first, and after the second parsing phase it becomes set "$$16384=arbitrary string", supposing %random% returns the number 16384.
Now let us take another example value for %a which contains a percent sign, like string with % sign: after the first parsing phase, we have call set "$$%random%=string with % sign"; after the second one, we get something like set "$$32767=string with sign". The percent sign disappears here, because the command line interpreter encounters a single % sign that cannot be paired with another one (where a variable name is expected in between them, like %random%), so it simply dismisses that character. In case %a contained two % signs, for instance, more % signs % here, the result would be more here (most likely), because SPACEsignsSPACE, being in between a pair of % signs, would be treated as the name of a (undefined, hence empty) variable.
To solve that issue, we need to avoid %a to be passed through a second parsing phase. This can be achieved by avoiding %%a to appear in the call command line.
So to overcome the loss of % signs in your script, you need to store %%a into an interim variable and to do the same double-% expansion within the call command line like you already do for random:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set /A "index=0"
for /f "usebackq delims=" %%a in ("G:\Playlists\Onesies.m3u8") do (
set "item=%%a"
call set "$$%%random%%.%%index%%=%%item%%"
set /A "index+=1"
)
> "G:\Playlists\Onesies.m3u8" (
for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b
)
echo Playlist "Onesies.m3u8" has been created in the folder "G:\Playlists\".
echo/
endlocal
pause
An alternative method is to apply delayed expansion:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Toggling delayed expansion in the loop is necessary to not lose exclamation marks.
set /A "index=0"
for /f "usebackq delims=" %%a in ("G:\Playlists\Onesies.m3u8") do (
setlocal EnableDelayedExpansion
rem // This loop passes `random` over the `endlocal` barrier:
for /f %%b in ("!random!.!index!") do (
endlocal
set "$$%%b=%%a"
)
set /A "index+=1"
)
> "G:\Playlists\Onesies.m3u8" (
for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b
)
echo Playlist "Onesies.m3u8" has been created in the folder "G:\Playlists\".
echo/
endlocal
pause
The original code from the question relies on the built-in random number variable random which may return duplicate values. This leads to the problem that some lines of the read file get lost.
To resolve this issue, both of the above approaches herein feature a counter called index which is of course unique for every single line that is read by the for /F loop. This counter value is appended to the random value so that the total concatenated string is unique. Therefore, none of the lines of the read file are lost any more, opposed to the original script.
This question already has answers here:
windows batch files: setting variable in for loop
(3 answers)
Closed 7 years ago.
I want to get the list of all files inside of c:\test. My attempt is this:
set loc=c:\test
for /f %%i in(dir "%loc%" /b') do (
#set variable=%%i
echo %variable%
)
...but I'm getting back only one file name, n times. How to get all the files in that folder.
1. The reason you get back only 1 file name all the time is that you did not Setlocal EnableDelayedExpansion.
2. Check again you code, you did not add a single quotation mark before dir "%loc%" /b'.
3. Check again your code once more, you can't stick "in" and "(" like in(, this will absolutely ruin your code.
#echo off
Setlocal EnableDelayedExpansion
set "loc=c:\test"
for /f %%i in ('dir "%loc%" /b') do (
set "variable=%%i"
echo !variable!
)
You need to enable "delayed expansion" for this to work. If it isn't enabled, variables are evaluated exactly once, when the script is parsed. This is why you get the one filename n times.
Some notes:
Enable delayed expansion with SETLOCAL EnableDelayedExpansion
When delayed expansion is enabled, to take advantage of it, you need to use ! instead of % as variable delimiter, so your %variable% becomes !variable!
Loop variables like your %%i are an exception in that they will change their value even when delayed expansion is not enabled. Try ECHOing %%i in your original script (i.e. without SETLOCAL EnableDelayedExpansion) to see what I mean
Edit: dark fang correctly points out syntax errors in your script, which I didn't even catch - but from the behaviour your described, these were not in your script when you were trying run it, because it would just have errored out.
In the end you get:
SETLOCAL EnableDelayedExpansion
set loc=c:\test
for /f %%i in ('dir "%loc%" /b') do (
#set variable=%%i
echo !variable!
)
I've got a script that does everything I expect it to do, apart from one line.
I've done similar before, but I can't get this one to work.
The code I've got is here
#echo off
SETLOCAL ENABLEDELAYEDEXPANSION
::Set Path to be folder of Sage Files
SET PATH=C:\Welcome\Progs\SitesDataSetups\GeorgeYarmouth
::set date variables
for /f "tokens=1" %%i in ('date /t') do set thedate=%%i
set mm=%thedate:~3,2%
set dd=%thedate:~0,2%
set yyyy=%thedate:~6,4%
::Set T_DAY variable to date in ddmmyy format
set T_DAY=%dd%%mm%%yyyy:~2%
c:
cd\
cd %path%
for /f "usebackq tokens=* delims= " %%P in (`dir sage*.csv /od /b`) do (
set SAGE=%%P
set SAGE2=!SAGE:~0,8!_EDITED
set EODNUM=!SAGE:~4,4!
for /f "tokens=* delims= " %%A in (%%P) do (
echo %EODNUM%
set S=%%A
***This line is the problem***
set S=!S:%T_DAY%=%EODNUM%!
echo.!S! >> %PATH%\TEST\!SAGE2!.csv
)
)
endlocal
I was expecting that is would take each line of the csv file and replace it with itself, except with a string replace of the current date with the variable EODNUM (which it does... only the variable is expanded before it is set, so is nothing)... The delayed expansion should solve this, but I can use this line of code
set S=!S:%T_DAY%=!EODNUM!!
because I think its too many !'s for CMD.
Am I missing something, or is there a better way to code this?? (I'm not a programmer of any kind, and most of the code I write comes from trial and error, and 'borrowing' from other scripts, so this may be a very messy way to do this).
Transfer the the value of !EODNUM! to a FOR variable, and then use your FOR variable as the replace string.
echo !EODNUM!
set "S=%%A"
for /f "delims=" %%E in ("!EODNUM!") do set "S=!S:%T_DAY%=%%E!"
echo.!S!>> %PATH%\TEST\!SAGE2!.csv
By way of explanation...
CMD reads (and does env var substitution), parses, and executes one top-level command at a time.
In your example, it reads the "for /f..." command all at once parsing and performing %var% substitution.
Once this is complete, it then executes the for loop, performing delayed !var! substitution.
Unfortunately, !var! substitution is not a do-substitution-until-none-left. This makes it hard (as in the answerer's solution) to perform the substitution into the !var:src=dst! value.
You will need a way that during execution you can get guaranteed substitution. This requires a for-statement, or something that involves reading and %var% substituting again. One way of doing this is to use the CALL :LABEL form where you can call to a specific label in your .cmd file and have this section do what you want:
...
call :GenS
...
and then:
:GenS
set S=!S:%T_DAY%=%EODNUM%!
goto :eof
BTW: I'm perplexed that you didn't notice the ECHO %EODNUM% not working in the loop as during the reading of the for loop all %var% substitutions are made.
A have a text file that contains the results of a dir
dir "%local%" /b /a:d /s >> FolderList.txt
But I want to iterate in a For loop going from the last to the first line.
Since I believe this cannot be done in the For command, how can I generate a new file containing the same lines but in a inverse order?
You can't using the For command. But you can reverse the order of the dir listing that created the text file, using dir "%local%" /o-n /b /a:d /s >> FolderList.txt; the - means "reversed".
I like the general strategy of both of Aacini's original solutions, but as written they have problems (some trivial, some significant)
Original Aacini solution 1 using temp file with SORT:
Corrupts lines containing exclamation point (!)
Strips leading colon(s) (:) from each line
temp file creation using >> not as efficient as >
Uses default SORT maximum line length of 4096 bytes
Line count unnecessarily capped at 1 million
Doesn't actually provide the asked for solution (an actual file output)
Leaves behind the temporary file
Modified solution 1
Here is a version that fixes the problems. The only practical limitation is a maximum line length of 8180 bytes (characters). I'm not sure how high FINDSTR can count, but this solution will handle up to 999 billion lines. (I agree with Aacini, no one would ever want to wait for such a large file to finish using a batch solution) The line limit can easily be adjusted.
#echo off
setlocal DisableDelayedExpansion
set file="%~1"
set revfile="%~1.rev"
set tempfile="%temp%\revfile%random%.txt"
(
for /f "delims=" %%a in ('findstr /n "^" %file%') do (
set "ln=%%a"
setlocal EnableDelayedExpansion
for /f "delims=:" %%n in ("!ln!") do set "prefix=000000000000%%n"
echo !prefix:~-12!!ln:*:=!
endlocal
)
)>%tempfile%
(
for /f "delims=" %%a in ('sort /rec 8192 /r %tempfile%') do (
set "ln=%%a"
setlocal EnableDelayedExpansion
echo(!ln:~12!
endlocal
)
)>%revfile%
del %tempfile%
Aacini modified solution 1
Aacini dramatically improved the robustness and performance with a modified solution 1 using SET /P and multiple TEMP files. The SET /P solution eliminates the need for a looped SETLOCAL/ENDLOCAL toggle, but it does have a few limitations.
Lines must be terminated by <LF><CR> (normal for Windows, but Unix style is sometimes encountered in Windows world).
Lines must be <= 1024 characters
Control characters at end of line will be stripped.
Modified solution 1 take 2
If any of the above limitations are a problem, here is an adaptation of my 1st solution that uses multiple temp files. Like Aacinis modified solution, it performs linearly with file size. It is about 40% slower than Aacinis modified version.
#echo off
setlocal DisableDelayedExpansion
set file="%~1"
set revfile="%~1.rev"
set "tempfile=%temp%\revfile%random%.txt"
findstr /n "^" %file% >"%tempfile%.1"
(
for /f "usebackq delims=" %%a in ("%tempfile%.1") do (
set "ln=%%a"
setlocal EnableDelayedExpansion
for /f "delims=:" %%n in ("!ln!") do set "prefix=000000000000%%n"
echo !prefix:~-12!!ln:*:=!
endlocal
)
)>"%tempfile%.2"
sort /rec 8192 /r "%tempfile%.2" >"%tempfile%.3"
(
for /f "usebackq delims=" %%a in ("%tempfile%.3") do (
set "ln=%%a"
setlocal EnableDelayedExpansion
echo(!ln:~12!
endlocal
)
)>%revfile%
del "%tempfile%*"
Original Aacini solution 2 using environment variables:
Corrupts lines containing exclamation point (!)
Strips blank lines
Doesn't actually provide the asked for solution (an actual file output)
Modified solution 2
Here is a version that fixes the problems. The only known limitations are
A maximum line length between 8181 and 8190, depending on line number
A maximum file size slightly under 64MB.
This was my favorite solution because the file output can probably be eliminated by processing the file in the variables directly, thus completely avoiding the creation of any temporary file. Edit But based on info provided by Aacini, I learned it has severe performance problems as the environment grows. The problem is worse than Aacini realized - Even a simple SET command suffers dramatically with large environment sizes. I've posted a question regarding this phenomenon at DosTips. http://www.dostips.com/forum/viewtopic.php?f=3&t=2597 (I originally posted on SO, but apparently the question is too open ended for this site)
#echo off
setlocal disableDelayedExpansion
set file="%~1"
set revfile="%~1.rev"
set num=0
for /f "delims=" %%a in ('findstr /n "^" %file%') do (
set /a "num+=1"
set "ln=%%a"
setlocal enableDelayedExpansion
for %%n in (!num!) do for /f "delims=" %%b in (""!ln:*:^=!"") do endlocal&set "ln%%n=%%~b"'
)
setlocal enableDelayedExpansion
(
for /l %%n in (!num! -1 1) do echo(!ln%%n!
)>%revfile%
There are two relatively easy ways to sort a file in reversed order. The first one is a direct method over file contents: add line numbers to all lines, sort the file in reversed order, eliminate line numbers:
#echo off
setlocal EnableDelayedExpansion
rem Insert line numbers in all lines
for /F "tokens=1* delims=:" %%a in ('findstr /n ^^ %1') do (
set /A lineNo=1000000+%%a
echo !lineNo!:%%b>> tempfile.txt
)
rem Sort the file and show the result
for /F "tokens=1* delims=:" %%a in ('sort /r tempfile.txt') do (
echo Line %%a is %%b
)
The other method consist in load the file lines in a Batch array, that may be processed in any way you wish:
#echo off
setlocal EnableDelayedExpansion
rem Load file lines in a Batch array
set lineNo=0
for /F "delims=" %%a in (%1) do (
set /A lineNo+=1
set "line[!lineNo!]=%%a"
)
rem Process array elements in reversed order:
for /L %%i in (%lineNo%,-1,1) do (
echo Line %%i is !line[%%i]!
)
This last method works only if the size of the file is below 64 MB, because this is the limit for Batch variables.
Both methods can be modified to correctly process special characters (> < |).
HOWEVER
If you want to delete all the tree contents of a folder in bottom-up order, the "right" way to do that is via a recursive subroutine...
EDIT Answer to dbenham
As I wrote in my answer, the two methods I proposed can be modified to correctly process special characters and blank lines. In my answer I showed a general method to "change the order of lines" in reversed order paying no special attention on create an output file because the OP said in his own answer that "The objective was to reorder a list of folders to prevent problems while deleting them in sequence", so I thought that was enough to show him how to process the folders in reversed order. I also assumed that the list of folders:
Have not exclamation points (!).
Have not leading colons (:).
Folder names are shorter than 4096 bytes.
Have less than 1000000 lines.
Have not blank lines.
I even thought (and still think) that the method the OP want to use to delete a list of folders is not adequate, and I mentioned this point under a big HOWEVER in my answer proposing to use a recursive subroutine instead.
However it seems that dbenham thought that the original question was something similar to "What is the most efficient method to sort a large file in reversed order?" and criticize my methods because they lack of such features. For this reason, I should reply in terms of this new question (efficient method), right?
In first place, it's funny to me that dbenham critizice my methods because "Doesn't actually provide the asked for solution (an actual file output)", but in his own Modified solution 2 he wrote that "This is my favorite solution because the file output can probably be eliminated by processing the file in the variables directly, thus completely avoiding the creation of any temporary file". ???
The two methods proposed by dbenham have a serious problem in terms of efficiency that was already discussed in this question: the pair of setlocal EnableDelayedExpansion and endlocal commands are executed with every line of the file. If the file is large (i.e. 200 000 lines and about 8 MB, as in the previous mentioned question) the environment will be copied to a new memory area and then deleted, and this will be repeated for 200000 times! Of course, this task is time-consuming. This problem becomes worse in dbenham's Modified solution 2: as the processing of lines go on, the environment grow as it store the file contents at that point. In the last lines of the file an environment almost equal to the size of the whole file will be copied to a new memory area for every remaining line of the file. Of course, this is the worst way to achieve this process in terms of efficiency!
There is another way to process empty lines and special characters that don't require the setlocal EnableDelayedExpansion - endlocal pair. For details on this method and further discussion on efficient ways to process large files, see the previously mentioned question.
The following Batch files are my modified versions on "How to sort a large file in reversed order in an efficient way".
Modified solution 1: using temp file with SORT
#echo off
setlocal EnableDelayedExpansion
set revfile="%~1.rev"
set tempfile=%temp%\revfile%random%
rem Insert line numbers in all lines
findstr /n ^^ %1 > "%tempfile%1.txt"
find /c ":" < "%tempfile%1.txt" > "%tempfile%2.txt"
set /P lines=< "%tempfile%2.txt"
call :JustifyLineNumbers < "%tempfile%1.txt" > "%tempfile%2.txt"
del "%tempfile%1.txt"
rem Sort the file in reversed order
sort /rec 8192 /r "%tempfile%2.txt" /o "%tempfile%3.txt"
del "%tempfile%2.txt"
rem Remove line numbers
call :RemoveLineNumbers < "%tempfile%3.txt" > %revfile%
del "%tempfile%3.txt"
goto :EOF
:JustifyLineNumbers
for /L %%i in (1,1,%lines%) do (
set /A lineNo=1000000000+%%i
set /P line=
echo !lineNo!!line:*:=!
)
exit /B
:RemoveLineNumbers
for /L %%i in (1,1,%lines%) do (
set /P line=
echo !line:~10!
)
exit /B
This solution still have a limit of "only" 1147483647 lines (the maximum 32-bits signed positive integer minus the initial seed). Although this limit can be easily increased in the way suggested by dbenham, that modification imply a slower execution speed. The conclusion is: if you really want to reverse-sort a very large file don't use a Batch file, but a more efficient programming language (like C).
Modified solution 2: using a Batch variable array
#echo off
setlocal EnableDelayedExpansion
set revfile="%~1.rev"
set tempfile=%temp%\revfile%random%
rem Load file lines in a Batch array
findstr /n ^^ %1 > "%tempfile%1.txt"
find /c ":" < "%tempfile%1.txt" > "%tempfile%2.txt"
set /P lines=< "%tempfile%2.txt"
del "%tempfile%2.txt"
call :CreateArray < "%tempfile%1.txt"
del "%tempfile%1.txt"
rem Process array elements in reversed order:
(for /L %%i in (%lines%,-1,1) do echo=!ln%%i!) > %revfile%
goto :EOF
:CreateArray
for /L %%i in (1,1,%lines%) do (
set /P line=
set ln%%i=!line:*:=!
)
exit /B
EDIT A possible solution for large environment problem.
I devised an idea that may solve, at least in part, the performance problems of SET command caused by a very large environment. Let's suppose that the internal operation of SET VAR=VALUE command follow these steps:
When a new variable is defined with a value that exceed the current environment size, the environment is copied to a new area if the area beyond it is not available.
The new area is just large enough to receive the new variable. No additional space is reserved.
The important one: When a large variable is deleted, the remaining free space is NOT released. The environment memory block is never shrunk.
If previous steps are true, then the performance problems may decrease if we first reserve the desired environment space via large (8 KB) variables with the same name of the working variables. For example, to reserve 1024 KB we define 128 large variables; I suppose that the time required to define these 128 variables will be less than the time required to fill the same 1024 KB with shorter variables.
When the process is running, the definition of the first 128 working variables will take the time necessary to delete an 8 KB variable and define a shorter one, but for the variable 129 on the process must be faster because it just define a new variable in an already available space. To aid to this process, the variables must have names that place them at the end of the environment as dbenham indicated.
:ReserveEnvSpace sizeInKB
rem Define the first large variable (reserving 6 bytes for variable name)
rem (this method may be done in larger chunks until achieve the fastest one)
set z1=X
for /L %%i in (1,1,8184) do set z1=!z1!X
rem Define the rest of large variables
set /A lastVar=%1 / 8
for /L %%i in (2,1,%lastVar%) do set z%%i=!z1!
exit /B
You may use MEM /P command to check the size and placement of the environment memory block. In old MS-DOS (command.com) days the environment was placed after command.com, but if a resident program was placed after the environment, then it can't grow anymore. For this reason, the /E:nnnnn switch was provided in command.com to reserve a certain size in bytes for the environment.
I have no time to check this method for the rest of the day, but here it is for you!
The objective was to reorder a list of folders to prevent problems while deleting them in sequence.
I came up with the following algorithm. I accept suggestions to make it more efficient or better.
#ECHO off
setLocal EnableDelayedExpansion
:: File that contains a list of folders
set file_from=%~1
:: Destination file, that will contain the sorted list
if "%2"=="" (
set replace=1
set file_to=_%file_from%
) else (
set file_to=%~2
)
:: Create empty destination file
if exist "%file_to%" del "%file_to%"
copy NUL "%file_to%"
:: Temporary file
if exist ".\~Remaining.txt" del ".\~Remaining.txt"
copy "%file_from%" .\~Remaining.txt
:: Sort the order of folders
:while
set untouched=1
For /f "tokens=* delims=" %%a in (.\~Remaining.txt) Do (
:: check if line was already added
FindSTR /X /C:%%a "%file_to%"
if errorlevel 1 (
set untouched=0
:: check if folder contains sub-folders to be added
FindSTR /B /C:%%a\ .\~Remaining.txt
if errorlevel 1 (
:: remove current line from "~Remaining.txt"
FindSTR /V /B /E /C:%%a .\~Remaining.txt> .\~Remaining_new.txt
move .\~Remaining_new.txt .\~Remaining.txt
:: add current line to destination file
>> "%file_to%" ECHO %%a
goto while
)
)
)
if untouched LSS 1 (
goto while
)
if exist .\~Remaining.txt del .\~Remaining.txt
if defined replace (
ECHO REPLACE!
:: destination was not provided, so replace
if exist "%file_from%" del "%file_from%"
move "%file_to%" "%file_from%"
)
This code will reverse a text file, but with a few limitations. Blank lines are omitted and lines containing special charaters cause it to fail: & < > |
#Echo Off
If "%1"=="" Goto Syntax
If "%2"=="" Goto Syntax
If Not Exist %1 (
Echo File not found: %1
Exit /B 2
)
SetLocal EnableDelayedExpansion
Set SOF=~StartOfFile~
Set InFile=%~snx1~in
Set OutFile=%2
Set TempFile=%~snx1~temp
If Exist %OutFile% Del %OutFile%
If Exist %TempFile% Del %TempFile%
Copy %1 %InFile% >nul
:Loop
Set "Line=%SOF%"
For /F "tokens=*" %%a In (%InFile%) Do (
If Not "!Line!"=="%SOF%" Echo !Line!>>%TempFile%
Set "Line=%%a"
)
Echo %Line%>>%OutFile%
Del %InFile%
If Not Exist %TempFile% (
EndLocal
Exit /B 0
)
Rename %TempFile% %InFile%
Goto Loop
:Syntax
Echo Usage:
Echo %~n0 input-file output-file
Echo.
Exit /B 1