I'm trying to get a test code working so I can make a more complex version of the script for my actual needs, but I'm struggling with the FIND command.
My code:
#echo off
SETLOCAL DISABLEDELAYEDEXPANSION
>"%~dpn1_b.txt" (
FOR /F "tokens=1* delims=]" %%j in ('find /V /N "" %1') DO (
SET "currentLine=%%k"
SETLOCAL ENABLEDELAYEDEXPANSION
IF "!currentLine:~0,3!"=="12/" (
SET string=!currentLine:~0,2!!currentLine:~3,4!
FIND "%string%" 111.txt > nul
IF %errorlevel% EQU 0 (SET "currentLine=!currentLine:~0,2!!currentLine:~3!")
)
ECHO(!currentline!
ENDLOCAL
)
)
This is executed by dragging test.txt onto the batch file.
111.txt holds 123456. test.txt has 12/3456 on the first line and 12/4456 on the second line. The script is supposed to remove the slash from the first line but not from the second. Any idea what went wrong?
Within the FOR loop body, all environment variable expansions using the %var% format will be pre-expanded. You still need to use the delayed expansion !var! format even for internal variables like errorlevel.
FOR /F ... (
...
FIND "!string!" 111.txt > nul
IF !errorlevel! EQU 0 (SET "currentLine=!currentLine:~0,2!!currentLine:~3!")
)
As Ryan mentioned, this is clearer if you leave ECHO enabled as you see that with the original code the last two statements in the loop were being pre-expanded:
FIND "" 111.txt 1>nul
IF 0 EQU 0 (SET "currentLine=!currentLine:~0,2!!currentLine:~3!" )
interesting: you get the line numbers (%%j) but then don't use it...
How about:
...
if "%%j"=="[1" (
...
) else (
echo %%k
)
...
Related
I saw that in batch, to delete a specific line from a text file, you need to do it with findstrthat allow you to find your line and then delete it. But is there a way to do it when you don't know the line ? I got another program that fills the file and the batch must delete the first line. Does anyone know how to do it ?
I tried with something thet reads a line from an index and then use what I got with findstr, but it doesn't work:
#echo off
setlocal EnableDelayedExpansion
set count=1
for /f "tokens=*" %%a in (test.txt) do (
if !count! equ 1 (set "TIMER=%%a")
if !count! equ 1 (type test.txt | findstr /v %TIMER%)
set /a count+=1
)
echo %TIMER%
timeout %TIMER%
for /f "tokens=*" %%a in (test.txt) do (
echo %%a
)
pause
It tells me : FINDSTR : wrong command line
(the piece of code for the loop on the lines of the file and find a specific line was found on the internet)
So where is the problem ? Or maybe does someone know something like delete(x) and it deletes the line ? Just something that takes an INDEX... ^^'
(The last for loop is used to check if the line was removed btw)
Thanks by advance for any help !
To remove the first n lines from a file without converting tabs into spaces:
#echo off
>RemoveFirstLine.txt <test.txt (
FOR /L %%N in (1 1 1) do set/p"=" SKIP N lines
%__APPDIR__%findstr.exe /R "^"
)
This works as FINDSTR moves the pointer from STDIN, while FIND and MORE do not.
Your code mainly doesn't work because the lack of delayed expansion.
But there are easier ways to achieve your goal. If Compo's suggestion (more.com" +1 "test.txt" > "modifiedtest.txt") doesn't work for you (need to keep TABs or need to remove another line than the first one), the following might work for you:
#echo off
setlocal
set file=test.txt
set lineToDelete=1
(for /f "tokens=1* delims=:" %%a in ('findstr /N "^" "%file%" ^|findstr /bv "%lineToDelete%:"') do echo/%%b) > "%file%.tmp
move /y "%file.tmp%" "%file%"
Not able to substring the dynamic variable inside forloop in Windows batch script.
I have the properties file in my git hub in the below format.
"collectionName=TestCollectionRun.json=test"
So I have written the below code to fetch this values.But the requirement is that I need to strip of the '.json' part from collection name.With the below code I am not able to set/echo that value.Can you please help on this!
#ECHO ON
:BEGIN
IF EXIST "test.properties" ECHO Found properties file, reading file..
SET props=test.properties
setlocal EnableDelayedExpansion
For /F "delims== tokens=1,2,3" %%G in (%props%) Do (
if "%%I" EQU "test" if "%%G" EQU "collectionName" SET collName=%%H(
SET finalCollName=%collName%:~0,-5
ECHO %finalCollName%
)
)
:END
We need the ECHO to return "TestCollectionRun".currently its not returning anything.
For /F "delims== tokens=1,2,3" %%G in (%props%) Do (
if "%%I" EQU "test" if "%%G" EQU "collectionName" SET "collName=%%~nH"&echo %%~nH
)
ECHO %CollName%
Note the second ) is now redundant. Your problem has to do with delayedexpansion which you are invoking but not using. call %%collname%% within the for loop would have shown the value after assignment if required.
This code works by interpreting %%H as a filename and assigning simply the name part of %%H (%%~nH)
Given a line content of collectionName=TestCollectionRun.json=test, here's a quick rewrite of what I think you're tring to do:
#Echo Off
Set "props=test.properties"
If Not Exist "%props%" (
Echo Properties file not found!
Echo Exiting..
Timeout /T 3 /NoBreak >NUL
Exit /B
)
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" Echo %%~nB
)
Pause
If you wanted to do something with the collection name within the loop then you would probably need to use delayed expansion:
#Echo Off
SetLocal DisableDelayedExpansion
Set "props=test.properties"
If Not Exist "%props%" (
Echo Properties file not found!
Echo Exiting..
Timeout /T 3 /NoBreak >NUL
Exit /B
)
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" (
Set "collName=%%B"
SetLocalEnableDelayedExpansion
Echo !collName!
Rem Perform substring task on the variable named collName
Set "finalCollName=!collName%:~0,-5!"
Echo !finalCollName!
EndLocal
)
)
Pause
Note, these answers will not work, as is, if your string is surrounded by doublequotes, (as in your question body), or if the line content differs (e.g. begins with spaces or tabs).
[Edit /]
Looking at your 'after the fact' question in the comments, it is clear that you do not need to substring the variable at all, so should use the first method posted:
Echo Found properties file, reading file..
For /F "UseBackQ Tokens=1-3 Delims==" %%A In ("%props%") Do (
If /I "%%C" == "test" If /I "%%A" == "collectionName" (
newman run "%%B" -e "%envName%" --insecure --reporters cli,htmlextra --reporter-htmlextra-export "newman\%BUILD_NUMBER%\%%~nB.html" --disable-unicode
)
)
Pause
This assumes that both %envName% and %BUILD_NUMBER% have been previously defined correctly.
So I am using a batch to go through a text file line by line. The reason why I can't do a regular search is because I need to see what is written on each line and then determine how I want to process the next line. Anyway, my basic code is:
for /f "usebackq delims=" %%a in (%SourceLicenseFile%) do (
Set LineofFile=%%a
echo !LineofFile! | find /i "TIMESTAMP" >nul
if !errorlevel! EQU 0 (
FOR %%b IN (!LineofFile!) DO SET LogDate=%%b
)
echo %%a | find /i "(LOGTIME) IN" >nul
if !errorlevel! EQU 0 (
FOR %%d IN (!LineofFile!) DO SET Username=%%d
FOR /f "tokens=1" %%c IN ("!LineofFile!") DO (
SET Logtime=%%c
)
echo !LogDate! !LogTime! !Username!
>> !ExportLicenseLog! echo !LogDate! !LogTime! !Username!
)
)
I believe the echo is slowing down this processing. I have a file with thousands of lines and it would take me over a minute to process. Is there another method I can use to go through each line quicker?
Yes, the echo ... | find ... slowdown the process, not because the echo, but for the pipe...
You have not clearly explained the purpose of your program. However, to check if !LineofFile! variable have the "TIMESTAMP" string, you may use this code:
if "!LineofFile:TIMESTAMP=!" neq "!LineofFile!" (
echo TIMESTAMP string found in LineofFile variable
)
This code means: "Try to remove TIMESTAMP string from LineofFile variable. If the string was found, it was removed, so the replaced value is different than the original".
You don't need the echo:
find /i "SOMETHING"<%SourceLicenseFile%
Escape thus ^< inside a for parentheses.
OK. I got it. Using substitution is MUCH faster
for /f "usebackq delims=" %%a in (%SourceLicenseFile%) do (
Set LineofFile=%%a
if not "x!LineofFile:TIMESTAMP=!" EQU "x!LineofFile!" (
FOR %%b IN (!LineofFile!) DO SET LogDate=%%b
)
if not "x!LineofFile:IN:=!" EQU "x!LineofFile!" (
FOR /f "tokens=1,5" %%c IN ("!LineofFile!") DO (
SET Logtime=%%c
SET Username=%%d
)
if "!LogDate!" NEQ "DATE NOT FOUND" (
echo !LogDate! !LogTime! !Username!
>> !ExportLicenseLog! echo !LogDate! !LogTime! !Username!
)
)
Set /A LineNumber=!LineNumber!+1
Set /A PercentDone=!LineNumber!/%TotalLines%
)
Is it possible to remove duplicate rows from a text file? If yes, how?
Sure can, but like most text file processing with batch, it is not pretty, and it is not particularly fast.
This solution ignores case when looking for duplicates, and it sorts the lines. The name of the file is passed in as the 1st and only argument to the batch script.
#echo off
setlocal disableDelayedExpansion
set "file=%~1"
set "sorted=%file%.sorted"
set "deduped=%file%.deduped"
::Define a variable containing a linefeed character
set LF=^
::The 2 blank lines above are critical, do not remove
sort "%file%" >"%sorted%"
>"%deduped%" (
set "prev="
for /f usebackq^ eol^=^%LF%%LF%^ delims^= %%A in ("%sorted%") do (
set "ln=%%A"
setlocal enableDelayedExpansion
if /i "!ln!" neq "!prev!" (
endlocal
(echo %%A)
set "prev=%%A"
) else endlocal
)
)
>nul move /y "%deduped%" "%file%"
del "%sorted%"
This solution is case sensitive and it leaves the lines in the original order (except for duplicates of course). Again the name of the file is passed in as the 1st and only argument.
#echo off
setlocal disableDelayedExpansion
set "file=%~1"
set "line=%file%.line"
set "deduped=%file%.deduped"
::Define a variable containing a linefeed character
set LF=^
::The 2 blank lines above are critical, do not remove
>"%deduped%" (
for /f usebackq^ eol^=^%LF%%LF%^ delims^= %%A in ("%file%") do (
set "ln=%%A"
setlocal enableDelayedExpansion
>"%line%" (echo !ln:\=\\!)
>nul findstr /xlg:"%line%" "%deduped%" || (echo !ln!)
endlocal
)
)
>nul move /y "%deduped%" "%file%"
2>nul del "%line%"
EDIT
Both solutions above strip blank lines. I didn't think blank lines were worth preserving when talking about distinct values.
I've modified both solutions to disable the FOR /F "EOL" option so that all non-blank lines are preserved, regardless what the 1st character is. The modified code sets the EOL option to a linefeed character.
New solution 2016-04-13: JSORT.BAT
You can use my JSORT.BAT hybrid JScript/batch utility to efficiently sort and remove duplicate lines with a simple one liner (plus a MOVE to overwrite the original file with the final result). JSORT is pure script that runs natively on any Windows machine from XP onward.
#jsort file.txt /u >file.txt.new
#move /y file.txt.new file.txt >nul
you may use uniq http://en.wikipedia.org/wiki/Uniq from UnxUtils http://sourceforge.net/projects/unxutils/
Some time ago I found an unexpectly simple solution, but this unfortunately only works on Windows 10: the sort command features some undocumented options that can be adopted:
/UNIQ[UE] to output only unique lines;
/C[ASE_SENSITIVE] to sort case-sensitively;
So use the following line of code to remove duplicate lines (remove /C to do that in a case-insensitive manner):
sort /C /UNIQUE "incoming.txt" /O "outgoing.txt"
This removes duplicate lines from the text in incoming.txt and provides the result in outgoing.txt. Regard that the original order is of course not going to be preserved (because, well, this is the main purpose of sort).
However, you sould use these options with care as there might be some (un)known issues with them, because there is possibly a good reason for them not to be documented (so far).
The Batch file below do what you want:
#echo off
setlocal EnableDelayedExpansion
set "prevLine="
for /F "delims=" %%a in (theFile.txt) do (
if "%%a" neq "!prevLine!" (
echo %%a
set "prevLine=%%a"
)
)
If you need a more efficient method, try this Batch-JScript hybrid script that is developed as a filter, that is, similar to Unix uniq program. Save it with .bat extension, like uniq.bat:
#if (#CodeSection == #Batch) #then
#CScript //nologo //E:JScript "%~F0" & goto :EOF
#end
var line, prevLine = "";
while ( ! WScript.Stdin.AtEndOfStream ) {
line = WScript.Stdin.ReadLine();
if ( line != prevLine ) {
WScript.Stdout.WriteLine(line);
prevLine = line;
}
}
Both programs were copied from this post.
set "file=%CD%\%1"
sort "%file%">"%file%.sorted"
del /q "%file%"
FOR /F "tokens=*" %%A IN (%file%.sorted) DO (
SETLOCAL EnableDelayedExpansion
if not [%%A]==[!LN!] (
set "ln=%%A"
echo %%A>>"%file%"
)
)
ENDLOCAL
del /q "%file%.sorted"
This should work exactly the same. That dbenham example seemed way too hardcore for me, so, tested my own solution. usage ex.: filedup.cmd filename.ext
Pure batch - 3 effective lines.
#ECHO OFF
SETLOCAL
:: remove variables starting $
FOR /F "delims==" %%a In ('set $ 2^>Nul') DO SET "%%a="
FOR /f "delims=" %%a IN (q34223624.txt) DO SET $%%a=Y
(FOR /F "delims=$=" %%a In ('set $ 2^>Nul') DO ECHO %%a)>u:\resultfile.txt
GOTO :EOF
Works happily if the data does not contain characters to which batch has a sensitivity.
"q34223624.txt" because question 34223624 contained this data
1.1.1.1
1.1.1.1
1.1.1.1
1.2.1.2
1.2.1.2
1.2.1.2
1.3.1.3
1.3.1.3
1.3.1.3
on which it works perfectly.
Did come across this issue and had to resolve it myself because the use was particulate to my need.
I needed to find duplicate URL's and order of lines was relevant so it needed to be preserved. The lines of text should not contain any double quotes, should not be very long and sorting cannot be used.
Thus I did this:
setlocal enabledelayedexpansion
type nul>unique.txt
for /F "tokens=*" %%i in (list.txt) do (
find "%%i" unique.txt 1>nul
if !errorlevel! NEQ 0 (
echo %%i>>unique.txt
)
)
Auxiliary: if the text does contain double quotes then the FIND needs to use a filtered set variable as described in this post: Escape double quotes in parameter
So instead of:
find "%%i" unique.txt 1>nul
it would be more like:
set test=%%i
set test=!test:"=""!
find "!test!" unique.txt 1>nul
Thus find will look like find """what""" file and %%i will be unchanged.
I have used a fake "array" to accomplish this
#echo off
:: filter out all duplicate ip addresses
REM you file would take place of %1
set file=%1%
if [%1]==[] goto :EOF
setlocal EnableDelayedExpansion
set size=0
set cond=false
set max=0
for /F %%a IN ('type %file%') do (
if [!size!]==[0] (
set cond=true
set /a size="size+1"
set arr[!size!]=%%a
) ELSE (
call :inner
if [!cond!]==[true] (
set /a size="size+1"
set arr[!size!]=%%a&& ECHO > NUL
)
)
)
break> %file%
:: destroys old output
for /L %%b in (1,1,!size!) do echo !arr[%%b]!>> %file%
endlocal
goto :eof
:inner
for /L %%b in (1,1,!size!) do (
if "%%a" neq "!arr[%%b]!" (set cond=true) ELSE (set cond=false&&goto :break)
)
:break
the use of the label for the inner loop is something specific to cmd.exe and is the only way I have been successful nesting for loops within each other. Basically this compares each new value that is being passed as a delimiter and if there is no match then the program will add the value into memory. When it is done it will destroy the target files contents and replace them with the unique strings
So the situation is like so... I have two nested if statements and then a loop inside them (using the GoTo command and an incremented variable - for loop simulation :D). As you probably know to assign new values to variables inside of parentheses (of an if statement) you have to use delayedexpansion. Also to use variables in the for command you have to double the percent marks like so %%. I want to set the tokens in a for /f command to be the value of the variables I'd like. The problem is doubling the exclamation marks has no effect. I also tried all sorts ... like using quotes, escaping those quotes, using quote alternatives, but it was all to no avail. If you can help in any way that would be just great, because I can't think of anything at all :(. Thank you in advance guys!
If that made no sense here's the code:
#echo off
set FilePath=test.bat
set RefreshRate=3
setlocal enabledelayedexpansion enableextensions
:GetData
if defined FilePath (
if exist "%FilePath%" (
:GetLines
cls
:: This is how I find out how many lines there is in the file
set "cmd=findstr /R /N "^^" "%FilePath%" | find /C ":""
for /f %%a in ('!cmd!') do set Lines=%%a
:ShowCode
cls
set LineNum+=1
if ""!LineNum!"" GTR ""!Lines!"" GoTo Refresh
::THIS IS THE MAIN PROBLEM
for /f "tokens=%%LineNum%% delims=$" %%b in ("%FilePath%") do (
set Line%LineNum%=%%b
echo !LineNum!. | !Line%LineNum%!
GoTo ShowCode
)
)
)
:Refresh
ping localhost -n %RefreshRate% >nul
GoTo GetData
I'm sorry that I didn't have enough time to make it more readable, but it should make the whole thing a little clearer.
First: do not use neither :: remark comments nor :label in a code block enclosed in () parentheses. A proof of harmfulness you could find in the labels.bat script encoded in output from a script provided thereinafter; an explanation here: Comments within bracketed code blocks.
In next script, non-empty lines of a particular plain text file (cf. set "FilePath=labels.bat") are saved to a pseudo-array LineAAA, where index AAA = line number. I do not know whether it isn't off topic according to your question but could give some useful clue...
#echo off
setlocal enabledelayedexpansion enableextensions
cls
set line
set "FilePath=labels.bat"
set /A "RefreshRate=3"
:GetData
if not defined FilePath (
echo %%FilePath%% not defined
goto :eof
)
if not exist "%FilePath%" (
echo %FilePath% does not exist
goto :eof
rem following 'else' (sub)statement seems to be superabundant
) else (
rem GetLines
rem This is how I find out how many lines there is in the file
set "cmd=findstr /R /N "^^" "%FilePath%" | find /C ":""
for /f %%a in ('!cmd!') do set /A "Lines=%%a"
set /A "LineNum=0"
set "Line000=#rem %FilePath%"
for /f "tokens=*" %%b in (%FilePath%) do (
set /A "LineNum+=1"
set "LineX=000000000!LineNum!"
set "LineX=!LineX:~-3!"
set "Line!LineX!=%%b"
)
call :Refresh
)
set line
rem pause
endlocal
goto :eof
:Refresh
ping localhost -n %RefreshRate% | findstr /I "Packets: statistics"
rem >nul
GoTo :eof
Output:
Environment variable line not defined
Ping statistics for ::1:
Packets: Sent = 3, Received = 3, Lost = 0 (0% loss),
Line000=#rem labels.bat
Line001=#SETLOCAL enableextensions enabledelayedexpansion
Line002=#ECHO ON >NUL
Line003=if ""=="" (
Line004=rem comment
Line005=#echo rem comment
Line006=)
Line007=if ""=="" (
Line008=:: comment
Line009=#echo :: comment
Line010=)
Line011=if ""=="" (
Line012=:label
Line013=#echo :label
Line014=)
Line015=#ENDLOCAL
Line016=#goto :eof
LineNum=16
Lines=16
LineX=016
labels.bat output:
d:\bat>labels.bat
d:\bat>if "" == "" (
rem comment
)
rem comment
d:\bat>if "" == "" (#echo :: comment )
'#echo' is not recognized as an internal or external command,
operable program or batch file.
d:\bat>if "" == "" (#echo :label )
'#echo' is not recognized as an internal or external command,
operable program or batch file.
d:\bat>
Tokens property don't go in the brackets they go in quotes and are wrong anyway. It's the nth to nth delimited term.
Type
for /?
for examples