I have a log file which I need to read in, line by line and pipe the line to a next loop.
Firstly I grep the logfile for the "main" word (like "error") in a separate file - to keep it small. Now I need to take the seperate file and read it in line by line - each line needs to go to another loop (in these loop I grep the logs and divide it in blocks) but I stuck here.
The log looks like
xx.xx.xx.xx - - "http://www.blub.com/something/id=?searchword-yes-no" 200 - "something_else"
with a for /f loop I just get the IP instead of the complete line.
How can I pipe/write/buffer the whole line? (doesn't matter what is written per line)
Try this:
#echo off
for /f "tokens=*" %%a in (input.txt) do (
echo line=%%a
)
pause
because of the tokens=* everything is captured into %a
edit:
to reply to your comment, you would have to do that this way:
#echo off
for /f "tokens=*" %%a in (input.txt) do call :processline %%a
pause
goto :eof
:processline
echo line=%*
goto :eof
:eof
Because of the spaces, you can't use %1, because that would only contain the part until the first space. And because the line contains quotes, you can also not use :processline "%%a" in combination with %~1. So you need to use %* which gets %1 %2 %3 ..., so the whole line.
The "call" solution has some problems.
It fails with many different contents, as the parameters of a CALL are parsed twice by the parser.
These lines will produce more or less strange problems
one
two%222
three & 333
four=444
five"555"555"
six"&666
seven!777^!
the next line is empty
the end
Therefore you shouldn't use the value of %%a with a call, better move it to a variable and then call a function with only the name of the variable.
#echo off
SETLOCAL DisableDelayedExpansion
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ t.txt"`) do (
set "myVar=%%a"
call :processLine myVar
)
goto :eof
:processLine
SETLOCAL EnableDelayedExpansion
set "line=!%1!"
set "line=!line:*:=!"
echo(!line!
ENDLOCAL
goto :eof
This has worked for me in the past and it will even expand environment variables in the file if it can.
for /F "delims=" %%a in (LogName.txt) do (
echo %%a>>MyDestination.txt
)
For those with spaces in the path, you are going to want something like this:
n.b. It expands out to an absolute path, rather than relative, so if your running directory path has spaces in, these count too.
set SOURCE=path\with spaces\to\my.log
FOR /F "usebackq delims=" %%A IN ("%SOURCE%") DO (
ECHO %%A
)
To explain:
(path\with spaces\to\my.log)
Will not parse, because spaces.
If it becomes:
("path\with spaces\to\my.log")
It will be handled as a string rather than a file path.
"usebackq delims="
See docs will allow the path to be used as a path (thanks to Stephan).
Related
I have file s_result.txt as:
AAA,BBB,CCC
DDD,EEE
FFF,GGG
HHH,III,JJJ
...
And I try to get the sf_result.txt like this:
AAA,BBB
AAA,CCC
DDD,EEE
FFF,GGG
HHH,III
HHH,JJJ
...
I used script as below:
REM Transfer s_result.txt to sf_result.txt
DEL sf_result.txt
Echo. 2>sf_result.txt
for /F "tokens=1,2,3 delims=," %%a in (s_result.txt) do (
If %%c EQU [] (
ECHO %%a,%%b>>sf_result.txt
) else (
ECHO %%a,%%b>>sf_result.txt
ECHO %%a,%%c>>sf_result.txt
)
)
I got this result.txt instead:
AAA,BBB
AAA,CCC
DDD,EEE
DDD,
FFF,GGG
FFF,
HHH,III
HHH,JJJ
...
How can I get the right result?
Thanks,
If you want to parse line-by-line, use for /F. If you want to tokenize word-by-word on a single line, use for without the /F. Also, in a basic for loop, Windows already tokenizes on unquoted commas with no need to specify a delimiter. (It also tokenizes on spaces, tabs, and semicolons.) With this in mind, the solution is actually pretty simple.
#echo off & setlocal
for /f "usebackq tokens=1* delims=," %%I in ("test.txt") do (
for %%x in (%%J) do (
echo(%%I,%%x
)
)
Output:
AAA,BBB
AAA,CCC
DDD,EEE
FFF,GGG
HHH,III
HHH,JJJ
There have already been great answers provided with some smart approaches.
However, I want to stick to the code you posted here.
The main problem is the line if %%c EQU [], because it compares the third token with the literal string []; the third token can be CCC, JJJ, or an empty string, according to your example, so the condition is never going to be fulfilled.
To correct that, you should write if "%%c"=="" instead.
You could further improve your script by doing a single redirection > to the output file rather than creating it in advance and appending multiple times; just put the entire for /F loop in between parentheses and redirect the whole block.
So here is the corrected and improved code:
rem Transfer s_result.txt to sf_result.txt
> "sf_result.txt" (
for /F "usebackq tokens=1-3 delims=," %%a in ("s_result.txt") do #(
if "%%c"=="" (
echo %%a,%%b
) else (
echo %%a,%%b
echo %%a,%%c
)
)
)
The # symbol prevents command echoes of the loop body to be written to the output file as well. If there is #echo off placed at the beginning of your script you do no longer need that symbol.
Of course this code cannot handle lines with more than three tokens correctly.
I do like rojos clever approach (+1).
To overcome the implications he mentions I think of a recursive approach.
:: Q:\Test\2018\06\28\SO_51073893.cmd
#echo off & setlocal
for /f "usebackq tokens=1-2* delims=," %%A in ("test.txt") do Call :Sub "%%A" "%%B" "%%C"
Goto :Eof
:Sub
Echo %~1,%~2
if "%~3" neq "" for /f "tokens=1* delims=," %%D in (%3) do Call :Sub %1 "%%D" "%%E"
With a slightly changed file test.txt I get this output:
> SO_51073893.cmd
AAA,B=B
AAA,C;C
DDD,EEE
FFF,GGG
HHH,I I
HHH,JJJ
folks!
I'm not sure if this is a very simple task or a very complicated one, but either way I'm struggling with it. Suppose I have a text file like this:
11111FOO
11111BAR
22222ZOOM
33333FOO
11111CAR
I want to figure out a command line in windows that I can plop into a batch file that will pull out text strings from this file and push them to a new file. I would pass in the leading string to search for, and it would take everything from the end of that string to the next new line.
So using the above example, if I said the leading string was 11111, I would get a new text file that looked like this:
FOO
BAR
CAR
Everything else would be ignored.
Thanks!
If there are no potential poison characters in the input file then perhaps this would suffice:
#(For /F "UseBackQ Delims=" %%A In ("input.txt") Do #(Set "an=%%A"
For /L %%B In (%%A 1 %%A) Do #Call Echo %%an:*%%B=%%))>"output.txt"
Magoo's additions:
#Echo Off
(For /F "UseBackQ Delims=" %%A In ("q46858215.txt") Do (Set "an=%%A"
For /L %%B In (%%A 1 %%A) Do Call Echo %%an:*%%B=%%))>"Output1.txt"
If "%~1"=="" GoTo :Next
(For /F "Delims=" %%A In ('FindStr/BLC:"%~1" "q46858215.txt"') Do (
Set "an=%%A"
For /L %%B In (%%A 1 %%A) Do Call Echo %%an:*%%B=%%))>"Output2.txt"
:Next
(For /F "UseBackQ Delims=" %%A In ("q46858215.txt") Do (Set "an=%%A"
For /L %%B In (%%A 1 %%A) Do Echo %%B))>"Output3.txt"
I used a file named q46858215.txt containing OP's data for my testing.
Produces three output files:
Output1.txt : Compo's original
Output2.txt : filtered to isolate lines beginning with (first parameter to routine)
Output3.txt : Compo's original in reverse, showing the numbers isolated
I have done some researching on this topic but everything I have tried doesn't work properly. I just want to add a counter to certain aspects of my batch file. The count.txt file contains:
Counters started on 2-18-15
opened: 0
actions: 0
The script that I have written so far is:
setlocal enabledelayedexpansion
for /F "usebackq tokens=2" %%r in (`findstr opened: counter.txt`) do (
echo %%r
set opened=%%r
set /a opened=!opened!+1
echo opened= !opened!
)
I would like to just edit the number of opened times and not change anything else in the file. I know this is rememdial but I am still very remedial in my batch abilities.
I have specified ! in place of %, using some debugging it is pulling the correct variable and adding 1 to it, the current issue is that it is not saving over that variable afterwards.
You need to use delayed expansion to properly display variables set inside of code blocks.
setlocal enabledelayedexpansion
for /F "usebackq tokens=2" %%r in (`findstr /C:"opened:" counter.txt`) do (
set opened=%%r
set /a opened=!opened!+1
echo opened=!opened!
)
You never write the information back to the file - you are simply echoing the value to the screen.
Batch does not have a mechanism to modify a value in a file directly. You must write a new file with the updated information, and then replace the old file with the new.
Assuming the order of the lines does not matter, I would write a batch script as follows:
#echo off
setlocal
for /f "tokens=2" %%N in ('findstr /bc:"opened: " counter.txt') do set /a opened=%%N+1
>counter.txt.new (
findstr /vbc:"opened: " counter.txt
echo opened: %opened%
)
move /y counter.txt.new counter.txt >nul
After running the script once, your counter.txt file would look like:
Counters started on 2-18-15
actions: 0
opened: 1
You might want to preserve the order of the rows. You could do so with a pure batch script, but I rarely use batch to modify text files. Instead, I would use my JREPL.BAT utility - it is faster, more reliable, and simpler (assuming you understand regular expressions and rudimentary JScript)
call jrepl "^(opened:\s*)(\d+)$" "$1+(Number($2)+1)" /j /f test.txt /o -
#ECHO OFF
SETLOCAL
(
for /F "delims=" %%a in (q28585447.txt) do (
for /F "tokens=1,2" %%q in ("%%a") do (
IF "%%q"=="opened:" (CALL :REPLACE %%r) ELSE (ECHO %%a)
)
)
)>newfile.txt
GOTO :EOF
:REPLACE
SET /a newvalue=%1+1
ECHO(opened: %newvalue%
GOTO :eof
I used a file named q28585447.txt containing your data for my testing.
Produces newfile.txt To replace your file, use move /y newfile.txt q28585447.txt (after modifying throughout for your filename, of course.
Simply, it reads each line from the data file to %%a then tokenises %%a using the default separators into %%q and %%r If %%q is the target string, then %%r contains the value to be incremented, otherwise regurgitate the line in %%a.
The subroutine :replace simlpy adds 1 to the parameter passed and reconstructs the target line.
By surrounding the entire for...%%a loop in parentheses and redirecting to a file, all of the output echoed is redirected to that file.
I have a file (MyFile.txt) with a string like this, for example:
var1,var2,asds,123,var6
This file is generated automatically, and I don't know how many words it will have.
I make a for to print all words, one in each line. And process them one by one
for /f "tokens=1-30 delims=," %%a in (MyFile.txt) do call Process.bat %%a
In Process.bat I have ping %1 for example.
This only processes the first token. My problem is, I need to process all tokens, but I need to do it automatically, is there a way to loop over all existing tokens?
In my example: var1,var2,asds,123,var6 it would be: do call Process.bat %%a, then %%b, then %%c, then %%d and %%e.
Thank you.
This will work even if you have various line in Myfile.txt
#echo off&cls
for /f "delims=" %%a in (MyFile.txt) do for %%b in (%%a) do call Process.bat %%b
What about this:
set /p var=<MyFile.txt
set var="%var%"
for %%a in ("%var:,=" "%) do call Process.bat %%a
That Should work.
Mona
you can combine for to read all your lines and call to parse the vars in the line
try this to get you started
#echo off
setlocal enabledelayedexpansion
for /f %%a in (file.txt) do (
set line=%%a
set line=!line:,= !
call :parms !line!
)
goto :eof
:parms
if "%1"=="" goto :eof
echo ping %1
shift
goto :parms
goto :eof
I am creating a code that strips through different MAC addresses randomly, but cannot figure out how to do this. My thought on how to approach this is to randomize or rearrange the order of the MAC address in the text file with this script, but I cannot figure out how to do this with a batch file. How this will work is that it will read "maclist.txt", then create a new temp file with the random order "maclist_temp.txt", that will be the rearranged file. Then, it will pull this randomized file in order.
I have tried Google and searching the web, but I haven't found anything too useful. I'm still actively looking, but any advice would be extremely useful.
Something as simple as extracting and deleting a random line and then adding to the bottom might work. Randomization would be better though, but I want to keep the original list. Something like:
Make a temp copy of maclist.txt called maclist_temp.txt
Take one random MAC address, remove it from maclist_temp.txt
Readd it to the bottom
That is all I want, but any suggestions are welcome.
You may try this batch file to help you to shuffle your maclist.txt. The usage of the batch code is
C:\> type list.txt | shuffle.bat > maclist_temp.txt
Here are the contents of shuffle.bat:
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET TmpFile=tmp%RANDOM%%RANDOM%.tmp
TYPE NUL >%Tmpfile%
FOR /F "tokens=*" %%i IN ('MORE') DO SET Key=!RANDOM!!RANDOM!!RANDOM!000000000000& ECHO !Key:~0,15!%%i>> %TmpFile%
FOR /F "tokens=*" %%i IN ('TYPE %TmpFile% ^| SORT') DO SET Line=%%i&ECHO.!Line:~15!
::DEL %TmpFile%
ENDLOCAL
After issuing the above command, maclist_temp.txt will contain a randomized list of MAC addresses.
Hope this helps.
Here is a simpler method to randomize/randomise a file, no temp files needed. You can even reuse the same input filename.
Limitations are: blank lines and line starting with ; will be skipped, and lines starting with = will have all leading = signs stripped and ^ characters are doubled.
#echo off
setlocal
for /f "delims=" %%a in (maclist.txt) do call set "$$%%random%%=%%a"
(for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b)>newmaclist.txt
endlocal
I really like foxidrive's approach. Nevertheless I want to provide a solution with all the listed limitations eliminated (although cmd-related restrictions like file sizes < 2 GiB and line lengths < ~ 8 KiB remain).
The key is delayed expansion which needs to be toggled to not lose explamation marks. This solves all the potential problems with special characters like ^, &, %, !, (, ), <, >, | and ".
The counter index has been implemented in order not to lose a single line of the original text file, which could happen without, because random may return duplicate values; with index appended, the resulting variable names $$!random!.!index! are unique.
The findstr /N /R "^" command precedes every line of the original file with a line number followed by a colon. So no line appears empty to the for /F loop which would ignore such. The line number also implicitly solves the issue with leading semicolons, the default eol character of for /F.
Finally, everything up to and including the first colon (remember the said prefix added by findstr) is removed from every line before being output, hence no more leading equal-to signs are dismissed.
So here is the code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set /A "index=0"
for /f "delims=" %%a in ('findstr /N /R "^" "%~dpn0.lst"') do (
setlocal EnableDelayedExpansion
for /F %%b in ("$$!random!.!index!") do (
endlocal
set "%%b=%%a"
)
set /A "index+=1"
)
> "%~dpn0.new" (
for /f "delims=" %%a in ('set $$') do (
set "item=%%a"
setlocal EnableDelayedExpansion
echo(!item:*:=!
endlocal
)
)
endlocal
exit /B
This seems to work. Feed it a command line parameter of the file to randomize.
#echo off
setlocal EnableDelayedExpansion
rem read the number of lines in the file
rem the find prepends the line number so we catch blank lines
for /f "delims=" %%n in ('find /c /v "" %1') do set "len=%%n"
set len=!len:*: =!
rem echo %1 has %len% lines
rem Relocate as many lines as there are lines in the file
for /l %%j in (1 1 !len!) do (
rem echo starting round %%j
rem geta random number between 1 and the number of lines in the file
set /a var=!random! %% !len! + 1
rem echo relocating line !var!
rem make sure there is no temp file
if exist %1.temp del %1.temp
rem read each line of the file, write any that don't match and then write the one that does
<%1 (
for /l %%i in (1 1 !len!) do (
rem if it is the target line then save it
if %%i == !var! (
set /p found=
rem echo saving !found!
)
rem if it is the target line then write it
if not %%i == !var! (
set /p other=
rem echo writing !other!
echo !other!>> %1.temp
)
)
rem now write the target line at the end
rem echo appending !found!
echo !found!>> %1.temp
)
rem replace the original with the temp version
move %1.temp %1>nul
)
rem print the result
type %1
Place in cmd file
for /f "tokens=2 delims=/" %%m in ('cmd /e:on /v:on /c "for /f %%f in (maclist.txt) do #echo !random!/%%f" ^| sort') do echo %%m
It spawns a cmd which reads the mac list in the inner for, prefixes a random value and a slash to the mac and sorts the list. Then this list is splitted in the outter for using the slash as delimiter and printing the mac address.