Findstr based on column1 only [batch file] - batch-file

I have been searching through StackOverflow but could not find an answer that hits the mark. I have 2 .txt files to compare and return a 3rd one where differences exist.
However, only the first column of the first 2 files need a comparison.
E:\Compare_flie\file_1.txt
GND ZERO
22XC44 XXYYZZ
33XC55 YYUUTT
E:\Compare_file\file_2.txt
GND ZERO
22XC44 KK77UU
33XC55 88JJ66
66NN77 HHOO99
99CC88 UU77RR
E:\Compare_file\file_3.txt (intended output)
66NN77 HH0099
99CC88 UU77RR
Tried the code below but it is only good at picking out the differences of all the strings in the line
%echo on
findstr /v /i /g:E:\Compare_files\file_1.txt E:\Compare_files\file_2.txt
> E:\Compare_files\file_3.txt
Refined it further but not hitting the mark yet.
%echo on
for /f "tokens=1 delims= " %%I in ("E:\Compare_files\file_1.txt") do
findstr /v /i "%%I"/g:"D:\Compare_files\file_2.txt"
> "D:\Compare_files\file_3.txt"
Appreciate if anyone can assist.

#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q48816766.txt"
SET "filename2=%sourcedir%\q48816766_2.txt"
SET "tempfile=%temp%\q48816766.tmp"
SET "outfile=%destdir%\outfile.txt"
(FOR /f "usebackq" %%a IN ("%filename1%") DO ECHO %%a )>"%tempfile%"
FINDSTR /b /v /g:"%tempfile%" "%filename2%">"%outfile%"
REM DEL "%tempfile%" /F /Q
GOTO :EOF
I've set up names to suit my system, but with the two files containing your data.
Obviously, the usebackq on the for/f is only required if the filename is quoted. The parentheses around the command permit the echoed output to be accumulated into the temporary file. What's important here is the space between the %%a and ). This ensures that the temporary file contains trailing spaces.
Then apply the temporary file to the second data file via /g as in OP's code. The presence of the trailing spaces in the tempfile ensure that the only lines selected for omission are those where the first column exactly matches so for instance had 66NN7 appeared in the first file, first column, then this would not match 66NN77 in the second file.

Here's a method using batch with the type command piping the first file's contents over to the findstr command then passing the arguments accordingly to redirect those results into a temp file.
Using a for /f loop with "usebackq tokens=1 delims= " it will iterate through the temp file and for each line in that file parsing accordingly, it will append the column one lines with an echo command using >> to redirect the results over to file_3.txt with the expected results.
Please note the addition of the if exist "%srcdir%\file_3.txt" del /q /f "%srcdir%\file_3.txt" to delete that file if it exists since the for /f echo commands will append one after the other to it.
#echo on
set srcdir=E:\Compare_files
set tmpfile=%temp%\%~N0.tmp
type "%srcdir%\file_1.txt" | findstr /vig:"%srcdir%\file_2.txt">"%tmpfile%"
if exist "%srcdir%\file_3.txt" del /q /f "%srcdir%\file_3.txt"
for /f "usebackq tokens=1 delims= " %%I in ("%tmpfile%") do (
echo %%~I>>"%srcdir%\file_3.txt"
)
Further Resources
FOR /F
FOR /?
usebackq - specifies that the new semantics are in force,
where a back quoted string is executed as a
command and a single quoted string is a
literal string command and allows the use of
double quotes to quote file names in
file-set.
Redirection
Comparing/finding the difference between two text files using findstr

As I understand your problem, you want the lines from file_2.txt whose first column are not contained in first column of file_1.txt, that is: file_2.txt minus file_1.txt. There is a simpler approach to get such result:
#echo off
setlocal
rem Fill "line" array with lines from file_2.txt
rem use the first column for the array keys
for /F "delims=" %%a in (file_2.txt) do for /F %%b in ("%%a") do set "line[%%b]=%%a"
rem Delete array elements with same key from file_1.txt
for /F %%b in (file_1.txt) do set "line[%%b]="
rem Show remaining elements
(for /F "tokens=1* delims==" %%a in ('set line[') do echo %%b) > file_3.txt

Related

Compare the contents of file1 and file2 and output the result

Batch file that retrieves the string of file1 from file2
file1.txt
aaa.dll
ccc.dll
ddd.sys
file2.txt
aaa.dll=c:\windows\aaa.dll
bbb.dll=c:\windows\bbb.dll
ccc.dll=c:\windows\system32\ccc.dll
ddd.sys=c:\windows\system32\drivers\ddd.sys
eee.log=c:\windows\debug\wia\eee.log
expected result
c:\windows\aaa.dll
c:\windows\system32\ccc.dll
c:\windows\system32\drivers\ddd.sys
test command
for /f "tokrns=*" %%i ('findstr file1.txt file2.txt') do (set result=%%i)
I am assuming that file2.txt is, essentially, a list of "configuration settings"... that is, a list of name-value pairs. Further, file1.txt is a list of names that you want to get the values of.
If that is the case, then you need to run the findstr command for each of the lines in file1.txt. For instance:
getvalues.bat
#echo off
setlocal
set "REQUIRED=file1.txt"
set "SETTINGS=file2.txt"
for /f "usebackq tokens=*" %%a in ( "%REQUIRED%" ) do (
for /f "tokens=1,* delims==" %%b in ( 'findstr /b "%%a=" "%SETTINGS%"' ) do (
echo %%c
)
)
produces the following:
C:\>getvalues.bat
c:\windows\aaa.dll
c:\windows\system32\ccc.dll
c:\windows\system32\drivers\ddd.sys
Notes
I've hard-coded REQUIRED (the list of names to get values for) and SETTINGS (the list of name-value pairs). Depending on your requirements, you could take one or more from the command-line.
The first loop runs over all lines in REQUIRED (=file1.txt). It uses usebackq because the filename is wrapped in double-quotes, just in case it contains spaces.
For each name from the first loop (%%a), we run a findstr command. The /b tells it to look for the pattern at the beginning of the line. I also append an = to the end of the name. Both help prevent accidental partial-matches.
The tokens=1,* delims== splits the output of findstr at the first (or only) equals-sign. The name (before the =) will be assigned to %%b and the value (all the rest) will be assigned to %%c.
You don't need to compare anything in order to get strings (that is, variable values) that have been defined previously:
#echo off
setlocal EnableDelayedExpansion
rem Define the strings from file2.txt
for /F "delims=" %%a in (file2.txt) do set "%%a"
rem Retrieve the strings indicated in file1.txt
for /F %%a in (file1.txt) do echo !%%a!
If you insist in using the "find strings" way:
for /F "tokens=2 delims==" %%a in ('findstr /G:file1.txt file2.txt') do echo %%a

Read a text file, find a character and include the previous 3 lines

I am trying to find a command or batch file that can find specific text in a line of a log file. When the text is found, I want to include the previous 3 lines before the text that is found. I saw some examples using a FOR loop but I can only get one token and not the previous lines. Is there a way to do this without utilities in a batch file?
#echo off
svn log https://sub.mycompany.com/svn/company-dev/ > c:\temp\svn.txt
setlocal EnableDelayedExpansion
rem Assemble the list of line numbers
set JiraID=%1
set numbers=
for /F "delims=:" %%a in ('findstr /I /N /C:"%JiraID%" c:\temp\svn.txt')
do (
set /A beforeo=%%a, before=%%a-1, before2=%%a-2
set "numbers=!numbers!: !beforeo!: !before!: !before2!:"
)
rem Search for the lines
(for /F "tokens=1* delims=:" %%a in ('findstr /N "^" c:\temp\svn.txt ^|
findstr /B "%numbers%"') do echo %%b)
del c:\temp\svn.txt
I actually do not want to dump the SVN log if there is a way to do it inline with SVN LOG command. The file created has spaces when outputed to text,
The output being read has spaces between lines as well and the output sometimes does not include the "-----" lines so it was tricky trying to collect the correct lines. Any ideas?
------------------------------------------------------------------------
r132279 | USERID | 2014-04-30 12:59:09 -0700 (Wed, 30 Apr 2014) | 3 lines
Removed unused "Calculated Fields" column entry.
Jira ID: JIRA-977
------------------------------------------------------------------------
No need for a FOR /F loop. FINDSTR has an undocumented ability to search across line breaks, so a single FINDSTR command can do it! See What are the undocumented features and limitations of the Windows FINDSTR command? for more info.
#echo off
setlocal enableDelayedExpansion
:: Define LF to contain a line feed character (0x0A)
set ^"LF=^
^" The above empty line is critical - DO NOT REMOVE
:: Define CR to contain a carriage return character (0x0D)
for /f %%A in ('copy /Z "%~dpf0" nul') do set "CR=%%A"
:: Define the string to search for
set "s=something"
findstr /r /c:"!s!" /c:"!lf!.*!s!" /c:"!lf!.*!cr!*!lf!.*!s!" /c:"!lf!.*!cr!*!lf!.*!cr!*!lf!.*!s!" test.txt
Since the above search uses a regular expression, any meta-characters in the search string must be escaped. For example, a period in the search string would be represented as \.
Update in response to edited question
It is not necessary to save the original file to a temp file - it can be piped directly into FINDSTR.
#echo off
setlocal enableDelayedExpansion
:: Define LF to contain a line feed character (0x0A)
set ^"LF=^
^" The above empty line is critical - DO NOT REMOVE
:: Define CR to contain a carriage return character (0x0D)
for /f %%A in ('copy /Z "%~dpf0" nul') do set "CR=%%A"
:: Define the string to search for
set "s=%~1"
svn log https://sub.mycompany.com/svn/company-dev/|findstr /ri /c:"!s!" /c:"!lf!.*!s!" /c:"!lf!.*!cr!*!lf!.*!s!" /c:"!lf!.*!cr!*!lf!.*!cr!*!lf!.*!s!"
(Answer before question was edited)
This might be what you're looking for:
#echo off
setlocal enabledelayedexpansion
set l1=
set l2=
set l3=
set findme=test
for /f %%l in (t.txt) do (
if "%%l"=="%findme%" echo.!l1! & echo.!l2! & echo.!l3!
set l1=!l2!
set l2=!l3!
set l3=%%l
)
Set findme to the value you want.

Concatenating files in a Batch script

I have a batch script that would concatenate the content of files that were listed in an index file. It used to work until there were spaces in the paths. I have edited it a bit, but it is something like this:
SET INPUT="C:\Has Spaces In Path\indexfile.txt"
SET ROOT="C:\Has Spaces In Path\inputdirectory\"
SET OUTPUT="C:\Has Spaces In Path\outputdirectory\mergedfile.txt"
FOR /F %%A IN (%INPUT%) DO TYPE "%ROOT%%%A" >> "%OUTPUT%"
The problem I have is that %INPUT% now appears to get tokenised in the for loop and if I put quotes around it (i.e. "%INPUT%") it does not work either. Is there any way I can get this loop to iterate over each line in the file specified by INPUT and concatenate the contents to the OUTPUT file?
Thanks.
Edit: Based on the answer, this did what I wanted:
FOR /F %%A IN ('type "%INPUT%"') DO TYPE "%ROOT%%%A" >> "%MERGED%"
FOR /F %%A IN ('type %INPUT%') DO echo %%A
' in the brackets will cause to execute the statement and uses the output like a file source
FOR /F "usebackq" %%A IN ("%INPUT%") DO TYPE "%ROOT%%%A" >> "%OUTPUT%"
or, better yet:
(FOR /F "usebackq" %%A IN ("%INPUT%") DO TYPE "%ROOT%%%A") > "%OUTPUT%"
For further details, see: FOR /?

How to randomly rearrange lines in a text file using a batch file

I am creating a code that strips through different MAC addresses randomly, but cannot figure out how to do this. My thought on how to approach this is to randomize or rearrange the order of the MAC address in the text file with this script, but I cannot figure out how to do this with a batch file. How this will work is that it will read "maclist.txt", then create a new temp file with the random order "maclist_temp.txt", that will be the rearranged file. Then, it will pull this randomized file in order.
I have tried Google and searching the web, but I haven't found anything too useful. I'm still actively looking, but any advice would be extremely useful.
Something as simple as extracting and deleting a random line and then adding to the bottom might work. Randomization would be better though, but I want to keep the original list. Something like:
Make a temp copy of maclist.txt called maclist_temp.txt
Take one random MAC address, remove it from maclist_temp.txt
Readd it to the bottom
That is all I want, but any suggestions are welcome.
You may try this batch file to help you to shuffle your maclist.txt. The usage of the batch code is
C:\> type list.txt | shuffle.bat > maclist_temp.txt
Here are the contents of shuffle.bat:
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET TmpFile=tmp%RANDOM%%RANDOM%.tmp
TYPE NUL >%Tmpfile%
FOR /F "tokens=*" %%i IN ('MORE') DO SET Key=!RANDOM!!RANDOM!!RANDOM!000000000000& ECHO !Key:~0,15!%%i>> %TmpFile%
FOR /F "tokens=*" %%i IN ('TYPE %TmpFile% ^| SORT') DO SET Line=%%i&ECHO.!Line:~15!
::DEL %TmpFile%
ENDLOCAL
After issuing the above command, maclist_temp.txt will contain a randomized list of MAC addresses.
Hope this helps.
Here is a simpler method to randomize/randomise a file, no temp files needed. You can even reuse the same input filename.
Limitations are: blank lines and line starting with ; will be skipped, and lines starting with = will have all leading = signs stripped and ^ characters are doubled.
#echo off
setlocal
for /f "delims=" %%a in (maclist.txt) do call set "$$%%random%%=%%a"
(for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b)>newmaclist.txt
endlocal
I really like foxidrive's approach. Nevertheless I want to provide a solution with all the listed limitations eliminated (although cmd-related restrictions like file sizes < 2 GiB and line lengths < ~ 8 KiB remain).
The key is delayed expansion which needs to be toggled to not lose explamation marks. This solves all the potential problems with special characters like ^, &, %, !, (, ), <, >, | and ".
The counter index has been implemented in order not to lose a single line of the original text file, which could happen without, because random may return duplicate values; with index appended, the resulting variable names $$!random!.!index! are unique.
The findstr /N /R "^" command precedes every line of the original file with a line number followed by a colon. So no line appears empty to the for /F loop which would ignore such. The line number also implicitly solves the issue with leading semicolons, the default eol character of for /F.
Finally, everything up to and including the first colon (remember the said prefix added by findstr) is removed from every line before being output, hence no more leading equal-to signs are dismissed.
So here is the code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set /A "index=0"
for /f "delims=" %%a in ('findstr /N /R "^" "%~dpn0.lst"') do (
setlocal EnableDelayedExpansion
for /F %%b in ("$$!random!.!index!") do (
endlocal
set "%%b=%%a"
)
set /A "index+=1"
)
> "%~dpn0.new" (
for /f "delims=" %%a in ('set $$') do (
set "item=%%a"
setlocal EnableDelayedExpansion
echo(!item:*:=!
endlocal
)
)
endlocal
exit /B
This seems to work. Feed it a command line parameter of the file to randomize.
#echo off
setlocal EnableDelayedExpansion
rem read the number of lines in the file
rem the find prepends the line number so we catch blank lines
for /f "delims=" %%n in ('find /c /v "" %1') do set "len=%%n"
set len=!len:*: =!
rem echo %1 has %len% lines
rem Relocate as many lines as there are lines in the file
for /l %%j in (1 1 !len!) do (
rem echo starting round %%j
rem geta random number between 1 and the number of lines in the file
set /a var=!random! %% !len! + 1
rem echo relocating line !var!
rem make sure there is no temp file
if exist %1.temp del %1.temp
rem read each line of the file, write any that don't match and then write the one that does
<%1 (
for /l %%i in (1 1 !len!) do (
rem if it is the target line then save it
if %%i == !var! (
set /p found=
rem echo saving !found!
)
rem if it is the target line then write it
if not %%i == !var! (
set /p other=
rem echo writing !other!
echo !other!>> %1.temp
)
)
rem now write the target line at the end
rem echo appending !found!
echo !found!>> %1.temp
)
rem replace the original with the temp version
move %1.temp %1>nul
)
rem print the result
type %1
Place in cmd file
for /f "tokens=2 delims=/" %%m in ('cmd /e:on /v:on /c "for /f %%f in (maclist.txt) do #echo !random!/%%f" ^| sort') do echo %%m
It spawns a cmd which reads the mac list in the inner for, prefixes a random value and a slash to the mac and sorts the list. Then this list is splitted in the outter for using the slash as delimiter and printing the mac address.

DOS batch FOR loop with FIND.exe is stripping out blank lines?

This DOS batch script is stripping out the blank lines and not showing the blank lines in the file even though I am using the TYPE.exe command to convert the file to make sure the file is ASCII so that the FIND command is compatible with the file. Can anyone tell me how to make this script include blank lines?
#ECHO off
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE.exe "build.properties" ^| FIND.exe /V ""`) DO (
ECHO --%%A--
)
pause
That is the designed behavior of FOR /F - it never returns blank lines. The work around is to use FIND or FINDSTR to prefix the line with the line number. If you can guarantee no lines start with the line number delimiter, then you simply set the appropriate delimiter and keep tokens 1* but use only the 2nd token.
::preserve blank lines using FIND, assume no line starts with ]
::long lines are truncated
for /f "tokens=1* delims=]" %%A in ('type "file.txt" ^| find /n /v ""') do echo %%B
::preserve blank lines using FINDSTR, assume no line starts with :
::long lines > 8191 bytes are lost
for /f "tokens=1* delims=:" %%A in ('type "file.txt" ^| findstr /n "^"') do echo %%B
::FINDSTR variant that preserves long lines
type "file.txt" > "file.txt.tmp"
for /f "tokens=1* delims=:" %%A in ('findstr /n "^" "file.txt.tmp"') do echo %%B
del "file.txt.tmp"
I prefer FINDSTR - it is more reliable. For example, FIND can truncate long lines - FINDSTR does not as long as it reads directly from a file. FINDSTR does drop long lines when reading from stdin via pipe or redirection.
If the file may contain lines that start with the delimiter, then you need to preserve the entire line with the line number prefix, and then use search and replace to remove the line prefix. You probably want delayed expansion off when transferring the %%A to an environment variable, otherwise any ! will be corrupted. But later within the loop you need delayed expansion to do the search and replace.
::preserve blank lines using FIND, even if a line may start with ]
::long lines are truncated
for /f "delims=" %%A in ('type "file.txt" ^| find /n /v ""') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
::preserve blank lines using FINDSTR, even if a line may start with :
::long lines >8191 bytes are truncated
for /f "delims=*" %%A in ('type "file.txt" ^| findstr /n "^"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
::FINDSTR variant that preserves long lines
type "file.txt" >"file.txt.tmp"
for /f "delims=*" %%A in ('findstr /n "^" "file.txt.tmp"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
del "file.txt.tmp"
If you don't need to worry about converting the file to ASCII, then it is more efficient to drop the pipe and let FIND or FINDSTR open the file specified as an argument, or via redirection.
There is another work around that completely bypasses FOR /F during the read process. It looks odd, but it is more efficient. There are no restrictions with using delayed expansion, but unfortunately it has other limitations.
1) lines must be terminated by <CR><LF> (this will not be a problem if you do the TYPE file conversion)
2) lines must be <= 1021 bytes long (disregarding the <CR><LF>)
3) any trailing control characters are stripped from each line.
4) it must read from a file - you can't use a pipe. So in your case you will need to use a temp file to do your to ASCII conversion.
setlocal enableDelayedExpansion
type "file.txt">"file.txt.tmp"
for /f %%N in ('find /c /v "" ^<"file.txt.tmp"') do set cnt=%%N
<"file.txt.tmp" (
for /l %%N in (1 1 %cnt%) do(
set "ln="
set /p "ln="
echo(!ln!
)
)
del "file.txt.tmp"
I wrote a very simple program that may serve as replacement for FIND and FINDSTR commands when they are used for this purpose. My program is called PIPE.COM and it just insert a blank space in empty lines, so all the lines may be directly processed by FOR command with no further adjustments (as long as the inserted space don't cares). Here it is:
#ECHO off
if not exist pipe.com call :DefinePipe
FOR /F "USEBACKQ delims=" %%A IN (`pipe ^< "build.properties"`) DO (
ECHO(--%%A--
)
pause
goto :EOF
:DefinePipe
setlocal DisableDelayedExpansion
set pipe=´)€ì!Í!ŠÐŠà€Ä!€ü.t2€ü+u!:æu8²A€ê!´#€ì!Í!².€ê!´#€ì!Í!²+€ê!´#€ì!Í!Šò€Æ!´,€ì!Í!"Àu°´LÍ!ëÒ
setlocal EnableDelayedExpansion
echo !pipe!>pipe.com
exit /B
EDIT: Addendum as answer to new comment
The code at :DefinePipe subroutine create a 88 bytes program called pipe.com, that basically do a process equivalent to this pseudo-Batch code:
set "space= "
set line=
:nextChar
rem Read just ONE character
set /PC char=
if %char% neq %NewLine% (
rem Join new char to current line
set line=%line%%char%
) else (
rem End of line detected
if defined line (
rem Show current line
echo %line%
set line=
) else (
rem Empty line: change it by one space
echo %space%
)
)
goto nextChar
This way, empty lines in the input file are changed by lines with one space, so FOR /F command not longer omit they. This works "as long as the inserted space don't cares" as I said in my answer.
Note that the pipe.com program does not work in 64-bits Windows versions.
Antonio
Output lines including blank lines
Here's a method I developed for my own use.
Save the code as a batch file say, SHOWALL.BAT and pass the source file as a command line parameter.
Output can be redirected or piped.
#echo off
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" ^< "%~1"') do echo.%%ba
exit /b
EXAMPLES:
showall source.txt
showall source.txt >destination.txt
showall source.txt | FIND "string"
An oddity is the inclusion of the '^<' (redirection) as opposed to just doing the following:
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" "%~1"') do echo.%%ba
By omitting the redirection, a leading blank line is output.
Thanks to dbenham, this works, although it is slightly different than his suggestion:
::preserve blank lines using FIND, no limitations
for /f "USEBACKQ delims=" %%A in (`type "file.properties" ^| find /V /N ""`) do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
As mentioned in this answer to the above question, it doesn't seem that lines are skipped by default using for /f in (at least) Windows XP (Community - Please update this answer by testing the below batch commands on your version & service pack of Windows).
EDIT: Per Jeb's comment below, it seems that the ping command, in at least Windows XP, is
causing for /f to produce <CR>'s instead of blank lines (If someone knows specifically why, would
appreciate it if they could update this answer or comment).
As a workaround, it seems that the second default delimited token (<space> / %%b in the example)
returns as blank, which worked for my situation of eliminating the blank lines by way of an "parent"
if conditional on the second token at the start of the for /f, like this:
for /f "tokens=1,2*" %%a in ('ping -n 1 google.com') do (
if not "x%%b"=="x" (
{do things with non-blank lines}
)
)
Using the below code:
#echo off
systeminfo | findstr /b /c:"OS Name" /c:"OS Version"
echo.&echo.
ping -n 1 google.com
echo.&echo.
for /f %%a in ('ping -n 1 google.com') do ( echo "%%a" )
echo.&echo.&echo --------------&echo.&echo.
find /?
echo.&echo.
for /f %%a in ('find /?') do ( echo "%%a" )
echo.&echo.
pause
.... the following is what I see on Windows XP, Windows 7 and Windows 2008, being the only three versions & service packs of Windows I have ready access to:

Resources