I need to split one file when ever the empty records starts,File contains multiple file structure in the same file need to split the files using batch script.
Important, that every sub file is separated by empty record
so the data look like: test.csv
sno,employee name,address,location,zip code
1,aaaa,12/34,Hyderabad,500089
2,bbbb,12/35/44,chennai,500079
3,cccc,12/31/11,pune,500069
Cardnumber,cardname,card type,card limit
12345,visa,diamond,10000
2345,master,platinum,50000
Accno,bank name,branch code,Branch location
98765,sbi,23456,hyd
12457,citi,8765,usa
4444,axis,78767,India
and I need separate file like below
Test1.csv
sno,employye name,address,location,zipcode
1,aaaa,12/34,Hyderabad,500089
2,bbbb,12/35/44,chennai,500079
3,cccc,12/31/11,pune,500069
Test2.csv
Cardnumber,cardname,card type,card limit
12345,visa,diamond,10000
2345,master,platinum,50000
Test3.csv
Accno,bank name,branch code,Branch location
98765,sbi,23456,hyd
12457,citi,8765,usa
4444,axis,78767,india
I have tried below script but it is not working as expected.
#echo off & setlocal enabledelayedexpansion
set c=0
for /f "tokens=*" %%a in (test.csv) do (
if "%%a" equ "\n" (
set /a c+=1
>f!c!.csv echo.
) else (
>> f!c!.csv echo %%a
)
)
Please help me to write the batch script for the above requirement. Thanks in Advance
for /f ignores empty lines. So you need a little trick for lines not to be empty. find /n adds a line number to each line, so it isn't empty any more.
#echo off
setlocal enabledelayedexpansion
set c=0
for /f "tokens=1,* delims=]" %%a in ('type t.txt^|find /n /v ""') do (
REM echo A %%a B %%b
if "%%b" equ "" (
set /a c+=1
>f!c!.csv echo.
) else (
>> f!c!.csv echo %%b
)
)
The REM line is left from troubleshooting, but I left it there because it may help you understand, what's going on.
Related
Test which among the text files in a specific path contains value on its 2nd line and save the filenames to another text file using windows batch file.
This is what Ive done so far.
#echo off
setlocal ENABLEDELAYEDEXPANSION
SET COUNT=0
FOR %%A IN (C:\Users\mark.jim.a.mercado\Desktop\*.txt) DO (
ECHO %%A
)
pause
It will list all the text files in the directory I used but when I tried to save the names of it to another text file (using the code below) only the name of the last text file will be saved.
#echo off
setlocal ENABLEDELAYEDEXPANSION
SET COUNT=0
FOR %%A IN (C:\Users\mark.jim.a.mercado\Desktop\*.txt) DO (
ECHO %%A #echo off
setlocal ENABLEDELAYEDEXPANSION
SET COUNT=0
FOR %%A IN (C:\Users\mark.jim.a.mercado\Desktop\*.txt) DO (
ECHO %%A >C:\Users\mark.jim.a.mercado\Desktop\names.txt
)
pause
I believe the easiest way to retrieve the second line of every text file is input redirection <:
> ".\logfile.log" (
for %%F in (".\*.txt") do (
set "FILE=%%~F"
setlocal EnableDelayedExpansion
< "!FILE!" (
for /L %%I in (1,1,2) do (
set "LINE=" & set /P LINE=""
)
echo 2nd line of "!FILE!": "!LINE!"
)
endlocal
)
)
Since you do not tell us what you want to do with the second lines or how you want to filter them, it is quite impossible to provide more help...
You don't say what value to look for. IMO the easiest is to use findstr to number the lines and another piped findstr to filter for :2: aka second line (if existent)
Suppose there are three file: one.txt two.txt, three.txt with analogous number of lines and content:
> type *.txt
one.txt
one
two.txt
one
two
three.txt
one
two
three
This batch:
#Echo off&SetLocal EnableExtensions EnableDelayedExpansion
For /f "tokens=1,2* delims=:" %%A in (
'findstr /N "^" *.txt ^|findstr ":2:"'
) Do Echo File:%%A Line:%%B Value:%%C
will outout:
File:two.txt Line:2 Value:two
File:three.txt Line:2 Value:two
This is my first posting so if the format is not as it supposed to be please excuse me for this. (Suggestions for
improvement are welcome.)
I am trying to create a batchfile that will read last lines from logfiles and copy them to a new file.
Until now I have found here a way to read the last line.
Code would be something like:
for /f %%i in ('find /v /c "" ^< someFile.txt') do set /a lines=%%i
set /a startLine=%lines% - 1
more /e +%startLine% someFile.txt > lastLines.txt
The above code works for one file at a time. What I need is to read the last line from all files in a known list and add this line to a new .csv file.
I have been using the following code for getting the 4th entry in the logfiles but it returns every line of every logfile:
for /f %%x in (%list%) do for /f "delims=.txt, tokens=4" %%i in (%%x.txt) do echo %%x, %%i >> output.csv
What I would need is a sort of combination of both but I don't know how to combine them and make the complete last line be copied to the .csv file.
===
#Magoo:
Thanx for your reaction.
In every logfile can be 1 to >100 lines with comma separated information. Something like:
"LOGON,6-1-2015,12:43:39,USERNAME,HOSTNAME,,,,192.168.209.242,00:21:5A:2E:64:5E"
The last code with the 4th entry was used to get a list of all accounts that had logged in to the computers. This code gave me a very large list of all logon/logoff events on all computerlogs I checked in %list%.
In %list$ I had all the names of logfiles I wanted to be checked. This returned all lines.
For a new batchfile I need only the last logon/logoff entry and I want the whole last line.
So I have a .txt file with the hostnames of all computers I need to examine.
This .txt file will be read line by line via the variable %list%.
From every logfile I need only the last line copied to an output file.
===
I just tried the solution offered by JosefZ. Unfortunately this does not work for me yet. No lastlines are copied to the resultfile. In the code I removed the extra entry for possible lastlines for there are no empty lines in the logs, I also added an entry for the hostname I want to be available in the result. JosefZ had the filename there:
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set "host=%%~x"
for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
set "filename=.\logs\%filename:&=^&%.txt"
echo %host%,%lastline%>>output.csv
goto :eof
The resultfile shows only the hostnames. I'll puzzle some more with this but all tips are welcome!
===
Got it!!!
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set filename= :: *empty previous filename*
set lastline= :: *empty previous lastline*
set "host=%%~x"
set "filename=.\logs\%host%.txt" :: *creating the filename from path+hostname+extention*
for /F "tokens=*" %%G in ('type "%filename%"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
echo %host%,%lastline%>>output.csv
goto :eof
Your approach with line numbering could fail if a file has more trailing empty lines. Fortunately for /F loop ignores (does not iterate) empty lines; let's put to use this feature: in the script used next practices:
disabledelayedexpansion to allow ! in file names
set "list=_listing.txt" where the _listing.txt contains list of file names (full path and extension .txt including), one file name on one line: got by dir /b /s *.txt>_listing.txt
type nul>files\output.csv to empty the output file (optional)
set "lastline=!!!file empty!!!" to initialize variable %lastline%; could be set "lastline=" as well
call :lline to process variables %filename% and %lastline%
set "filename=%filename:&=^&%" to allow & in file names
The script is as follows:
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>files\output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set "filename=%%~x"
set "lastline=!!!file empty!!!"
rem the whole line
for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"
rem the fourth token only
rem for /F "tokens=4" %%G in ('type "%%~x"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
set "filename=%filename:&=^&%"
echo %filename% %lastline%
rem >>files\output.csv
goto :eof
Sample _listing.txt file:
d:\bat\files\1exclam!ation.txt
d:\bat\files\2exc!lam!ation.txt
d:\bat\files\11per%cent.txt
d:\bat\files\12per%cent%.txt
d:\bat\files\17per%Gcent.txt
d:\bat\files\18per%%Gcent.txt
d:\bat\files\21ampers&nd.txt
d:\bat\files\22ampers&&nd.txt
Output:
d:\bat>lastlines
d:\bat\files\1exclam!ation.txt 0 15.01.2015 1:52:28.48 -15072 20465
d:\bat\files\2exc!lam!ation.txt 6 15.01.2015 1:52:28.50 3250 16741
d:\bat\files\11per%cent.txt -8 15.01.2015 1:52:28.50 -3692 27910
d:\bat\files\12per%cent%.txt !!!file empty!!!
d:\bat\files\17per%Gcent.txt 0 15.01.2015 1:52:28.56 14508 12374
d:\bat\files\18per%%Gcent.txt 1 15.01.2015 1:52:28.56 30540 26959
d:\bat\files\21ampers&nd.txt 15.01.2015 1:22:50.18
d:\bat\files\22ampers&&nd.txt 15.01.2015 1:22:50.18
Honestly, all that ballast is for (possibly) trailing empty lines in files and for (possibly) ! and & in file names only; all could be done with
for /f %%x in (%list%) do for /f "skip=%startLine% tokens=4" %%i in (%%x) do echo %%x, %%i >> output.csv
You should use a simple FOR to iterate a list of values, not FOR /F.
Something like the following should work:
#echo off
setlocal enableDelayedExpansion
>>output.csv (
for %%F in (
"file1.log"
"file2.log"
"file3.log"
etc.
) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
more +!skip! %%F
)
)
The quotes around the file names are there in case you get a name with spaces.
You could use your LIST variable if it looks something like
set LIST="file1.log" "file2.log" "file3.log" etc.
#echo off
setlocal enableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.
>>output.csv (
for %%F in (%LIST%) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
more +!skip! %%F
)
)
If any of your file names contain the ! character, then you must toggle delayed expansion ON and OFF within your loop. Otherwise the delayed expansion will corrupt the names when %%F is expanded.
#echo off
setlocal disableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.
>>output.csv (
for %%F in (%LIST%) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
setlocal enableDelayedExpansion
more +!skip! %%F
endlocal
)
)
Need to copy only selected portion of the text file. A sample txt file is given below.
aaa
bbb
ccc
ddd
eee
ccc
fff
ggg
Kindly notice that 'ccc' is repeated twice here(in 3rd and 6th line). I need to copy the lines of this text file after the last repetition of 'ccc'(i.e.. on 6th line here) OR to be more clear i want to copy 7th and 8th line in this sample text file to a new text file. I have tried it to the extent given below. The below given program doesn't manage to populate my new text file i.e.. check_txt_edited.txt Kindly help me to fulfill my task.
#echo off
cd E:\Projects\Retrieve-data
set file=check.txt
type nul > check_txt_edited.txt
IF EXIST %file% (
for /f %%a in ('type "%file%"^|find "" /v /c') do set /a ttl_cnt=%%a
for /f "tokens=1 delims=:" %%i in ('findstr /n /c:ccc %file%') do set /a line=%%i
set /a startline=%line%+1
for /f %%b in (%file%) do (
set /a startline+=1
if %startline% leq %ttl_cnt% echo %* >> check_txt_edited.txt
)
) ELSE (
ECHO %file% does not exist
pause
)
Any ideas are welcome. Thanks in advance.
Thank you,
Regards,
Prasad Poduval
First step: search for last occurence of "ccc" (%x%=linenumber):
for /f "tokens=1 delims=:" %%i in ('findstr /n /x /c:"ccc" %file%') do set x=%%i
Second step: write all lines below that line:
more +%x% %file%>check_txt_edited.txt
#ECHO OFF
SETLOCAL enabledelayedexpansion
SET "target=ccc"
SET occur=2
SET "repro="
(
FOR /f "delims=" %%a IN (q24865244.txt) DO (
IF defined repro ECHO(%%a
IF "%%a"=="%target%" (
SET /a occur-=1
IF !occur!==0 SET "repro=Y"
)
)
)>newfile.txt
GOTO :EOF
I used a file named q24865244.txt containing your data for my testing.
Produces newfile.txt
Simply set the target and occur count to suit.
I've assumed that you want the target to match the line.
If you want to include the target line, moe the if defined after the closing parenthesis of the if "%%a" command.
I have a file which I need to load into a database. It has a delimiter of pipe (|) however each line contains different number of pipes. Using a batch script, how can I remove pipes from each line so the same number of pipes are on each line?
Example of file:
1|2|3||||||
4|5|6|||
7|8||||||
Let's say I'd like 5 pipes on each line only so it looks like:
1|2|3|||
4|5|6|||
7|8||||
Update See second solution and limitation updates.
Example file.txt contents
A|B|C|D|E|F|G
1|2|3|4|5|6|7
!|#|#|$|%|^|&
]1|]2|]3|]4|]5|]6|]7
|Two||Four||||Eight
!#$%^&%^*(){}|[]';/.,<>/|
Lonely||||||||||||||||||
Sep|er|ate| From| Th|e |W||orld | |
First Solution
Here is a simple way to do what you want. It should not have any problems with special characters.
Limitations
It only supports up to 24 25 columns as it is currently written. %%A to %%Y
The first value may not begin with ]. Replaced for /F "tokens=1,* delims=]" %%Y in ('type file.txt ^| find /v /n ""') do ( with for /F "delims=" %%Z in ('type file.txt') do (.
"Empty" fields may only appear at the end of every line. See second solution.
Does not preserve blank lines in the file. (This can be fixed if desired.)
Just specify how many and which columns you want to keep. For example tokens=3-5,12,48-50 will select only columns 3,4,5,12,48,49,50. Make sure you add on or remove the variables to match the output you want. echo %%A^|%%B^|%%D^|%%C^|%%G^|%%E^|%%F. Note that the columns can be reordered as well in the echo statement.
#echo off
setlocal DisableDelayedExpansion
for /F "delims=" %%Z in ('type file.txt') do (
for /F "tokens=1-5 delims=|" %%A in ("%%Z") do (
echo %%A^|%%B^|%%C^|%%D^|%%E
)
)
endlocal
pause >nul
You can either redirect the output of the .bat file into a new file Script.bat>output.txt or output the echo command to a file by appending >>output.txt to the echo line.
Example Output:
A|B|C|D|E
1|2|3|4|5
!|#|#|$|%
]1|]2|]3|]4|]5
Two|Four|Eight|| <-- Note that this line exhibits limit 3.
!#$%^&%^*(){}|[]';/.,<>/|||
Lonely||||
Sep|er|ate| From| Th
Second Solution
Shares only limitations 1 and 4. Currently adds spaces into existing blank columns to preserve all columns. They can be removed with a further code change, but will not add unless desired by the OP.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F "delims=" %%Z in ('type file.txt') do (
set "xLine=|%%Z"
call :Parse xLine
)
endlocal
pause >nul
goto :eof
:Parse
call set "xLine=%%%~1:||=| |%%"
for /F "tokens=1-5 delims=|" %%A in ("%xLine%") do (
echo %%A^|%%B^|%%C^|%%D^|%%E
)
goto :eof
Example Output:
A|B|C|D|E
1|2|3|4|5
!|#|#|$|%
]1|]2|]3|]4|]5
|Two| |Four|
!#$%^&%^*(){}|[]';/.,<>/|||
Lonely| | | |
Sep|er|ate| From| Th
There is no direct way to achieve this process, so each character must be revised in order to count the number of pipes in each line. It works, but it is somewhat slow.
#echo off
setlocal EnableDelayedExpansion
rem Number of desired pipes
set limit=5
for /F "delims=" %%a in (input.txt) do (
set "line=%%a"
rem Get position of last character
set last=0
for /L %%b in (12,-1,0) do (
set /A "last|=1<<%%b"
for %%c in (!last!) do if "!line:~%%c,1!" equ "" set /A "last&=~1<<%%b"
)
rem Copy each character to result, but just %limit% number of pipes
set pipes=0
set result=
for /L %%c in (0,1,!last!) do (
if "!line:~%%c,1!" neq "|" (
set "result=!result!!line:~%%c,1!"
) else (
set /A pipes+=1
if !pipes! leq %limit% set "result=!result!|"
)
)
echo !result!
)
Previous program will fail if the input line contain exclamation marks.
Output:
1|2|3|||
4|5|6|||
7|8||||
Antonio
How to delete last n lines from file using batch script
I don't have any idea about batch files, I am writing batch file for the first time.
How should I write this batch file?
For Windows7
Try it for
<Project_Name>
<Noter>
<Common>
<File>D:\Project_Name\Util.jar</File>
<File>D:\Project_Name\Noter.bat</File>
<File>D:Project_Name\Noter.xml</File>
<File>D:Project_Name\Noter.jar</File>
</Common>
<Project_Name>
<File>D:\Util.bat</File>
<File>D:\Util.xml</File>
<File>D:\log.bat</File>
</Project_Name>
</Noter>
<CCNET>
This the complete script for remove last N line
count the total line
set Line = Line - N , remain just processing lines number
#echo OFF
setlocal EnableDelayedExpansion
set LINES=0
for /f "delims==" %%I in (infile.txt) do (
set /a LINES=LINES+1
)
echo Total Lines : %LINES%
echo.
:: n = 5 , last 5 line will ignore
set /a LINES=LINES-5
call:PrintFirstNLine > output.txt
goto EOF
:PrintFirstNLine
set cur=0
for /f "delims==" %%I in (infile.txt) do (
echo %%I
::echo !cur! : %%I
set /a cur=cur+1
if "!cur!"=="%LINES%" goto EOF
)
:EOF
exit /b
Here call:PrintFirstNLine > output.txt will give the output in an external file name as output.txt
Output for sample Input
<Project_Name>
<CBA_Notifier>
<Common>
<File>D:\CBA\CBA_Notifier\Project_Name\IPS-Util.jar</File>
<File>D:\CBA\CBA_Notifier\Project_Name\Notifier.bat</File>
<File>D:\CBA\CBA_Notifier\Project_Name\Notifier.xml</File>
<File>D:\CBA\CBA_Notifier\Project_Name\Notifier.jar</File>
</Common>
<Project_Name>
<File>D:\CBA\CBA_Notifier\IPS-Util.bat</File>
remove last 5 line
Update
:PrintFirstNLine
set cur=0
for /F "tokens=1* delims=]" %%I in ('type "infile.txt" ^| find /V /N ""') do (
if "%%J"=="" (echo.) else (
echo.%%J
set /a cur=cur+1
)
if "!cur!"=="%LINES%" goto EOF
)
This script will takes 1 arguement, the file to be trunkated, creates a temporary file and then replaces the original file with the shorter one.
#echo off
setlocal enabledelayedexpansion
set count=
for /f %%x in ('type %1 ^| find /c /v ""') do set /a lines=%%x-5
copy /y nul %tmp%\tmp.zzz > nul
for /f "tokens=*" %%x in ('type %1 ^| find /v ""') do (
set /a count=count+1
if !count! leq %lines% echo %%x>>%tmp%\tmp.zzz
)
move /y %tmp%\tmp.zzz %1 > nul
If the original file is 5 or less lines, the main output routine will noT create a file. To combat this, I use the copy /y null to create a zero byte file.
If you would rather not have an empty file, just remove the copy /y nul line, and replace it with the following line:
if %lines% leq 0 del %1
You should use one method or the other, otherwise source files with 5 or less lines will remain untouched. (Neither replaced or deleted.)
to delete last lines from your file,
1 copy starting lines that are needed from file like from- e:\original.txt
2 paste them in new file like- e:\new\newfile1.txt
code is thanks to the person giving me this code:
remember all may be done if you have motive and even blood hb =6. but help of nature is required always as you are a part of it
#echo off & setLocal enableDELAYedeXpansion
set N=
for /f "tokens=* delims= " %%a in (e:\4.txt) do (
set /a N+=1
if !N! gtr 264 goto :did
e:\new4.txt echo.%%a
)
:did
if you have 800 files then use excel to make code for 800 and then copy it to notepad and using Ctrl+h replace space with no space. then rename file as haha.bat . run in folder with files numbered 1.txt 2.txt 3.txt etc. any enquirers welcome Erkamaldev#gmail.com " Long Live Bharata"
A slow method with less coding:
set input=file.txt
set remove=7
for /f "delims=" %i in ('find /c /v "" ^< "%cd%\%input%"') do set lines=%i
set /a lines-=remove
for /l %i in (1,1,!lines!) do findstr /n . log.txt | findstr /b %i:
May be redirected to a file.
Each line is prefixed with the line number; may be removed with extra coding.
A faster version with /g flag in my answer at:
How to split large text file in windows?
Tested in Win 10 CMD, on 577KB file, 7669 lines.