I've written a batch file to modify the attribute of an XML file. The script works and the attribute if modified, however I'm having an issue with escaping some characters.
I've tried every solution I can possibly find online with no luck.
My intended output is:
<xs:import schemaLocation="2e9dd7db-f58b-4c91-8575-3b3af05d3178.xsd" namespace="urn:verastar:veracore:types" />
However I'm getting:
<xs:import schemaLocation="="2e9dd7db-f58b-4c91-8575-3b3af05d3178.xsd"" namespace="urn:verastar:veracore:types
I've tried escaping the quotes with ^, this works when using echo to the console however doesn;t work when writing to the file.
Why am I getting ="=" also I'm getting double quotes, but when I remove one nothing gets written to the file.. and finally the last quote seems to be messing things up and therefore the output is missing the XML closing tag.
How can I escape these characters properly?
My code is:
#echo on
setlocal EnableExtensions DisableDelayedExpansion
rem check if the XSD to modify exists in the batch directory
set "XSDFile=%~dp0test.xsd"
if not exist "%XSDFile%" goto EndBatch
rem environment variables
set "LineNumber="
set "LineCount=0"
set "TmpFile=%TEMP%\%~n0.tmp"
rem Search for the line containing attribute schemaLocation and get its
rem line number and the line itself loaded into environment variables
for /F "tokens=1* delims=:" %%I in ('%SystemRoot%\System32\findstr.exe /L /N /C:schemaLocation= "%XSDFile%" 2^>nul') do (
set "LineNumber=%%I"
set "FileLine=%%J"
)
rem If no line with attribute schemaLocation found, exit this batch file
if not defined LineNumber goto EndBatch
setlocal EnableDelayedExpansion
set "FileName=!FileLine:*schemaLocation=!"
for /f "tokens=1 delims=?" %%a in ("%FileName%") do (set test=%%a)
set "test=<xs:import schemaLocation="%test%.xsd"" namespace="urn:verastar:veracore:types" />"
pause
endlocal & set "FileLine=%test%
rem Make sure the temporary file used next does not already exist.
del "%TmpFile%" 2>nul
rem Copy all lines from XML file to a temporary file including empty
rem lines with the exception of the line containing attribute schemaLocation
rem which is copied to temporary file with the modified schemaLocation.
for /F "tokens=1* delims=:" %%I in ('%SystemRoot%\System32\findstr.exe /R /N "^" "%XSDFile%" 2^>nul') do (
set "XmlLine=%%J"
set /A LineCount+=1
setlocal EnableDelayedExpansion
if not !LineCount! == %LineNumber% (
echo/!XmlLine!>>"%TmpFile%"
) else (
echo/!FileLine!>>"%TmpFile%"
)
endlocal
)
rem Overwrite original file with temporary file automatically deleted on success.
move /Y "%TmpFile%" "%XSDFile%" >nul
:EndBatch
endlocal
Related
I usually get the current working directory by giving the batch command %~dp0 for combine multiple csv files.
But I encountered ampersand (&) symbol while getting current working directories which makes batch file broke after the '&' saying the path after & is not recognized as an internal or external command.
Can any of you guys please help me in modifying my below script in identifying & and replacing it with ^& (as this can escape the & and run the batch file). Any of your suggestions is appreciated;
Here is my code:
#echo off
ECHO Set working directory
pushd %~dp0 - How to escape & in this line???
ECHO Deleting existing combined file
del combined.csv
setlocal ENABLEDELAYEDEXPANSION
REM set count to 1
set cnt=1
REM for each file that matches *.csv
for %%i in (*.csv) do (
REM if count is 1 it's the first time running
if !cnt!==1 (
REM push the entire file complete with header into combined.csv - this will also create combined.csv
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
REM otherwise, make sure we're not working with the combined file and
) else if %%i NEQ combined.csv (
REM push the file without the header into combined.csv
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
)
REM increment count by 1
set /a cnt+=1
)
cmd \k
Putting quotation marks around %~dp0 should be enough:
...
pushd "%~dp0"
...
Btw, I don't understand why you use pushd. You don't popd anywhere in your code so pushd seems useless. If I understand ECHO Set working directory correctly, you should replace pushd with CD %~dp0.
how can I remove the duplicate entry from the text file using batch script. All i want to remove the duplicates before "=" sign and "%%" is exist in every single text file. Text file look likes below
%%B05AIPS_CDDOWNLOAD_IBDE_UNC=\\%%B05AIPS_UPLOAD_NODE.\F$\DATA\IPSL\CDFILES\B05_NAG\CD\INCOMING
%%B05AIPS_CDDOWNLOAD_FTS_UNC=\\%%B05AIPS_UPLOAD_NODE.\B05_NAG\FTS\To_Clearpath\%%DATE_CCYYMMDD.
%%B05AIPS_CDDOWNLOAD_FTS_UNC=%%B05AIPS_CDDOWNLOAD_FTS_UNC.
I got about 30 plus different text files which contains above kind of entries and want to remove the duplicate line and want to keep the first occurrence. Remember duplicate line should be identified before "=" sign only and removal required for the entire line.Each of the different text files have got "%%" sign. Please guide me if there is way to do through batch script or vbscript? Thanks
Here is a simple batch-file solution; let us call the script rem-dups.bat. Supposing your input file is test.txt and your output file is result.txt, you need to provide these files as command line arguments, so you need to call it by: rem-dups.bat "test.txt" "results.txt". Here is the script:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "INFILE=%~1"
set "OUTFILE=%~2"
if not defined INFILE exit /B 1
if not defined OUTFILE set "OUTFILE=con"
for /F "usebackq tokens=1,* delims==" %%K in ("%INFILE%") do (
set "LEFT=%%K"
set "RIGHT=%%L"
set "LEFT=!LEFT:*%%%%=__!"
rem Remove `if` query to keep last occurrence:
if not defined !LEFT! set "!LEFT!=!RIGHT!"
)
> "%OUTFILE%" (
for /F "delims=" %%F in ('set __') do (
set "LINE=%%F"
echo(!LINE:*__=%%%%!
)
)
endlocal
exit /B
The script is based on the fact that there cannot occur duplicate environment variables, that are such with equal names.
This code only works if the following conditions are fulfilled:
the file content is treated in a case-insensitive manner;
the order of lines in the output file does not matter;
the partial strings before the first = sign start with %% and contain at least one more character other than %;
the partial strings before the first = contain only characters which may occur within environment variable names, besides the leading %%;
the partial strings after the first = must not be empty;
the partial strings after the first = must not start with = on their own;
no exclamation marks ! are allowed within the file, because they may get lost or lead to other unexpected results;
Here is an alternative method using a temporary file:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "INFILE=%~1"
set "OUTFILE=%~2"
if not defined INFILE exit /B 1
if not defined OUTFILE set "OUTFILE=con"
set "TEMPFILE=%TEMP%\%~n0_%RANDOM%.tmp"
> "%TEMPFILE%" break
> "%OUTFILE%" (
for /F usebackq^ delims^=^ eol^= %%L in ("%INFILE%") do (
for /F tokens^=1^,*^ delims^=^=^ eol^= %%E in ("%%L") do (
> nul 2>&1 findstr /I /X /L /C:"%%E" "%TEMPFILE%" || (
echo(%%L
>> "%TEMPFILE%" echo(%%E
)
)
)
)
> nul 2>&1 del "%TEMPFILE%"
endlocal
exit /B
Every unique (non-empty) token left to the first = sign is written to a temporary file, which is searched after having read each line from the input file. If the token is already available in the temporary file, the line is skipped; if not, it is written to the output file.
The file content is treated in a case-insensitive manner, unless you remove the /I switch from the findstr command.
Update: Improved Scripts
Here are two scripts which are improved so that no special character can bring them to fail. They do not use temporary files. Both scripts remove lines with duplicate keywords (such is the partial string before the first = sign).
This script keeps the first line in case of duplicate keywords have been encountered:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "INFILE=%~1"
set "OUTFILE=%~2"
if not defined INFILE exit /B 1
if not defined OUTFILE exit /B 1
> "%OUTFILE%" break
for /F usebackq^ delims^=^ eol^= %%L in ("%INFILE%") do (
for /F tokens^=1^ delims^=^=^ eol^= %%E in ("%%L") do (
set "LINE=%%L"
set "KEY=%%E"
setlocal EnableDelayedExpansion
if not "!LINE:~,1!"=="=" (
set "KEY=!KEY: = !"
set "KEY=!KEY:\=\\!" & set "KEY=!KEY:"=\"!"
more /T1 "%OUTFILE%" | > nul 2>&1 findstr /I /M /B /L /C:"!KEY!=" || (
>> "%OUTFILE%" echo(!LINE!
)
)
endlocal
)
)
endlocal
exit /B
This script keeps the last line in case of duplicate keywords have been encountered:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "INFILE=%~1"
set "OUTFILE=%~2"
if not defined INFILE exit /B 1
if not defined OUTFILE exit /B 1
> "%OUTFILE%" (
for /F delims^=^ eol^= %%L in ('findstr /N /R "^" "%INFILE%"') do (
set "LINE=%%L"
for /F "delims=:" %%N in ("%%L") do set "LNUM=%%N"
setlocal EnableDelayedExpansion
set "LINE=!LINE:*:=!"
if defined LINE if not "!LINE:~,1!"=="=" (
for /F tokens^=1^ delims^=^=^ eol^= %%E in ("!LINE!") do (
setlocal DisableDelayedExpansion
set "KEY=%%E"
setlocal EnableDelayedExpansion
set "KEY=!KEY: = !"
set "KEY=!KEY:\=\\!" & set "KEY=!KEY:"=\"!"
more /T1 +!LNUM! "%INFILE%" | > nul 2>&1 findstr /I /M /B /L /C:"!KEY!=" || (
echo(!LINE!
)
endlocal
endlocal
)
)
endlocal
)
)
endlocal
exit /B
For both scripts, the following rules apply:
the order of lines with non-duplicate keywords is maintained;
empty lines are ignored and therefore removed;
empty keywords, meaning lines starting with =, are ignored and therefore removed;
non-empty lines that do not contain an = at all are treated as they would be ended with an = for the check for duplicates, hence the entire line is used as the keyword;
for the check for duplicates, each TAB character is replaced by a single SPACE;
every line that is transferred to the returned file is copied from the original file without changes (hence the aforementioned attachment of = or replacement of TAB is not reflected there);
the check for duplicates is done in a case-insensitive manner, unless you remove the /I switch from the findstr command;
Amendment: Processing Multiple Files
All of the above scripts are designed for processing a single file only. However, if you need to process multiple files, you could simply write a wrapper that contains a for loop enumerating all the input files and calls one of the scripts above (called rem-dups.bat) for every item -- like this:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Define constants here:
set "INPATH=D:\Data\source" & rem (location of input files)
set "OUTPATH=D:\Data\target" & rem (location of output files)
set INFILES="source.txt" "test*.txt" & rem (one or more input files)
set "OUTSUFF=_no-dups" & rem (optional suffix for output file names)
set "SUBBAT=%~dp0rem-dups.bat"
pushd "%INPATH%" || exit /B 1
for %%I in (%INFILES%) do if exist "%%~fI" (
call "%SUBBAT%" "%%~fI" "%OUTPATH%\%%~nI%OUTSUFF%%%~xI"
)
popd
endlocal
exit /B
You must not specify the same locations for the input and output files. If you want to overwrite the original input files, you need to write the modified output files to another location first, then you can move them back to the source location -- supposing you have set OUTSUFF in the wrapper script to an empty string (set "OUTSUFF=" instead of set "OUTSUFF=_no-dups"). The command line to overwrite the original input files would be: move /Y "D:\Data\target\*.*" "D:\Data\source".
You could read the file into Excel without splitting it into multiple columns. Use Excel functionality to eliminate duplicates and save it back. You could do all this in VBScript.
Create an Excel Object
Loop
Load text file
Remove duplicates
Save text file
Until there are no more files
Dispose of the Excel Object
Code for the individual pieces should be easily available on the web. Do ask for any additional, specific, pointers you might need.
I downloaded about 34000 books in .txt format from Project Gutenberg. Now I want to rename all of them by its content. For example every text file includes its "Title" and "Author's Name" so I want to rename all the text files on its "Title" and "Author's Name" by some commands.
I created a batch file. It runs but is not renaming the files. This is my code:
#echo off&setlocal
cd E:\Test
for /f "delims=" %%i in ('dir /a-d/b *.txt') do (
set "nname="
set "fname=%%~i"
for /f "usebackqskip=7delims=" %%f in ("%%~i") do if not defined nname
set "nname=%%f"
setlocal enabledelayedexpansion
set "nname=!nname:~0,40!"
echo rename "!fname!" "!nname!"
endlocal
)
You can use this as a base
#echo off
setlocal enableextensions disabledelayedexpansion
rem Change to source folder
pushd "e:\test" && (
rem Where the renamed files will be placed to avoid re-rename
if not exist renamed\ md renamed
rem For each input file
for %%f in (*.txt) do (
rem Retrieve the data from inside the file
set "author=" & set "title="
for /f "tokens=1,* delims=: " %%a in ('
findstr /b "Author: Title:" "%%~ff"
') do if not defined %%a set "%%a=%%b"
rem If the fields have been retrieved then do the rename
if defined author if defined title (
setlocal enabledelayedexpansion
for /f "delims=" %%a in ("!author! - !title!") do (
endlocal
echo move "%%~ff" "renamed\%%a%%~xf"
rem NOTE: operation is only echoed to console
rem if console output seems correct, then
rem remove the echo command
)
)
)
rem Done. Return to previous active directory
popd
)
Of course, filesystem has rules about what is allowed in a file name and, not knowing what kind of characters can be found, this code could and probably will fail to rename some files.
Your current script will just print the rename commands, not execute them. You should remove echo (after checking what it produces) in this line:
echo rename "!fname!" "!nname!"
Your script also has a few formatting issues. There should be spaces like this:
for /f "usebackq skip=7 delims=" %%f in ("%%~i") do
And there should be no newline just after:
if not defined nname
This is my first posting so if the format is not as it supposed to be please excuse me for this. (Suggestions for
improvement are welcome.)
I am trying to create a batchfile that will read last lines from logfiles and copy them to a new file.
Until now I have found here a way to read the last line.
Code would be something like:
for /f %%i in ('find /v /c "" ^< someFile.txt') do set /a lines=%%i
set /a startLine=%lines% - 1
more /e +%startLine% someFile.txt > lastLines.txt
The above code works for one file at a time. What I need is to read the last line from all files in a known list and add this line to a new .csv file.
I have been using the following code for getting the 4th entry in the logfiles but it returns every line of every logfile:
for /f %%x in (%list%) do for /f "delims=.txt, tokens=4" %%i in (%%x.txt) do echo %%x, %%i >> output.csv
What I would need is a sort of combination of both but I don't know how to combine them and make the complete last line be copied to the .csv file.
===
#Magoo:
Thanx for your reaction.
In every logfile can be 1 to >100 lines with comma separated information. Something like:
"LOGON,6-1-2015,12:43:39,USERNAME,HOSTNAME,,,,192.168.209.242,00:21:5A:2E:64:5E"
The last code with the 4th entry was used to get a list of all accounts that had logged in to the computers. This code gave me a very large list of all logon/logoff events on all computerlogs I checked in %list%.
In %list$ I had all the names of logfiles I wanted to be checked. This returned all lines.
For a new batchfile I need only the last logon/logoff entry and I want the whole last line.
So I have a .txt file with the hostnames of all computers I need to examine.
This .txt file will be read line by line via the variable %list%.
From every logfile I need only the last line copied to an output file.
===
I just tried the solution offered by JosefZ. Unfortunately this does not work for me yet. No lastlines are copied to the resultfile. In the code I removed the extra entry for possible lastlines for there are no empty lines in the logs, I also added an entry for the hostname I want to be available in the result. JosefZ had the filename there:
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set "host=%%~x"
for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
set "filename=.\logs\%filename:&=^&%.txt"
echo %host%,%lastline%>>output.csv
goto :eof
The resultfile shows only the hostnames. I'll puzzle some more with this but all tips are welcome!
===
Got it!!!
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set filename= :: *empty previous filename*
set lastline= :: *empty previous lastline*
set "host=%%~x"
set "filename=.\logs\%host%.txt" :: *creating the filename from path+hostname+extention*
for /F "tokens=*" %%G in ('type "%filename%"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
echo %host%,%lastline%>>output.csv
goto :eof
Your approach with line numbering could fail if a file has more trailing empty lines. Fortunately for /F loop ignores (does not iterate) empty lines; let's put to use this feature: in the script used next practices:
disabledelayedexpansion to allow ! in file names
set "list=_listing.txt" where the _listing.txt contains list of file names (full path and extension .txt including), one file name on one line: got by dir /b /s *.txt>_listing.txt
type nul>files\output.csv to empty the output file (optional)
set "lastline=!!!file empty!!!" to initialize variable %lastline%; could be set "lastline=" as well
call :lline to process variables %filename% and %lastline%
set "filename=%filename:&=^&%" to allow & in file names
The script is as follows:
#ECHO OFF >NUL
#SETLOCAL enableextensions disabledelayedexpansion
type nul>files\output.csv
set "list=_listing.txt"
for /F "tokens=*" %%x in ('type "%list%"') do (
set "filename=%%~x"
set "lastline=!!!file empty!!!"
rem the whole line
for /F "tokens=*" %%G in ('type "%%~x"') do set "lastline=%%G"
rem the fourth token only
rem for /F "tokens=4" %%G in ('type "%%~x"') do set "lastline=%%G"
call :lline
)
:endlocal
#ENDLOCAL
goto :eof
:lline
set "filename=%filename:&=^&%"
echo %filename% %lastline%
rem >>files\output.csv
goto :eof
Sample _listing.txt file:
d:\bat\files\1exclam!ation.txt
d:\bat\files\2exc!lam!ation.txt
d:\bat\files\11per%cent.txt
d:\bat\files\12per%cent%.txt
d:\bat\files\17per%Gcent.txt
d:\bat\files\18per%%Gcent.txt
d:\bat\files\21ampers&nd.txt
d:\bat\files\22ampers&&nd.txt
Output:
d:\bat>lastlines
d:\bat\files\1exclam!ation.txt 0 15.01.2015 1:52:28.48 -15072 20465
d:\bat\files\2exc!lam!ation.txt 6 15.01.2015 1:52:28.50 3250 16741
d:\bat\files\11per%cent.txt -8 15.01.2015 1:52:28.50 -3692 27910
d:\bat\files\12per%cent%.txt !!!file empty!!!
d:\bat\files\17per%Gcent.txt 0 15.01.2015 1:52:28.56 14508 12374
d:\bat\files\18per%%Gcent.txt 1 15.01.2015 1:52:28.56 30540 26959
d:\bat\files\21ampers&nd.txt 15.01.2015 1:22:50.18
d:\bat\files\22ampers&&nd.txt 15.01.2015 1:22:50.18
Honestly, all that ballast is for (possibly) trailing empty lines in files and for (possibly) ! and & in file names only; all could be done with
for /f %%x in (%list%) do for /f "skip=%startLine% tokens=4" %%i in (%%x) do echo %%x, %%i >> output.csv
You should use a simple FOR to iterate a list of values, not FOR /F.
Something like the following should work:
#echo off
setlocal enableDelayedExpansion
>>output.csv (
for %%F in (
"file1.log"
"file2.log"
"file3.log"
etc.
) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
more +!skip! %%F
)
)
The quotes around the file names are there in case you get a name with spaces.
You could use your LIST variable if it looks something like
set LIST="file1.log" "file2.log" "file3.log" etc.
#echo off
setlocal enableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.
>>output.csv (
for %%F in (%LIST%) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
more +!skip! %%F
)
)
If any of your file names contain the ! character, then you must toggle delayed expansion ON and OFF within your loop. Otherwise the delayed expansion will corrupt the names when %%F is expanded.
#echo off
setlocal disableDelayedExpansion
set LIST="file1.log" "file2.log" "file3.log" etc.
>>output.csv (
for %%F in (%LIST%) do (
for /f %%A in ('find /v /c "" <%%F') do set /a skip=%%A-1
setlocal enableDelayedExpansion
more +!skip! %%F
endlocal
)
)
I am trying to write a batch file to find and replace a string in multiple files within a folder. But I am getting this error:
Cannot perform a cyclic copy
Any idea why that happens?
#echo off
SETLOCAL
for %%* in (.) do set foldername=%%~n*
SET stringtofindreplace=XXXX
for %%f in (*.fmw) do (
echo Processing %%f...
fOR /F "delims=" %%l IN (%%f) DO (
SET "line=%%l"
SETLOCAL ENABLEDELAYEDEXPANSION
set "x=!line:%stringtofindreplace%=%foldername%!"
echo(!x!
ENDLOCAL)
)>%%~nf.new
)
GOTO:EOF
#ECHO OFF
SETLOCAL
:: no idea what this is aimed at doing...??
for %%* in (.) do set new=%%~n*
SET new=newstring
SET old=XXXX
for %%f in (*.fmw) do (
echo Processing %%f...
(
FOR /F "delims=" %%l IN (%%f) DO (
SET "line=%%l"
SETLOCAL ENABLEDELAYEDEXPANSION
set "x=!line:%old%=%new%!"
ECHO(!x!
ENDLOCAL
)
)>%%~nf.new
)
GOTO :EOF
I've no idea what you are trying to do with the first for, so I just made an obvious replacement string.
You need to add the "delims=" option to deliver the entire line to %%l.
Make sure there are no trailing spaces on the ECHO(!x!
This will make a new file called *.new from each *.fmw file.
Cannot perform a cyclic copy error occurs when the source folder includes the target folder, and so is trying to copy all the files, including the files it has already copied.
This will give you that error.
xcopy c:\apple\*.* c:\apple\backup\ /s