I'm looking for a way to copy from specified line to the EOF?
I have two or more text.log which are filled from time to time and I need to append all these data to one textall.log but remembering the last line in text.log because otherwise are copied again all data in textall.log.
So - it would appear you wish to accumulate all of the updates to *.log into textall.log
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir\t w o"
SET "history=q28129095.history"
PUSHD "%sourcedir%"
:: force history file to eist
ECHO(>>"%history%"
FOR /f "tokens=1*delims=" %%a IN (
'dir /b /a-d "*.log" '
) DO IF /i "%%a" neq "textall.log" (
SET "filename=%%a"
REM find lines in target logfile
FOR /f "tokens=2delims=:" %%q IN ('find /c /v "" "%%a"') DO SET /a lines=%%q
REM locate record of this file in history
SET "hist="
FOR /f "usebackqtokens=1,2delims=:" %%q IN ("%history%") DO IF "%%q"=="%%a" SET "hist=skip=%%r"
REM append new lines to textall.log
CALL :addlines
)
popd
GOTO :EOF
:addlines
FOR /f "usebackq %hist% delims=" %%q IN ("%filename%") DO >>textall.log ECHO(%%q
:: Now maintain the history file
(FOR /f "delims=" %%r IN ('FINDSTR /V /B /L /c:"%filename%" "%history%"') DO echo(%%r)>"%history%.temp"
:: record new start-line
>>"%history%.temp" ECHO(%filename%:%lines%
MOVE /y "%history%.temp" "%history%" >nul
GOTO :eof
You would need to change the setting of sourcedir to suit your circumstances.
For lack of further information, I've assumed testxall.log is in the same directory as all of the logfiles.
You would need to set history to a filename (possibly including a pathname) to suit you. This file retains a record of how far has already been recorded to the accumulated log.
You need to consider what would happen if your logfiles are purged - you'd need to maintain the history file manually.
Also consider that there's noindication of the source log in the accumulated.
And empty lines will be expunged.
Response to problem report
I can't reproduce your symptoms.
The echo keyword should not appear in the history file. Note that the coding is very precise - echo( not echo (. Although traditionally the character following echo is a space, there is a set of characters which may succeed the o and produce virtually the same result. Of these, ( is paricularly useful because echo %something% where something is undefined will report echo is on/off but echo(%something% where something is undefined will produce a new line
This is the result I obtained by running the routine twice; the first time with 4 lines in each of x1..5.log and the second with x5.log deleted and the others with 6 lines:
X5.log:4
X1.log:6
X2.log:6
X3.log:6
X4.log:6
What should happen with regard to the history file is:
An empty line is added at the start by ECHO(>>"%history%"
This should simply ensure the file exists so findstr has nothing to complain about
for each logfile, hist is set to nothing then the history file is examined and tokenised. The name is assigned to %%q and the previous-linecount to %%r. If the filename matches then the hist string is set to skip=(previous-linecount)
within the :addlines procedure, the findstr command allows through all of the lines which do not match (/v) the literal (/L) of the filename (/c:"string"), which should remove any existing record for the file %filename%. This is mechanically read by a for/f command assigning the entire line to %%r (because of the delims= - and this also removes empty lines.
There is a little technique however that you may have "corrected." The echo( I've already covered. As I've coded it, the line then reads
(for...etc...echo(%%r)>"temporaryfilename"
The parentheses here are important as they serve to output the entire response from for into a new file. The name and linecount for the log file are then appended to this tempfile and it's renamed for the next cycle.
I've modified the findstr here by adding in a /b switch to cause the filename match to be applied at the beginning of the line. In this way, "X2.log" won't delete the record for "anotherlogx2.log" by matching the x2.log part.
If you have further trouble, please edit your original question to include formatted data if required. As you've seen, using comments doesn't make the situation clear...
Related
I have a custom service that automatically generates files every 60 mins into a particular directory with part of the filename incrementing numerically, Eg:
File_00004.job
File_00003.job
File_00002.job
File_00001.job
Currently I have an issue where on occasion a file isn't generated, which results in gaps in the file sequence. This issue then causes a number of issues if not identified ASAP.
I'd like a batch file to identify if I have a gap in the file name sequence.
Tried looking for solutions from existing posts, but haven't found something that fits, so apologies if this has been covered elsewhere.
#ECHO OFF
SETLOCAL enabledelayedexpansion
rem The following settings for the source directory, destination directory, target directory,
rem batch directory, filenames, output filename and temporary filename [if shown] are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files\t w o"
SET "mask=file_??????.job"
SET "lowest="
SET "highest="
FOR /f "delims=" %%a IN (
'dir /b /a-d /on "%sourcedir%\%mask%" '
) DO (
IF NOT DEFINED lowest SET "lowest=%%~na"
SET "highest=%%~na"
)
SET "lowest=%lowest:*_=1%"
SET "highest=%highest:*_=1%"
ECHO checking range %lowest:~1% to %highest:~1%
:: See whether an entry in the range is missing; report&create an empty file if so.
FOR /L %%a IN (%lowest%,1,%highest%) DO SET "name=%%a"&SET "name=file_!name:~1!.job"&IF NOT EXIST "%sourcedir%\!name!" echo !name! missing&(copy nul "%sourcedir%\!name!" >nul)
GOTO :EOF
Alternative structure for the for /L loop:
FOR /L %%a IN (%lowest%,1,%highest%) DO (
SET "name=%%a"
SET "name=file_!name:~1!.job"
IF NOT EXIST "%sourcedir%\!name!" (
echo !name! missing
copy nul "%sourcedir%\!name!" >nul
copy "d:\path to\template.file" "wherever\!name!" >nul
copy "d:\path to\template.file" "anotherplace\!name!" >nul
echo Batch is fun and powerful
copy "d:\path to\template.file" "a third place\!name!" >nul
)
)
The critical point is the positioning of the ( - must be directly after and on the same line as do or else or the logical comparison clause of if and must be matched by a ) (which doesn't need to be on its own line - I find it easier that way, to align indentation.) )s that are not intended to close a block need to be escaped with ^, thus: ^)
I need to rename a filename like this 7612372 filename 50x50.jpg into this filename 50x50.jpg, removing all digits at the beginning of the filename.
The number of digit can be variable.
I need to integrate this into an existing batch file run from the Windows command prompt.
If the format of the filename would be the same for all the files in the folder, then you can try:
#echo off
for /F "delims=" %%A IN ('dir /b /A-D') do (
for /F "tokens=2-3" %%B IN ("%%A") do (
ren "%%~fA" "%%B %%C"
)
)
This is the shortest way, but not the most accurate one. It is unsecure, because if the filename contains spaces, the file will be rename incorrectly. I suggest the following code for the task:
#echo off
setlocal EnableDelayedExpansion
for /F "delims=" %%A IN ('dir /b /A-D') do (
set filename=%%A
for /F "tokens=1" %%B IN ("%%A") do (
ren "%%~fA" "!filename:%%B =!"
)
)
which is more accurate and renames all files correctly only if they have the format mentioned in the beginning.
#echo off turns command-echoing off.
setlocal EnableDelayedExpansion enables delayed expansion. We use it only here, as we have to access variables inside a for loop which is a code block. You must use delayed expansion always inside these code blocks.
Now we make a for loop to parse the output (/F) of the dir /b /A-D command which lists all items in current working directory (%cd%), excluding directories (/A-D).
We need to set a variable here with the filename. We could use the variable name of the loop (%%A), but variables have an advantage: %var:search=replace%, or even !var:search=replace! which we need here.
Now we make another for loop to parse a string (/F): the filename (%%A). We need to access the first token to substract it later. We don't really need to specify it here, but it is good to make it clearer.
We rename files now: %%~fA is the full path where filename currently processed is and !filename:%%B =! means to take filename environment variable, search for string "%%B " (first part of filename [digits] and a space) and replace it with an empty string; actually nothing!
An easier solution is to use
all digits and space as delims and
tokens=*
:: Q:\Test\2019\01\06\SO_54054587.cmd
for /F "delims=" %%A in (
'dir "* *" /A-D-H /B 2^>nul'
) do for /F "tokens=* delims=0123456789 " %%B in (
"%%A"
) do ren "%%A" "%%B"
this will remove all leading delimiters while not splitting the remainder of the file name.
Like the other answers this will not account for the shorted file name already being present.
Your question is not specific enough for us to provide a solution, you really need to provide the section of code into which you wish this to be integrated.
This one expects only one file, as in your question, and that file must be named in the format you've indicated, i.e. the required part is separated from the non-required part by a space:
#Set "name=7612372 filename 50x50.jpg"
#Ren "%name%" "%name:* =%"
[Edit /]
I have noted from your comments that you were indeed looking to parse several files and those files did not match the naming scheme you provided in your question.
Here therefore is an updated potential solution based on those changed parameters.
#For %%A In (*.*) Do #For /F "Tokens=* Delims=0123456789 " %%B In ("%%A") Do #Ren "%%~A" "%%B"
Apologies to LotPings, who I've noticed has posted a very similar method/solution
It's very simple with the basic DOS command rename.
7612372 filename 50x50.jpg
If this is your sample file in the folder, it contains 7 digits and 1 blank space. Totally 8 characters.
We can do this by simply running this command on the particular folder
rename "*.mp3" "////////*.mp3"
each / represents a character you want to remove. That's it.
I suggest following batch code for this task:
#echo off
for /F "delims=" %%A in ('dir "* *" /A-D-H /B 2^>nul') do for /F "tokens=1*" %%B in ("%%A") do ren "%%A" "%%C"
pause
The command FOR runs with cmd.exe /C (more precise %ComSpec% /C) in a separate command process in background the command line:
dir "* *" /A-D-H /B 2>nul
DIR outputs to handle STDOUT of this background command process
just the names of all non-hidden files because of option /A-D-H (attribute not directory and not hidden)
in bare format because of option /B without file path
matching the wildcard pattern * * which matches any file name with at least one space inside
in current directory which can but must not be the directory of the batch file.
DIR would output an error message to handle STDERR if it can't find any directory entry matching these criteria. This error message is redirected to device NUL to suppress it.
Read the Microsoft article about Using Command Redirection Operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.
FOR captures all lines output to handle STDOUT of started command process and processes those lines after started cmd.exe terminated itself. It is very important for this file renaming task that FOR runs on a list of file names captured before doing the file renames as otherwise the directory entries would change while FOR is accessing them. For that reason for can't be used directly in this case because of for would process the list of * * directory entries while this list changes on each successful file rename. The result would be files not renamed or renamed multiple times or even an endless running loop depending on file system (NTFS or a FAT file system like FAT32 or ExFAT).
FOR with option /F ignores empty lines which do not occur here. FOR ignores also lines starting with a semicolon because of end of line option eol=; is the default. But all lines output by DIR should start with a number and for that reason the default end of line definition can be kept for this task.
FOR with option /F splits up a line by default to substrings using normal space and horizontal tab as delimiters and assigns just first space/tab separated string to specified loop variable. This line splitting behavior is not wanted here in outer FOR loop because loop variable A should hold complete file name with all spaces. Therefore delims= is used to define an empty list of delimiters to disable the line splitting behavior. Safer would be "delims= eol=" which defines also no end of line character.
The file name assigned to loop variable A is referenced with %%A as string in inner FOR loop which splits up the file name into two substrings (tokens). The first substring is the number assigned to specified loop variable B. The second substring after first sequence of spaces (tabs not possible in a file name) is assigned without any further splitting to next loop variable C according to ASCII table. In other words on file name 7612372 filename 50x50.jpg loop variable B holds 7612372 and filename 50x50.jpg is assigned to loop variable C.
The command REN renames the file by referencing complete file name as assigned to loop variable A to the part after first sequence of spaces as assigned to loop variable C.
The command PAUSE at end is added to see the error message output by command REN if renaming a file failed. There is nothing output except the prompt by PAUSE on all files could be renamed successfully.
The batch code can be enhanced further by using FINDSTR as filter to make sure that a file to rename starts really with one or more digits up to first space by using this code:
#echo off
for /F "delims=" %%A in ('dir "* *" /A-D-H /B 2^>nul ^| %SystemRoot%\System32\findstr.exe /R /C:"^[0123456789][0123456789]* "') do for /F "tokens=1*" %%B in ("%%A") do ren "%%A" "%%C"
pause
One more variant for renaming a file with name 03T30 NAME T ALL 40X40X2 - Copy.JPG to T30 NAME T ALL 40X40X2 - Copy.JPG:
#echo off
for /F "delims=" %%A in ('dir /A-D-H /B 2^>nul ^| %SystemRoot%\System32\findstr.exe /R "^[0123456789][0123456789]*"') do for /F "tokens=* delims=0123456789 " %%B in ("%%A") do ren "%%A" "%%B"
pause
DIR outputs the names of all non-hidden files in current directory. This output is redirected as input for FINDSTR which checks if the file name starts with one or more digits. Only those file names are output to STDOUT of background command process to be processed next by FOR.
The inner FOR interprets all digits and space character as string delimiters because of delims=0123456789 and assigns everything after first sequence of digits or spaces to loop variable B because of tokens=*. So loop variable B holds filename 50x50.jpg with 7612372 filename 50x50.jpg assigned to A and T30 NAME T ALL 40X40X2 - Copy.JPG for file name 03T30 NAME T ALL 40X40X2 - Copy.JPG.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
dir /?
echo /?
findstr /?
for /?
pause /?
ren /?
PS: I recommend the usage of the shareware file manager Total Commander which has a built-in multi-rename tool for renaming files and folders for people with no coding experience. Download, install and start Total Commander, navigate to the folder containing all these files, press Ctrl+A to select the files, press Ctrl+M to open multi-rename tool window and the rest is self-explaining. If you need nevertheless help, press key F1 to open the help page for multi-rename tool.
I’m trying to make a script that removes the ” – Shortcut” from shortcut names and have discovered an odd phenomenon, if the name is under 6 characters not including the “ – Shortcut.lnk” the loop goes through an extra cycle for that file and makes its name blank. However this only applies to the first file not any file after the first.
So if we have two lnk files one is “12345 – Shortcut.lnk” and the other is “C1 – Shortcut.lnk” the output is a blank lnk file and a “C1.lnk”
But “123456 – Shortcut.lnk” and “C1 – Shortcut.lnk” gives “123456.lnk” and “C1.lnk” (the way its suppose to work)
“x1 – Shortcut.lnk” and “c1 – Shortuct.lnk” gives a blank lnk file and and “x1.lnk”
Here is the script I’m using
#echo off
setlocal enabledelayedexpansion
for %%i in ("*.lnk") do (
set CurrentFile=%%i
set NewName=!CurrentFile:~0,-15!.lnk
ren "%%i" "!NewName!"
)
pause
What is happening is that when the file is renamed, the new name is placed later in the directory than the old name, so the for finds the filename again as it processes the names mechanically as it encounters them.
Three solutions
You could change your mask to
for %%i in ("* - shortcut.lnk") do (
You could change your processing to ensure that the shortcut text is still there before renaming by gating the rename
if /i "!CurrentFile:~0,-15!"=="- shortcut.lnk" (
(
set NewName=!CurrentFile:~0,-15!.lnk
ren "%%i" "!NewName!"
)
Or you use for /f which builds a list in memory, then processes the list (hence only the "old" names are present)
for /f "delims=" %%i in ('dir /b/a-d "*.lnk" ') do (
or preferably
for /f "delims=" %%i in ('dir /b/a-d "* - shortcut.lnk" ') do (
The second is preferable since the dir command will only select names ending appropriately, so the process can be run repeatedly despite having rnamed files on a prior run.
Since you're trying to delete a specific string (rather than generally shorten the filename), you're probably safer using the substitution operator to explicitly remove - Shortcut if present:
#echo off
setlocal enabledelayedexpansion
for %%i in ("*.lnk") do (
set "CurrentFile=%%i"
set "NewName=!CurrentFile: - Shortcut=!"
ren "%%i" "!NewName!"
)
pause
I have got a system that generates CSV files containing time based data.
Some files have data from two different dates. I want to break up these files into two files, one containing the data from the first day, the other containing the data from the next day. The original file looks like this:
09.01.2015 00:00:00,0385 - Veerhaven,VP01 in bedrijf hoog toerental,K,Process message.
09.01.2015 00:00:00,0385 - Veerhaven,VP01 in bedrijf laag toerental,G,Process message.
08.01.2015 23:59:55,1475 - Schaatsbaan,PO01 in bedrijf,G,Process message.
08.01.2015 23:59:52,0311 - Abraham van Stolkweg,PO01 in bedrijf,G,Process message.
The first 10 Characters are the date of the event. I want to break up the file in two output files seperating the data from the two days. I have to do this using batch processing because it has to be done every day over a lot of files.
I hope someone can help me on my way. Thanks in advance.
#echo off
setlocal enableextensions disabledelayedexpansion
set "file=c:\somewhere\data.txt"
for %%f in ("%file%") do for /f "usebackq" %%a in ("%%~ff") do (
if not defined %%a (
findstr /b /c:"%%a" "%%~ff" > "%%~dpnf.%%a%%~xf"
set "%%a=1"
)
)
The first for command is used only to retrieve a reference to the file and being able to separate path, filename and extension (that will be used later to generate the output files).
Second for loop reads the input file and for each line retrieves the first token/field in the line using spaces as delimiters (the default behaviour in for /f command). This value is used to filter the input file and declare environment variables:
If the variable is not defined, it is the first time the value is seen, matching records are extracted from the input file to a new output file and the variable is defined.
If the variable is defined, this value has been seen and the corresponding output file generated, the extraction is skipped and the process continues reading the next line.
edited to adapt to comments
#echo off
setlocal enableextensions disabledelayedexpansion
set "files=c:\somewhere\*.txt"
set "outputFolder=c:\where\to\put\files"
for %%f in ("%files%") do (
setlocal
for /f "usebackq" %%a in ("%%~ff") do if not defined %%a (
findstr /b /c:"%%a" "%%~ff" > "%outputFolder%\%%~nf.%%a%%~xf"
set "%%a=1"
)
endlocal
)
The wildcard management in the input needs no changes: for %%f iterates over the indicated set, being it only a file or a set of files.
The output folder is stored in a environment variable. The redirection is changed to use the variable insted of the path of the input file.
As the variables used to determine if the indicated token has been processed needs to be deleted for each file processed, the loop that processes the file contents is wrapped in a pair of setlocal/endlocal that cleans the flag variables after each file has been processed
read HELP FOR to learn how to use the FOR command to loop over the lines of a file and parse its contents. Then, try
for /f "tokens=1,*" %%a in (timedata.txt) do (
echo %%a ... %%b
)
you see that you may use %%a to split the files by date, so you could figure out something like
for /f "tokens=1,*" %%a in (timedata.txt) do (
echo %%b >>timedata.%%a.txt
)
or more generically
set fn=%~dpn1
set fx=%~x1
for /f "tokens=1,*" %%a in (%~1) do (
echo %%b >>%fn%.%%a%fx%
)
I need to work two things into a .bat file I am working on for a little project. First things first, I have to know if any filename contained into the same folder (recursively) I launch my .bat in is any longer than 100 characters. If so, I need to make it 92 characters long and keep the extensions.
For example, I have this filename:
IncrediblyLongFileNameIAmSorryForThisItLooksLikeSomeDamnSpamJesusIAintEvenCloseTo100yetalmostwaitforitYEAH.omg
The above filename is 110 characters. I need to keep the extension, therefore the program should rename the file as this:
IncrediblyLongFileNameIAmSorryForThisItLooksLikeSomeDamnSpamJesusIAintEvenCloseTo100yetalmos.omg
So far, my main problem is that I don't know how to work with filename strings in batch. I used this code:
#echo off & setlocal enableextensions
FOR /R %%i IN (*.*) DO (
ECHO %%~nxi
FOR /f "delims=:" %%a in ('
^(echo."%%~nxi"^& echo.^)^|findstr /o .'
) DO set lenght=%%a-5
echo The length of "%%~nxi" is %lenght%
)
endlocal & goto :EOF
But I can't SET inside a FOR, and it can't do basic math either (i.e. it can't do the -5 operation).
The second thing, which I believe should be easier once the first one is done, is simply to compare all the filenames in the folder (recursive, once again) and make sure no filenames are the same. If the program finds any filenames that are the same, the second occurrence should be renamed to add something like l1l at the end. (I can't use parentheses here, therefore I use two ls instead to cover the number.) The only thing you need to take care of is the file extensions, because I can't add anything after the file extensions, lest they become unusable.
Can anyone offer explanations for how to accomplish this? I would really like to be able to work this out myself, but I simply lack experience in batch programming.
#ECHO OFF
SETLOCAL
SET "sourcedir=c:\sourcedir"
SET "tempfile=%temp%\##fn##.92"
ECHO ::>"%tempfile%"
FOR /f "delims=" %%a IN (
'dir /s /b /a-d "%sourcedir%\*" '
) DO (
SET "fullname=%%a"
SET "name=%%~na"
SET "ext=%%~xa"
CALL :chgname
)
del "%tempfile%"
GOTO :EOF
:chgname
:: Proposed new name part - first 92 characters of existing name
:: also prepare for adding modifier
SET "newname=%name:~0,92%"
SET /a modifier=0
:modl
:: See whether this name has already been found
ECHO %newname%%ext%|FINDSTR /b /e /i /g:"%tempfile%" >NUL
IF ERRORLEVEL 1 GOTO makechange
:: existing name - modify it
SET "newname=%name:~0,92%#%modifier%#"
SET /a modifier+=1
GOTO modl
:makechange
IF "%name%" NEQ "%newname%" ECHO REN "%fullname%" "%newname%%ext%"
>>"%tempfile%" ECHO %newname%%ext%
GOTO :eof
Reasonably simple problem.
Get a directory-list in basic form (full-filename only) and apply the full filename, name part and extension part to appropriately-named variables.
Manipulate the filename to a new name consisting of the first 92 characters of the original name part. Anticipate the need to modify this new name by establishing a modifier to optionally be applied.
See whether the proposed new name already exists in the temporary file of NEW names already processed. If not found on that file, safe to rename (if required) and record name used.
If the filename has already been used, modify it to the original first 92+ "#anumber#", increment the modifier in anticipation and try again.
Only two comments required further - first, I used # rather than ! because ! has a special meaning to batch. Second, writing :: to the tempfile (the name of the tempfile is irrelevant - I chose one that's unlikely to exist...) means that findstr doesn't complain because the file is empty, but :: can't possibly be a real filename.
The /b /e /i options to findstr mean that the name echoed in must exactly match a line (matches both /b - begin and /e - end) but /i - case is irrelevant.