I have got a system that generates CSV files containing time based data.
Some files have data from two different dates. I want to break up these files into two files, one containing the data from the first day, the other containing the data from the next day. The original file looks like this:
09.01.2015 00:00:00,0385 - Veerhaven,VP01 in bedrijf hoog toerental,K,Process message.
09.01.2015 00:00:00,0385 - Veerhaven,VP01 in bedrijf laag toerental,G,Process message.
08.01.2015 23:59:55,1475 - Schaatsbaan,PO01 in bedrijf,G,Process message.
08.01.2015 23:59:52,0311 - Abraham van Stolkweg,PO01 in bedrijf,G,Process message.
The first 10 Characters are the date of the event. I want to break up the file in two output files seperating the data from the two days. I have to do this using batch processing because it has to be done every day over a lot of files.
I hope someone can help me on my way. Thanks in advance.
#echo off
setlocal enableextensions disabledelayedexpansion
set "file=c:\somewhere\data.txt"
for %%f in ("%file%") do for /f "usebackq" %%a in ("%%~ff") do (
if not defined %%a (
findstr /b /c:"%%a" "%%~ff" > "%%~dpnf.%%a%%~xf"
set "%%a=1"
)
)
The first for command is used only to retrieve a reference to the file and being able to separate path, filename and extension (that will be used later to generate the output files).
Second for loop reads the input file and for each line retrieves the first token/field in the line using spaces as delimiters (the default behaviour in for /f command). This value is used to filter the input file and declare environment variables:
If the variable is not defined, it is the first time the value is seen, matching records are extracted from the input file to a new output file and the variable is defined.
If the variable is defined, this value has been seen and the corresponding output file generated, the extraction is skipped and the process continues reading the next line.
edited to adapt to comments
#echo off
setlocal enableextensions disabledelayedexpansion
set "files=c:\somewhere\*.txt"
set "outputFolder=c:\where\to\put\files"
for %%f in ("%files%") do (
setlocal
for /f "usebackq" %%a in ("%%~ff") do if not defined %%a (
findstr /b /c:"%%a" "%%~ff" > "%outputFolder%\%%~nf.%%a%%~xf"
set "%%a=1"
)
endlocal
)
The wildcard management in the input needs no changes: for %%f iterates over the indicated set, being it only a file or a set of files.
The output folder is stored in a environment variable. The redirection is changed to use the variable insted of the path of the input file.
As the variables used to determine if the indicated token has been processed needs to be deleted for each file processed, the loop that processes the file contents is wrapped in a pair of setlocal/endlocal that cleans the flag variables after each file has been processed
read HELP FOR to learn how to use the FOR command to loop over the lines of a file and parse its contents. Then, try
for /f "tokens=1,*" %%a in (timedata.txt) do (
echo %%a ... %%b
)
you see that you may use %%a to split the files by date, so you could figure out something like
for /f "tokens=1,*" %%a in (timedata.txt) do (
echo %%b >>timedata.%%a.txt
)
or more generically
set fn=%~dpn1
set fx=%~x1
for /f "tokens=1,*" %%a in (%~1) do (
echo %%b >>%fn%.%%a%fx%
)
Related
From a dir command that displays a number of sub-folders within a main folder, I am trying to extract the total number of files only. I am trying to use the FIND command with the string "File(s)", however this lists all of the file totals for each of the sub-folders, in addition to the overall total on the last line. I would like to limit my extraction to only this overall total, and none of the sub-folder file counts.
Whilst I've provided a simple one liner already in the comment section, it does not technically provide a method of doing what you wanted. The task you've explained is really to retrieve a substring of the last but one line of output from the DIR command with its /S option. This method prevents the need to use the FIND command to search for the language dependent string File(s) too.
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
Set "MainFolder=C:\Users\CoastalGuy\Documents"
Set "CommandLine=Dir "%MainFolder%" /S /A:-D"
Set "Last="
Set "LastButOne="
For /F Delims^=^ EOL^= %%G In (
'%CommandLine% 2^>NUL'
) Do (
If Defined Last (
SetLocal EnableDelayedExpansion
For /F "Tokens=1 Delims= " %%H In ("!Last!") Do (
EndLocal
Set "LastButOne=%%H"
)
)
Set "Last=%%G"
)
If Defined LastButOne (
Echo %LastButOne%
Pause
)
For the specific task you've posted, this script is a little over the top, but I've tried to create it more like a template, so that you could more easily adapt it for similar tasks, (by changing the variable values on lines 4 and 5, and modifying the Tokens and Delims values in the inner For /F loop as required).
Found this simple result on an internet search:
After doing a dir /s of the main folder and then exporting a file (Files.txt) of each of the file results using FIND "File(s)", the following command extracts just the last line of that file, which has the grand total of Files from all sub-folders.....
For /F "UseBackQ Delims==" %%A In ("Files.txt") Do Set "lastline=%%A"
Echo - %lastline%
Works Great,
Thanks,
I have two .txt files. One contains numbers, and the other one contains filepaths. I want to combine these two files to a .csv. The combination is based on wether the number (from nrs.txt) is in the string of the filepath (nodups.txt).
Now I have the following code for this:
#setlocal enableextensions enabledelayedexpansion
for /F %a IN (Output\nrs.txt) DO (
SET "nrs=%a"
for /F %b IN (Output\nodups.txt) DO (
SET "pathstring=%b"
SET csvdelim=,
IF NOT x!pathstring:%nrs%=""!==x%pathstring% %nrs%,%pathstring%>>new2017.txt
)
)
#endlocal
However, I keep having the following issues with the code:
The pathstring never seems to get set. (when I run the code without the if statement, The nrs variable gets set but the pathstring is set to %b). I've seen a lot of possible solutions on here already but none seem to work for me (setting variables like !var! and using usebackq).
The IF statement in the second for loop gets the following error message =""!==x%pathstring% was unexpected at this time. The ="" should remove the nr. from the path (if its there). When I replace "" with something else it still does not work.
The file contents are:
File nrs.txt:
12345
12245
16532
nodubs.txt:
C:\tmp\PDF_16532_20170405.pdf
C:\tmp\PDF_1234AB_20170405.pdf
C:\tmp\PDF_12345_20170506.pdf
Desired output:
12345, C:\tmp\PDF_12345_20170506.pdf
16532, C:\tmp\PDF_16532_20170405.pdf
I really hope someone can help me out with this !
This solution use a different approach, based on arrays:
#echo off
setlocal EnableDelayedExpansion
rem Load array from nodubs.txt file
pushd "Output"
for /F "tokens=1,2* delims=_" %%a in (nodubs.txt) do set "nodubs[%%b]=%%a_%%b_%%c"
rem Process nrs.txt file and show output
(for /F %%a in (nrs.txt) do (
if defined nodubs[%%a] echo %%a, !nodubs[%%a]!
)) > new2017.txt
In a batch file for variables need two percent signs.
There is no need to put %%A into a variable, use it directly.
#setlocal enableextensions enabledelayedexpansion
for /F %%a IN (Output\nrs.txt) DO (
findstr /i "_%%a_" Output\nodups.txt >NUL 2>&1 || >>new2017.txt Echo %%a
)
#endlocal
Instead of a second for, I'd use findstr to search for the entry of nrs.txt enclosed in underscores.
if no find use condiotonal execution on failure || to write to the new file.
According to changed preliminaries another answer.
#Echo on
Pushd Output
for /F "tokens=1-3 delims=_" %%A IN (
' findstr /G:nrs.txt nodubs.txt'
) DO >>"..\new2017.txt" Echo %%B, %%A_%%B_%%C
Popd
sample output:
> type ..\new2017.txt
16532, C:\tmp\PDF_16532_20170405.pdf
12345, C:\tmp\PDF_12345_20170506.pdf
I wanted to take list of files to delete from user as a argument. One line per argument.
How can store the list of files separated by new line in a variable.
I am using below command.
Set DeletionFiles=${p:DeleteFiles}"
for %%i in (%DeletionFiles%) do (
echo %%i
)
Then i wanted to iterated them on a loop.
${p:DeleteFiles} will get replaced by it's value from external app, which will contain list of files separated by new line.I can not change it.
#ECHO OFF
SETLOCAL
SET "deletionfiles="
:dloop
SET "deleteme="
SET /p "deleteme=Which file to delete ? "
IF DEFINED deleteme SET "deleteme=%deleteme:"=%"
IF DEFINED deleteme SET "deletionfiles=%deletionfiles%,"%deleteme%""&goto dloop
ECHO delete %deletionfiles:~1%
GOTO :EOF
There is no need to use a newline. Your for command (or a del command) will operate perfectly happily on a comma-(or space-)separated list.
Note that there are certain characters that batch uses for special purposes and batch string-processing may not process them in the expected manner. These characters include % ^ and &.
${p:DeleteFiles} will get replaced by it's value from external app,
which will contain list of files separated by new line.I can not
change it.
After the replacement the batch file looks like:
Set DeletionFiles=file1.jpg
file2.jpg
file3.jpg
"
This isn't a valid batch file anymore.
Furthermore it's a bad idea to modify the batch file itself, as this works only once.
You could place the ${p:DeleteFiles} into another file, like input.txt.
Your batch would look like
echo ${p:DeleteFiles} > input.txt
<external program for replacing the DeleteFiles> input.txt
for /F "tokens=*" %%A in (input.txt) do (
echo File: %%A
)
If I understand you correctly, your external program will generate a list of files. You then want to store this multi-line list to a variable. What do you want to do with the variable once you have it? I assume you want to delete the files, but your question isn't clear on that point, so I'll try to over-answer to cover it.
for /f "delims=" %%a in ('{command that generates your list}') do (
echo Doing stuff to %%a...
echo %%a>>listOfFilesToDelete.txt
set var=%%a
if "%var:~0,7%"="DoNotDelete" copy "%%a" \someArchiveFolder\
del "%%a"
)
This will read each line in your generated list as variable %%a. It will then do whatever command(s) you specify. This way, you can run a command on each of the files in the list. In the above code it's
Printing each line to the console embedded in some text
Outputting it to a file
Checking the first 7 characters of the line against a specified string and then copying it to a folder if it matches
And then deleting it
If you still need to reference each line from your generated list, you can even setup an array-like structure. (See Create list or arrays in Windows Batch)
setlocal EnableDelayedExpansion
:: Capture lines in an 'array'
set /a i=0
for /f "delims=" %%a in ('dir /b') do (
set /a i+=1
set var!i!=%%a
)
:: Loop through the 'array'
for /L %%a in (1,1,%i%) do (
echo Do more stuff with !var%%a!
)
Just like above, this will read each line in your generated list as variable %%a. It will then set a variable var!i! equal to the value of the current line. You can then reference each line as var1, var2, and so on, or, as the second section shows, you can loop through them all using for /L. You'll need to get a grasp on working with delayed expansion variables, though.
I have several files in a folder with names like
"prefix (S-N 12RE3123) suffix.pdf"
I would like to rename them to just "12RE3123.pdf", removing the prefix and suffix. The issue is that both the prefix and suffix are of varying lengths/ characters, so I cannot just rename by removing the first/last xx characters, but have to use the only commonality where only the characters inside the parenthesis following "S-N" are those to be kept.
There is one special case where a few of the serial numbers are named as WD-12RE3123, and I need to remove the WD- as well. If I had to do it manually, there aren't a lot of them like that so it wouldn't be the end of the world, but having it automated would be nice. I thought of maybe doing an if statement after the removal of prefix/suffix to check if the first 3 characters match WD- then remove those if true, but I am not sure on the implementation.
I am still a novice in batch or vbscript, so I was wondering if this can be done in either of those. Batch has the method "REN" to rename files, but since the final name depends upon what the current name is I am not sure how to set up delimiters or how to approach this.
Any assistance would be greatly appreciated.
Here is a simple solution for your request. It relies on the following facts:
the prefix portion does not contain any parenthesis ( or ) on its own (suffix might though);
the serial number does not contain any parentheses ( or ) on its own;
there are no duplicate serial numbers, also with respect to removal of potentional WD- strings;
This is the code (after having tested the code on the files in your target folder, you need to remove the upper-case ECHO command to actually rename the files):
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Set constants here:
set "LOCATION=.\test"
set "PATTERN=*.pdf"
set STRINGS="S-N " "WD-"
cd /D "%LOCATION%"
for /F "eol=| delims=" %%K in ('dir /B "%PATTERN%"') do (
for /F "eol=| tokens=2 delims=()" %%L in ("%%~nK") do (
set "NAME=%%L"
setlocal EnableDelayedExpansion
for %%J in (%STRINGS%) do (
if "!NAME!"=="%%~J!NAME:*%%~J=!" (
set "NAME=!NAME:*%%~J=!"
)
)
ECHO ren "%%~fK" "!NAME!%%~xK"
endlocal
)
)
endlocal
exit /B
How it works:
the first section specifies the folder location, the file pattern and the strings to remove (after having extracted the portion within ()); adapt the values to your needs;
there are several nested for loops; the outermost one simply iterates all the files matching the given pattern (it basically reads and parses the output of a dir command applied on the given files and loops through the returned items; opposed to a standard for loop, this for /F method ensures that the entire folder is read before the loop starts iterating, which is necessary when modifying the enumerated folder content like we do here by renaming files; see also this thread about that issue);
the next for /F loop extracts the file name portion of interest, that is everything between the first pair of parenthesis, and stores it in variable NAME; this loop iterates once only per file;
the is another for loop which walks though all items in the STRINGS variable;
the if clause checks whether the current item of STRINGS occurs at the very beginning of the NAME value; if so, it is removed, otherwise not; this is just a safety query because perhaps a serial number might also contain a given STRINGS item in the middle or at the end (for instance, 123-WD-45A);
at this point, the renaming is performed (after having removed ECHO, of course);
the toggling of delayed expansion is intended to avoid trouble with some special characters in the file names;
And here is another script that uses a more complex method for extracting the interesting parts of the file names. It relies on the following facts:
there occurs only a single substring (S-NSPACE in the file name
the serial number is followed by a );
the serial number does not contain any parentheses ( or ) on its own;
there are no duplicate serial numbers, also with respect to removal of potentional WD- strings;
The code looks like this:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Set constants here:
set "LOCATION=.\test"
set "PATTERN=*.pdf"
set "FILTER=(S-N [^()][^()]*)"
set "IDENTIFYER=S-N "
set STRINGS="WD-"
cd /D "%LOCATION%"
for /F "eol=| delims=" %%K in ('
dir /B "%PATTERN%" ^| findstr /I /R /C:"%FILTER%"
') do (
set "NAME=%%K"
setlocal EnableDelayedExpansion
set "NAME=!NAME:*(%IDENTIFYER%=!"
for /F "eol=| tokens=1 delims=)" %%L in ("!NAME!") do (
setlocal DisableDelayedExpansion
set "NAME=%%L"
setlocal EnableDelayedExpansion
for %%J in (%STRINGS%) do (
if "!NAME!"=="%%~J!NAME:*%%~J=!" (
set "NAME=!NAME:*%%~J=!"
)
)
ECHO ren "%%~fK" "!NAME!%%~xK"
endlocal
endlocal
)
endlocal
)
endlocal
exit /B
Basically, this script works similar to the above one, with a few deviations:
the first section specifies a file name filter and the serial number identifyer (S-N) in addition;
the dir command in the outermost for loop is piped into a findstr command to filter out files not containing (S-N and ) and a string (not containing (/)) in between already at the beginning;
the part (S-N and everything before is removed, the result is stored in NAME;
the next for /F loop extracts everything up to but not including the first ) from the NAME value, which constitutes the file name portion of interest; this loop iterates once only per file;
the is another for loop which walks though all items in the STRINGS variable, which does not contain the S-N portion here as this has already been removed before; the rest in this loop is the same as in the above script;
at this point, the renaming is performed (after having removed ECHO, of course);
#ECHO Off
SETLOCAL
SET "sourcedir=U:\sourcedir"
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\*(*)*.*" '
) DO (
FOR /f "tokens=1,2delims=()" %%c IN ("%%a") DO (
FOR /f "tokens=1-3delims=- " %%m IN ("%%d") DO (
IF "%%o"=="" (ECHO(REN "%sourcedir%\%%a" "%%n%%~xa"
) ELSE (
ECHO(REN "%sourcedir%\%%a" "%%o%%~xa"
)
)
)
)
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
The required REN commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(REN to REN to actually rename the files.
Apply each filename matching the mask "()." to %%a
Tokenise using ( and ) as delimiters so that the parenthesised portion is in %%d
Re-tokenise using - and as delimiters so that the required number is in %%o for S-N patterns and %%n for WD- pattern.
Show the rename line.
Note: this should work provided the prefix portion does not contain parentheses and the parenthesised portion is exactly as described.
Magoo and aschipfl both provided good pure batch solutions.
I find development of custom batch scripts for every complex renaming task to be tedious, which is why I wrote JREN.BAT - a regex find/replace renaming utility. JREN.BAT is pure script (hybrid JScript/batch) that runs natively on any Windows machine from XP onward. Full documentation is available from the command line via jren /?, or use jren /?? for paged help.
With JREN.BAT, the solution is as simple as (untested):
jren "^.*\((?:S-N (?:WD-)?)(.+?)\).*" "$1.jpg" /fm *.jpg
If I got the regex wrong, it can easily be fixed. You should probably add the /T option to run the command in test mode first, and then remove it when everything looks good.
If you put the command in a batch script, then you must use CALL JREN.
Happy Friday Think-Tank!
I need some assistance with a Batch .BAT script. Specifically I need help with some "IF statement syntax"
I have a script that is renaming files. There are two files, one ending in four digits and the other ending in five digits. The files will be renamed with variables I have already pre-set earlier within my script.
So here is a scenario: We have two files in a directory located at
c:\Users\username\Desktop\test-dir
There are two files within test-dir:
file1.12345
file2.1234
A four digit ending is one variable type (VAR1), whereas a file ending in five digits is another variable type (VAR2).
I need an if statement to:
a) read all the files(s) with the chosen directory (without using a wildcard if possible).
b) determine based on the number of digits after the "." which variable to use.
c) once making that determination rename the file with the appropriate variables.
The final re-naming convention is as so: yyyymmddtype.1234/12345
So basically it would use the datestamp variable I already created, the type variable I already created to be injected by the if statement, and append with the original ending digits of the file.
I know this seems like a lot, but I am more so a bash script guy. I have all the elements in place, I just need the if statement and what feels like a for loop of some kind to tie it all together.
Any help would be great!
Thank you!
Sorry, not the option you where asking for. Instead of iterating over the full list checking each file for extension conformance, iterate over a list of patterns that will filter file list, renaming matching files with the asociated "type"
for %%v will iterate over variable list, for %%a will split the content of the variable in pattern and type, for %%f will generate the file list, filter with findstr using the retrieved pattern and rename matching files with the corresponding "type"
Rename command is preceded with a echo to output commands to console. If the output is correct, remove the echo to rename the files.
#echo off
rem Variables defined elsewhere
set "folder=c:\somewhere"
set "timestamp=yyyymmdd"
rem Rename pattern variables in the form pattern;type
set "var1=\.....$;type1"
set "var2=\......$;type2"
set "var1=\.[^.][^.][^.][^.]$;type1"
set "var2=\.[^.][^.][^.][^.][^.]$;type2"
setlocal enableextensions disabledelayedexpansion
for %%v in ("%var1%" "%var2%") do for /f "tokens=1,* delims=;" %%a in ("%%~v") do (
for /f "tokens=*" %%f in ('dir /a-d /b "%folder%" ^| findstr /r /c:"%%~a"') do (
echo ren "%folder%\%%~f" "%timestamp%%%~b%%~xf"
)
)
endlocal
#ECHO OFF &SETLOCAL
set "yyyymmdd=yyyymmdd"
set "VAR1=VAR1"
set "VAR2=VAR2"
for /f "delims=" %%a in ('dir /b /a-d^|findstr /re ".*\....."') do echo(ren "%%~a" "%yyyymmdd%%VAR1%%%~xa"
for /f "delims=" %%a in ('dir /b /a-d^|findstr /re ".*\......"') do echo(ren "%%~a" "%yyyymmdd%%VAR2%%%~xa"
remove echo( to get it working.
If I understand you then this will rename the two files using preset variables for each one:
for %%a in ("%userprofile%\Desktop\test-dir\*") do (
if "%%~xa"==".12345" ren "%%a" "%variableA%-%variableB%%%~xa"
) else (
ren "%%a" "%variableC%-%variableD%%%~xa"
)
)