I have a huge csv file with more columns than allowed in excel so i need to split the file into 2 in order to use it.
I found a code which i can put in a batch file and it works too but the problem is if data is not available in a cell it takes data from the next available cell.
Also it is taking a long time.
Please let me know how the code can be edited?
#echo off
setlocal ENABLEDELAYEDEXPANSION
REM Edit this value to change the name of the file that needs splitting. Include the extension.
SET BFN=BigFile.csv
REM Edit this value to change the number of lines per file.
SET LPF=5000
REM Edit this value to change the name of each short file. It will be followed by a number indicating where it is in the list.
SET SFN=SplitFile
REM Do not change beyond this line.
SET SFX=%BFN:~-3%
SET /A LineNum=0
SET /A FileNum=1
For /F "delims==" %%l in (%BFN%) Do (
SET /A LineNum+=1
echo %%l >> %SFN%!FileNum!.%SFX%
if !LineNum! EQU !LPF! (
SET /A LineNum=0
SET /A FileNum+=1
)
)
endlocal
Pause
Also my company does not allow me to download any csv splitters or any other data.
All help would be appreciated.
Related
Good evening. I enjoy modding a game in my free time, it's one those games where individual mods are packed into zip files and loaded in by simply placing said zips into a certain folder. I have hundreds of these zips and was hoping to make a batch that would pick out a random mod for me each time I use it instead of having to pick through them all myself, while placing all the unused ones into a disabled folder that I made to keep everything not being used all in one place. Every mod was prefixed with a name referring to what category it'd belong to, for example 'weapon_shotgun_GiantAngryGun9000.zip' or 'character_huck_Clown.zip'
How this batch script would work is that first it'd take every file with the specified prefixes currently in the mod folder and put them all into the disabled folder. It would then go through every single file in the disabled folder that has all the specified prefixes, pick a random file from each category, and move them into the mod folder. Currently I've gotten my batch to do all that is explained, except rather than pick a random of each, it keeps picking the alphabetically last one from each prefix list. I believe it's because the Num var being used to designate the index for the entries in the File array is not being modified or read, and in-spite my best efforts I cannot get it to work the way I'm hoping.
Any advice or solutions is appreciated, thank you for reading.
setlocal EnableDelayedExpansion
SET modFolder="C:\Game\Mods\"
SET disabledFolder="C:\Game\Mods\Disabled\"
SET itemListIndex=0
::Prefix names
SET Prefix1=Weapon_Shotgun_
set itemList[%itemListIndex%]=%Prefix1%
set /A itemListIndex+=1
SET Prefix2=Character_ThisPerson_
set itemList[%itemListIndex%]=%Prefix2%
set /A itemListIndex+=1
SET Prefix3=Weapon_Rifle_
set itemList[%itemListIndex%]=%Prefix3%
set /A itemListIndex+=1
for /l %%G in (0 1 %itemListIndex%) do (
if not "!itemList[%%G]!" == "" (
set num=0
for %%f in (!itemList[%%G]!*.zip) do (
set /A num+=1
set file[%num%]=%%f
)
set /A pick = !random! * %num% / 32768 + 1
echo Moving "!file[%pick%]!" to the mod folder
move "%disabledFolder%!file[%pick%]!" %modFolder%
)
)
set file[%num%]=%%f is echoed as set file[]='[name of file]' instead of the expected set file['(index number)']='[name of file]'
the written echo says Moving !file[]! to the mod folder instead of the expected Moving (name of file) to the mod folder
The pick variable echos as blank, implying num or random are not valid
#ECHO OFF
setlocal EnableDelayedExpansion
SET modFolder="C:\Game\Mods\"
SET disabledFolder="C:\Game\Mods\Disabled\"
SET itemListIndex=0
::Prefix names
SET Prefix1=Weapon_Shotgun_
set itemList[%itemListIndex%]=%Prefix1%
set /A itemListIndex+=1
SET Prefix2=Character_ThisPerson_
set itemList[%itemListIndex%]=%Prefix2%
set /A itemListIndex+=1
SET Prefix3=Weapon_Rifle_
set itemList[%itemListIndex%]=%Prefix3%
set /A itemListIndex+=1
for /l %%G in (0 1 %itemListIndex%) do (
if not "!itemList[%%G]!" == "" (
set num=0
for %%f in (!itemList[%%G]!*.zip) do (
set /A num+=1
set file[!num!]=%%f
)
set /A pick = !random! * !num! / 32768 + 1
CALL SET "PICKFILE=%%file[!pick!]%%"
REM echo Moving "!file[%pick%]!" to the mod folder
REM ECHO move "%disabledFolder%!file[%pick%]!" %modFolder%
echo Moving "!pickfile!" to the mod folder
ECHO move "%disabledFolder%!pickfile!" %modFolder%
)
)
GOTO :EOF
Your issue is that %num% and %pick% are the values of num and ``pick` when the outer loop (%%G) was parsed, that is, nothing.
The classic method is to use !num! for the run-time value, but you are using indirection, so you'd be tempted to use !file[!pick!]! which is ambiguous and resolved to pick. The call uses a subshell to resolve the ambiguity.
Move command disarmed by being echoed.
Despite having already received and accepted an answer, I have decided to post this to show you how I would do it.
#Echo Off
SetLocal EnableDelayedExpansion
Set "modFolder=C:\Game\Mods"
Set "disabledFolder=C:\Game\Mods\Disabled"
Rem Undefine existing variables
For /F "Delims==" %%A In ('Set item 2^>NUL') Do Set "%%A="
Rem Define filename prefixes
Set "itemListIndex=0"
For %%A In (
Weapon_Shotgun
Character_ThisPerson
Weapon_Rifle
) Do (Set /A itemListIndex+=1
Set "itemList[!itemListIndex!]=%%A_")
Rem Randomize itemListIndex
Set /A itemListIndex=(%RANDOM% %% itemListIndex)+1
Rem Define files based upon itemList[%itemListIndex%]
Set "itemFileIndex=0"
For %%A In ("!itemList[%itemListIndex%]!*.zip") Do (Set /A itemFileIndex+=1
Set "itemFile[!itemFileIndex!]=%%A")
Rem Randomize itemFileIndex
Set /A itemFileIndex=(%RANDOM% %% itemFileIndex)+1
Rem Move itemFile[%itemFileIndex%] to modFolder.
Echo Moving "!itemFile[%itemFileIndex%]!" to the mod folder
Move /Y "%disabledFolder%\!itemFile[%itemFileIndex%]!" "%modFolder%"
Pause
This way, you can easily append to the parenthesized list of prefixes, (which both significantly reduces your code and makes it simpler to maintain). It also makes more sense to randomize itemListIndex first, to reduce the number of returned .zip files for selection/moving.
I'm kind of new to batch commands and have been trying to automate something we do manually. I have a log file that is downloaded every day, then I search for certain items in it using Findstr (output to another file) and then split what was found into smaller files.
I'm having trouble with making file names and file name variables and using it throughout my code. The splitter code was given to me, so I'm just trying to incorporate Findstr into it. Any tips or see what I'm doing wrong?
Example:
Campaign ID: 1234
Campaign Name: Pepsi
Impression Filename: 10-06-16_file.log
Day of week: 2
It will look through 10-06-16_file.log for any rows with Campaign ID 1234 and output them to fnd_10-06-16_file.log.
If fnd_10-06-16_file.log filesize is greater than 177000kb then split the file into smaller files with name 2SplitFile1_Pepsi.log, 2SplitFile2_Pepsi.log, 2SplitFile3_Pepsi.log, etc.
setlocal ENABLEDELAYEDEXPANSION
#echo off
REM Ask for Campaign information to find
SET /P campaignid="Campaign ID(s): "
SET /P campaignname="Campaign Name: "
SET /P impressionfile="Impression Filename: "
SET /P dayofweek="Day of week: "
SET fnd_impressionfile=%campaignname%_%impressionfile%
SET maxbytesize=177000
SET fnd_impressionfile_sz=%%~zfnd_impressionfile
REM Find campaigns inside log file
findstr "%campaignname%" %impressionfile% > %fnd_impressionfile%
REM Split log file if greater than 177000 kb
if fnd_impressionfile_sz > maxbytesize (
REM Edit this value to change the name of the file that needs splitting. Include the extension.
SET BFN=%fnd_impressionfile%
REM Edit this value to change the number of lines per file.
SET LPF=1000000
REM Edit this value to change the name of each short file. It will be followed by a number indicating where it is in the list.
SET SFN=SplitFile
REM Do not change beyond this line.
SET SFX=%BFN:~-3%
SET /A LineNum=0
SET /A FileNum=1
For /F "delims=" %%l in (%BFN%) Do (
SET /A LineNum+=1
echo %%l >> %dayofweek%%SFN%!FileNum!_%campaignname%.%SFX%
if !LineNum! EQU !LPF! (
SET /A LineNum=0
SET /A FileNum+=1
)
)
)
endlocal
pause
First, change SET fnd_impressionfile_sz=%%~zfnd_impressionfile to FOR %%a in (%fnd_impressionfile%) DO SET fnd_impressionfile_sz=%%~za
Then, double check you maxbytesize. Your description says kb, but the batch implies byte. fnd_impressionfile_sz contains the byte value.
The other thing I see is there you're using the redirection operator > in if fnd_impressionfile_sz > maxbytesize. You should change it to GTR or GEQ. See HELP IF.
I have a text.txt file that is filled with data that is not separated by a delimiter. I need to add a delimiter based on columns in the text file.
The data currently looks like
00067800000000000000000000N00006N 00000125463150050000012546315012
00067800000000000000000000N00006N 00000125463150810000012546315098
and I need to be add a delimiter such as "," at specific columns in the text so it comes out looking like.
000678,0000000,000000,0000000,N,00006,N, ,00000,12546315005,00000,12546,315012
000678,0000000,000000,0000000,N,00006,N, ,00000,12546315081,00000,12546,315098
I would like to create a batch file that goes through and adds the delimiter , at each column I need it for every line in the file.
Is this possible?
Thanks in advance!
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q39666479.txt"
SET "outfile=%destdir%\outfile.txt"
:: set delimiter and read column-sizes from supplied command-line
SET "delimiter=,"
CALL :colsize %*
(
FOR /f "usebackqdelims=" %%a IN ("%filename1%") DO (
SET "line=%%a"
CALL :report
)
)>"%outfile%"
GOTO :EOF
:colsize
SET "magic="
:: offset position
SET /a size=0
:colsizelp
IF DEFINED magic (
SET "magic=%magic%%delimiter%`line:~%size%,%1`"
) ELSE (
SET "magic=`line:~%size%,%1`"
)
SET /a size +=%1
SHIFT
IF "%1" neq "" GOTO colsizelp
SETLOCAL enabledelayedexpansion
SET "magic=!magic:`=%%!"
endlocal&SET "magic=set "line=%magic%""
GOTO :eof
:report
CALL %magic%
ECHO %line%
GOTO :eof
You would need to change the settings of sourcedir, filename1 and destdir to suit your circumstances.
I used a file named q39666479.txt containing your data for my testing.
Produces the file defined as %outfile%
Run as thisbatchname 6,7,6,7,1,5,1,1,5,11,5,5,6
to produce the above result. Each number is simply the column-width.
Essentially, build the string magic as appropriate substring commands strung together to assemble the output line as required. With each line from the file, execute the magic command and echo the result.
I'm an amateur on the usage of the FOR command. I need a batch file that will run one of 5 file conversion tools based on a file's extension. I want to drop a file onto the batch file icon and have it converted.
Since my list is huge, I can't use nested IF's.
What I've tried so far:
#ECHO OFF
SET cadfile=.dwg .dxf .dwf
SET gsfile=.ps .eps .epi .epsp
SET xxxxxx=.xx .xx and goes on
FOR %%~x1 in (%cadfile%) do (
Do some action
FOR %%~x1 in (%gsfile%) do (
Do some other action
)
)
The %%~x1 variable is used for file extension of file, which dragged and dropped over the batch file.
(edited to make more clear)
FOR %%a in (%cadfile%) do (
if /i "%~x1"=="%%a" some_action "%~1"
)
... and follow the bouncing ball for the rest of the utilities/lists
I think this will work for you. It looks through all your groups of extensions in a single For loop and when the matching extension is found, calls a label where you can do the conversion and any related tasks. You'll need to finish the "groupN" variables and labels.
#echo off
SETLOCAL EnableDelayedExpansion
set file="%1"
set ext=%~x1
:: Set the 5 groups of extensions that have different converters
set group1=.dwg, .dxf, .dwf
set group2=.ps, .eps, .epi, .epsp
For %%A in (1 2 3 4 5) do (
set groupnum=group%%A
call set thisgroup=%%!groupnum!%%
:: Look for extension in this group
echo.!thisgroup!|findstr /i /C:"%ext%" >nul 2>&1
if not errorlevel 1 call :group%%A
:: else go loop next group
)
echo Extension not found in any group &pause &goto end
:group1
echo group1 file to convert is %file%
goto end
:group2
echo group2 file to convert is %file%
goto end
:end
pause
exit
The following method allows you to easily add and modify your list of extensions/applications. Please note that you just need to edit the values placed inside the first FOR command; the rest of the program is the solution you don't need to care of...
#echo off
setlocal EnableDelayedExpansion
rem Define the list of extensions per application:
rem (this is the only part that you must edit)
for %%a in ("cadfile=.dwg .dxf .dwf"
"gsfile=.ps .eps .epi .epsp"
"xxxxxx=.xx .xx1 .xx2") do (
rem The rest of the code is commented just to be clear,
rem but you may omit the reading of this part if you wish
rem Separate application from its extensions
rem and create a vector called "ext" with an element for each pair
for /F "tokens=1,2 delims==" %%b in (%%a) do (
rem For example: %%b=cadfile, %%c=.dwg .dxf .dwf
for %%d in (%%c) do set "ext[%%d]=%%b"
rem For example: set "ext[.dwg]=cadfile", set "ext[.dxf]=cadfile", set "ext[.dwf]=cadfile"
rem In the next line: set "ext[.ps]=gsfile", set "ext[.eps]=gsfile", etc...
)
)
rem Now process the extension of the file given in the parameter:
if defined ext[%~x1] goto !ext[%~x1]!
echo There is no registered conversion tool for %~x1 extension
goto :EOF
:cadfile
echo Execute cadfile on %1 file
rem cadfile %1
goto :EOF
:gsfile
echo Execute gsfile on %1 file
rem gsfile %1
goto :EOF
etc...
If each conversion tool is executed in the same way and don't require additional parameters (just the filename), then you may omit the individual sections and directly execute the conversion tools this way:
if defined ext[%~x1] !ext[%~x1]! %1
For further explanations on array concept, see this post.
I am creating an MS DOS batch script that needs to list every .bat file in the current directory, but not show autoexec.bat or other utilities or systems .bat files that shouldn't be run by the user.
I currently have DIR "*.bat" /B /P
This lists all .bat files appropriately, but it shows autoexec.bat. How would I exclude that from the list? Also slightly important, how could I chop off the file extensions and show more than the 7-characters DOS limits files to?
Constraints: I am not able to use a DOS version above WinME. That is the version I am using.
Thanks for any help.
EDIT:
There is plenty of information on the internet about doing this, but it is all in the windows command processor, not MS DOS. Please understand that DOS and the Command Prompt are not the same thing.
#echo off
setlocal EnableDelayedExpansion
rem Add more names separated with slashes here:
set exclude=/autoexec/
for %%a in (*.bat) do (
if "!exclude:/%%~Na/=!" equ "%exclude%" (
echo %%~Na
)
)
EDIT: Some explanations added
Batch file processing is slow, so you should use techniques that allows a Batch file to run faster. For example:
Try to use the minimum lines/commands to achieve a certain result. Try to avoid external commands (*.exe files) like find, findstr, fc, etc. specially if they work on small amounts of data; use if command instead.
Use for %%a in (*.bat)... instead of for /F %%a in ('dir /B *.bat').... The second method requires to execute cmd.exe and store its output in a file before for command can process its lines.
Avoid pipes and use redirections instead. A pipe require the execution of two copies of cmd.exe to process the command at each side of the pipe.
A simple way to check if a variable contain a given string is trying to delete the string from the variable: if the result is different then the string exists in the variable: if "!variable:%string%=!" neq "%variable%" echo The string is in the variable.
Previous method may also be used to check if a variable have anyone of a list of values: set list=one two three, if "!list:%variable%=!" neq "%list%" echo The variable have one value from the list. If the values of the list may have spaces, they must be separated by another delimiter.
EDIT: New version added as answer to new comments
The easiest way to pause one page at a time is to use more filter this way:
theBatchFile | more
However, the program must reorder the output in order to show it in columns. The new version below achieve both things, so it does not require more filter; you just need to set the desired number of columns and rows per page.
#echo off
setlocal EnableDelayedExpansion
rem Add more names separated with slashes here:
set exclude=/autoexec/
rem Set the first two next variables as desired:
set /A columns=5, rows=41, wide=(80-columns)/columns, col=0, row=0
rem Create filling spaces to align columns
set spaces=
for /L %%a in (1,1,%wide%) do set spaces= !spaces!
set line=
for %%a in (*.bat) do (
if "!exclude:/%%~Na/=!" equ "%exclude%" (
rem If this column is less than the limit...
set /A col+=1
if !col! lss %columns% (
rem ... add it to current line
set name=%%~Na%spaces%
set "line=!line!!name:~0,%wide%! "
) else (
rem ... show current line and reset it
set name=%%~Na
echo !line!!name:~0,%wide%!
set line=
set /a col=0, row+=1
rem If this row is equal to the limit...
if !row! equ %rows% (
rem ...do a pause and reset row
pause
set row=0
)
)
)
)
rem Show last line, if any
if defined line echo %line%
Antonio
attrib +h autoexec.bat
should hide autoexec.bat and it should thus not appear in the list
DIR "*.bat" /B /P | find /v "autoexec" | for %i in (*.bat) do #echo %~ni
Using for to process each file name individually:
setlocal enabledelayedexpansion
for /f %%i in ('dir "*.bat" /b') do (
set system=0
if "%%i"=="autoexec.bat" set system=1
if "%%i"=="somesystem.bat" set system=1
if !system!==0 echo %%i
)
Another method without variables:
for /f %%i in ('dir "*.bat" /b') do call :test %%i
goto continue
:test
if "%1"=="autoexec.bat" goto :eof
if "%1"=="somesystem.bat" goto :eof
echo %1
goto :eof
:continue
For both, you can add new filenames to exclude from the list.