I'm trying to remove a string (%1, %~p1 = path of commandline parameter 1) from another string (%i, %%i = another path). I have tried the following:
SET relpath = !!%%i:%~p1=!!
But that returns:
relpath = \ISO til braending\filelist.bat:\ISO til braending\=
So nothing is removed; instead the value of %~p1 is added to relpath.
I'm trying to what is written at e.g. here: https://ss64.com/nt/syntax-replace.html
Any ideas as to how to achieve this?
Related
Using VBS on Windows 2012 R2 I am trying to pass a command line parameter isActive="false" but I cannot get the equal sign to appear in the command line.
Create a dummy batch file like test.bat
echo %1
Pause
Then in the created VBScript
Set oShell = WScript.CreateObject("WSCript.Shell")
oShell.CurrentDirectory = "C:\test"
'strEqual = Chr(61)
strCommand = "test.bat" & " " & "isActive=" &"""false"""
return = oShell.Run(strCommand, 1, True)
Set oShell = Nothing
I get isActive "false" but no equal sign.
I have tried separating out as a unique value
like & Chr(61) & and have tried escaping with / and \ and // and \\ before and after the equal sign. I have tried to use as a variable, strEqual = Chr(61).
I am at a loss as to how to get the = to be part of the string when passed to the command shell. I can write it to a text file and the equal sign is written, but not in the shell.
You observe this behavior because CMD uses not only spaces and tabs, but also commas, semicolons, and the = character as parameter delimiters. Meaning that isActive="false" is parsed as 2 distinct arguments: isActive and "false". If you want it to be parsed as a single argument you need to put the whole key/value pair in quotes: "isActive=false".
Note that the double quotes inside VBScript string literals must be escaped by doubling them. If you require double quotes around the value part of the argument simply add another set of escaped double quotes.
Set oShell = CreateObject("WScript.Shell")
strCommand = "test.bat ""isActive=""false"""""
return = oShell.Run(strCommand, 1, True)
There is also no need to concatenate string literals (except for readability reasons when you want to wrap a long string). Just define your command as a single string.
You may pass a value from VBS to BAT/CMD using process environment variable.
Save the below code as test.vbs:
strCurDir = CreateObject("Scripting.FileSystemObject").GetParentFolderName(WScript.ScriptFullName) & "\"
CreateObject("WScript.Shell").Environment("process").Item("myvar") = "isActive=""false"""
CreateObject("WSCript.Shell").Run strCurDir & "test.bat"
And this code save as test.bat in the same folder:
echo %myvar%
pause
Run test.vbs and console output will be
C:\Windows\system32>echo isActive="false"isActive="false"
i`m trying to scan for a string in all files under a directory and several sub directories , replace it , and when i preformed the replacement save it into a new file,
I managed to
1) search for the string in all the files in the folder and sub folders ,
2) count the amount of appearances of the string
3) create new files for those who had the string .
But for some reason the new files i get are empty , instead of containing the lines of the old files with the replaced string .
Here is my Code :
import os
parentDirectory = "E:\ProjectPython\Test"
oldString = "oldString"
newString = "newString"
if (os.path.isdir(parentDirectory)):
for dirName,directories,files in os.walk(parentDirectory):
for fileName in files:
fullPath = os.path.join(dirName,fileName)
with open(fullPath) as file:
counter=0
for line in file:
if line.__contains__(oldString) or line.__contains__(oldString.lower()):
counter=counter+1
if counter>>0 :
with open(fullPath+".New","w") as newFile:
for line in file:
newFile.write(line.replace(oldString,newString))
print("File:"+fullPath+"\nStrings that were replaced:"+str(counter))
else:
print(parentDirectory+"is missing!")
I have multiple text file with about 100,000 lines and I want to split them into smaller text files of 5000 lines each.
I used:
split -l 5000 filename.txt
That creates files:
xaa
xab
aac
xad
xbe
aaf
files with no extensions. I just want to call them something like:
file01.txt
file02.txt
file03.txt
file04.txt
or if that is not possible, i just want them to have the ".txt" extension.
I know the question was asked a long time ago, but I am surprised that nobody has given the most straightforward Unix answer:
split -l 5000 -d --additional-suffix=.txt $FileName file
-l 5000: split file into files of 5,000 lines each.
-d: numerical suffix. This will make the suffix go from 00 to 99 by default instead of aa to zz.
--additional-suffix: lets you specify the suffix, here the extension
$FileName: name of the file to be split.
file: prefix to add to the resulting files.
As always, check out man split for more details.
For Mac, the default version of split is dumbed down. You can install the GNU version using the following command. (see this question for more GNU utils)
brew install coreutils
and then you can execute the above command by replacing split with gsplit. Check out man gsplit for details.
Here's an example in C# (cause that's what I was searching for). I needed to split a 23 GB csv-file with around 175 million lines to be able to look at the files. I split it into files of one million rows each. This code did it in about 5 minutes on my machine:
var list = new List<string>();
var fileSuffix = 0;
using (var file = File.OpenRead(#"D:\Temp\file.csv"))
using (var reader = new StreamReader(file))
{
while (!reader.EndOfStream)
{
list.Add(reader.ReadLine());
if (list.Count >= 1000000)
{
File.WriteAllLines(#"D:\Temp\split" + (++fileSuffix) + ".csv", list);
list = new List<string>();
}
}
}
File.WriteAllLines(#"D:\Temp\split" + (++fileSuffix) + ".csv", list);
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET /a fcount=100
SET /a llimit=5000
SET /a lcount=%llimit%
FOR /f "usebackqdelims=" %%a IN ("%sourcedir%\q25249516.txt") DO (
CALL :select
FOR /f "tokens=1*delims==" %%b IN ('set dfile') DO IF /i "%%b"=="dfile" >>"%%c" ECHO(%%a
)
GOTO :EOF
:select
SET /a lcount+=1
IF %lcount% lss %llimit% GOTO :EOF
SET /a lcount=0
SET /a fcount+=1
SET "dfile=%sourcedir%\file%fcount:~-2%.txt"
GOTO :EOF
Here's a native windows batch that should accomplish the task.
Now I'll not say that it'll be fast (less than 2 minutes for each 5Kline output file) or that it will be immune to batch character-sensitivites. Really depends on the characteristics of your target data.
I used a file named q25249516.txt containing 100Klines of data for my testing.
Revised quicker version
REM
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET /a fcount=199
SET /a llimit=5000
SET /a lcount=%llimit%
FOR /f "usebackqdelims=" %%a IN ("%sourcedir%\q25249516.txt") DO (
CALL :select
>>"%sourcedir%\file$$.txt" ECHO(%%a
)
SET /a lcount=%llimit%
:select
SET /a lcount+=1
IF %lcount% lss %llimit% GOTO :EOF
SET /a lcount=0
SET /a fcount+=1
MOVE /y "%sourcedir%\file$$.txt" "%sourcedir%\file%fcount:~-2%.txt" >NUL 2>nul
GOTO :EOF
Note that I used llimit of 50000 for testing. Will overwrite the early file numbers if llimit*100 is gearter than the number of lines in the file (cure by setting fcount to 1999 and use ~3 in place of ~2 in file-renaming line.)
You can maybe do something like this with awk
awk '{outfile=sprintf("file%02d.txt",NR/5000+1);print > outfile}' yourfile
Basically, it calculates the name of the output file by taking the record number (NR) and dividing it by 5000, adding 1, taking the integer of that and zero-padding to 2 places.
By default, awk prints the entire input record when you don't specify anything else. So, print > outfile writes the entire input record to the output file.
As you are running on Windows, you can't use single quotes because it doesn't like that. I think you have to put the script in a file and then tell awkto use the file, something like this:
awk -f script.awk yourfile
and script.awk will contain the script like this:
{outfile=sprintf("file%02d.txt",NR/5000+1);print > outfile}
Or, it may work if you do this:
awk "{outfile=sprintf(\"file%02d.txt\",NR/5000+1);print > outfile}" yourfile
Syntax looks like:
$ split [OPTION] [INPUT [PREFIX]]
where prefix is
PREFIXaa, PREFIXab, ...
Just use proper one and youre done or just use mv for renameing.
I think
$ mv * *.txt
should work but test it first on smaller scale.
:)
This "File Splitter" Windows command line program works nicely: https://github.com/dubasdey/File-Splitter
It's open source, simple, documented, proven, and worked for me.
Example:
fsplit -split 50 mb mylargefile.txt
My requirement was a bit different. I often work with Comma Delimited and Tab Delimited ASCII files where a single line is a single record of data. And they're really big, so I need to split them into manageable parts (whilst preserving the header row).
So, I reverted back to my classic VBScript method and bashed together a small .vbs script that can be run on any Windows computer (it gets automatically executed by the WScript.exe script host engine on Window).
The benefit of this method is that it uses Text Streams, so the underlying data isn't loaded into memory (or, at least, not all at once). The result is that it's exceptionally fast and it doesn't really need much memory to run. The test file I just split using this script on my i7 was about 1 GB in file size, had about 12 million lines of test and made 25 part files (each with about 500k lines each) – the processing took about 2 minutes and it didn’t go over 3 MB memory used at any point.
The caveat here is that it relies on the text file having "lines" (meaning each record is delimited with a CRLF) as the Text Stream object uses the "ReadLine" function to process a single line at a time. But hey, if you're working with TSV or CSV files, it's perfect.
Option Explicit
Private Const INPUT_TEXT_FILE = "c:\bigtextfile.txt" 'The full path to the big file
Private Const REPEAT_HEADER_ROW = True 'Set to True to duplicate the header row in each part file
Private Const LINES_PER_PART = 500000 'The number of lines per part file
Dim oFileSystem, oInputFile, oOutputFile, iOutputFile, iLineCounter, sHeaderLine, sLine, sFileExt, sStart
sStart = Now()
sFileExt = Right(INPUT_TEXT_FILE,Len(INPUT_TEXT_FILE)-InstrRev(INPUT_TEXT_FILE,".")+1)
iLineCounter = 0
iOutputFile = 1
Set oFileSystem = CreateObject("Scripting.FileSystemObject")
Set oInputFile = oFileSystem.OpenTextFile(INPUT_TEXT_FILE, 1, False)
Set oOutputFile = oFileSystem.OpenTextFile(Replace(INPUT_TEXT_FILE, sFileExt, "_" & iOutputFile & sFileExt), 2, True)
If REPEAT_HEADER_ROW Then
iLineCounter = 1
sHeaderLine = oInputFile.ReadLine()
Call oOutputFile.WriteLine(sHeaderLine)
End If
Do While Not oInputFile.AtEndOfStream
sLine = oInputFile.ReadLine()
Call oOutputFile.WriteLine(sLine)
iLineCounter = iLineCounter + 1
If iLineCounter Mod LINES_PER_PART = 0 Then
iOutputFile = iOutputFile + 1
Call oOutputFile.Close()
Set oOutputFile = oFileSystem.OpenTextFile(Replace(INPUT_TEXT_FILE, sFileExt, "_" & iOutputFile & sFileExt), 2, True)
If REPEAT_HEADER_ROW Then
Call oOutputFile.WriteLine(sHeaderLine)
End If
End If
Loop
Call oInputFile.Close()
Call oOutputFile.Close()
Set oFileSystem = Nothing
Call MsgBox("Done" & vbCrLf & "Lines Processed:" & iLineCounter & vbCrLf & "Part Files: " & iOutputFile & vbCrLf & "Start Time: " & sStart & vbCrLf & "Finish Time: " & Now())
here is one in c# that doesn't run out of memory when splitting into large chunks! I needed to split 95M file into 10M x line files.
var fileSuffix = 0;
int lines = 0;
Stream fstream = File.OpenWrite($"{filename}.{(++fileSuffix)}");
StreamWriter sw = new StreamWriter(fstream);
using (var file = File.OpenRead(filename))
using (var reader = new StreamReader(file))
{
while (!reader.EndOfStream)
{
sw.WriteLine(reader.ReadLine());
lines++;
if (lines >= 10000000)
{
sw.Close();
fstream.Close();
lines = 0;
fstream = File.OpenWrite($"{filename}.{(++fileSuffix)}");
sw = new StreamWriter(fstream);
}
}
}
sw.Close();
fstream.Close();
I have created a simple program for this and your question helped me complete the solution...
I added one more feature and few configurations.
In case you want to add a specific character/ string after every few lines (configurable). Please go through the notes.
I have added the code files :
https://github.com/mohitsharma779/FileSplit
If you run Anaconda on windows, you have an activate.bat file which concludes with this line to put your current conda env on the prompt:
set PROMPT=[%CONDA_DEFAULT_ENV%] $P$G
If you run cmder on windows, there is a nice lua script to customize your prompt:
function lambda_prompt_filter()
clink.prompt.value = string.gsub(clink.prompt.value, "{lamb}", "λ")
end
clink.prompt.register_filter(lambda_prompt_filter, 40)
These two scripts do not play very nicely with each other. Clink has an API that seems like I could use to incorporate the change from activate.bat, but I cannot figure out how to call it from a batch file.
My overall goal is to merge these two prompts into the nicer Cmder-style. My thought is to create an environment variable, change activate.bat to check for the existence of the variable, and, if so, call the Clink API to change the prompt instead of set PROMPT. At that point I would think I could create a new filter to cleanly merge the value in. I can't figure out how to call the API from the batch file, though.
Other solutions welcome.
EDIT: Partial, non-working solution
require "os" -- added to top of file, rest in filter function
local sub = os.getenv("CONDA_DEFAULT_ENV")
if sub == nil then
sub = ""
end
print(sub)
clink.prompt.value = string.gsub(clink.prompt.value, "{conda}", sub)
I added a {conda} in the prompt definition at the very beginning; removed the prompt statement from activate.bat, and added this to git_prompt_filter. Prior to using activate, everything is fine - the {conda} gets suppressed by the ''. However, if I use activate and switch into a folder with a git repo to trigger the change, I see:
{conda}C:\...
Does os.getenv not get user set variables? Don't know what else the problem would be. I also tried adding a print, it doesn't print out the contents of CONDA... either.
This is what I do to reset the prompt and add the conda env name to the prompt:
---
-- Find out the basename of a file/directory (last element after \ or /
-- #return {basename}
---
function basename(inputstr)
sep = "\\/"
local last = nil
local t={} ; i=1
for str in string.gmatch(inputstr, "([^"..sep.."]+)") do
--t[i] = str
--i = i + 1
last = str
end
return last
end
---
-- Find out current conda env
-- #return {false|conda env name}
---
function get_conda_env()
env_path = clink.get_env('CONDA_DEFAULT_ENV')
if env_path then
basen = basename(env_path)
return basen
end
return false
end
---
-- after conda activate: reset prompt and add conda env name
---
function conda_prompt_filter()
-- reset to original, e.g. after conda activate destroyed it...
if string.match(clink.prompt.value, "{lamb}") == nil then
-- orig: $E[1;32;40m$P$S{git}{hg}$S$_$E[1;30;40m{lamb}$S$E[0m
-- color codes: "\x1b[1;37;40m"
cwd = clink.get_cwd()
prompt = "\x1b[1;32;40m{cwd} {git}{hg} \n\x1b[1;30;40m{lamb} \x1b[0m"
new_value = string.gsub(prompt, "{cwd}", cwd)
clink.prompt.value = new_value
end
-- add in conda env name
local conda_env = get_conda_env()
if conda_env then
clink.prompt.value = string.gsub(clink.prompt.value, "{lamb}", "["..conda_env.."] {lamb}")
end
end
clink.prompt.register_filter(conda_prompt_filter, 10)
I want to build on #Jan-Schulz answer since it didn't exactly work for me in April 2017.
Instead of editing cmder/vendor/clink/clink.lua I added custom code into cmder/config/prompt.lua which is not overwritten on upgrade (You can add additional modifications to cmder prompt in this file as well using the clink lua api)
I was having an issue where the {lamb} was not being replaced with the proper λ character so I added another filter to run at the end of all processing.
---
-- Find out the basename of a file/directory (last element after \ or /
-- #return {basename}
---
function basename(inputstr)
sep = "\\/"
local last = nil
local t={} ; i=1
for str in string.gmatch(inputstr, "([^"..sep.."]+)") do
--t[i] = str
--i = i + 1
last = str
end
return last
end
---
-- Find out current conda env
-- #return {false|conda env name}
---
function get_conda_env()
env_path = clink.get_env('CONDA_DEFAULT_ENV')
if env_path then
basen = basename(env_path)
return basen
end
return false
end
---
-- after conda activate: reset prompt and add conda env name
---
function conda_prompt_filter()
-- reset to original, e.g. after conda activate destroyed it...
if string.match(clink.prompt.value, "{lamb}") == nil then
-- orig: $E[1;32;40m$P$S{git}{hg}$S$_$E[1;30;40m{lamb}$S$E[0m
-- color codes: "\x1b[1;37;40m"
cwd = clink.get_cwd()
prompt = "\x1b[1;32;40m{cwd} {git}{hg} \n\x1b[1;30;40m{lamb} \x1b[0m"
new_value = string.gsub(prompt, "{cwd}", cwd)
clink.prompt.value = new_value
end
-- add in conda env name
local conda_env = get_conda_env()
if conda_env then
clink.prompt.value = string.gsub(clink.prompt.value, "{lamb}", "["..conda_env.."] {lamb}")
end
end
function fix_lamb()
if string.match(clink.prompt.value, "{lamb}") ~= nil then
clink.prompt.value = string.gsub(clink.prompt.value, "{lamb}", "λ")
end
end
clink.prompt.register_filter(conda_prompt_filter, 1)
clink.prompt.register_filter(fix_lamb, 999)
Why not just delete the line from activate.bat and do all the logic in your cmder profile? CONDA_DEFAULT_ENV will be empty if no environment is active.
I have a textfile which contains the following text.
"Module"
{
"ModuleSignature" = "8:MergeModule.6F1248514B3047E99E4EE8A129CB8605"
"Version" = "8:1.0.0.0"
"Title" = "8:uoipmsm"
"Subject" = "8:"
"Author" = "8:Microsoft"
"Keywords" = "8:"
"Comments" = "8:"
"SearchPath" = "8:"
"UseSystemSearchPath" = "11:TRUE"
"TargetPlatform" = "3:1"
"PreBuildEvent" = "8:"
"PostBuildEvent" = "8:"
"RunPostBuildEvent" = "3:0"
}
In the above; I want to change the Version number which I will give when I trigger a build from a tool.
I wanna pass a parameter $Version in batch file, it has to take the version number from the tool I use and update the same in that text file.
For ex: in the above text i wanna code it as "Version" = "8:$Version" hence when ever I provide a version number while triggering a build, it has to update the same in this text file.
Could you please guide me how to edit the specific line. I am new to windows batch scripting.
and i hav to add one more point... in the text file i have to modify the version in the line number 399. So the batch file has to jump to line num 399 in that text file and modify the same. Kindly help me to fix the same ...
I had saved the above script in a text pad and saved the same as ver.bat; and also in the same folder I saved the Intext file. When I mention the line number which to be replaced, it is removing the contents which are present after "=" symbol, from line 1 to 399 .
Before running the batch file:
"ModuleSignature" = "8:MergeModule.6F1248514B3047E99E4EE8A129CB8605"
"Version" = "8:1.0.0.0"
"Title" = "8:uoipmsm"
"Subject" = "8:"
"Author" = "8:Microsoft"
"Keywords" = "8:"
"Comments" = "8:"
"SearchPath" = "8:"
"UseSystemSearchPath" = "11:TRUE"
"TargetPlatform" = "3:1"
"PreBuildEvent" = "8:"
"PostBuildEvent" = "8:"
"RunPostBuildEvent" = "3:0"
I mentioned the line number as 10 and after tat if I run the batch file; i get the following output;
"ModuleSignature" =
"Version" = ""
"Title" =
"Subject" =
"Author" =
"Keywords" =
"Comments" =
"SearchPath" =
"UseSystemSearchPath" =
"TargetPlatform" =
"PreBuildEvent" =
"PostBuildEvent" =
"RunPostBuildEvent" =
Any idea??
This should do it. Of course this will have to change if your input file ever changes. You may also have to make an adjustment if there are blank lines in your input file (they don't count). Note that you only need to pass the version (not the entire new line 399). In your example that would be "8:1.0.0.0". Best to quote the string in case there are ever spaces in there.
By way of explanation, this will:
This bat file must be called with an argument
Use a FOR LOOP to echo the first 398 lines to a temp file
Add a new line 399 using the version passed as an argument
Use MORE to append the remaining lines to the temp file
Copy the temp file to the original file
Delete the temp file
x
#echo off
REM %1=Version (use quotes if there are spaces in version)
set ReplaceLine=399
set InFile=Test.txt
set TempFile=TempTest.txt
if exist "%TempFile%" del "%TempFile%"
if "%~1"=="" (
color CF
echo.This program must be called with an argument!
pause
goto :eof
)
setlocal enabledelayedexpansion
set /A Cnt=1
for /F "tokens" %%a in (%InFile%) do (
echo.%%a>> "%TempFile%"
set /A Cnt+=1
if !Cnt! GEQ %ReplaceLine% GOTO :ExitLoop
)
:ExitLoop
endlocal
echo."Version" = "%~1">> "%TempFile%"
more +%ReplaceLine% < "%InFile%">> "%TempFile%"
copy /y "%TempFile%" "%InFile%"
del "%TempFile%"
goto :eof