I want to access STDIN from inside a batch file after some other commands. I know that the first command in a .BAT file receives STDIN but I want to first run some other commands and then capture STDIN. I also want this to work with streamed STDIN i.e. it is not acceptable to capture STDIN to a file at the start with (see workaround below).
Now, I understand that CON is the "file" representing STDIN and that TYPE CON would output (echo) STDIN. This does not seem to work at all inside a batch file. Indeed, it appears not to represent STDIN but user/host input by keyboard.
test.bat
TYPE CON > output.txt
Test run:
C:>TYPE myfile.txt | test.bat
Expected result: myfile.txt is copied into output.txt.
Actual result: The batch waits for user input (ignores what is piped to it) and writes user input typed on the keyboard to output.txt.
Workaround
As a workaround: the following test.bat works but does not support streamed input (e.g. from a tail command):
findstr "^" STDIN.txt
:: I can now run some other commands
:: And finally access my STDIN via STDIN.txt
TYPE STDIN.txt | AWK /e/ > output.txt
UPDATE: Back Story:
I have a neat CMD which uses powershell to download (via HTTP) an arbitrary .ps1 script (like a package manager would) and execute it on the fly. If I call REMEXEC.bat mymodule foo bar it loads and executes mymodule.ps1 with the parameters foo and bar.
This works wonderfully for every scenario except piped, streamed input. Using the findstr "^" works for piped input but not for an open stream. Using say AWK /.*/ as the first line of my BAT gets me that streamed input but just pushes the problem down the road.
Ultimately I want a something.bat which looks like this (pseudocode):
downloadPSModule( "http://myrepo.com/modules/%1.ps1" )
STDIN | executePSModule %2 %3 %4
The catch 22 is that downloadPSModule happens BEFORE executePSModule and thus has no access to STDIN (a privelege reserved for the first line of a BAT).
If you need to retrieve input from console or isolate reading from the stdin stream to not consume piped data, I would try directly reading from console with something like
#echo off
setlocal enableextensions disabledelayedexpansion
rem Part that reads from console, not piped input
< con (
set "data="
set /p "data=Type something: "
)
echo(
echo You have typed: [%data%]
echo(
rem Part that reads piped input
find /v ""
When executed
W:\>type test.cmd | test.cmd
Type something: this is a test
You have typed: [this is a test]
#echo off
setlocal enableextensions disabledelayedexpansion
rem Part that reads from console, not piped input
< con (
set "data="
set /p "data=Type something: "
)
echo(
echo You have typed: [%data%]
echo(
rem Part that reads piped input
find /v ""
Related
#echo off
(for /f "skip=1 tokens=3,5,11" %%a in (Data.txt) do (if /i "%%c"=="%1" (if %%b==%2 echo %%a))
echo EXIT
)|Sum.exe
I'm trying to write a simple batch script that would take the .txt file with columns of data (Data.txt), find some values using 'if' and redirect all found values to stdin input of "Sum.exe".
"EXIT" is also redirected as it means that there's no more input to be given.
When I run above code first found value is printed in console and then "The process tried to write to a nonexistent pipe" message error is printed multiple times. Therefore echo EXIT somehow must be messing up with |Sum.exe. How to properly redirect both for and echo Exit into Sum?
EDIT:
Ok, so here's the input part of the Sum program (written in c++)
std::string a;
while (a != "EXIT")
{
std::cin >> a;
if (isNumber(a))
add(sum, std::stoi(a));
}
I added cout to see whether the data was being processed and it seems that the commands in batch script were treted as input aswell.
My first suggestion would be procedure call: call :PROCEDURE. But the call doesn't work with pipe redirection.
For example call :PROCEDURE | SORT return error: Invalid attempt to call batch label outside of batch script.
So I suggest to use input parameter as switch flag to call self batch. For example it would be third parameter: %3. Batch calls itself while third parameter equals x.
So I made this code
#echo off
if "%3"=="x" (%~nx0 %1 %2|sort )& GOTO :EOF
for /f "skip=1 tokens=3,5,11" %%a in (Data.txt) do (
if /i "%%c"=="%1" (if %%b==%2 echo %%a)
echo EXIT
)
I use sort.exe utility in my example. Change it to your sum.exe
To call batch-file use syntax: my_batch.cmd [value11] [value5] x
P.S. I think you do not need word EXIT in output but I left it in example code.
Say I have the following batch script:
For ... DO (
SET VAL=%%B
IF defined VAL echo %%A=%%B >> %OUTPUT_FILEPATH%
)
How could I get the echo to output using Unix (just line feed) line endings?
Alternatively, could I write the file as-is then convert it from the batch script afterwards? (some kind of find /r/n and replace with /n? If so, how would I do that?)
I'd like a self-contained solution (i.e. one that doesn't involve downloading extra utilities, and can be done from within the batch script itself [Windows 7]).
The suitable way to perform this conversion is not via a Batch file, but using another programming language, like JScript; this way, the conversion process is fast and reliable. However, you don't need a hundreds lines program in order to achieve a replacement as simple as this one. The two-lines Batch file below do this conversion:
#set #a=0 /* & cscript //nologo //E:JScript "%~F0" < input.txt > output.txt & goto :EOF */
WScript.Stdout.Write(WScript.Stdin.ReadAll().replace(/\r\n/g,"\n"));
EDIT: I added a modification to the original code that allows to include more commands in the Batch part in the standard way.
#set #a=0 /*
#echo off
set "OUTPUT_FILEPATH=C:\Path\Of\The\File.txt"
cscript //nologo //E:JScript "%~F0" < "%OUTPUT_FILEPATH%" > output.txt
move /Y output.txt "%OUTPUT_FILEPATH%"
goto :EOF */
WScript.Stdout.Write(WScript.Stdin.ReadAll().replace(/\r\n/g,"\n"));
The first line is a trick that hide the cscript command from the JScript code, so the compilation of this hybrid .BAT file don't issue errors.
In the JScript code: WScript.Stdin.ReadAll() read the whole redirected input file; this may cause problems if the file is huge. The replace method use a regex to identify the text to replace and put in its place the second string; you may read a further description of this topic at this link. The WScript.Stdout.Write just take the output from replace and send it to the screen. Easy! Isn't it? ;-)
If you are OK with using PowerShell, you can produce a Unix newline like this:
PowerShell -Command Write-Host
You can combine this with the SET /P trick to output text without newlines and add newlines manually. For example:
( ECHO | SET /P="Hello World!" & PowerShell -Command Write-Host ) > output.txt
After this, output.txt will contain the text Hello World! with a single 0x0a character appended.
Taken from Macros with parameters appended:
Formatting is tricky, but try
set ^"LF=^
^" Don't remove previous line & rem line feed (newline)
set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"& rem Define newline with line continuation
For ... DO (
SET VAL=%%B
IF defined VAL <nul set/P^=%%A=%%B%\n%>> %OUTPUT_FILEPATH%
)
Or, to avoid the leading space after first line:
<nul set/P^=%%A=%%B%\n%^>> %OUTPUT_FILEPATH%
So I wrote a batch that has some code to check how many times it has been run by reading a textfile and then writing back into that textfile the new, increased number.
#ECHO OFF
for /f "delims=" %%x in (TimesRun.txt) do set Build=%%x
set Build=%Build%+1
#echo Build value : %Build%
echo %Build%>>TimesRun.txt
Pause
That does append the textfile allright, but it adds "1+1" to it. Silly me! I forgot to use the /a switch to enable arithmetic operations! But when I change the code accordingly...
#ECHO OFF
for /f "delims=" %%x in (TimesRun.txt) do set Build=%%x
set /a Build=%Build%+1
#echo Build value : %Build%
echo %Build%>>TimesRun.txt
Pause
... something funny happens: Instead of appending my file, ECHO is off. gets written on the console. Now, I know that this usually happens when ECHO is used without text or with an empty variable. I have added the first #echo Build value : %Build% specifically to see whether the variable Build is empty or not; it is not, and the calculation was carried out correctly.
I already figured out that
>>TimesRun.txt (echo %Build%)
does bring the desired result. I still do not understand why
echo %Build%>>TimesRun.txt
does not, however. What am I missing?
You are unintentionally specifying a redirection handle.
Redirection allows you to specify a certain handle that defines what is to be redirected:
0 = STDIN (keyboard input)
1 = STDOUT (text output)
2 = STDERR (error text output)
3 ~ 9 = undefined
For the input redirection operator <, handle 0 is used by default; for the output redirection operators > and >>, the default handle is 1.
You can explicitly specify a handle by putting a single numeric figure in front of the redirection operator; for instance 2> defines to redirect the error text output.
In your echo command line you are doing exactly this unintentionally, when %Build% is a single numberic digit, like 1 for example:
echo 1>>TimesRun.txt
To avoid that, you have the following options:
To reverse the statement so that the redirection definition comes first:
>>TimesRun.txt echo %Build%
This is the most general and secure way of doing redirections.
To enclose the redirected command in parentheses:
(echo %Build%)>>TimesRun.txt
This also works safely.
To put a SPACE in front of the redirection operator:
echo %Build% >>TimesRun.txt
This works too, but the additional SPACE is included in the output of echo.
See also this great post: cmd.exe redirection operators order and position.
Batch file redirection can be customized to specify where you're outputting to.
command 1>file.txt redirects the output of STDOUT to file.txt
command 2>file.txt redirects the output of STDERR to file.txt
Your build value was 1, so you inadvertently told CMD to send the output of echo to TimesRun.txt - when you run echo by itself, it prints it's status (ON or OFF).
You also could have said echo %Build% >>TimesRun.txt and the space would prevent the value of Build from being treated as a redirection command.
The Microsoft article Using command redirection operators explains the 3 standard handles and how to redirect them to another handle, command, device, file or console application.
Redirection of output written to handle 1 - STDOUT - to a file should be done with just
using > ... create file if not already existing or overwrite existing file, or
using >> ... create file if not already existing or append to existing file.
The redirection operators are usually appended at end of a command line. But this is problematic in case of using command ECHO and the string output to STDOUT ends with 1 to 9.
One of several solutions is to specify in this case the redirection at beginning of the command line:
#for /F "delims=" %%x in (TimesRun.txt) do #set Build=%%x
#set /A Build+=1
#echo Build value : %Build%
>>TimesRun.txt echo %Build%
Executing this small batch file without #echo off at top from within a command prompt window shows what Windows command processor executes after preprocessing each line with text file TimesRun.txt containing currently the value 0 or does not exist at all.
echo 1 1>>TimesRun.txt
It can be seen that Windows command interpreter moved the redirection to end of line with inserting a space and 1 left to >>.
With above batch code the line with >> really executed after preprocessing is:
echo 2 1>>TimesRun.txt
Specifying the redirection at end with 1>>, i.e. use in the batch file
echo %Build%1>>TimesRun.txt
is also no good idea as this would result on first run in executing the line:
echo 11 1>>TimesRun.txt
So 11 is written into the file instead of 1. This wrong output could be avoided by inserting a space before >> or 1>>, i.e. use one of those two:
echo %Build% >>TimesRun.txt
echo %Build% 1>>TimesRun.txt
But then the space after %Build% is also written into the file as really executed is:
echo 1 1>>TimesRun.txt
The trailing space would be no problem here, but should be nevertheless avoided.
Note: On using arithmetic operations, i.e. set /A ... any string not being a number or operator is automatically interpreted as variable name and the current value of this variable is used on evaluating the arithmetic expression. Therefore after set /A with environment variable names consisting only of word characters and starting with an alphabetic character as usually used for environment variables no %...% or !...! must be used inside the arithmetic expression. This is explained in help of command SET output into console window on running set /? within a command prompt window.
I'm writing my own simple system allowing me to automatically sign APKs before they are uploaded to GPlay. I've got a batch file that does the signing; and a "wrapper" batch file, the content of which will be run on the command line by Jenkins post-build.
sign_apks.bat:
#echo off
set /p job= "Enter job name: "
set /p alias= "Enter key alias: "
set /p mobile= "Sign mobile? (y/n): "
set /p wear= "Sign wear? (y/n): "
echo.
echo "%job%"
echo "%alias%"
echo "%mobile%"
echo "%wear%"
[the rest of the code is sensitive so is emitted, but these variables are used later]
wrapper code:
#echo off
(echo test
echo test
echo y
echo y)| call sign_apks.bat
This article showed me how to pipe values into a program. To quote from the answer,:
Multiple lines can be entered like so:
(echo y
echo n) | executable.exe
...which will pass first 'y' then 'n'.
However, this doesn't work. This is the output I get when running the wrapper code:
Enter job name: Enter key alias: Sign mobile? (y/n): Sign wear? (y/n):
"test "
""
""
""
Help?
There is something very odd with how SET /P interacts with your piped input that I do not fully understand.
But I do have some solutions :-)
The simplest solution is to write your responses to a temporary file, and then use that temp file as redirected input.
#echo off
(
echo test
echo test
echo y
echo y
)>responses.temp
call sign_apks.bat <responses.temp
delete responses.temp
That is how I would solve your problem. But some people do not like to use temporary files (why I don't know). So I decided I would attempt to solve it using a pipe without a temp file.
I discovered an odd variation of your code that almost solves the problem - but it appends an extra unwanted space at the end of each value.
#echo off
(
call echo test1
call echo test2
call echo y1
call echo y2
) | sign_apks.bat
--OUTPUT--
Enter job name: Enter key alias: Sign mobile? (y/n): Sign wear? (y/n):
"test1 "
"test2 "
"y1 "
"y2 "
I cannot explain why the CALL enables each of the SET /P statements to work properly. But I can explain why the space is appended to each value. It has to do with why CALL is not needed when you use a batch script with a pipe.
Each side of a pipe is executed in a brand new cmd.exe session. For example, the right side of the pipe becomes a command that looks something like:
C:\Windows\system32\cmd.exe /S /D /c" sign_apks.bat"
This is the reason why CALL is not needed - control will return after the new cmd.exe session terminates.
The unwanted spaces are an artifact of how pipes process parenthesized blocks. The parser must capture the entire piped code block and transform it into a single line that can be incorporated into the CMD.EXE /C argument. The CMD.EXE parser does this by putting an & between each command. Unfortunately, the parser also inserts some extra spaces. So the left side of the pipe is transformed into something like:
C:\Windows\system32\cmd.exe /S /D /c" ( call echo test & call echo test & call echo y & call echo y )"
Now you can easily see where the unwanted trailing spaces are coming from. See Why does delayed expansion fail when inside a piped block of code? for more information about how pipes are implemented.
I finally came up with one more solution. I created a helper batch script called WriteArgs.bat that simply ECHOs each argument passed to it.
WriteArgs.bat
#echo off
:loop
if .%1 equ . exit /b
echo %1
shift /1
goto loop
With this simple batch script, you can now solve your problem using:
WriteArgs.bat test test y y | sign_apks.bat
Again, I don't understand why SET /P works properly here, yet doesn't work with your original command. But this does solve the problem :-)
Update - Well, it solves the problem on my machine. But it seems to be a timing issue, and I don't have confidence that any given piped solution will always work. The only solution I feel is robust is the one that uses a temp file and redirection.
You know, the easiest solution would be to supply job, alias, mobile, and wear as script arguments rather than trying to pipe them into stdin. You can still set /p if not defined, if you wish to run interactively without arguments.
#echo off
setlocal
set "job=%~1"
set "alias=%~2"
set "mobile=%~3"
set "wear=%~4"
if not defined job set /p "job=Enter job name: "
if not defined alias set /p "alias=Enter key alias: "
if not defined mobile set /p "mobile=Sign mobile? (y/n): "
if not defined wear set /p "wear=Sign wear? (y/n): "
echo.
echo "%job%"
echo "%alias%"
echo "%mobile%"
echo "%wear%"
Then when you call sign_apks.bat, just call it like this:
call sign_apks.bat test test y y
It's a problem of set /p, it reads the input buffer, but it fails to split this buffer when multiple lines are available, it simply takes the first line from the buffer and the rest will be discarded.
This isn't a problem for a single echo piped to a single set/p, but when you pipe more lines to multiple set/p you got random results.
The solution of dbenham can work, but it's depends on your system!
As both processes (line producer and the set/p consumer) are asnchronously running in an own cmd.exe task, it depends on the cpu time each process gets.
But you can ensure a correct consuming by splitting the content by another program like more or findstr.
As these split the input buffer proberly at the line boundarys.
Input redirection is working for .exe files or internal windows commands.
app.exe < ListOfNames.txt
sort < input.txt
However it isn't working when I try to redirect it into a batch script.
test.bat :-
#echo off
echo %1 %2
Running it using :-
test.bat<input.txt
where input.txt has two strings.
However, it is working fine for redirecting output even in case of batch scripts.
Is this the expected behavior or I am making some syntax mistake? Is there any other way to read arguments from a file instead of manually parsing it?
Parameters that are provided on the command line are completely different than stdin ( where your redirected input goes). This is true for both batch scripts as well as .exe programs.
Some programs are designed to accept the same values via command line arguments or stdin. But that is not the norm. That is a feature that is provided by the developer of the program.
If you want to read redirected input within a batch script, then you must do one of the following.
To read a single line:
set /p "ln="
echo %ln%
To read all lines in a loop:
for /f "delims=" %%A in ('findstr "^"') do (
echo %%A
)
Additionally to dbenhams answer, you could also read multiple lines with set/p for a input redirection, like myBatch.bat < commands.txt
#echo off
set "line_1="
set "line_2="
set "line_3="
set /p line_1=
set /p line_2=
set /p line_3=
set line_
But this would fail with an input pipe like type commands.txt | myBatch.bat