How to escape pipe in curl in a batch script? - batch-file

I have a Windows batch script like this:
#echo off
setlocal EnableDelayedExpansion
SET fruit1=apple
SET fruit2=banana
SET "payload={\"name\":\"value\",\"name\":\"%fruit1%^|%fruit2%\"}"
echo %payload%
curl -X POST -H "Content-Type: application/json" -d "%payload%" "<serverURL>"
endlocal
The output is:
{\"name\":\"value\",\"name\":\"apple|banana\"}
Cannot find the specified path
So Curl is not doing anything, that error message is due to the pipe I guess, while the echoed payload is displayed correctly.
As you can see I already tried escaping the pipe with ^ , and setting EnableDelayedExpansion , but nothing works.
Is there a way to pass the correct payload to curl?

I suggest to use:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "fruit1=apple"
set "fruit2=banana"
set "payload={\"name\":\"value\",\"name\":\"%fruit1%^|%fruit2%\"}"
setlocal EnableDelayedExpansion
echo !payload!
curl.exe -X POST -H "Content-Type: application/json" -d "!payload!" "<serverURL>"
endlocal
endlocal
The environment variables are defined while delayed environment variable expansion is disabled which results in processing each command line with command SET at beginning only once by the Windows command processor.
See also: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
The vertical bar must be escaped nevertheless with a caret character to be interpreted as literal character and not as pipe redirection operator.
Then delayed expansion is enabled and used on referencing the string value assigned to the environment variable payload by using ! around the environment variable name instead of %.
It would be also possible to use in this special case:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "fruit1=apple"
set "fruit2=banana"
set "payload={\"name\":\"value\",\"name\":\"%fruit1%^|%fruit2%\"}"
echo !payload!
curl.exe -X POST -H "Content-Type: application/json" -d "!payload!" "<serverURL>"
endlocal
The three lines defining the three environment variables do not contain an exclamation mark and there is also no other string get from somewhere outside the batch file. For that reason it is possible to enable delayed environment variable expansion with the second command line and let the Windows command processor double process all command lines. The environment variables fruit1 and fruit2 could be referenced in this case also with using delayed expansion by using as fifth command line:
set "payload={\"name\":\"value\",\"name\":\"!fruit1!^|!fruit2!\"}"
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
echo /?
endlocal /?
set /?
setlocal /?
curl -h or curl --help

Related

Extract URL from curled one-line text file using Batch

I'm new in batch files and I need help extracting an URL from a single-line txt-file using Batch.
This is the curl command:
curl -X "GET" %runone% -H "accept: application/json" -o runone.txt
The output stored in the file (runone.txt) is a single line like this:
{"meta":{"terms-and-conditions":{"href":"https://apps.censored.int/datasets/licences/general/"},"license":"CC-BY-4.0","copyright":"censored"},"data":{"link":{"href":"https://apps.censored.int/webapps/opencharts/content/20220620211737-879c62d3db4f29947daaf8140a3f9261a35e4c5b.png","type":"image/png"},"attributes":{"description":"","name":"opencharts","title":"Chart"},"type":"graphical_product"},"tracker":"tracker-dd80b7a5299e46ef8aa8de4041b12aeb","uid":""}
I only want to pick the URL to the png-file from the runone.txt (in this example: https://apps.censored.int/webapps/opencharts/content/20220620211737-879c62d3db4f29947daaf8140a3f9261a35e4c5b.png) and store it to a variable in batch.
Thanks for your help.
#ECHO OFF
SETLOCAL
rem The following settings for the source directory and filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "filename1=%sourcedir%\q72693254.txt"
SET "pngurl="&FOR /f "usebackqtokens=11,*delims={:}" %%b IN ("%filename1%") DO FOR %%e IN (%%c) DO IF NOT DEFINED pngurl SET "pngurl=%%~e"
ECHO pngurl=%pngurl%
GOTO :EOF
Note that if the filename does not contain separators like spaces, then %filename1% does not need to be quoted, and then the usebackq can be omitted.
Solution depends on the format of the output file being as described.
You can try to parse the JSON File and exctract a link from it using Powershell and Batch like this code below :
#echo off
Title Parse JSON File And Exctract A Link From It Using Powershell And Batch
Set "JSON_File=%~dp0runone.txt"
set psCmd="&{(GC "%JSON_File%" | ConvertFrom-Json).data.link.href}"
Call :RunPS %psCmd% URL
Echo "%URL%"
pause & Exit
::-------------------------------------------------------------------
:RunPS <PassPSCMD> <RetValue>
#for /F "usebackq tokens=*" %%i in (`Powershell %1`) do set "%2=%%i"
Goto:eof
:: End of :RunPS function
::-------------------------------------------------------------------
Since the output is JSON, I'd suggest you have a look at the JSON-parser xidel. It'll save you a lot of trouble and headache, especially because of Batch its limitations.
xidel -s --method=GET -H "accept: application/json" "%runone%" -e "$json/data/link/href"
This opens the url and extracts the url to the png-file from the JSON. As xidel can open urls, there's no need for curl, or to save the JSON to a file.
"GET" is the default method, so you can probably do without --method=GET. Maybe the header isn't necessary either.
To assign the png-file-url to a variable:
FOR /F "delims=" %A IN ('
xidel -s --method=GET -H "accept: application/json" "%runone%" -e "$json/data/link/href"
') DO SET "pngurl=%A"
or alternatively:
FOR /F "delims=" %A IN ('
xidel -s --method=GET -H "accept: application/json" "%runone%" -e "pngurl:=$json/data/link/href" --output-format^=cmd
') DO %A
(if you want use this command in a Batch-file, use %%A instead of %A)

Parse ffmpeg output into batch variable

help please.
I am on Windows command line and I want ffmpeg error returned as variable.
Tried many things, still not working.
#echo off&setlocal disabledelayedexpansion
SET S="path\to\ffmpeg.exe" -hide_banner -i InputFile.FLV -vframes 1 -an -s 400x222 -ss 30 OutputFile.jpg
!S!
for /f "usebackq delims=" %%A in ('!S!') do set var=%%A
echo !var!
The result is that the statement !S! produces the correct output on the console (InputFile.FLV: No such file or directory) but does not put that into the variable. The output of echo !var! statement simply shows as "path\to\ffmpeg.exe" -hide_banner -i InputFile.FLV -vframes 1 -an -s 400x222 -ss 30 OutputFile.jpg
How do I get the output of !S! into !var!?
The command setlocal disabledelayedexpansion explicitly disables delayed expansion and so !S! is not expanded delayed at all. For that reason the command line with just !S! results in an error message on execution of the batch file and definitely not in output InputFile.FLV: No such file or directory. And last line just outputs the string !var!.
The usage of for option usebackq results in interpreting the string between '...' as string and not as command line to execute with %ComSpec% /c in a background command process. For that reason the string !S! is assigned to variable var.
Therefore it can be expected that batch file was executed with setlocal EnableDelayedExpansion in real to get the outputs as written in question.
This not tested batch file should work for this task.
#echo off
set "var="
for /F delims^=^ eol^= %%I in ('""path\to\ffmpeg.exe" -hide_banner -i InputFile.FLV -vframes 1 -an -s 400x222 -ss 30 OutputFile.jpg 2>&1"') do set "var=%%I"
if defined var set var
Command FOR with option /F without using option usebackq and with a command line defined between the two ' executes in background %ComSpec% /c and the string between the two ' appended. So executed with Windows being installed in C:\Windows is:
C:\Windows\System32\cmd.exe /c ""path\to\ffmpeg.exe" -hide_banner -i InputFile.FLV -vframes 1 -an -s 400x222 -ss 30 OutputFile.jpg 2>&1"
The started Windows command processor instanced running in background removes in this case first and last " before executing the remaining command line:
"path\to\ffmpeg.exe" -hide_banner -i InputFile.FLV -vframes 1 -an -s 400x222 -ss 30 OutputFile.jpg 2>&1
ffmpeg.exe not installed by me at all outputs information like this one as far as I know to handle STDERR (standard error) instead of STDOUT (standard output). But FOR captures just output written to handle STDOUT of started command process. For that reason 2>&1 is needed to redirect output written to handle STDERR of background command process by ffmpeg.exe to handle STDOUT of background command process for being also captured by FOR of command process which is processing the batch file.
With FOR options argument string delims^=^ eol^= an empty list of string delimiters and no end of line character is defined to really get always the entire line as captured by FOR assigned to specified loop variable I. The two options are specified here by way of an exception not enclosed in " as otherwise it is not possible to define an empty list of delimiters and no end of line character. The two equal signs and the space must be escaped with caret character ^ to be interpreted as literal characters and not as argument string separators because of not being enclosed in a double quoted argument string.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cmd /? ... explains how double quotes in string(s) after option /C or /K are interpreted by Windows command processor and when a file name (or any other argument string) must be enclosed in " on containing a space or one of these characters &()[]{}^=;!'+,`~<|>.
for /?
if /?
set /?
See also the Microsoft article about Using command redirection operators.

set variable and call result during FOR loop; batch script windows

I have read several posts here on Stackoverflow about binding a variable during a FOR loop. While I figure most of the help provided here has been for Linux/Unix, I'm reaching out for help with batch scripting in Windows. My main goal is to extract the "date created" from a mp4-file and "overlay the date on my video" using ffmpeg (and or ffprobe).
I have experimented a lot, but my latest attempt has been trying to bind the result from ffprobe onto a variable, and use the variable later. My latest and simplest attempt looks like this:
SETLOCAL ENABLEDELAYEDEXPANSION
for %%a in ("*.mp4") do (
for /F "tokens=*" %%G in ('ffprobe -v quiet %%a -print_format compact -show_entries format_tags=creation_time') do (
set DateC=%%G
echo !DateC!)
)
I was hoping to be able to print the tag result from ffprobe using that code, but apparently not. So helping me bind that variable, and how to call it again later inside the following code snippet in Windows, would be deeply appreciated:
ffmpeg -i %%a -filter_complex "drawtext=fontfile=/Windows/Fonts/Arial.ttf:x=28:y=650:fontsize=45:fontcolor=white:box=1:boxcolor=black#0.4:text='!DateC!'" -c:a copy output.mp4
I must also mention I've seen the following code on StackOverflow:
ffmpeg -i %%a -filter_complex "drawtext=fontfile=/Windows/Fonts/Arial.ttf:x=28:y=650:fontsize=45:fontcolor=white:box=1:boxcolor=black#0.4:text='%{metadata\:creation_time}'" -c:a copy output.mp4
But I have the same problem making Windows recognize and print the metadata.
I am certain the file in question contains this metadata.
I suggest this not tested code for the batch file:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
for %%a in ("*.mp4") do (
for /F "delims=" %%G in ('ffprobe.exe -v quiet "%%a" -print_format compact -show_entries format_tags^=creation_time 2^>^&1') do (
set "DateC=%%G"
echo !DateC!
)
)
endlocal
The inner FOR runs in a separate command process started with cmd /C the command line:
ffprobe.exe -v quiet "%%a" -print_format compact -show_entries format_tags=creation_time 2>&1
"%%a" is already replaced by name of current *.mp4 file. The double quotes are necessary in case of current *.mp4 file name contains a space or one of these characters &()[]{}^=;!'+,`~.
It is necessary to escape the equal sign with ^ in arguments list to get the command line correct passed to cmd.exe started by FOR in background.
2>&1 results in redirecting output written to handle STDERR to handle STDOUT because of FOR captures only everything written to STDOUT of the started command process.
The redirection operators > and & must be escaped with caret character ^ on FOR command line to be interpreted as literal characters when Windows command interpreter processes this command line before executing command FOR which executes the embedded ffprobe.exe command line in the separate command process started in background.
I don't have ffprobe.exe and ffmpeg.exe installed, but I think those console applications write information about files to handle STDERR (standard error) instead of STDOUT (standard output) which is the reason for using 2>&1 to get the wanted information captured by FOR and assigned to an environment variable.
"tokens=*" is the same as "delims=". Both result in getting the entire line captured by FOR assigned to loop variable G without splitting it up into substrings (tokens) using space/tab as delimiters, except the line starts with a semicolon in which case FOR would ignore that line completely for processing because of internal default eol=;.

Batch file CURL String with Spaces and Special Characters

I'm trying to get a string from an url using a batch file.
String example:
e-e --ser u.g --p 3 --f 0 x,ss
I am using the command below to CURL output directly to a variable:
FOR /F %%I IN ('curl.exe -s -S %URL%') DO (SET W=%%I)
The problem is, when I echo the variable [W] after the command runs, most of the string is missing...
e-e
What is the best method to get around this issue?
By default, the FOR /F command delimits the output based on a space and tab. That is stated in the help file. To keep that from happening use the DELIMS option to tell the FOR command to not use any delimiters.
FOR /F "delims=" %%I IN ('curl.exe -s -S %URL%') DO (SET W=%%I)

How to put a single PowerShell output string into a cmd variable?

I have a PowerShell script that outputs a single string value. I have a cmd batch script that needs to execute the PowerShell script and place that single PowerShell output value into a variable in the batch script. I'm finding all sorts of methods for exporting to a file or reading a file but that's not what I want. Thanks!
(edit) Here's where I'm trying to use it (in response to posting the script):
#echo off
REM The next line puts the .ps1 output into the variable
REM and, obviously, this does not work
set pass_word=<C:\temp\PullPassword.ps1
tabcmd login -s "http://myserver.net" -u mylogon -p %pass_word%
( edit )
I saw the answer by the OP here Getting Powershell variable value in batch script and also looked at foxdrive's answer so I started playing with the FOR...DO statement. I thought I was pretty good with cmd and couldn't figure out why what I had wasn't working:
for /f "delims=" %%a in ('powershell . "C:\temp\PullPassword.ps1"') do set val=%%a
echo %a%
When I was looking at foxdrive's full answer in the other post it struck me: %a% was wrong, I needed %val%! Oh, the shame! The below works:
#echo off
set mypath=C:\temp\PullPassword.ps1
for /f "delims=" %%a in ('powershell . "C:\temp\PullPassword.ps1"') do set pass_word=%%a
tabcmd login -s "http://myserver.net" -u mylogon -p %pass_word%
So I'll credit where it's due and mark foxdrive's answer correct, even though it was the other post that clarified my mistake.
This may help: it expects that the powershell script outputs the text to STDOUT which is the normal place for it to appear.
#echo off
for /f "delims=" %%a in (' powershell "script.ps1" ') do set "var=%%a"

Resources