Batch to loop through variables and send API request - batch-file

Question

I am facing an issue to get some data from an API using the command lines in Windows. Basically, this is what I want to do:
1- Get a list of clients from an API (using curl and jq),
2- Save this list in a .txt (I wanted to save it in a variable but did not manage),
3- Loop through the list of clients within the .txt and send a new API request to download a csv specific to this client
The first 2 steps are working fine, but I'm stuck on the last bit to loop through the clients. Here is my code:
setlocal ENABLEDELAYEDEXPANSION
Set tok=XXXXX
Set hURL="https://api.website.com/v2/clients?token=%tok%"
Set IDPath="C:\Users\My Self\subIDs.txt"
Set cuPath=%~dp0
del %IDPath%
curl -sS %hURL% | jq ".clients[].id" > %idPath%
FOR /F %%i in (%IDPath%) do (
echo %%i
Set subID=%%i
Set rURL="https://api.website.com/v2/data?token=%tok%&subscriptionId=!subID!"
curl -sS -o cuPath!subID!.csv !rURL!
)
endlocal
pause
What looks odd to me:
- echo %%i returns: C:\Users\My
It looks like the space within the path is a problem for this loop, but it didn't cause any issue to save the file, so I'm a bit lost
- I am using !subID! as I understand it's the only way to get the variable ajusted each time however when looking at the output of the !rURL! variable, while %tok% is successfuly passed as XXXXX, !subID! remains as !subID!
- I finally get, of course, a curl: (6) Could not resolve host: rURL
I am completely new to APIs, batch or JSON and even though I feel I am getting closer, I am now stuck on this last bit. Any idea how to solve this?
Thanks,

Answer 1

You had some mistakes in your code which is fixed in the following code.
But since you have not specified the exact format that is returned by jq ".clients[].id" I could only guess that it returns a single token without any spaces between the characters, but it may have leading or trailing spaces.
So if this is not the case you have to setup the FOR /F options accordingly.
#echo off
setlocal DisableDelayedExpansion
Set "tok=XXXXX"
Set "hURL=https://api.website.com/v2/clients?token=%tok%"
Set "rURL=https://api.website.com/v2/data?token=%tok%&subscriptionId="
Set "IDPath=C:\Users\My Self\subIDs.txt"
:: No need to delete %IDPath%, It will be overwrriten by the below redirection
curl -sS "%hURL%" | jq ".clients[].id" > "%IDPath%"
FOR /F "usebackq" %%i in ("%IDPath%") do (
echo subID=%%i
curl -sS -o "%~dp0%%i.csv" "%rURL%%%i"
)
REM del "%IDPath%"
endlocal
pause
Some points:
You have assigned the variables with embedded surrounding quotes in their values, while this will work in your particular case, It is a best practice to not embed the quotes in variables values, So instead of Set hURL="https://..." use Set "hURL=https://..." and you can later surround the variable in quotes when actually using them: "%hURL%"
Think of the situation where you want to access the value without quotes, then you have to write additional code to remove the quotes from the value if you assigned the value by set a="value" . But by set "a=value" you can access to both quoted ("%a%" ) and unquoted (%a%) value at the same time.
You have used Set cuPath=%~dp0 without quotes, it should be Set "cuPath=%~dp0" but it is not needed either, you could just use %~dp0 directly in the loop.
And later you've used cuPath in the FOR loop without expanding it and without surrounding the cURL output file in quotes.
When reading a file contents by FOR /F, if the path to the is surrounded in quotes then you have to use the UseBackq option to change the interpretation of quotes in the IN clause of the FOR to not interpret it as literal string.
Based on what you are doing, delayed expansion is not needed at all.
But as side note you have to be more careful when using delayed expansion. when delayed expansion is enabled, any literal string that you may use in your code, including file and url paths, will get corrupted if they contain the ! character.
So It is better setup the required variables and strings while delayed expansion is off and later access them by !var! syntax when delayed expansion is enabled.