Until very recently I was able to use postman just fine, but since its latest updates I'm no longer able.
If I copy a POST request from a website as cURL (bash), I get this:
curl 'https://spectate-web.888sport.es/spectate/betslip/betslipFetchData' \
-H 'authority: spectate-web.888sport.es' \
-H 'accept: */*' \
-H 'accept-language: es-ES,es;q=0.9' \
-H 'cache-control: max-age=0' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundaryBfiA5TvRRorRMOGo' \
-H 'cookie: 888Cookie=lang%3Des%26OSR%3D485697%26RefType%3DNoReferrer%26TestData%3D%7B%22orig-lp%22%3A%22https%3A%2F%2Fwww.888sport.es%2F%22%2C%22referrer%22%3A%22NULL%22%7D; _gid=GA1.2.995815426.1666996817; 888TestData=%7B%22orig-lp%22%3A%22https%3A%2F%2Fwww.888sport.es%2F%22%2C%22referrer%22%3A%22NULL%22%2C%22datecreated%22%3A%222022-10-28T22%3A40%3A16.792Z%22%7D; bbsess=kN4YjaaDCW-nJWa4OxHQIRCc364; lang=esp; anon_hash=7b00acc96faf5119781cc6693e88e1bf; spectate_session=4d88ffab-af5c-452a-b0b4-df46f129d8c4%3Aanon; odds_format=DECIMAL; _ga=GA1.2.688931953.1666996817; FPID=FPID2.2.Tj33ha0XQmoaNPw2I5C29pCS2NIxwFa9J8Oic4Mfvs4%3D.1666996817; OptanonAlertBoxClosed=2022-10-28T22:40:21.763Z; _gcl_au=1.1.68988761.1666996822; OptanonConsent=isGpcEnabled=0&datestamp=Sat+Oct+29+2022+00%3A40%3A21+GMT%2B0200+(hora+de+verano+de+Europa+central)&version=6.29.0&isIABGlobal=false&hosts=&consentId=9d994a59-6978-4ff7-b84c-165e3ba44024&interactionCount=1&landingPath=NotLandingPage&groups=C0001%3A1%2CC0002%3A1%2CC0003%3A1%2CC0004%3A1; FPLC=1jxuCIujpmMEA%2B8UtQq4s3DV2dlZAI1BmbBRUYq7XbKBcFDtuXvIo0dzxqGtFeFNPmaTRKmOWUveUdiDJe6FJlGavhhPc8K9cIkAhPKropkJIbGkzkicMBb9Efbj5g%3D%3D; FPAU=1.2.1718678257.1666996822; _fbp=fb.1.1666996823784.1844480782; _sp_srt_ses.7bad=*; _sp_srt_id.7bad=1643e122-62eb-42e4-8b86-c915ac6c026c.1666996824.1.1666996824.1666996824.d981301d-7471-408b-aba2-8a2956d27e63; mzzietknzr=2a2e16c60d6d6820d5f25ff1c0853a; _gat_UA-125725186-2=1; _ga_QFPLQJ4PLW=GS1.1.1666996818.1.1.1666997839.60.0.0' \
-H 'newrelic: eyJ2IjpbMCwxXSwiZCI6eyJ0eSI6IkJyb3dzZXIiLCJhYyI6IjI1MDk2NzQiLCJhcCI6IjMwNzYyODM3IiwiaWQiOiIxMjc4OWM1MTc2ODU5ZjZlIiwidHIiOiJlNDg5YjA4OGUyYzIzZWMxMWQ5YjkxZTRmZjY3ZDc2MSIsInRpIjoxNjY2OTk3ODQ1MzkwfX0=' \
-H 'origin: https://www.888sport.es' \
-H 'referer: https://www.888sport.es/' \
-H 'sec-ch-ua: "Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Windows"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36' \
-H 'x-spectateclient-v: 1.12' \
--data-raw $'------WebKitFormBoundaryBfiA5TvRRorRMOGo\r\nContent-Disposition: form-data; name="selectionDataForFetch"\r\n\r\n[{"id":"11411785409","is_scorecast":false,"is_cast_market":false,"is_any":false}]\r\n------WebKitFormBoundaryBfiA5TvRRorRMOGo\r\nContent-Disposition: form-data; name="allSelectionData"\r\n\r\n[{"stake":0,"each_way_selected":false,"free_bet_id":null,"sp_selected":false,"is_scorecast":false,"is_cast_market":false,"is_any":false,"survey_id":null,"recommended":false,"recommendation_location":"","bet_locate":"unknown_component","id":"11411785409"}]\r\n------WebKitFormBoundaryBfiA5TvRRorRMOGo\r\nContent-Disposition: form-data; name="extraData"\r\n\r\n{"channel":"WEB"}\r\n------WebKitFormBoundaryBfiA5TvRRorRMOGo--\r\n' \
--compressed
It returns an error because " please check your curl string for malformed url". Apparently, if you replace " with \" and ' with " you are able to import the request.
Another problem, it is imported as GET. This you can bypass by adding to the cURL the following
-X POST
but if you do, the POST request is loaded without the payload.
So, how do I accomplish importing the whole request? Why is this happening?
A couple of your headers have double quotes that are not escaped.
Try importing this;
curl -X POST "https://spectate-web.888sport.es/spectate/betslip/betslipFetchData"
-H "authority: spectate-web.888sport.es"
-H "accept: */*"
-H "accept-language: es-ES,es;q=0.9"
-H "cache-control: max-age=0"
-H "content-type: multipart/form-data; boundary=----WebKitFormBoundaryBfiA5TvRRorRMOGo"
-H "origin: https://www.888sport.es"
-H "referer: https://www.888sport.es/"
-H "sec-ch-ua: \"Chromium\";v=\"106\", \"Google Chrome\";v=\"106\", \"Not;A=Brand\";v=\"99\""
-H "sec-ch-ua-mobile: ?0"
-H "sec-ch-ua-platform: \"Windows\""
-H "sec-fetch-dest: empty"
-H "sec-fetch-mode: cors"
-H "sec-fetch-site: same-site"
-H "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
-H "x-spectateclient-v: 1.12"
-d "$------WebKitFormBoundaryBfiA5TvRRorRMOGornContent-Disposition: form-data; name=\"selectionDataForFetch\"rnrn[{\"id\":\"11411785409\",\"is_scorecast\":false,\"is_cast_market\":false,\"is_any\":false}]rn------WebKitFormBoundaryBfiA5TvRRorRMOGornContent-Disposition: form-data; name=\"allSelectionData\"rnrn[{\"stake\":0,\"each_way_selected\":false,\"free_bet_id\":null,\"sp_selected\":false,\"is_scorecast\":false,\"is_cast_market\":false,\"is_any\":false,\"survey_id\":null,\"recommended\":false,\"recommendation_location\":\"\",\"bet_locate\":\"unknown_component\",\"id\":\"11411785409\"}]rn------WebKitFormBoundaryBfiA5TvRRorRMOGornContent-Disposition: form-data; name=\"extraData\"rnrn{\"channel\":\"WEB\"}rn------WebKitFormBoundaryBfiA5TvRRorRMOGo--rn"
Related
I get a request via chrome F12 => Network. I click on the RMB request, click Copy => Copy as cURL (cmd). I get something like this:
curl 'https://account.mail.ru/api/v1/user/signup' \
-H 'authority: account.mail.ru' \
-H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="97", "Chromium";v="97"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundaryPQdK1qxMovrr8yGQ' \
-H 'accept: application/json, text/plain, */*' \
-H 'x-requested-with: XMLHttpRequest' \
-H 'x-request-id: ef1e213c-26a6-332d-bf1b-8be1a971f6fe' \
-H 'sec-ch-ua-platform: "Windows"' \
-H 'origin: https://account.mail.ru' \
-H 'sec-fetch-site: same-origin' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-dest: empty' \
-H 'referer: https://account.mail.ru/signup?from=main&rf=auth.mail.ru&app_id_mytracker=58519' \
-H 'accept-language: ru-RU,ru;q=0.9' \
-H 'cookie:' \
--data-raw $'------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="extended"\r\n\r\ntrue\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="more_password_strength"\r\n\r\n1\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="context"\r\n\r\nsignup\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="browser"\r\n\r\n{"screen":{"availWidth":"1920","availHeight":"1040","width":"1920","height":"1080","colorDepth":"24","pixelDepth":"24","availLeft":"0","availTop":"0"},"navigator":{"vendorSub":"","productSub":"20030107","vendor":"Google Inc.","maxTouchPoints":"0","doNotTrack":"inaccessible","hardwareConcurrency":"8","cookieEnabled":"true","appCodeName":"Mozilla","appName":"Netscape","appVersion":"5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36","platform":"Win32","product":"Gecko","userAgent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36","language":"ru-RU","onLine":"true","webdriver":"false","pdfViewerEnabled":"true","deviceMemory":"8"},"flash":{"version":"inaccessible"}}\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="from"\r\n\r\nmain\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="sent_me_ads"\r\n\r\ntrue\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="sent_me_ads_common"\r\n\r\ntrue\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="name"\r\n\r\n{"first":"imechko","last":"famochka"}\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="birthday"\r\n\r\n{"day":3,"month":6,"year":1992}\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="sex"\r\n\r\nfemale\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="login"\r\n\r\nfamochka\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="domain"\r\n\r\ninbox.ru\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="password"\r\n\r\n111zzz111zzz=\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="restore"\r\n\r\n{"secret":"-","secret_answer":"-","additional_email":"ardacan54524#gmail.com"}\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="htmlencoded"\r\n\r\nfalse\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ\r\nContent-Disposition: form-data; name="utm"\r\n\r\n{}\r\n------WebKitFormBoundaryPQdK1qxMovrr8yGQ--\r\n' \
--compressed
But it does not work if you write this code in start.bat, save it in the same directory as curl.exe and run start.bat. Bat starts to execute this code line by line. What is the correct way to use this code with cURL?
copy as PowerShell (Windows) instead of curl
We've come across a strange issue with fuzzy search requests that have begun to start failing in some browsers.
Running the search from test page here works. If you view this request in the network tab, and copy it as CURL, you get something like:
curl 'https://atlas.microsoft.com/search/fuzzy/json?api-version=1.0&query=New%20York&radius=100000&view=Auto&subscription-key=tTk1JVEaeNvDkxxnxHm9cYaCvqlOq1u-fXTvyXn2XkA' \
-H 'authority: atlas.microsoft.com' \
-H 'dnt: 1' \
-H 'accept-language: en-US' \
-H 'content-type: application/json; charset=utf-8' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36' \
-H 'ms-am-request-origin: ServiceModule' \
-H 'map-agent: ServiceModule/2.0.5 (Web)' \
-H 'accept: */*' \
-H 'origin: https://azuremapscodesamples.azurewebsites.net' \
-H 'sec-fetch-site: cross-site' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-dest: empty' \
-H 'referer: https://azuremapscodesamples.azurewebsites.net/Services%20Module/Fuzzy%20Search%20using%20Services%20Module.html' \
--compressed
If you right click on the search request and click "open in new tab", it fails with 400/Bad Request. If you copy this request as CURL, it will give you something like:
curl 'https://atlas.microsoft.com/search/fuzzy/json?api-version=1.0&query=New%20York&radius=100000&view=Auto&subscription-key=tTk1JVEaeNvDkxxnxHm9cYaCvqlOq1u-fXTvyXn2XkA' \
-H 'authority: atlas.microsoft.com' \
-H 'cache-control: max-age=0' \
-H 'dnt: 1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
-H 'sec-fetch-site: none' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-user: ?1' \
-H 'sec-fetch-dest: document' \
-H 'accept-language: en,nl;q=0.9,en-GB;q=0.8' \
--compressed
which will fail. I've narrowed down the problem to the accept-language param 'en,nl;q=0.9,en-GB;q=0.8' which Chrome seems to have added as a default when I opened in the GET request in the new tab. Replacing this val with 'en-US' makes the request work again.
I've reproduced this on:
Ubuntu 18.04.4 LTS, Chrome Version 81.0.4044.129 (Official Build) (64-bit)
Windows Version 10.0.18362 Build 18362, Chrome Version 81.0.4044.138 (Official Build) (64-bit)
and we've also had customers report it on Edge and Chrome.
So, assuming this default value for accept-language is valid, it seems there's a bug in the azure api parsing it.
A workaround for this problem is to chrome://settings/?search=language and set just one language.
I believe the issue is that an invalid language parameter is being passed in. A list of supported languages can be found here: https://learn.microsoft.com/en-us/azure/azure-maps/supported-languages Passing anything other than one of these would result in an error. To address this issue, you can either override the accept-language header, or add a language parameter to the request "&language=en-US".
The BigCommerce API documentation suggests that image files can be uploaded through the API, without having to upload it elsewhere first:
POST /catalog/products/{product_id}/images
Creates an image on a product. Publically accessible URLs and files (form post) are valid parameters
Emphasis mine. My attempts, variations on the below, mostly come back with 422 image_url must be present if uploading by url.
curl -X POST \
https://api.bigcommerce.com/stores/redacted/v3/catalog/products/123/images \
-H 'accept: application/json' \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'x-auth-client: redacted' \
-H 'x-auth-token: redacted' \
-F productImage=#img_123.jpg \
-F image_url=image_123.jpg
What does a correctly formed request look like, that POSTs an image file to a product?
Related:
Bigcommerce Python API, how do I create a product with an image?
A correctly formed request looks like this:
curl -X POST \
https://api.bigcommerce.com/stores/js......7j/v3/catalog/products/32011/images \
-H 'accept: application/json' \
-H 'cache-control: no-cache' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-H 'x-auth-client: ts2.........................r0r' \
-H 'x-auth-token: ihq.........................5b2' \
-F 'image_file=#Downloads/img_2405.jpg'
However, certain images can cause the misleading error; such as this one.
It's not clear what property of the file causes the error, but compressing or otherwise re-saving the image resolves the problem.
I have a device Stiebel Eltron heat pump home and I would like to use Linux shell curl (not php) curl to login (POST) and retrieve (GET) data once logged
Here is how my curl login POST call looks like (I used Firebug + persist option to copy/paste the below):
curl --data-urlencode 'userName=tutu&password=xx' \
'https://thesite/api/login?noCacheDummyValue=1459356436185' -X POST \
-H 'Host: thesite' \
-H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0' \
-H 'Accept: application/json, text/plain, */*' \
-H 'Accept-Language: en-US,en;q=0.7,fr-FR;q=0.3' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'Content-Type: application/json;charset=utf-8' \
-H 'Referer: https://thesite/mobile/app/app.html' \
-H 'Content-Length: 44' \
-H 'Cookie: JSESSIONID=qhs02mfeip2p1n5n4t2rj1huu'
But this gives me nothing in the output:
sh myproj.sh
curl: (52) Empty reply from server
Q1: What should I collect from this first POST, and how to do it?
I tried to get the cookie with --cookie-jar cookie.txt but nothing showed up in my current directory. However I sent cookies with the header... shouldn't I get them back?
After that I would like to continue the session I (think I) opened so to collect data from a GET that is (same method as above):
curl 'https://thesite/api/data/1036493/heatEnergy/today?noCacheDummyValue=1459356438440' \
-H 'Host: thesite' \
-H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0' \
-H 'Accept: application/json, text/plain, */*' \
-H 'Accept-Language: en-US,en;q=0.7,fr-FR;q=0.3' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'Referer: https://thesite/mobile/dashboard/dash.html' \
-H 'Cookie: JSESSIONID=qhs02mfeip2p1n5n4t2rj1huu'
Q2: How do I keep the session opened to perform the GETabove?
I tried --next option idea from here to call the GET after the POST; without success, --next option is not recognized on my Ubuntu Linux.
If it can help understanding, the page I should get is full of javascript (angularjs I think).
From your POST call it is obvious that you are trying to do an ajax XMLHttpRequest. Likely this is expecting json encoded data. However you are providing url-encoded data.
Probably you should try using something along
... -X POST -H "Content-Type: application/json" -d '{"userName":"tutu","password":"xx"}' 'https://thesite/api/login?noCacheDummyValue=1459356436185'
To make the login work, you need to match the way the server is maintaining the session. If session info is referred to via a cookie yo may just extract the cookie from the POST call (as you said you tried already, but on an error related response)
If the server does not use a straight way of referencing the session you might not succeed in getting your GET call working. If you are lucky maybe you then just need to parse a string from the page (fragment) or json data returned.
So monitor what a working handshake (e.g using a browser) is exchanging over the wire. And try identifing the important pieces of data. Then, put those together to form your GET call.
Thanks to rpy, I was able to do what I wanted:
Collect the session cookie:
curl -a -X POST -d '{"userName":"tutu","password":"xx"}' \
--trace-ascii debugdump.txt --cookie-jar cookies.txt \
-H 'Content-Type: application/json;charset=utf-8' \
-H 'Referer: https://thesite/mobile/app/app.html' \
'https://thesite/api/login?noCacheDummyValue=1459356436185' -o my.post
todo: check result is "ok" in my.post
todo: get content of cookies.txt into ${COOKIE}
Use it to retrieve the raw data I wanted:
curl -a \
-H 'Referer: https://thesite/mobile/dashboard/dash.html' \
-H 'Cookie:JSESSIONID='${COOKIE} \
'https://thesite/api/data/1036493/outTemp/lastMonth?noCacheDummyValue=1459972646687' \
-o lm_outTemp.get \
-L 'https://thesite/api/data/1036493/heatEnergy/lastMonth?noCacheDummyValue=1459972646694' \
-o lm_heatEnergy.get
And I already started working on how to process the data here: Firefox - Collecting the data used by LAB.min.js
I want this cURL
curl https://api.cardinity.com/v1/payments\
-H "Content-Type: application/json" \
-H 'Authorization: OAuth oauth_consumer_key="<your_consumer_key>", \
oauth_signature_method="HMAC-SHA1", \
oauth_timestamp="<timestamp>", \
oauth_nonce="<unique_random_string>", \
oauth_version="1.0", \
oauth_signature="<computed_oauth_signature>"'
transformed into Angular, I've tried everything.