How to handle GZip/Deflate on HTTP Actions - azure-logic-apps

I have an HTTP Action in Azure Logic Apps that calls a StackExchange API, fundamentally, it could be any API that returned GZip or Deflate content by default:
Because the response is neither Plain Text nor JSON the output from the HTTP Action is:
{
"$content-encoding": "gzip",
"$content-type": "application/json; charset=utf-8",
"$content": "H4sIAAAAAAAEAGWPzY7CMAyE38XnqsrPFtq+ClpFoXghEkm6iVtWQrw7Lt3mAEd/Hs+M7+AIfYb+cIeAN2MHcjOaKWNiKCqgSPa6zZ2SFRzt6YzZjJiMd2EiZF0tvjbpuoZe7rrdxuZIC9KNLo5D9B4DLUIl2gpsyDfOeLflvN8JM7kYPnbF6/+WA/ZNYcOAI+Hp5V/eCKv0heVGSwD0SnRcZXQm4ewyM+hBCbmvda3aWjVaS3h8V3Cx2fiYuMePvWZcSrKV8faPSyzF1jmhty64cGbnVjyeRIHcnG0BAAA="
}
If you went to the trouble of passing the $content field through #base64toString() you would end up with the Gzip binary representation of the JSON, and that is as far as I can take it.
Question: How can I either force the HTTP Action to behave like a HttpClient and Accept GZip data and emit the JSON from the Action, or more laboriously take the GZip Base64/Binary data and decompress it before acting further upon it?

I wasn't able to find a way to "solve" the problem directly, so I created an Azure Function that made the request to the URL with the appropriate HttpClientHandler passed into the HttpClient:
#r "Newtonsoft.Json"
using System;
using System.Net;
using Newtonsoft.Json;
public static async Task<object> Run(HttpRequestMessage req, TraceWriter log) {
string jsonContent = await req.Content.ReadAsStringAsync();
dynamic data = JsonConvert.DeserializeObject(jsonContent);
if (data.url == null) {
return req.CreateResponse(HttpStatusCode.BadRequest, new {
error = "Please pass a URL in the input object"
});
}
HttpClientHandler handler = new HttpClientHandler() {
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
var client = new HttpClient(handler);
var result = await client.GetAsync((string)data.url);
var response = req.CreateResponse(HttpStatusCode.OK);
response.Content = result.Content;
return response;
}
By swapping the built-in HTTP Action from the original question, I was able to get this working:
And the configuration for the Azure Function is as follows:

Related

How to use Axios.post() in a post request containing a json object request-body and a multipart form data (MP4)

Hi I Was wondering how I can send a a single axios post request containing a json object as the request body and also multipart form data (Mp4 file).
In my example I want to send 'details' and 'file'. I have tried sending details and file as 2nd and 3rd arguments to the axios.post() method but from what I can tell axios.post only accepts 2 args.
I have also tried appending the details and then the file, to the form data, but this does not work either.
If I split these into 2 seperate post calls, it works fine, but my application requires these to happen together.
I am getting the following error in my spring console:
[org.springframework.web.HttpMediaTypeNotSupportedException: Content type 'multipart/form-data;boundary=----WebKitFormBoundaryqKyZ0R2SyFeDNCVp;charset=UTF-8' not supported]
Here is the error in my web dev tools console:
xhr.js:210 POST http://localhost:9191/api/123/file/upload 415
Id really appreciate any suggestions
const FileUpload = () => {
const [file, setFile]= useState(null)
const[details, setDetails] = useState({consent:false,
idConfirmed:false,
label:"",
roundId:""})
const changeHandler=(e)=>{
setFile(e.target.files[0]);
setDetails(prevDetails=>({
...prevDetails,
consent:true,
idConfirmed:true,
label:"test_Label"
}));
};
const handleSubmission=(e)=>{
e.preventDefault();
const data = new FormData();
data.append("file", file)
data.append("file", details)
console.log("Data: ", data)
axios.post(`${process.env.REACT_APP_URL_NEW_ROUND_VID}/123/file/upload`, data,
{
headers:{
"Content-Type":"multipart/form-data"
}
})
.then(res=>{
console.log("Data: ",res.data)
console.log("success")
})
.catch((e)=>{
console.log("Error", e)
})
//})
};
Here is my rest end point in Springboot:
#PostMapping(
path = "{patientId}/file/upload",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE)
public void addWardRound(#PathVariable("patientId") String patientId,
#RequestParam("file") MultipartFile file,
#RequestBody WardRequest wardRequest){
WardRoundService.isFileEmpty(file);
WardRound round = service.saveRound(wardRequest);
String roundId = round.getRoundId();
service.uploadVid(patientId, roundId, file);
}
you can stringify and send the JSON data. but by using this method you need to parse it in the server!. it could be difficult
The best method is splitting it into 2 APIs
you can merge both API requests in UI by merging both APIs using promise.All
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
const promise1 = axios.get(URL1);
const promise2 = axios.post(URL2, data);
Promise.all([promise1, promise2]).then(function(values) {
console.log(values);
});
or else if you need to use the result of first API
then call the second API inside the response of first API itself

How can I make an authenticated http request and return a stream of objects with dart?

I have to put the API Key in the request header as:
Authorization: Bearer "YOUR API KEY"
This is my code (I'm not sure where to put the header and how)
Future<Stream<Book>> getBooks() async {
var url = ‘example_url’
var client = http.Client();
var streamedResponse = await client.send(
http.Request(‘get’, Uri.parse(url))
);
return streamedResponse.stream
.transform(utf.decoder)
.transform(json.decoder)
.expand(jsonBody) => (jsonBody as Map)[‘results’] )
.map((jsonBook) = Book.fromJson(jsonBook));
}
The Flutter docs https://flutter.io/cookbook/networking/authenticated-requests/ says to use this format for authenticated requests but this is not for streams, this returns a future of an object (Book)
Future<Book> fetchPost() async {
final response = await http.get(
'https://jsonplaceholder.typicode.com/posts/1',
headers: {HttpHeaders.authorizationHeader: "Place your_api_token_here"},
);
final responseJson = json.decode(response.body);
return Book.fromJson(responseJson);
}
You can add custom headers after you created the Request
final request = http.Request('GET'), url)
..headers.addAll(myHeaders);
I have made a custom header using http.Request as follow bellow :
final url =
'https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.6.0-amd64-xfce-CD-1.iso';
final request = Request('GET', Uri.parse(url));
request.headers.clear();
request.headers.addAll({"content-type":"application/json"});

DART & GAE : Why a POST method send from dart can't be evaluate in GAE?

I have a Dart code used to send an HttpRequest with a POST method to my GAE WepApp2 application. The dart code is executed in chromium and serve by Chrome dev editor. I add in my GAE code some headers to avoid the XHR error in the client side.
The dart code send the datas to my GAE app but I can't read the data with self.request.POST.get("language")) and the app never enter in def post(self): section but with self.request.body I can read the data.
Could you explain that and provide some correction to have a full POST compliant code?
dart:
void _saveData() {
HttpRequest request = new HttpRequest(); // create a new XHR
// add an event handler that is called when the request finishes
request.onReadyStateChange.listen((_) {
if (request.readyState == HttpRequest.DONE &&
(request.status == 200 || request.status == 0)) {
// data saved OK.
print(request.responseText);
}
});
// POST the data to the server
var url = "http://127.0.0.1:8080/savedata";
request.open("POST", url, async: false);
String jsonData = JSON.encode({"language":"dart"});
request.send(jsonData);
}
GAE code in my handler:
def savedata(self):
logging.info("test")
logging.info(self.request.body)
logging.info(self.request.POST.get("language"))
def post(self):
logging.info("test 2")
logging.info(self.request.POST.get("language"))
self.response.headers["Access-Control-Allow-Origin"] = "http://127.0.0.1:49981"
self.response.headers["Access-Control-Allow-Methods"] = "POST, GET, OPTIONS"
In Dart, if you don't specify request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded") in your HttpRequest, the data is considered by GAE like a bite stream and you can only read them with self.request.body
If you add the Content-Type header in Dart you need also to change the data formating. In my case I mimic a form sending with POST method so I change String jsonData = JSON.encode({"language":"dart"}); by String jsonData = "language=dart2";
IN GAE python I can now read the data with self.request.POST.get("language")
If you need to send a JSON from DART to GAE, you can encode the string like this:
String jsonData = JSON.encode({"test":"valuetest1"});
String datas = "datas=$jsonData";
request.send(datas);
In GAE you can read the datas like this:
my_json = json.loads(self.request.POST.get("datas"))
logging.info(my_json["test"])
The complete code:
Dart
void _saveData2() {
String url = "http://127.0.0.1:8080/savedata";
HttpRequest request = new HttpRequest()
..open("POST", url, async: true)
..setRequestHeader("Content-Type", "application/x-www-form-urlencoded")
..responseType = "arraybuffer";
String jsonData = JSON.encode({"test":"valuetest1"});
String datas = "datas=$jsonData";
request.send(datas);
}
GAE
class PageHandler(webapp2.RequestHandler):
def savedata(self):
self.response.headers.add_header('Access-Control-Allow-Origin', '*')
self.response.headers['Content-Type'] = 'application/json'
#logging.info(self.request)
my_json = json.loads(self.request.POST.get("datas"))
logging.info(my_json["test"])

AngularJS ajax resulting in two calls for html & json

So I turned on network capture in my web browser to have a look at my ajax calls coming out of AngularJS. For every call made, there seems to be two results:
URL|Protocol|Method|Result|Type|Received|Taken
/p/jobs/03512dc8-6f25-49ea-bdff-0028ac2023cb|HTTP|GET|301|text/html|408 B|< 1 ms
/p/jobs/03512dc8-6f25-49ea-bdff-0028ac2023cb|HTTP|GET|200|application/json|0.79 KB|15 ms
It looks like it's attempting to request HTML first, getting a 301 and then requesting the JSON. How can I eliminate the request for HTML? I'm using $resource to do this but I'd also like to see an example for $http.
Also, the receiving framework is NancyFX for .NET. Perhaps there's a header I need to specify to make sure it always returns JSON? I've tried the "Accept" header but it seems to make no difference. It's as if Nancy is always trying to return a View first before switching to JSON.
Javascript code (translated slightly from TypeScript):
$resource(jobUrl, {}, { get: {method: "GET", isArray: false }});
Nancy code:
public class JobService : NancyModule
{
public static readonly string Prefix = "/p/jobs";
WebLogger logger;
public JobService(WebLogger logger)
: base(Prefix)
{
this.logger = logger;
Get[""] = _ => GetJobs();
Get["/{id}"] = _ => GetJob(_.id);
Get["/{id}/nodes"] = _ => GetNodes(_.id);
Get["/{id}/faults"] = _ => GetFaults(_.id);
}
Job GetJob(string id)
{
lock (logger)
{
if (logger.JobGuid != id)
{
Context.Response.StatusCode = HttpStatusCode.NotFound;
return null;
}
return MakeJob();
}
}
Some part of your code will be helpful, but lets try - you have put character / at the end of $resource definition. Angular has asked server for content for directory, server response with header 301 - redirect to file as Angular expects some data response than directory listing.

OAuth issue in Google AppEngine developing

i have encountered a weird problem in Google App Engine developing, every time is carry a body content in my post request, app engine failed to auth my account, but get request works.
can anybody help me? i'm using oauth library ChromeExOAuth in chrome extension developing.
oauth.authorize(function(){
var request = {
'method': 'POST',
'headers': {
"Content-Type" : "application/x-www-form-urlencoded"
},
'parameters': {
},
'body': "a=b"
};
oauth.sendSignedRequest("http://mytabshub.appspot.com/tabshub", function(resp, xhr){
console.log("responding from test server", xhr.responseText);
}, request);
});
For POST requests you must pass the oauth parameter url-encoded in the request body. The relavant code in the SDK is this (dev_appserver_oauth.py):
def _Parse(self, request, base_env_dict):
"""Parses a request into convenient pieces.
Args:
request: AppServerRequest.
base_env_dict: Dictionary of CGI environment parameters.
Returns:
A tuple (method, path, headers, parameters) of the HTTP method, the
path (minus query string), an instance of mimetools.Message with
headers from the request, and a dictionary of parameter lists from the
body or query string (in the form of {key :[value1, value2]}).
"""
method = base_env_dict['REQUEST_METHOD']
path, query = dev_appserver.SplitURL(request.relative_url)
parameters = {}
if method == 'POST':
form = cgi.FieldStorage(fp=request.infile,
headers=request.headers,
environ=base_env_dict)
for key in form:
if key not in parameters:
parameters[key] = []
for value in form.getlist(key):
parameters[key].append(value)
elif method == 'GET':
parameters = cgi.parse_qs(query)
return method, path, request.headers, parameters
See that the query is only parsed in GET requests. For POST, it must be in the body.

Resources