cURL post json data, json array and image files REST api testing - arrays

I'm facing a problem with complex http request via cURL.
I'm building REST API with NODEjs, using Express routers and Multer middleware to handle multiple body data and files.
My endpoint route 127.0.0.1/api/postData
expects:
json data with fields, one of which is array of json objects (I'm having nested mongoose schema) and 2 named images (png/jpg).
I need to send Post request via cURL with the following 5-object data structure:
name String
description String
usersArray Array of json objects like: [{"id": "123"}, {"id": "456}]
imgIcon Png/Image providing /path/to/imageIcon.png
imgHeader Png/Image providing /path/to/imageHeader.png
I've read a lot of threads on stackoverflow, but all of them are answers to a particular singular problem, one thread about curl post images, another cURL post arrays, but not altogether.
I've tried REST API test tools like POSTMAN and DHC (google chrome), and there everything's fine except for arraysArray field
I used the fields like:
usersArray[0] {"id": "123"}
usersArray[1] {"id": "456"}
But validation didn't pass, cuz the json object value is parsed somehow incorrectly.
So I decided to put everything in cURL script.
I tried to write my cURL request in following way:
#!/bin/bash
curl -H 'Content-Type: application/json'
-H 'Accept: application/json' -X POST
-F "name=Foo Name Test"
--data '[{"id": "a667cc8f-42cf-438a-b9d8-7515438a9ac1"}, {"id": "7c7960fb-eeb9-4cbf-9838-bcb6bc9a3878"}]'
-F "description=Super Bar"
-F "imgIcon=#/home/username/Pictures/imgIcon.png"
-F "imgHeader=#/home/username/Pictures/imgHeader.png" http://127.0.0.1:7777/api/postData
When I run my cUrl script in bash
./postData
I got this:
$ Warning: You can only select one HTTP request!
You can help with:
1) Any Idea how to write such a complex HTTP REST request in cURL
2) Or with suggestion of tools (like DHC, POSTMAN) to solve this complex http request.
3) or with any idea how to write this request with the help of request.js node http request library.
Thank you all in advance for all answers, thoughts and ideas!!
Regards, JJ

You can try POSTMAN or google chrome ext app for REST api testing DHC.
But you should use multidimensional array instead of using JSON objects as a value, that can cause problem during validation.
Try this in DHC:
|FIELD| |VALUE|
name 'Some name'
description 'Some description'
usersArray[0][id] 89a7df9
usersArray[1][id] dskf28f
imgIcon (select type of field "file" and upload image)
imgHeader (select type of field "file" and upload image)
The way it works above is:
usersArray[0][id] specifies multidimensional array and places on position 0 an object {} with key "id" and value which you sepcify in value part.
So usersArray[0][id] "123" creates [{"id": 123}]
usersArray[1][id] "456" adds another element to array, so array becomes: [{"id": "123"},{"id": "456"}]

Sometimes you would want to use shell+curl to make RestAPI Calls, and you may want to pass complex json as data. And in case you want to use variables and create that data during execution time, you probably end up using lot of escape characters () and code would look ugly. I was doing the same thing as well, Windows powershell has a method to convert Dictionary|Associative Array to JSON which really helps in that realm but looked for something similar but couldn't find anything. So this is a sample code that would help:
#!/bin/Bash
# Author Jayan#localhost.com
## Variable Declaration
email="jayan#localhost.com"
password="VerySecurePassword"
department="Department X"
## Create JSON using variables in Bash
creds="{^email^:^${email}^, ^password^:^${password}^}"
department="{^department^:^${department}^}"
authdata_crippled="{^Authenticate^:[${creds},${department}]}"
# Replace ^ with "
authdata_json_for_RestAPI_Call=$(echo ${authdata_crippled}|tr '^' '"')
# Testing syntax
# Get "jq": yum install jq OR https://stedolan.github.io/jq/
echo ${authdata_json_for_RestAPI_Call} | jq .
# Then you make API call using curl
# Eg:
curl http://hostname:port/right/endpoint/for/api/Authenticate -X POST --header "Content-Type:application/json" -d ${authdata_json_for_RestAPI_Call} --cookie-jar cookie.data
Hope this helps someone.

Related

Best method to send a complex json request with Convertigo

I have a complex json request that I'd like to POST to an api but I can't find the proper way to do this inside Convertigo Studio. Could anyone please indicate me the best way to do this?
Here is the request I'm send through curl that gives me the results. These result will be used by the front to display data.
curl -k -H "Accept: application/json" --compressed -XPOST https://myserverurl/api/search -d #- << EOF
{
"api-key":"somekey",
"usage":"someusage",
"criteria":{
"timestamp":{"from-to":{"date-pattern":"yyyy/MM/dd-HHmmss","from":"2019/07/28-000000","to":"2019/08/27-235959"}},
"timestamp-field":"timestamp",
"metric":"*",
"filter":{
"and":
[
{"eq":{"attribute":"type","value":"sometype"}},
{"simple-query":{"query":"_exists_:city"}},
{"neq":{"attribute":"status","value":"1"}}
]
}
},
"info":"someinfo",
"size":10000,
"mode":"last-hits",
"format":{"tabular":{"columns":["col1", "col2","col3"],
"last-hits-columns":["name"],"order-by":[{"name":"name","direction":"ASC"}]},
"timestamp":{"date-pattern":"dd/MM/yyyy HH:mm:ss"}},
"index":"someindex",
"last-hits-count":"1"
}
EOF
I now would like to transpose that into a Convertigo approach using the proper connector and transactions but so far I'm hitting a wall. Any help is appreciated.
update: So I've managed to contact the API, i.e. reproducing the first part of the curl, by implementing a HTTP_Connector and then a HTTP_Transaction. The server is answering in the expected way.
Now what I can't do is posting the json string. I've tried implementing a http_single_variable which default value is that json string but it doesn't work, I get the following error:
HTTP result {ContentType: application/json, Length: 277}
{"error":{"request":"http://localhost:8550/api/search","message":"Unexpected character ('H' (code 72)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: java.io.InputStreamReader#6c195833; line: 1, column: 2]","target":"/search"}}
The error seems to be coming from the header, which has been defined as Accept, application/json. When I remove it I get a HTTP 500 Error from the server.
To post a JSON body in a Convertigo request you have to add a variable "__body" to your transaction:
HTTP single-valued variable
If your API returns a JSON response, you should use a JSON_HTTP_transaction instead of your HTTP_Transaction transaction.
Set "HTTP verb" property transaction to POST and "HTTP headers" property to "Content-type, application/json".
The value of the __body variable is set in a sequence by a Sequence_JS step like this:
var data = {
"param1": "value1",
"param2": "value2",
...
};
Then, use a jElement step to transform "data" to a JSON string source:
JSON.stringify(data)
in "Expression" property.
Next step is the Call of your transaction. In the __body Source point to the jElement text.
Here is a link to a Convertigo (7.5.7+) sample:
useBody.car
Hope that Helps.

Aggregate Jenkins build logs stored in ElasticSearch

I'm storing my Jenkins build logs in ElasticSearch with the Jenkins Logstash plugin.
My configuration looks sort of like this:
That part works great, but I'd like to view the full log in Kibana.
The plugin incrementally sends the results to ES and breaks on each newline. That means a long log can look something like this in Kibana:
Where each line is a massive JSON output containing tons of fields I do not care about. I really only care about the message field.
I'm reading about aggregators right now that appear to be what I need, but my results are not coming out to what I'd like.
curl -X GET "localhost:9200/_search" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"buildLog" : {
"terms" : {
"field" : "data.url"
}
}
}
}'
Prints out a large glob of json that does not have what I need.
In a perfect world, I'd like to concatenate every message field from each data.url and fetch that.
In SQL, an individual query for this might look something like:
SELECT message FROM jenkins-logstash WHERE data.url='job/playground/36' ORDER BY ASC
Where 'job/playground/36' is one example of every data.url.
How can I go about doing this?
Update: Better answer than before.
I still ended up using FileBeat, but with ELK v6.5.+ Kibana has a logs UI! https://www.elastic.co/guide/en/kibana/current/logs-ui.html
The default config from FileBeat works fine with it.
__
Old answer:
I ended up solving this by using FileBeat to harvest all logs, and then using the Kibana Log Viewer to watch each one. I filtered based on source and then used the path where the log was going to be.

How to create a Perl API to pass data back to an angularjs $http request?

I'm using angular 1.5.7 and perl 5.16.2. I don't have the ability to use any external libraries or switch tools - I'm stuck just using what's included with these two.
The angular controller is making calls to a perl file. I'm able to make POST requests successfully, and my GET requests are returning status 200 but aren't showing the data that I am expecting to see. Right now I'm just working with a simple example to try and narrow down where things are going wrong, and I think I just don't know the format to return values from the perl file.
My controller GET method looks like so:
$http({
method : "GET",
url : "filename.pl",
params:{}
}).then(function successCallback(response) {
console.log(response.records);
}, function errorCallback(response) {
console.log(response);
});
and the perl code being called in filename.pl is:
my $string = q{{"records":{"2":{"City":"México D.F.","Country":"Mexico","Name":"Ana Trujillo Emparedados y helados"},"1":{"Name":"Alfreds Futterkiste","Country":"Germany","City":"Berlin"}}}};
return $string;
The string is just a dummy variable to see if passing things back works - it won't be the final data. I have verified it is valid JSON as well. Here's what the response from the server looks like:
{"data":"","status":200,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"url":"glwizardutils.esp","params":{"FUNCTION":"initcheckboxes","CONTEXTID":"432"},"headers":{"Accept":"application/json, text/plain, */*"}},"statusText":"OK"}
The data field is totally blank, and the response field doesn't work at all. I can verify it's hitting the perl file - like I said, POST requests work fine and I included some printouts that verify it's in the correct method, but it's just not returning anything. What should I be doing here instead to get this to work?
If filename.pl is invoked with HTTP, then it should be writing an HTTP response to standard output. At a minimum:
print "HTTP/1.1 200 OK\n";
print "Content-type: application/json\n";
print "\n";
my $string = q{{"records":{"2":{"City":"México ..."}}}};
print $string;
Several frameworks and modules exist in Perl and virtually every other language to handle the repetitive aspects of writing a proper response, which you will want to look into as your Perl server-side script gets more demanding.

No response Data Angularjs POST / TypeScript

I'm trying to test AngularJS's post method out, but so far I have not figured out how to get it work.
Here is my code snippet.
parameter = { categoryName: '' }; <- just a dummy parameter purposely set to '' for testing purposes
this.httpService({
method: 'POST',
url: '/svc_templates/svc_fetch_category.php',<- Just echoing some JSON string.
data: parameter
}).then(function (response) {
console.log(response);
The response I'm getting is shown below.
It seems that a call to the php file is going through, but the actual data(JSON) is not returning for some reason...
Could anyone help me out?
Make sure that the server .php file is actually working using curl e.g.:
curl -X POST -d #filename.txt http://example.com/svc_templates/svc_fetch_category.php --header "Content-Type:application/json"
Where filename.txt contains the JSON data you want to send.

Backbone.js fetch GET with data

I have a REST server that takes a query string in the request body of a GET statement.
It's similar to the Parse REST api that does the same. As seen in the curl statement below.
curl -X GET \
-H "X-Parse-Application-Id: ${APPLICATION_ID}" \
-H "X-Parse-REST-API-Key: ${REST_API_KEY}" \
-G \
--data-urlencode 'where={"playerName":"Sean Plott","cheatMode":false}' \
https://api.parse.com/1/classes/GameScore
My question is: How do I send a Backbone fetch (essentially a -X GET) with a data string.
Ive tried the following;
fetch: function(options) {
options = _.extend({data: 'Active is true' }, options);
return Backbone.Collection.prototype.fetch.apply(this, arguments);
}
Currently, that appends the string on the URL as parameters as such
http://restserver.com/collection/Customer?Active%20eq%20true
In your collection class, implement the url property as a function that returns the URL path plus the query string you want. The query string can be built up from properties stored on your collection instance like collection.playerName, etc. This pattern works well for a search query string type use case such as yours.

Resources