Cannot start Geospatial Analytics - analytics

I am new to Bluemix I and was trying out the IoT Connected Vehicle Tutorial http://m2m.demos.ibm.com/dl/iot-connected-vehicle-tutorial.pdf
Everything is working fine until I try to start the Geospatial Analytics Service by accessing the link for my app:
http://lm-trafficsim.eu-gb.mybluemix.net/GeospatialService_start
After approximately 1 minute it shows:
502 Bad Gateway: Registered endpoint failed to handle the request.
and the logs are:
[App/0]OUTAbout to call /GeospatialService_start
[App/0]OUTNO BODY
[App/0]OUTOptions prepared: { host: 'streams-broker.eu-gb.bluemix.net',
[App/0]OUT port: '443',
[App/0]OUT headers:
[App/0]OUT method: 'PUT',
[App/0]OUT path: '/jax-rs/geo/start/service_instances/538a9b3d-7160-4235-8e47-9be62d873842/service_bindings/2709488e-2310-4628-acef-df0313877bb5',
[App/0]OUT 'Content-Type': 'application/json',
[App/0]OUT { Authorization: 'Basic YWIxM2I3ODQtMzdmZi00ZGI2LWJkYTctYTgwYzc3MmMwNDY1OjA3YjU0NzMwLWYxM2MtNGYxYi1iZjkzLWY4ZWNlMDEwNDFhYg==',
[App/0]OUTWriting json:
[App/0]OUT 'Content-Length': 448 } }
[App/0]OUTDo the GeospatialService_start call
[App/0]OUT {
[App/0]OUT "mqtt_pw": "xxxxx",
[App/0]OUT "mqtt_uid": "xxxxx",
[App/0]OUT "mqtt_client_id_notify": "a:ybv0lr:geoNotify627",
[App/0]OUT "mqtt_client_id_input": "a:ybv0lr:geoInput393",
[App/0]OUT "mqtt_uri": "ybv0lr.messaging.internetofthings.ibmcloud.com:1883",
[App/0]OUT "device_id_attr_name": "id",
[App/0]OUT "mqtt_notify_topic": "iot-2/type/api/id/geospatial/cmd/geoAlert/fmt/json",
[App/0]OUT "mqtt_input_topics": "iot-2/type/vehicle/id/+/evt/telemetry/fmt/json",
[App/0]OUT "latitude_attr_name": "lat",
[App/0]OUT "longitude_attr_name": "lng"
[App/0]OUT}
[App/0]ERR{ [Error: socket hang up] code: 'ECONNRESET' }
[App/0]OUT[0mGET /GeospatialService_start [0m- [0m- ms - -[0m
[RTR/0]OUTlm-trafficsim.eu-gb.mybluemix.net - [12/05/2016:20:47:37 +0000] "GET /GeospatialService_start HTTP/1.1" 502 0 67 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" 159.122.215.10:19754 x_forwarded_for:"188.26.151.29" x_forwarded_proto:"http" vcap_request_id:b8812e5c-98dd-4d5a-6c2b-75780c6f0996 response_time:120.113068106 app_id:81052294-dfad-43c6-980f-1f058011632d x_global_transaction_id:"333879271"
I have stopped my firewall as I read in some other posts that it may prevent accessing the port, but that did not help. Any ideas of what could be wrong?
The only difference from the tutorial is that the Geospatial Analytics plan I am using is Standard, not Free, because Standard is the only one I could choose.

The article at the PDF you mentioned is old. I'll explain what is wrong with the code and how to fix it.
The example node.js application was written when the Geospatial Analytics service in Bluemix was in beta. The application uses http requests to call the service's REST APIs. When the service switched from beta to production it required using https for REST API access. The service no longer supports REST API access using http. That is what is causing the ECONNRESET in your application log.
To get the existing code working, in the app.js source file change line 462 from:
http = require('http'),
to:
https = require('https'),
And change line 617 from:
var reqPut = http.request(options, function(res) {
to:
var reqPut = https.request(options, function(res) {
After making the changes use cf push to update your application in Bluemix. When the application restarts you should see successful calls to the Geospatial Analytics REST APIs.
Having said all that, you can also use the final version of the connected car demo article at http://www.ibm.com/developerworks/library/mo-connectedcar-app/index.html. This updated article has a link to the current application code at jazz.net with the fix above in addition to some other enhancements and fixes.

Related

React-call to "Asp.Net Core with identity server" gives CORS-errors

Ill try to keep this short to save digital rain forest. Please ask If I missed any details.
I have an "asp .net 3.1 core + react"-project template in VS, with built in Identity server. This works ok, but I now want to do my react project in a separate project. So I started a new create-react-app-project.
So, from my new react project, when I call OidcConfigurationController. The controller method is called and I can step through the code on server side. Then I get a client error "Failed to fetch", which, by internet wizdom, seems to indicated CORS-error.
This is what I got when I inspect the header in chrome dev toolbar->network
Request URL: https://localhost:5001/authentication/_configuration/MyProject.Web
Referrer Policy: strict-origin-when-cross-origin
:authority: localhost:5001
:method: GET
:path: /authentication/_configuration/MyProject.Web
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,sv;q=0.8
origin: http://localhost:3000
referer: http://localhost:3000/
sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"
sec-ch-ua-mobile: ?0
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
These are relevant lines in startup.cs
ConfigureServices()
services.AddCors(options =>
{
options.AddPolicy(name: MyAllowSpecificOrigins,
builder =>
{
//builder.WithOrigins("http://localhost:3002/", "https://localhost:3001")
builder.AllowAnyOrigin()
.AllowAnyMethod()
.AllowAnyHeader();
});
});
services.AddSingleton<ICorsPolicyService>((container) => {
var logger = container.GetRequiredService<ILogger<DefaultCorsPolicyService>>();
return new DefaultCorsPolicyService(logger)
{
AllowAll = true
};
});
Configure()
app.UseCors(MyAllowSpecificOrigins); // I also tried to switch order on these 2 rows
app.UseIdentityServer();
Nothing I do here seems to change the Referrer Policy in the header, still get the exact same message
The React-call is just a plain fetch(address-of-the-controller-that-it-hits).
I have also tried to start a new Server Side-project (asp net core api) and set same CORS-policy, I can call this api from my react client without getting any errors)
So, in the request, you see the origin: http://localhost:3000 header is used. That is the source for the CORS request. But the request is for this URL:
https://localhost:5001/authentication/_configuration/MyProject.Web
Could it not be that there's a redirect from insecure HTTP to HTTPS that is interfering?
Do make sure you set the CORS settings in IdentityServer as well.
See the CORS documentation for more details.
As side note, IIS might cause CORS issues as well, see this answer for details:
IIS hijacks CORS Preflight OPTIONS request

Removing header (User-agent) from make_fetch_call while requesting from GAE

I have an application of Google App Engine(GAE) and I am using Python 2.7. This application receives an GET(ajax) request from user portal(say Chrome). Upon receiving the request, I prepare Asynchronous connections for requesting data from multiple websites(say X1, X2, etc) outside GAE using urlfetch.make_fetch_call() - GET request.
This worked fine for X1 website but not for X2. Started probing on local dev server. Upon probing I suspected that X2 is checking {'User-Agent':'Python-urllib/2.7'} tag in header. This is my best guess since changing this field to {'User-Agent': 'Mozilla/5.0'} returns the desired results.
So I uploaded the code to GAE and started the process with urlfetch.make_fetch_call(). Upon intercepting this call i found that no matter what i do, the default header added by GAE is not removed.
Here is the default header added by GAE.
302 218ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36 AppEngine-Google; (+http://code.google.com/appengine; appid: s~xxx-etching-112014) module=default version=1
107.178.194.96 - - [06/Feb/2016:19:57:04 -0800] "GET / HTTP/1.1" 302 383 "http://www.mywebbsite.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36 AppEngine-Google; (+http://code.google.com/appengine; appid: s~xxx-etching-112014)" "1.usedForIntercepting.appspot.com" ms=218 cpu_ms=224 cpm_usd=0.000043 loading_request=1 app_engine_release=1.9.32 trace_id=fd7b7420e7f8c23371a5b0ea7e9651 instance=00c61b117ce5ebac2a2eba44f26a01d4f2
This is what i have tried
for portal in self.searchPortals:
spoofHeader = {
'User-agent':'Mozilla/5.0----------------------',
'Host':portal.getURL(),
'Accept-Encoding': 'identity',
'Connection': 'close',
'Accept': 'application/json, text/plain, */*',
'Origin': 'http://www.mywebsite.com'
}
logging.info(spoofHeader)
rpc = urlfetch.create_rpc(deadline=5)
rpc.callback = lambda: self.handleCallBack(rpc, portal)
#urlfetch.make_fetch_call(rpc, portal.getSearchURL(searchKeyword), headers={'User-agent':'Mozilla/5.0'})
urlfetch.make_fetch_call(rpc, url='http://1.usedforintercepting.appspot.com', headers=spoofHeader)
rpcs.append(rpc)
for rpc in rpcs:
rpc.wait()
This is what i received.
2016-02-07 13:01:21.306 / 302 59ms 0kb Mozilla/5.0---------------------- AppEngine-Google; (+http://code.google.com/appengine; appid: s~xxx-etching-112014) module=default version=1
107.178.194.20 - - [06/Feb/2016:23:31:21 -0800] "GET / HTTP/1.1" 302 383 - "Mozilla/5.0---------------------- AppEngine-Google; (+http://code.google.com/appengine; appid: s~xxx-etching-112014)" "1.usedForIntercepting.appspot.com" ms=59 cpu_ms=6 cpm_usd=0.000043 app_engine_release=1.9.32 trace_id=a4a1f521c5a6fa65ed0295835dd175 instance=00c61b117ce5ebac2a2eba44f26a01d4f2
What i want is something like this.
GET http://somelink/search/abc HTTP/1.1
Accept-Encoding: identity
Host: somelink.com
Connection: close
User-Agent: Mozilla/5.0
I want to remove everything form header other than User-Agent:Mozilla/5.0 ??
Note - for intercepting the request made from GAE using urlfetch i am using another instance of GAE.
In the documentation, URL Fetch Python API Overview: Request Headers, it says
For security reasons, the following headers cannot be modified by the application:
Content-Length
Host
Vary
Via
X-Appengine-Inbound-Appid
X-Forwarded-For
X-ProxyUser-IP
It also says:
The following headers indicate the app ID of the requesting app:
User-Agent. This header can be modified but App Engine will append an identifier string to allow servers to identify App Engine requests. The appended string has the format "AppEngine-Google; (+http://code.google.com/appengine; appid: APPID)", where APPID is your app's identifier.
If you want custom headers, you will have to write your own urlfetch code or use an outside server that makes the call for you with your headers.

CSRF Cookie not set when posting request with AngularJs - Django Backend

I'm building a web app with angularjs and django and I'm submitting form via Ajax request.
My problem is that when posting an Ajxa request with angular (ng-file-upload precisely) the csrfmiddlewaretoken expected by django is not set.
From my lectures on angular documentation and other forums I ended up with the following configuration.
In the config part of angular :
$httpProvider.defaults.xsrfCookieName = 'csrftoken';
$httpProvider.defaults.xsrfHeaderName = 'X-CSRFToken';
$httpProvider.defaults.withCredentials = true;
and in my controller the code for sending the request is :
Upload.upload({
url: 'http://localhost:8000/image',
headers: {
'X-CSRFToken': $cookies['csrftoken']
},
data: {file: file}
})
With that code the request send has the following headers :
Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate
Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4,de;q=0.2,fi;q=0.2
Connection:keep-alive
Content-Length:16582
Content-Type:multipart/form-data; boundary=----WebKitFormBoundarybWo821vSwcejTATP
Cookie:csrftoken=bC2UpXurGXAg3AUZgSqMVlUs8TKfussS
Host:localhost:8000
Origin:http://127.0.0.1:8000
Referer:http://127.0.0.1:8000/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36
X-CSRFToken:UeSt4LoqgU9L28JQBdVbS2IJJXOMQK6n
However for django to be able to handle csrf protection correctly the following header is missing
Cookie:_ga=GA1.1.1358238168.1447425523; XSRF-TOKEN=zWIM6q7O2Nz3PLm8TMUJSLFVRF8bKUbr; csrftoken=UeSt4LoqgU9L28JQBdVbS2IJJXOMQK6n
So far and despite having seen a lot of forums about this topic I didn't manage to set this header. if I try to set it programmatically via :
Upload.upload({
url: 'http://localhost:8000/image',
headers: {
'Cookie': 'csrftoken='+$cookies['csrftoken']
},
data: {file: file}
})
I end up with the following error in my console :
Refused to set unsafe header "Cookie"
My problem is really how to configure the cookie header from the client side. My django code is fine.
I have been struggling with this for quite a time now. Any help would be appreciated ! Thanks
If you added in the csrftoken to client headers: {'X-CSRFToken': $cookies['csrftoken']} means your client is most probably ready, but for security matter if you interact with django api from external application he will still block the request returning unsafe header "Cookie".
try the following configuration to allow the cross site request over your app:
pip install django-cors-headers
and then add it to your installed apps in your settings.py:
INSTALLED_APPS = (
...
'corsheaders',
...
)
You will also need to add a middleware class to listen in on responses and make sure you respect the order as follow:
MIDDLEWARE_CLASSES = (
...
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
...
)
and finally add this settings variable:
CORS_ORIGIN_ALLOW_ALL = True
This should be enough but for more customized configuration you can check django-cors-headers

Angularjs using JWT breaks CORS to Amazon S3 on login

I've got an interesting issue happening on my angularjs app. When I login, all of the template requests in angular made to Amazon S3 stop working, and return a 400 Bad Request. They work completely fine before you login. The only thing that should change when logged in, is a json web token is sent in the headers to verify the person logged in. My thoughts are maybe the interceptor that is sending the jwt in the headers is somehow affecting CORS on Amazon S3. Seems strange.
Here is the interceptor code:
.factory('TokenInterceptor', function ($q, $window) {
return {
request: function (config) {
config.headers = config.headers || {};
if ($window.sessionStorage.token) {
config.headers.Authorization = 'Bearer ' + $window.sessionStorage.token;
}
return config;
},
response: function (response) {
return response || $q.when(response);
}
};
});
EDIT: It was giving me an Access Origin error but I changed my CORS file on Amazon and it seemed to change to a 400 error now. My CORS file looks like:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
EDIT: Including a sample of the response when trying to access the file after logging in:
Remote Address:1.2.3.4:443
Request URL:https://s3.amazonaws.com/bucket/path/to/file/template.html
Request Method:GET
Status Code:400 Bad Request
Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-GB,en;q=0.8,en-US;q=0.6,fr;q=0.4,es;q=0.2
Authorization:Bearer xxxXXXxxxXXXxxXXxxxXXXXxxXXxx
Connection:keep-alive
Host:s3.amazonaws.com
Origin:http://domain.com
Referer:http://domain.com/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Response Headersview source
Access-Control-Allow-Methods:GET
Access-Control-Allow-Origin:*
Access-Control-Max-Age:3000
Connection:close
Content-Type:application/xml
Date:Sun, 21 Dec 2014 01:49:16 GMT
Server:AmazonS3
Transfer-Encoding:chunked
Vary:Origin, Access-Control-Request-Headers, Access-Control-Request-Method
x-amz-id-2:xxxXXXxxxXXXxx
x-amz-request-id:xxxXXXxxxXXxxxXXXxx
Just for the record. I was also having the same problem (or a very similar one which is: None files would be found after i did the login on my AngularJS app) and after hours digging on the problem i found what was the issue.
On my specific case it was nothing really related to S3 or CORS but in fact the cookies (and I'm still not really sure how it did cause the problem of not finding the files, but it did). As i saw on my application, i've drastically increased the information stored on my user after i did the login and it was exceeding the 4kb limit for the cookies and for some reason it was breaking down my whole website.
I'm checking the possibility to change from cookies to localStorage, but that is another history. the main thing is that after i reduce the number of information stored on the cookie it started working as it was supposed to.
You need to add transformRequest to http post to delete the Authorization header for that specific request.
transformRequest: function (data, headersGetter) {
//Headers change here
var headers = headersGetter();
delete headers['Authorization'];
return data;
},
Got the same problem on uploading to Amazon S3 the same issue so i added this and it worked.

AngularJS - How to connect to Twitter application-only authentication via Oauth2?

I try to receive an accessToken from the Twitter application-only authentication but I keep receiving a 405 (Method Not Allowed) response from the twitter api. Anybody knows how to solve this? I'm desperately stuck..
I am aware of the fact that:
- best practice is doing this from serverside, but I wanted to try this out with angular on client side
- X-Requested-With should be deleted from the header
This is the factory I created:
twitterServices.factory('Connect', function($http){
var factory = {};
var baseUrl = 'https://api.twitter.com/';
var bearerToken = function(){
var consumerKey = encodeURIComponent('****');
var consumerSecret = encodeURIComponent('****');
var tokenCredentials = btoa(consumerKey + ':' + consumerSecret);
return tokenCredentials;
};
factory.fetchAccessToken = function(scope){
var oAuthurl = baseUrl + "oauth2/token";
var headers = {
'Authorization': 'Basic ' + bearerToken(),
'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'
};
$http.defaults.useXDomain = true;
delete $http.defaults.headers.common['X-Requested-With'];
$http({method: 'POST', url: oAuthurl, headers: headers, data: 'grant_type=client_credentials'}).
success(function(data, status){
scope.status = status;
scope.data = data;
}).
error(function(data, status){
scope.status = status;
scope.data = data || "Request failed";
});
};
factory.fetchTimeLine = function(scope){
scope.fetchAccessToken();
//the rest
};
return factory;
});
This is the header request/response in Chrome:
Request URL:`https://api.twitter.com/oauth2/token`
Request Method:OPTIONS
Status Code:405 Method Not Allowed
Request Headersview source
:host:api.twitter.com
:method:OPTIONS
:path:/oauth2/token
:scheme:https
:version:HTTP/1.1
accept:*/*
accept-encoding:gzip,deflate,sdch
accept-language:en-US,en;q=0.8
access-control-request-headers:accept, authorization, content-type
access-control-request-method:POST
origin:`http://localhost`
referer:`http://localhost/test/app/
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.69 Safari/537.36
Response Headersview source
content-length:0
status:405 Method Not Allowed
version:HTTP/1.1
My Console shows the following:
OPTIONS https://api.twitter.com/oauth2/token 405 (Method Not Allowed) angular.js:9312
OPTIONS https://api.twitter.com/oauth2/token Origin http://localhost is not allowed by Access-Control-Allow-Origin. angular.js:9312
XMLHttpRequest cannot load https://api.twitter.com/oauth2/token. Origin http://localhost is not allowed by Access-Control-Allow-Origin. (index):1
Check:
Twitter :Application-only authentication error Origin null is not allowed by Access-Control-Allow-Origin
and:
https://dev.twitter.com/discussions/1291
I ran into a similar issue when working with the google API where in request from localhost are denied even if you register it with the system.
We got around this issue by adding a multipart name to our /etc/hosts file and binding it to 127.0.0.1
for example
127.0.0.1 www.devsite.com
This has resolved the most basic issues that I have had writing angular services for APIs
update by request:
One of the ways that companies control access to their APIs is through whitelisting. When you register an application with the service that platform will typically add the domain you list in your application to its whitelist. This is Generally done to force you in to using separate API keys for separate services. This can make work on the dev side difficult when you are testing locally.
In this case I believe that twitter has specifically banned requests using localhost to prevent the use of 3rd party tools and bots.
Binding the domain you registered with your API key into your hosts file will cause any web requests on your machine to that domain to skip a dns lookup and instead route the request to your local dev server. This means that locally you will test your code by visiting:
www.devsite.com:[what ever port your local server is running on]
This may not be the solution to 100% of api access problems but it is one of the most common that I have experienced.
Note based on other responses:
There are Multiple reasons why you might experience a CORS related error. But just because you have received one doesn't mean that it isn't possible to implement your code on the front end. Generally in Angular CORS is encountered when:
a) you have failed to format your request correctly
-- one example of this might be you have added a header to indicate json is an expected result when infact the response it text.
b) the service or API is configured with a whitelist that needs to include explicitly either "localhost" or some other domain as discussed in this post.

Resources