Amazon S3 - ERR_INSECURE_RESPONSE - angularjs

This question has already been asked here but that answer didn't help me as I know that the URL is correct as it points to a US Standard url as detailed in the amazon guide here
So I'm attempting to upload a file to my bucket via the angular library ng-file-upload. I believe that I've setup the bucket correctly:
The buckets name is ac.testing
The CORS configuration for this:
<?xml version="1.0" encoding="UTF-8"?>
< CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
< CORSRule>
< AllowedOrigin>*</AllowedOrigin>
< AllowedMethod>POST</AllowedMethod>
< AllowedHeader>*</AllowedHeader>
< /CORSRule>
< /CORSConfiguration>
I had to put a space after the opening brackets above as SO didn't let me add it if I didn't.
I've understood this as allow POSTS from any location regardless of heading - is that correct?
And my policy which I generated:
{
"Version": "2012-10-17",
"Id": "Policy<numbers>",
"Statement": [
{
"Sid": "Stmt<numbers>",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::ac.testing/*"
}
]
}
Finally - in my actual code that attempts to upload:
var policy = {
"expiration": "2020-01-01T00:00:00Z",
"conditions": [
{"bucket": "ac.testing"},
["starts-with", "$key", ""],
{"acl": "private"},
["starts-with", "$Content-Type", ""],
["starts-with", "$filename", ""],
["content-length-range", 0, 524288000]
]
}
var uploadDetails = {
url: 'https://ac.testing.s3.amazonaws.com/',
method: 'POST',
data: {
key: file.name,
acl: 'public-read',
// application/octet-stream is the default format used by S3
'Content-Type': file.type != '' ? file.type : 'application/octet-stream',
AWSAccessKeyId: '<my-access-key-id>',
policy: policy,
signature: '<my-access-signature>',
filename: file.name,
},
file: file
};
if (file) {
// Upload to Amazon S3
file.upload = Upload.upload(uploadDetails).then(function(success) {
console.log(JSON.stringify(success) );
},function(error) {
console.log(error);
},function(progress) {
console.log(JSON.stringify(progress) );
});
}
So where am I going wrong here?
EDIT
A couple of things that I wasn't doing was encoding the policy and the signature according to Base64 encoding.
That process involves encoding your policy and then using that generated code, along with your AWS Secret Access Key to generate an SHA-1 HMAC code. Even after this it still returns the same error.
I also added another Permission for the bucket. It matches the policy and CORS settings already defined here but is defined as everyone:
EDIT 2
I created a new bucket called ac-k24-uploads and updated the url to match this:
uploadDetails = {
url: 'https://ac-k24-uploads.s3.amazonaws.com/',
Now the error coming back is this:
EDIT 3
Looking around it seems that headers that amazon doesn't like are being added to the request. I tried it on ng-file-uploads and it works but on my own machine it doesn't. The fix for this seems to be removing these headers like so:
method: 'POST',
headers: {
'Authorization': undefined
},
but I still get the same error. Why are these headers being attached to this request?
EDIT 4
Even with the code above the headers were still getting added at a different place so I located it and remove them. The error which I recieve now is this:

This is not related to CORS.
It is because of the dot in the name of your bucket.
If your bucket is indeed in US-Standard (example-bucket.s3.amazonaws.com is not necessarily a "US Standard" URL) then this is what you need:
var uploadDetails = {
url: 'https://s3.amazonaws.com/ac.testing/',
...
This is referred to as the path-based or path-style URL where the bucket is expressed as the first element of the path, rather than as including it in the hostname.
This is necessary if the bucket name has a dot, and s3. needs to be s3-region. for buckets not in US Standard.
This is a limitation of wildcard SSL/TLS certificates. S3 presents a certificate valid for *.s3[-region].amazonaws.com and the rules for such certificates state that * cannot match a dot... therefore the certificate is considered invalid and therefore it's an "insecure response."
Or, just create a bucket with a dash instead of a dot, and your code should work as-is.
Since you are using Signature V2, either hostname construct results in an identical signature for an otherwise-identical request. Not true for Sig V4.

The actual problem was two fold:
The headers were incorrect as the "Authorization" header was still present even after deleting it from the Upload.upload portion of the code.
To fix this I had to delete the object from my api.injector code - this was where it was being added.
delete config.headers['Authorization']
Following on from EDIT3 the signature that I generated was wrong - so I had to regenerate it.
Thanks to Michael for the help.

Related

Graph API: Using $filter with schema extensions data not working when request sent as part of a batch operation

According to the Graph API documentation, making a GET request to get groups with extension data that includes a filtered response is acceptable. For example, according to the doc referenced the following request should be valid:
GET https://graph.microsoft.com/v1.0/users/${id}/memberOf?$filter=graphlearn_courses/courseId eq ‘123’&$select=displayName,id,description,graphlearn_courses
This works when making the request as a singleton but fails and returns no response when the same request is made as part of a batch request:
POST https://graph.microsoft.com/v1.0/$batch
Accept: application/json
Content-Type: application/json
{
"requests": [
{
"id": "1",
"method": "GET",
"url": "/users/${id}/memberOf?$filter=graphlearn_courses/courseId eq ‘123’&$select=displayName,id,description,graphlearn_courses"
}
...
]
}
Can this be looked into and the issue resolved by someone at MS support please? Thank you in advance.
Schema extensions (legacy) are not returned with $select statement, but are returned without $select. So i would recommend you to try that and see if it helps. Documentation available # Microsoft Graph API limitations.

Deep insert in Odata v4 in Business Central AL Extension

We are creating an Extension in AL to import orders.
For this question we have simplified our extension to explain the challenges we are facing.
What we would like to do is import header + lines in Dynamics 365 Business Central.
To achive this we we have:
- Created a table (Header)
- Created a table (Lines)
- Created a page of Type API (Doc)
- Created a listPart (SalesLine)
Scenario 1
We have published page DOC and are trying to do a post request to this odata url.
POST https://api.businesscentral.dynamics.com/v1.0/{tennant}/Sandbox/ODataV4/Company('CRONUS%20NL')/Doc/ HTTP/1.1
Content-Type: application/json
Authorization: Basic {{username}} {{password}}
{
"name": "Description",
"SalesLines" : [{"lineno" : 1000}]
}
The response:
{
"error": {
"code": "BadRequest",
"message": "Does not support untyped value in non-open type."
}
}
Scenario 2
When we post:
POST https://api.businesscentral.dynamics.com/v1.0/3{tennant}/Sandbox/ODataV4/Company('CRONUS%20NL')/Doc/ HTTP/1.1
Content-Type: application/json
Authorization: Basic {{username}} {{password}}
{
"name": "Description"
}
We get the following response:
{
"#odata.context": "https://api.businesscentral.dynamics.com/v1.0/3ddcca3d-d343-4a06-95f9-f32dbf645199/Sandbox/ODataV4/$metadata#Company('CRONUS%20NL')/Doc/$entity",
"#odata.etag": "W/\"JzQ0O3BKUzExSUMrQUl4UXFQc2R6V1J1ellvZEttRTJoa2xhanNtV0M0K3Ezajg9MTswMDsn\"",
"id": 4,
"name": "Description",
"DateTime": "2019-05-20T19:33:13.73Z"
}
I have published our extension on GitHub
Help would be appriciated.
I have worked on a similar solution and found that the following things were required for it to work:
The part containing your lines must be placed inside the repeater on your header Page.
You must set the EntityName and EntitySetName on your part to the same values as on the actual page.
When calling the API you must append the parameter $expand=[EntitySetName of your lines part] e.g $expand=orderLines.
In the JSON body the property name of the array containing the lines must match the EntitySetName of the lines part.
I can provide some examples if the instructions above do not suffice.

Configuring Auth0 authentication on Google App Engine Standard and the Cloud Endpoints Frameworks with Python

I am using Cloud Endpoints Frameworks with Python in a Google Cloud App Engine Standard environment to provide an API.
As far as I can tell, I should be able to use python decorators from the Endpoints Frameworks in combination with the endpointscfg.py command-line tool to automatically set up token-based authentication with Auth0; the endpointscfg.py command-line automatically creates the openapi.json file that is used to configure the Google Endpoints proxy.
Here's an example of my decorator for an API that echos stuff back:
# # [START echo_api]
#endpoints.api(
name='echo',
version=_VERSION,
api_key_required=True,
audiences={'auth0': ['https://echo.<my-project>.appspot.com/_ah/api/echo/v1/echo']},
issuers={'auth0': endpoints.Issuer(
'https://<my-project>.auth0.com',
'https://<my-project>.auth0.com/.well-known/jwks.json')}
)
class EchoApi(remote.Service):
...
When I run the endpointscfg.py command-line tool, I get something in my openapi.json file that looks about right:
"paths": {
"/echo/v1/echo": {
"post": {
"operationId": "EchoApi_echo",
"parameters": [
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/MainEchoRequest"
}
}
],
"responses": {
"200": {
"description": "A successful response",
"schema": {
"$ref": "#/definitions/MainEchoResponse"
}
}
},
"security": [
{
"api_key": [],
"auth0_jwt": []
}
]
}
}
"securityDefinitions": {
"api_key": {
"in": "query",
"name": "key",
"type": "apiKey"
},
"auth0_jwt": {
"authorizationUrl": "https://<my-project>.auth0.com/authorize",
"flow": "implicit",
"type": "oauth2",
"x-google-issuer": "https://<my-project>.auth0.com",
"x-google-jwks_uri": "https://<my-project>.auth0.com/.well-known/jwks.json",
"x-google-audiences": "https://echo.<my-project>.appspot.com/_ah/api/echo/v1/echo"
}
}
So, the problem is that this set-up appears to do nothing and does not check incoming tokens to prevent access if no token is present or if the token is invalid.
I have been able to set-up manual processing of the bearer token within the API echo function using the python-jose library (sorry if it's not well done, but I'm just testing and comments are welcome):
authorization_header = self.request_state.headers.get('authorization')
if authorization_header is not None:
if authorization_header.startswith('Bearer '):
access_token = authorization_header[7:]
logging.info(access_token)
else:
logging.error("Authorization header did not start with 'Bearer '!")
raise endpoints.UnauthorizedException(
"Authentication failed (improperly formatted authorization header).")
else:
logging.error("Authorization header did not start with 'Bearer '!")
raise endpoints.UnauthorizedException("Authentication failed (bearer token not found).")
r = urlfetch.fetch(_JWKS_URL)
jwks_content = json.loads(r.content)
keys = jwks_content['keys']
public_key = jwk.construct(keys[0])
logging.info(public_key)
message, encoded_signature = str(access_token).rsplit('.', 1)
# decode the signature
decoded_signature = base64url_decode(encoded_signature.encode('utf-8'))
# verify the signature
if not public_key.verify(message.encode("utf8"), decoded_signature):
logging.warning('Signature verification failed')
raise endpoints.UnauthorizedException("Authentication failed (invalid signature).")
else:
logging.info('Signature successfully verified')
claims = jwt.get_unverified_claims(access_token)
# additionally we can verify the token expiration
if time.time() > claims['exp']:
logging.warning('Token is expired')
raise endpoints.UnauthorizedException("Authentication failed (token expired).")
# and the Audience (use claims['client_id'] if verifying an access token)
if claims['aud'] != _APP_CLIENT_ID:
logging.warning('Token was not issued for this audience')
raise endpoints.UnauthorizedException("Authentication failed (incorrect audience).")
# now we can use the claims
logging.info(claims)
This code works, but I assumed that the whole point of setting up the decorator and configuring the openapi.json file was to off-load these checks to the proxy so that only valid tokens hit my code.
What am I doing wrong?
UPDATE:
It may be that I need to check endpoints.get_current_user() in my code to control access. However, I have just noticed the following in my logs:
Cannot decode and verify the auth token. The backend will not be able to retrieve user info (/base/data/home/apps/e~<my-project>/echo:alpha23.414400469228485401/lib/endpoints_management/control/wsgi.py:643)
Traceback (most recent call last):
File "/base/data/home/apps/e~<my-project>/echo:alpha23.414400469228485401/lib/endpoints_management/control/wsgi.py", line 640, in __call__
service_name)
File "/base/data/home/apps/e~<my-project>/echo:alpha23.414400469228485401/lib/endpoints_management/auth/tokens.py", line 75, in authenticate
error)
UnauthenticatedException: (u'Cannot decode the auth token', UnauthenticatedException(u'Cannot find the `jwks_uri` for issuer https://<my-project>.auth0.com/: either the issuer is unknown or the OpenID discovery failed',))
However, I think everything is configured ok. Any idea why 'jwks_uri' cannot be found despite the fact that path in the openapi.json file is correct?
I'm the current maintainer of these Frameworks. You do need to check endpoints.get_current_user() to control access, yes. I'm working on a feature to make this much simpler.
As for that UnauthenticatedException, you can ignore it. That's coming from the 'management framework', which attempts to check auth tokens even though it's not involved in the Frameworks' oauth security (only the api key security).

Create datasource in azure search returns 403 Forbidden

I have a search service running on azure in a free tier. On this service I already have a datasource, and indexer and an index defined.
I'd like to add another datasource (and index + indexer). When I do this (using postman) I get 403 Forbidden without any other error message.
This is the POST I made to this url - https://my-search-service-name.search.windows.net/datasources?api-version=2016-09-01:
"Content-Type": "application/json",
"api-key": "API-KEY-HERE"
{
"name": "datasource-prod",
"description": "Data source for search",
"type": "azuresql",
"credentials": { "connectionString" : "Server=tcp:xxxx.database.windows.net,1433;Initial Catalog=xxxxx_prod;Persist Security Info=False;User ID=xxxxxx;Password=xxxxxx!;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;" },
"container": {"name": "DataAggregatedView"},
"dataChangeDetectionPolicy": {
"#odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
"highWaterMarkColumnName" : "ChangeIndicator"
},
"dataDeletionDetectionPolicy": {
"#odata.type" : "#Microsoft.Azure.Search.SoftDeleteColumnDeletionDetectionPolicy",
"softDeleteColumnName" : "isDeleted",
"softDeleteMarkerValue" : "0"
}
}
Using the same request, with different name and database name worked perfectly and generated the existing (first) datasource. This error (403) - not even got the error message - happens only when I try to define a second datasource.
As I can understand from documentation, free search tier allows 3 datasources. Anyone had this issue? Any help/direction is appreciate!
Thank you.
Make sure you're using the admin API key. It looks like you may be using a query key.

Google drive api invalid client

I am trying to get access to the google drive content of my users.
I can redeem a code on my domain using the google drive user consent url with correct parameters:
https://accounts.google.com/o/oauth2/v2/auth?response_type=code&client_id='+$scope.googledriveApi.client_id+'&scope='+$scope.googledriveApi.scopes+'&redirect_uri='+$scope.googledriveApi.redirect_uri
I am trying to do an angular $http request to the https://www.googleapis.com/oauth2/v4/token endpoint.
The request looks like this:
$http({
method: 'POST',
headers: {"Content-Type" : "application/x-www-form-urlencoded"},
data: $.param({
"code" : $routeParams.code,
"client_id" : $scope.googledriveApi.client_id,
"client_secret" : $scope.googledriveApi.secret,
"redirect_uri" : $scope.googledriveApi.redirect_uri,
"grant_type" : "authorization_code"
}),
url: 'https://www.googleapis.com/oauth2/v4/token'
})
However the response I get from the request is as follows:
{
"error": "invalid_client",
"error_description": "The OAuth client was not found."
}
Does anyone know why this happens? I have tried changing product name and client id name to be the same. I have checked this for spaces. The reason I'm mentioning this is because this seemed to be the case for other people who asked a question for the same error, however my error happens at the $http request.
I am getting back the user consent code and I am trying to exchange for an access token in this request. This is when the error comes in and I am stuck.
Try these:
Under OAuth consent screen, make sure Product name is not the same with project name as stated in this SO thread:
Try to include an authorization header in your URI request:
headers: { 'Authorization': 'bearer ' + accessToken }

Resources