InvalidSignatureException from POST request - reactjs

I have a Lambda function that handles reading data from a file(stored inside S3 bucket) as well as inserting data to a Dynamodb table. This Lambda function is exposed as a REST endpoint using API gateway. The function accepts GET request as well as POST request. I'm making GET/POST requests from my REACT project using axios and aws4(for signing) libraries. GET request is to read data from a file stored inside S3 and it works just fine. And POST request is for inserting data into Dynamodb table. However, it doesn't work and AWS returns InvalidSignatureException error as a respond. This is an excerpt of my code :
createAWSSignedRequest(postData) {
let request = {};
if (postData) {
request = {
host: process.env.AWS_HOST,
method: 'POST',
url: process.env.AWS_URL,
path: process.env.AWS_PATH,
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(postData)
}
} else {
request = {
host: process.env.AWS_HOST,
method: 'GET',
url: process.env.AWS_URL,
path: process.env.AWS_PATH
}
}
let signedRequest = aws4.sign(request, {
secretAccessKey: process.env.AWS_SECRET_KEY,
accessKeyId: process.env.AWS_ACCESS_KEY
});
return signedRequest;
}
This is how GET request is made :
let signedRequest = this.createAWSSignedRequest('GET');
axios(signedRequest)
.then(response => {
})
.catch((error) => {
console.log("error",error);
});
This is how POST request is made :
const data = {
uuid: "916b7d90-0137-11e8-94e6-116965754e23", //just a mock value
date : "22/jan/2018",
user_response: [
{
question:"this is quesiton1",
choice:"user selected A"
},
{
question:"this is quesiton2",
choice: "user selected b"
},
{
question:"this is quesiton3",
choice: "user selected C"
}
]
};
let signedRequest = this.createAWSSignedRequest(data);
axios(signedRequest)
.then(response => {
......
})
.catch((error) => {
console.log("error",error);
});
As you can see, the code for both GET and POST requests are exactly the same (except payload and method type). I'm singing with the same secret access key and access key id for both requests. I'm not sure why one request results in "InvalidSignatureException" when the other doesn't. Can anyone shed a light on this issue for me.
Thanks

After having discussion with AWS4 lib developer, I figured out what I did wrong. AWS4 uses "body" as a payload attribute to compute signature. However, Axios uses "data" attribute as payload. My mistake was only setting either one of them. So when I set just "data" attribute, the payload was present in the request and content-length is computed correctly. However, the signature was incorrect since the payload was not taken into consideration when computing signature. When I set just "body", payload was not present in the request because Axios does not use "body" attribute for payload. The solution is to set both attributes with payload. I hope this helps to anyone who are having the same issue I have.

If you use the AWS Amplify library it has a module called API which should fit your use cases, and it will perform Sigv4 signing for you either with authenticated or unauthenticated roles. The Auth category uses Cognito as the default implementation. For instance:
npm install aws-amplify --save
Then import and configure the lib:
import Amplify, { API } from 'aws-amplify';
Amplify.configure({
Auth: {
identityPoolId: 'XX-XXXX-X:XXXXXXXX-XXXX-1234-abcd-1234567890ab',
region: 'XX-XXXX-X'
},
API: {
endpoints: [
{
name: "APIName",
endpoint: "https://invokeURI.amazonaws.com"
}
]
}
});
Then for your API Gateway endpoint calling a Lambda:
let apiName = 'MyApiName';
let path = '/path';
let options = {
headers: {...} // OPTIONAL
}
API.get(apiName, path, options).then(response => {
// Add your code here
});
More info here: https://github.com/aws/aws-amplify

Related

Unable to access lambda function from s3 hosted react application

I have a react application which has a post method requesting data from a lambda function. It is working fine in my local machine. But when I deployed the application on s3 bucket and try to access it through s3 endpoint, it gives me 405 Method not allowed error. After search I found that post method will not work with s3 endpoint, So again I tried with rest api endpoint, but it shows file name xml. I am able to access application when I add index page to rest api endpoint, But here also I am getting 412 error in post method. So in any case I am getting error with post method. Can someone help me with that. I don't know what I am missing. Below is my code:
const apiUrl = process.env.NODE_ENV === 'production' ? process.env.REACT_APP_PROD_API_URL : process.env.REACT_APP_DEV_API_URL;
const params = (query: string) => ({
headers: {
"Accept": "application/sparql-results+json",
"Accept-Encoding": "gzip",
"Content-Type": "application/x-www-form-urlencoded"
},
method: "POST",
body: `query=${encodeURIComponent(query)}`
})
const fetchClinicalStudies = async () => {
const clinicalStudiesQuery = `
PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX disco: <http://rdf-vocabulary.ddialliance.org/discovery#>
SELECT ?id ?label
WHERE {
?id a disco:Study ;
dcterms:title ?label .
}
ORDER BY ?label`
const response = await fetch(apiUrl ?? '', params(clinicalStudiesQuery))
return response;
}

403 when upload file to S3 bucket using axios

I'm using axios to upload an audio file to AWS s3 bucket.
The workflow is: React => AWS API Gateway => Lambda.
Here is the backend Lambda code where generates the S3 presigned URL:
PutObjectRequest putObjectRequest = PutObjectRequest.builder()
.bucket(AUDIO_S3_BUCKET)
.key(objectKey)
.contentType("audio/mpeg")
.build();
PutObjectPresignRequest putObjectPresignRequest = PutObjectPresignRequest.builder()
.signatureDuration(Duration.ofMinutes(10))
.putObjectRequest(putObjectRequest)
.build();
PresignedPutObjectRequest presignedPutObjectRequest = s3Presigner.presignPutObject(putObjectPresignRequest);
AwsProxyResponse awsProxyResponse = new AwsProxyResponse();
awsProxyResponse.setStatusCode(HttpStatus.SC_OK);
awsProxyResponse.setBody(
GetS3PresignedUrlResponse.builder()
.s3PresignedUrl(presignedPutObjectRequest.url().toString())
.build().toString());
return awsProxyResponse;
Here is the java code to create the bucket:
private void setBucketCorsSettings(#NonNull final String bucketName) {
s3Client.putBucketCors(PutBucketCorsRequest.builder()
.bucket(bucketName)
.corsConfiguration(CORSConfiguration.builder()
.corsRules(CORSRule.builder()
.allowedHeaders("*")
.allowedMethods("GET", "PUT", "POST")
.allowedOrigins("*") // TODO: Replace with domain name
.exposeHeaders("ETag")
.maxAgeSeconds(3600)
.build())
.build())
.build());
log.info("Set bucket CORS settings successfully for bucketName={}.", bucketName);
}
In my frontend, here is the part that try to upload file:
const uploadFile = (s3PresignedUrl: string, file: File) => {
let formData = new FormData();
formData.append("file", file);
formData.append('Content-Type', file.type);
const config = {
headers: {
"Content-Type": 'multipart/form-data; boundary=---daba-boundary---'
//"Content-Type": file.type,
},
onUploadProgress: (progressEvent: { loaded: any; total: any; }) => {
const { loaded, total } = progressEvent;
let percent = Math.floor((loaded * 100) / total);
if (percent < 100) {
setUploadPercentage(percent);
}
},
cancelToken: new axios.CancelToken(
cancel => (cancelFileUpload.current = cancel)
)
};
axios(
{
method: 'post',
url: s3PresignedUrl,
data: formData,
headers: {
"Content-Type": 'multipart/form-data; boundary=---daba-boundary---'
}
}
)
.then(res => {
console.log(res);
setUploadPercentage(100);
setTimeout(() => {
setUploadPercentage(0);
}, 1000);
})
.catch(err => {
console.log(err);
if (axios.isCancel(err)) {
alert(err.message);
}
setUploadPercentage(0);
});
};
However, when try to upload the file, it return 403 error.
And if I use fetch instead of axios instead and it works, like this:
export async function putToS3(presignedUrl: string, fileObject: any) {
const requestOptions = {
method: "PUT",
headers: {
"Content-Type": fileObject.type,
},
body: fileObject,
};
//console.log(presignedUrl);
const response = await fetch(presignedUrl, requestOptions);
//console.log(response);
return await response;
}
putToS3(getPresignedUrlResponse['s3PresignedUrl'], values.selectdFile).then(
(putToS3Response) => {
console.log(putToS3Response);
Toast("Success!!", "File has been uploaded.", "success");
}
);
It seems to me that the only difference between these two is that: when using fetch the request's Content-Type header is Content-Type: audio/mpeg, but when using axios it is Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryClLJS3r5Xetv3rN7 .
How can I make it work with axios? I'm switching to axios for its ability to monitor request progress as I want to show an upload progress bar.
I followed this blog and not sure what I missed: https://bobbyhadz.com/blog/aws-s3-presigned-url-react
You are using POST in your axios. Should be PUT instead.
Also I think the content type has to match the one specified during requesting the pre-signed URL, which is audio/mpeg as you rightly pointed out.
Correspondingly, your data should be just file, instead of formData.
axios(
{
method: 'put',
url: s3PresignedUrl,
data: file,
headers: {
"Content-Type": 'audio/mpeg'
}
}
...
You didn't mark any answers as accepted so I guess you didn't solve it.
For any future viewers out there. The reason why you are getting 403 forbidden error is because your Content-Type in your server and client side are not matching. I'm assuming you set up the AWS policies correctly.
Your code in the backend should look like this:
const presignedPUTURL = s3.getSignedUrl("putObject", {
Bucket: "bucket-name",
Key: String(Date.now()),
Expires: 100,
ContentType: "image/png", // important
});
and in the front-end (assuming you are using axios):
const file = e.target.files[0]
const result = await axios.put(url, file, {
withCredentials: true,
headers: { "Content-Type": "image/png" },
});
In practical, you would normally have to send the file type to generate the pre-signed url in the POST body or whatever and then in axios you do file.type to get the file type of the uploaded file.
Check your Lambda execution role. It may be the culprit. Perhaps it does not grant enough permissions to allow PUTting files into your bucket.
URL signing is a delegation of power on behalf of the signer, which is restricted to a specified object, action... Signing does not magically grants full read/write permissions on S3, even on the specific object related to the presigned URL.
The "user" who generates the signature requires sufficient permissions to allow the actions you want to delegate through that presigned URL. In this case, this is the execution role of your Lambda function.
You can add the AmazonS3FullAccess managed policy to the execution role and see if it solves your situation. This change took me out of a blocked situation me after days of struggle. Afterwards, before going to production, restrict that rule to the specific bucket you want to allow uploads into (least privilege principle).
If you develop using SAM local emulation, those execution roles seem not to be taken into account as long as you run your functions locally; the signed links work in that context even without S3 permissions.

Why I am getting the fetch is not defined error?

I have tried to create a custom email receipt template with SendGrid using commercejs webhooks, by following this tutorial. I have uploaded this github repository to
this test Netlify site. The main code is /functions/email.js I am pretty sure my .env variables are correct, still no receipt emails are received and probably send. When checking the Netlify function email log says:
5:55:03 PM: f7c003e5 ERROR Invoke Error {"errorType":"ReferenceError","errorMessage":"fetch is not defined","stack":["ReferenceError: fetch is not defined"," at Runtime.exports.handler (/var/task/email.js:30:22)"," at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"]}
5:55:03 PM: f7c003e5 Duration: 2.24 ms Memory Usage: 52 MB
However my package.json dependencies look like this:
"dependencies": {
"#chec/commerce.js": "2.2.0",
"#hookform/error-message": "0.0.5",
"#sendgrid/mail": "^7.4.7",
"#stripe/react-stripe-js": "1.1.2",
"#stripe/stripe-js": "1.11.0",
"autoprefixer": "10.0.4",
"classcat": "4.1.0",
"framer-motion": "2.9.4",
"next": "10.0.2",
"next-google-fonts": "1.2.1",
"node-fetch": "^3.0.0",
"pdf-lib": "^1.16.0",
"postcss": "8.1.14",
"react": "17.0.1",
"react-dom": "17.0.1",
"react-hook-form": "6.11.5",
"react-toastify": "6.1.0",
"use-debounce": "^7.0.0"
},
Therefore it's confusing why I am getting the fetch is not defined error. Also, I am also confused about how to implement the headers correctly, because the tutorial did not specify how. So I just added the headers like this, without knowing if this is the way to do it:
let headers = {
"Accept": "application/json",
"Content-Type": "application/json",
};
let sgheaders = {
Authorization: `Bearer ${process.env.SENDGRID_SECRET}`,
"Content-Type": "application/json",
};
In the code currently uploaded to my netlify site, I had to change the export default async function handler(req, res) {
to
exports.handler = async function(req, res) { according to the Netlify functions documentation the Netlify functions documentation. (because of "errorMessage": "SyntaxError: Unexpected token 'export'")
// Create the API endpoint function with a req and res parameter
exports.handler = async function(req, res) {
const checSecretAPIKey = process.env.CHEC_SECRET_KEY;
let headers = {
"Accept": "application/json",
"Content-Type": "application/json",
};
//export default async function handler(req, res) {
if (!req.body || req.httpMethod !== 'POST') {
return {
status: 405,
headers,
body: JSON.stringify({
status: 'Invalid HTTP method',
}),
}
}
const { data } = JSON.parse(req.body);
// Request for your merchant information so that you can use your email
// to include as the 'from' property to send to the SendGrid API
const merchant = fetch(`${process.env.CHEC_API_URL}/v1/merchants`, {
headers: {
'X-Authoriza†ion': process.env.CHEC_SECRET_KEY,
},
}).then((response) => response.json);
// Extract the signature from the registered `orders.create` webhook
const { signature } = data;
delete data.signature;
// Your Chec webhook signing key, from the Chec Dashboard webhooks view
const webHookSigningKey = 'KEJlxz6cIlrWIpsX5jypcMeGl2uh7jJg';
// Verify the signature
const expectedSignature = crypto.createHmac('sha256', webHookSigningKey)
.update(JSON.stringify(data))
.digest('hex');
if (expectedSignature !== signature) {
console.error('Signature mismatched, skipping.')
}
// Verify the age of the request to make sure it isn't more than 5 minutes old.
if (new Date(data.created * 1000) < new Date() - 5 * 60 * 1000) {
console.error('Webhook was sent too long ago, could potentially be fake, ignoring');
}
// Because you will need to list out the order line items, map through the returned line items
// and structure out the data you need to display in the email receipt for your customer
// Note that we are keeping the data structure minimal here
const orderLineItems = data.payload.order.line_items.map((lineItem) => ({
text: lineItem.product_name,
price: lineItem.line_total.formatted_with_symbol,
}));
// Signature is verified, continue to send data to SendGrid
// Create the email object payload to fire off to SendGrid
const emailPayload = {
to: data.payload.customer.email,
from: merchant.support_email,
subject: `Thank you for your order ${data.payload.customer.firstname}`,
text: `Your order number is ${data.payload.customer_reference}`,
// SendGrid expects a JSON blog containing the dynamic order data your template will use
// More information below in 'What's next?' on how to configure your dynamic template in SendGrid
// The property key names will depend on what dynamic template you create in SendGrid
dynamic_template_data: {
total: data.payload.order.subtotal.formatted_with_symbol,
items: orderLineItems,
receipt: true,
name: data.payload.shipping.name,
address01: data.payload.shipping.street,
city: data.payload.shipping.town_city,
state: data.payload.shipping.county_state,
zip : data.payload.shipping.postal_zip_code,
orderId : data.payload.id,
},
// In addition to specifying the dynamic template data, you need to specify the template ID. This comes from your SendGrid dashboard when you create you dynamic template
// https://mc.sendgrid.com/dynamic-templates
template_id: 'd-xxx'
}
let sgheaders = {
Authorization: `Bearer ${process.env.SENDGRID_SECRET}`,
"Content-Type": "application/json",
};
let response = {};
try {
// Call the SendGrid send mail endpoint
response = await sgMailClient.send(emailPayload);
return {
statusCode: 200,
headers: sgheaders,
body: 'Email sent!'
}
} catch (err) {
console.error('Error', err)
}
// Return the response from the request
return res.status(200).json(response);
}
Need some advice on how to get this code to actually work, the tutorial seems to be uncompleted or I might have misunderstood some curtail details.
UPDATE (Working code below)
Had to use axios instead of node.fetch (thanks #hotcakedev) wehn deplodey on netlify. Also other changes to the code in order to make it work with commerce.js (see working code for detalis)
const axios = require('axios');
const sgMailClient = require("#sendgrid/mail");
sgMailClient.setApiKey(process.env.SENDGRID_API_KEY);
// Includes crypto module
const crypto = require('crypto');
// Create the API endpoint function with a req and res parameter
exports.handler = async function(req, res) {
//export default async function handler(req, res) {
if (!req.body || req.httpMethod !== 'POST') {
return {
status: 405,
headers: {},
body: JSON.stringify({
status: 'Invalid HTTP method',
}),
}
}
const data = JSON.parse(req.body);
// Request for your merchant information so that you can use your email
// to include as the 'from' property to send to the SendGrid API
const merchant = axios.get(`${process.env.CHEC_API_URL}/v1/merchants`, {
headers: {
'X-Authorization': process.env.CHEC_SECRET_KEY,
"Accept": "application/json",
"Content-Type": "application/json",
},
}).then((response) => response.json);
//console.log(data);
// Extract the signature from the registered `orders.create` webhook
const { signature } = data;
delete data.signature;
// Your Chec webhook signing key, from the Chec Dashboard webhooks view
const webHookSigningKey = 'KEJlxz6cIlrWIpsX5jypcMeGl2uh7jJg';
// Verify the signature
const expectedSignature = crypto.createHmac('sha256', webHookSigningKey)
.update(JSON.stringify(data))
.digest('hex');
if (expectedSignature !== signature) {
console.error('Signature mismatched, skipping.')
}
// Verify the age of the request to make sure it isn't more than 5 minutes old.
if (new Date(data.created * 1000) < new Date() - 5 * 60 * 1000) {
console.error('Webhook was sent too long ago, could potentially be fake, ignoring');
}
// Because you will need to list out the order line items, map through the returned line items
// and structure out the data you need to display in the email receipt for your customer
// Note that we are keeping the data structure minimal here
const orderLineItems = data.payload.order.line_items.map((lineItem) => ({
text: lineItem.product_name,
price: lineItem.line_total.formatted_with_symbol,
}));
// Signature is verified, continue to send data to SendGrid
// Create the email object payload to fire off to SendGrid
const emailPayload = {
to: data.payload.customer.email,
from: data.payload.merchant.support_email,
subject: `Thank you for your order ${data.payload.customer.firstname}`,
text: `Your order number is ${data.payload.customer_reference}`,
// SendGrid expects a JSON blog containing the dynamic order data your template will use
// More information below in 'What's next?' on how to configure your dynamic template in SendGrid
// The property key names will depend on what dynamic template you create in SendGrid
dynamic_template_data: {
total: data.payload.order.subtotal.formatted_with_symbol,
items: orderLineItems,
receipt: true,
name: data.payload.billing.name,
address01: data.payload.billing.street,
city: data.payload.billing.town_city,
state: data.payload.billing.county_state,
zip : data.payload.billing.postal_zip_code,
orderId : data.payload.id,
},
// In addition to specifying the dynamic template data, you need to specify the template ID. This comes from your SendGrid dashboard when you create you dynamic template
// https://mc.sendgrid.com/dynamic-templates
template_id: 'd-xxx'
};
/*let sgheaders = {
Authorization: `Bearer ${process.env.SENDGRID_SECRET}`,
"Content-Type": "application/json",
};*/
let response = {};
try {
// Call the SendGrid send mail endpoint
response = await sgMailClient.send(emailPayload);
return {
statusCode: 200,
headers: {},
body: JSON.stringify({
status: 'Email sent!',
}),
}
} catch (err) {
console.error('Error from function: ', err)
console.error(err.response.body);
console.log("Payload content: ", emailPayload );
}
// Return the response from the request
//return res.status(200).json(response);
}
If you want to integrate sendgrid using Rest API, I suggest you to use axios.
So in your case,
import axios from 'axios';
...
const merchant = axios.get(`${process.env.CHEC_API_URL}/v1/merchants`, {
headers: {
'X-Authoriza†ion': process.env.CHEC_SECRET_KEY,
},
}).then((response) => response.json);
...
Twilio SendGrid developer evangelist here.
You have installed node-fetch to the project, but the tutorial did not include requiring the library into the function to use it. So you need to require node-fetch.
The tutorial also fails to require the SendGrid library and set the API key. You should set your SendGrid API key in the environment in Netlify, called something like SENDGRID_API_KEY. Then add the following to the top of your function:
const fetch = require("node-fetch");
const sgMailClient = require("#sendgrid/mail");
sgMailClient.setApiKey(process.env.SENDGRID_API_KEY);
As for the headers that you are asking about, they are the response headers that your function returns in response to the incoming HTTP request. What you return depends on how you are calling this function, but you should not return your SendGrid API Key in the headers.
While you are getting the function working, I would recommend setting the return headers to an empty object until you work out what Content-Type you want to set (the response body is "Email sent!" right now, which would be text/plain but that's not super useful in the front end) and what other headers you may or may not need.

How to post data when you have javascript object using multipart/form-data content-type

So I have never post a data using FormData and multipart/form-data as Content-Type in my React project. But now I'm kinda forced by backend to send it this way since we have some images in payload.
The problem is that the whole data is a JS object and can be parsed to JSON, and nothing more. So how can I convert my JS object into a valid FormData that backend would accept it? Everything works without a problem in Postman, but in the app I always get the same error.
Here is some more detail:
The working Postman sample:
What I expect to be working (but obviously doesn't):
const createProd = () =>
HttpRequest.post('/api/prod', {
body: {
Product: {
title: 'Test Prod',
shop: null,
description: "My new ad's description",
category: { id: '5d8c6aa6fadeaf26b0194667' },
condition: 'USED'
}
});
HttpRequest is a helper function which uses ES6 fetch for requests.
I always get the same error: "Required request part 'Product' is not present" with/without JSON.stringify.
I even tried to create a sample FormData to at least change the error:
cont payload = new FormData();
payload.append('Product', {foo: 'bar'});
But still same error. I also copied the code which is generated by Postman. But still no chance.
I would be thankful if you share your suggestions or workarounds.
Thanks in advance.
const formData = new FormData();
const product = { //your product object };
formData.append('product', JSON.stringify(product));
const config = {
headers: {
'Content-Type': 'multipart/form-data; charset=utf-8; boundary="another cool boundary";'
}
};
axios.post(`/api/ads`, formData, config).then((res) => {
console.log(res);
}).catch(err => {
console.log(err);
});
Maybe you should set header.
Try this one. In my case I used Axios. It worked for me in my project.

How to add additional request body values to Apollo GraphQL request?

I'm using Apollo with the Scaphold.io service and for writing blobs to that service I need to be able to add additional options to the request body.
The full Scaphold example can be found here: https://scaphold.io/docs/#uploading-files
But it does something like this:
form.append("variables", JSON.stringify({
"input": {
"name": "Mark Zuck Profile Picture",
"userId": "VXNlcjoxMA==",
"blobFieldName": "myBlobField"
}
}));
// The file's key matches the value of the field `blobFieldName` in the variables
form.append("myBlobField", fs.createReadStream('./mark-zuckerberg.jpg'));
fetch("https://us-west-2.api.scaphold.io/graphql/scaphold-graphql", {
method: 'POST',
body: form
}).then(function(res) {
return res.text();
}).then(function(body) {
console.log(body);
});
Where it adds a blobField to the request body.
Based on this, I need to pass into the variables a blobFieldName property, and then add to the request body the value passed to that property with the blob. However, using Apollo I can't add to the request body. I tried the following, but it's absent from the request body when I check in the network inspector:
export const withCreateCoverPhoto = graphql(CreateCoverPhoto, {
props: ({mutate}) => ({
createCoverPhoto: (name, file) => mutate({
variables: {
input: {
blobFieldName: name,
},
},
[name]: file
}),
}),
});
Please advise.
Thanks for the question. Currently the best way to upload files is by [appending it to FormData and sending it up to the server using fetch] (https://medium.com/#danielbuechele/file-uploads-with-graphql-and-apollo-5502bbf3941e). Basically, you won't be able to have caching here for the file itself, but this is the best way to handle uploading files on Scaphold using a multipart request.

Resources