I have tried implementing the superagent way of uploading multiple files in axios. But somehow, I'm getting an error in console
Failed to load https://api.cloudinary.com/v1_1/xxxx/image/upload:
Request header field Authorization is not allowed by
Access-Control-Allow-Headers in preflight response.
My upload handler looks like this
uploadFile(){
const uploaders = this.state.filesToBeSent.map(file => {
const formData = new FormData();
formData.append("file", file);
formData.append("upload_preset", "xxxxx");
formData.append("api_key", "xxxxx");
formData.append("timestamp", (Date.now() / 1000) | 0);
return axios.post(url, formData, {
headers: { "X-Requested-With": "XMLHttpRequest" },
}).then(response => {
const data = response.data;
const fileURL = data.secure_url
console.log(data);
})
});
// Once all the files are uploaded
axios.all(uploaders).then(() => {
// ... perform after upload is successful operation
console.log("upload completed ", uploaders);
});
}
I have got this example from here
Another thing is confusing to me. In superagent we can attach parameters to the request field which includes API Secret Key of Cloudinary like this:
const paramsStr = 'timestamp='+timestamp+'&upload_preset='+uploadPreset+secretKey;
const signature = sha1(paramsStr);
const params = {
'api_key': 'xxxx',
'timestamp': timestamp,
'upload_preset': uploadPreset,
'signature': signature
}
Object.keys(params).forEach(key => {
uploadRequest.field(key, params[key])
});
But in that example, it is not mentioned how to append the secret key and other params to axios.
You will need to generate the signature on your backend, and then perform the upload with the generated signature.
You can generate a signature via the following instructions- https://support.cloudinary.com/hc/en-us/articles/203817991-How-to-generate-a-Cloudinary-signature-on-my-own-
You can also take a look at the following example on how to append the signature to your request. It's in PHP, however, the guidelines still apply.
https://gist.github.com/taragano/a000965b1514befbaa03a24e32efdfe5
Related
I'm using axios to upload an audio file to AWS s3 bucket.
The workflow is: React => AWS API Gateway => Lambda.
Here is the backend Lambda code where generates the S3 presigned URL:
PutObjectRequest putObjectRequest = PutObjectRequest.builder()
.bucket(AUDIO_S3_BUCKET)
.key(objectKey)
.contentType("audio/mpeg")
.build();
PutObjectPresignRequest putObjectPresignRequest = PutObjectPresignRequest.builder()
.signatureDuration(Duration.ofMinutes(10))
.putObjectRequest(putObjectRequest)
.build();
PresignedPutObjectRequest presignedPutObjectRequest = s3Presigner.presignPutObject(putObjectPresignRequest);
AwsProxyResponse awsProxyResponse = new AwsProxyResponse();
awsProxyResponse.setStatusCode(HttpStatus.SC_OK);
awsProxyResponse.setBody(
GetS3PresignedUrlResponse.builder()
.s3PresignedUrl(presignedPutObjectRequest.url().toString())
.build().toString());
return awsProxyResponse;
Here is the java code to create the bucket:
private void setBucketCorsSettings(#NonNull final String bucketName) {
s3Client.putBucketCors(PutBucketCorsRequest.builder()
.bucket(bucketName)
.corsConfiguration(CORSConfiguration.builder()
.corsRules(CORSRule.builder()
.allowedHeaders("*")
.allowedMethods("GET", "PUT", "POST")
.allowedOrigins("*") // TODO: Replace with domain name
.exposeHeaders("ETag")
.maxAgeSeconds(3600)
.build())
.build())
.build());
log.info("Set bucket CORS settings successfully for bucketName={}.", bucketName);
}
In my frontend, here is the part that try to upload file:
const uploadFile = (s3PresignedUrl: string, file: File) => {
let formData = new FormData();
formData.append("file", file);
formData.append('Content-Type', file.type);
const config = {
headers: {
"Content-Type": 'multipart/form-data; boundary=---daba-boundary---'
//"Content-Type": file.type,
},
onUploadProgress: (progressEvent: { loaded: any; total: any; }) => {
const { loaded, total } = progressEvent;
let percent = Math.floor((loaded * 100) / total);
if (percent < 100) {
setUploadPercentage(percent);
}
},
cancelToken: new axios.CancelToken(
cancel => (cancelFileUpload.current = cancel)
)
};
axios(
{
method: 'post',
url: s3PresignedUrl,
data: formData,
headers: {
"Content-Type": 'multipart/form-data; boundary=---daba-boundary---'
}
}
)
.then(res => {
console.log(res);
setUploadPercentage(100);
setTimeout(() => {
setUploadPercentage(0);
}, 1000);
})
.catch(err => {
console.log(err);
if (axios.isCancel(err)) {
alert(err.message);
}
setUploadPercentage(0);
});
};
However, when try to upload the file, it return 403 error.
And if I use fetch instead of axios instead and it works, like this:
export async function putToS3(presignedUrl: string, fileObject: any) {
const requestOptions = {
method: "PUT",
headers: {
"Content-Type": fileObject.type,
},
body: fileObject,
};
//console.log(presignedUrl);
const response = await fetch(presignedUrl, requestOptions);
//console.log(response);
return await response;
}
putToS3(getPresignedUrlResponse['s3PresignedUrl'], values.selectdFile).then(
(putToS3Response) => {
console.log(putToS3Response);
Toast("Success!!", "File has been uploaded.", "success");
}
);
It seems to me that the only difference between these two is that: when using fetch the request's Content-Type header is Content-Type: audio/mpeg, but when using axios it is Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryClLJS3r5Xetv3rN7 .
How can I make it work with axios? I'm switching to axios for its ability to monitor request progress as I want to show an upload progress bar.
I followed this blog and not sure what I missed: https://bobbyhadz.com/blog/aws-s3-presigned-url-react
You are using POST in your axios. Should be PUT instead.
Also I think the content type has to match the one specified during requesting the pre-signed URL, which is audio/mpeg as you rightly pointed out.
Correspondingly, your data should be just file, instead of formData.
axios(
{
method: 'put',
url: s3PresignedUrl,
data: file,
headers: {
"Content-Type": 'audio/mpeg'
}
}
...
You didn't mark any answers as accepted so I guess you didn't solve it.
For any future viewers out there. The reason why you are getting 403 forbidden error is because your Content-Type in your server and client side are not matching. I'm assuming you set up the AWS policies correctly.
Your code in the backend should look like this:
const presignedPUTURL = s3.getSignedUrl("putObject", {
Bucket: "bucket-name",
Key: String(Date.now()),
Expires: 100,
ContentType: "image/png", // important
});
and in the front-end (assuming you are using axios):
const file = e.target.files[0]
const result = await axios.put(url, file, {
withCredentials: true,
headers: { "Content-Type": "image/png" },
});
In practical, you would normally have to send the file type to generate the pre-signed url in the POST body or whatever and then in axios you do file.type to get the file type of the uploaded file.
Check your Lambda execution role. It may be the culprit. Perhaps it does not grant enough permissions to allow PUTting files into your bucket.
URL signing is a delegation of power on behalf of the signer, which is restricted to a specified object, action... Signing does not magically grants full read/write permissions on S3, even on the specific object related to the presigned URL.
The "user" who generates the signature requires sufficient permissions to allow the actions you want to delegate through that presigned URL. In this case, this is the execution role of your Lambda function.
You can add the AmazonS3FullAccess managed policy to the execution role and see if it solves your situation. This change took me out of a blocked situation me after days of struggle. Afterwards, before going to production, restrict that rule to the specific bucket you want to allow uploads into (least privilege principle).
If you develop using SAM local emulation, those execution roles seem not to be taken into account as long as you run your functions locally; the signed links work in that context even without S3 permissions.
Hellooooo.
I am trying to upload images to AWS S3 and I've stumbled upon something that irritates me quite a lot. I simply can't seem to understand why it would act like this.
Aight so.. I am using formdata to send data to my api endpoint. The API gets called without any issues, no errors, nothing.. Like srly, nothing. Before getting to image upload I was just using a basic body post request with fetch but now I am using formdata in order to upload images.
Here's my "fetch/axios/sendthingy"
const formData = new FormData();
formData.append('file', validFiles[0]);
formData.append('postTitle', postTitle);
formData.append('postProduct', postProduct);
formData.append('postDescription', postDescription);
formData.append('postPrice', postPrice);
formData.append('postCoins', cryptocoins);
formData.append('postCategory', Cate);
formData.append('postSubCategory', subCat);
formData.append('postCryptoDiscount', cryptoDiscount);
for (const entry of formData.entries())
{
// debugging
console.log(entry)
}
var myHeaders = new Headers();
myHeaders.append("Accept", "application/json"); // or form data?
const requestOptions = {
method: 'POST',
body: formData,
headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, // or use myHeaders
// body: JSON.stringify({ postTitle: postTitle, postProduct: postProduct, postDescription: postDescription, postPrice: postPrice, postCoins: cryptocoins, postImage: validFiles[0], postCategory: Cate, postSubCategory: subCat, postCryptoDiscount: cryptoDiscount})
};
// const res = await fetch(`http://localhost:3000/api/posts/create`, requestOptions)
// const data = await res.json();
axios.post('http://localhost:3000/api/posts/create', formData).then(console.log).catch(console.error)
My comments are from before using formdata. That worked.
As you can see, I have installed Axios. I wanted to see if axios would fix the return stuff. No luck. Any ideas on how to solve this or how to send images to api with or without formdata?
Right, let's move on. Next issue is that I can see it calling my API endpoint but my api endpoint does nothing. It returns nothing. Does nothing. But when I call the api from postman it works just as it should. How come? It creates the post and returns the correct data. Any ideas?
My API:
export default async function handler(req, res) {
console.log('body', req.body);
/*
Redacted
*/
res.status(200).json({response: 'test', code: 200, message:"Successfully created the post."})
})
Why is my API not doing anything locally... It will only work with formdata if I call the endpoint from postman.
This is what I mean.. The dark one is Firefox, white is Postman:
Firstly , you should use multipart/form-data for uploading files. And to see the response you have to console.log the response like this:
axios.post(
'http://localhost:3000/api/posts/create',
formData,
{headers:{'Content-Type':'multipart/form-data'}}
).then(res => console.log(res))
.catch(err => console.error(err))
We are trying to download large files say 1GB or 2 GB but after certain time though the backend still goes on the UI gives error as Failed to fetch for large files.
So how can we handle large file downloads using React js
Please help!
Code as below:
getFile = async (endpoint: string, id: string, params?: any) => {
const response = await fetch(
this.createUrl(endpoint + "/" + id, params),
this.getRequest("get", {
Accept: "application/octet-stream",
"Content-Type": "application/octet-stream",
}),
);
if (response.status === 200) {
return await response.blob();
} else {
throw Error(errorObj.error);
}
};
downloadFile = (filepath: any) => {
this.props.api.getFile(resource, filepath, {}).then((res: any) =>
this.setState(() => {
const url = window.URL.createObjectURL(new Blob([res]));
const link = document.createElement("a");
link.href = url;
link.setAttribute("download", path.basename(filepath));
document.body.appendChild(link);
link.click();
link.parentNode!.removeChild(link);
toaster.success("Successfully downloaded ");
}),
);
};
Using fetch will buffer the response into memory, and you can't quite expect to buffer 1 to 2 gigabytes in memory. (You could do something clever with IndexedDB like e.g. Mega does, but it's likely not worth it.)
Instead of fetching the data from an URL (let's call it URL A) and creating an Object URL from the content blob to put in a download link you click, simply put URL A in the download link.
If the endpoint at URL A requires some authentication or similar, you will need to change that to something that can be encoded into query parameters; maybe a token with a signature akin to what AWS S3 does with presigned URLs.
I am trying to send an API request through my React frontend to Spring backend. When I send the request I get this error:
Could not resolve parameter [0] in private org.springframework.http.ResponseEntity com.example.bottlecap.controllers.BottlecapController.entryForm(com.example.bottlecap.domian.Bottlecap,org.springframework.web.multipart.MultipartFile): Content type 'application/octet-stream' not supported
However, when I set up the request using Postman it goes through fine. I presume there may be an issue with how I am setting up my FormData on the React end. However, I have had little luck figuring it out.
My API is supposed to recieve an object that holds data about my submission as well as an image that goes with the submission. In my Postman request, I am creating a form data that holds a JSON file that holds all the object data and a random image just for testing. As I said, the requets goes through fine with this. However, in the frontend code, I am parsing through the object data as Json and adding it to a FormData as well as adding the image to the FormData.
Here is my Spring Controller:
#RequestMapping(path ="/bottlecap/entry", method = RequestMethod.POST, consumes = {MediaType.MULTIPART_FORM_DATA_VALUE, MediaType.APPLICATION_OCTET_STREAM_VALUE})
private ResponseEntity entryForm(#RequestPart("cap") Bottlecap cap, #RequestPart("file") MultipartFile image){
System.out.println(cap);
cap.toString();
System.out.println("Test");
return ResponseEntity.ok().build();
}
Here is my react Frontend form submission handler:
handleSubmit = event =>{
console.log(this.state.page);
console.log(this.state.page);
event.preventDefault();
const cap ={
"name":this.state.name,
"brand":this.state.brand,
"yearMade":parseInt(this.state.yearMade),
"country": this.state.country,
"description":this.state.description,
"yearFound":parseInt(this.state.yearFound),
"isAlcoholic":"true"
};
const stringCap = JSON.stringify({cap});
console.log(cap);
var formData = new FormData();
formData.append('cap', JSON.parse(stringCap));
formData.append('file',this.state.imageFile)
axios.post('http://localhost:8080/bottlecap/entry', formData, {headers:{'Content-Type':'multipart/form-data'}})
.then(res=>{
console.log(res);
console.log(res.data);
//window.location = "/success"
this.setState({pageDone:true})
this.setState({pageLoading:true})
})
}
Here is a screenshot of my Postman request if it may help.
Also here is the contents of the json file I am sending through on Postman, if it may help as well.
{"name":"post-test",
"brand":"post-test",
"yearMade":1000,
"country":"post-test",
"description":"post-test",
"yearFound":1000,
"isAlcoholic":"true"}
The last change I did was adding a header to the axios API request, but still no luck.
In postman, for parameter named cap, you're sending a .json file. But in your reactjs code, you're doing
formData.append('cap', JSON.parse(stringCap));
JSON.parse will create a javascript object which is not what your backend is expecting. You need to send it as a JSON file.
Not tested, but this might give you the idea.
const json = JSON.stringify(cap);
const blob = new Blob([json], {
type: 'application/json'
});
var formData = new FormData();
formData.append('cap', blob);
formData.append('file', this.state.imageFile)
axios.post('http://localhost:8080/bottlecap/entry', formData, {headers:{'Content-Type':'multipart/form-data'}})
.then(res=>{
console.log(res.data);
}
This is my fetch sample in Vue3, and it works thanks to you. Thanks!
let formData = new FormData();
formData.append('productJsonData', new Blob([JSON.stringify(productJsonObject)], {type: 'application/json'}));
formData.append('file', image); // This image comes from an <v-file-input> TAG
const response = await fetch(
`http://.../addProduct`,
{
headers: { 'Accept': 'application/json' },
method: 'POST',
body: formData
}
);
const responseData = await response.json();
if (!response.ok) {
console.log('REST error: [' + responseData.error + ']')
throw error;
}
I am trying to get an presigned url image upload working correctly. Currently the upload succeeds when selecting an image from the IOS simulator, however when I actually try to view the file it seems the file is corrupted and will not open as an image. I suspect it has something to do with my FormData but not sure.
export async function receiptUpload(file) {
const date = new Date();
const headers = await getAWSHeaders();
const presignUrl = await request.post(
urls.fileUpload.presignUpload,
{file_name: `${date.getTime()}.jpg`},
{headers}
)
.then(res => res.data);
const formData = new FormData();
formData.append('file', {
name: `${date.getTime()}.jpg`,
uri: file.uri,
type: file.type
});
const fileUpload = presignUrl.presignUrl && await request.put(
presignUrl.presignUrl,
formData
)
.then(res => res.status === 200);
}
I have tried from other fixes to change the file uri like so...
Platform.OS === 'android' ? file.uri : file.uri.replace('file://', '');
however this does not seem to work either.
I did this just recently in my current project and the following code is a working example for my use case. I didn't need to convert to a blob either though I am uploading to AWS S3 so if you are uploading elsewhere that may be the issue.
export const uploadMedia = async (fileData, s3Data, setUploadProgress = () => {}) => {
let sendData = { ...fileData };
sendData.data.type = sendData.type;
let formData = new FormData();
formData.append('key', s3Data.s3Key);
formData.append('Content-Type', fileData.type);
formData.append('AWSAccessKeyId', s3Data.awsAccessKey);
formData.append('acl', 'public-read');
formData.append('policy', s3Data.s3Policy);
formData.append('signature', s3Data.s3Signature);
formData.append('file', sendData.data);
return axios({
method: 'POST',
url: `https://${s3Data.s3Bucket}.s3.amazonaws.com/`,
data: formData,
onUploadProgress: progressEvent => {
let percentCompleted = Math.floor((progressEvent.loaded * 100) / progressEvent.total)
setUploadProgress(percentCompleted);
}
})
}
I would first check to see where the issue is occurring. After uploading can you view it on whatever storage service you are trying to upload it to. If so it's something on React Native side. If it doesn't ever get uploaded to the location you know its an error in your upload process. Might help you track the exact location of the error.
I had to do this recently for a project. I believe the data is a base64 string when coming directly from the file input. So the issue is your are uploading a base64 string not the image by simply passing the data field. I had to process it before uploading to the signed URL with the following method.
private dataUriToBlob(dataUri) {
const binary = atob(dataUri.split(',')[1]);
const array = [];
for (let i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
return new Blob([new Uint8Array(array)], { type: 'image/jpeg' });
}
This answer fixed it for me: How can I upload image directly on Amazon S3 in React Native?
I had tried uploading with axios and fetch with FormData. The download went through but the image file was not readable, even when downloaded to my Mac from the S3 console:
The file "yourfile.jpg" could not be opened. It may be damaged or use a file format that Preview doesn’t recognize.
Only after trying to upload with XHR with the correct Content-Type header did it work. Your signedUrl should be correct as well, which seems to be the case if the download goes through.