We are trying to download large files say 1GB or 2 GB but after certain time though the backend still goes on the UI gives error as Failed to fetch for large files.
So how can we handle large file downloads using React js
Please help!
Code as below:
getFile = async (endpoint: string, id: string, params?: any) => {
const response = await fetch(
this.createUrl(endpoint + "/" + id, params),
this.getRequest("get", {
Accept: "application/octet-stream",
"Content-Type": "application/octet-stream",
}),
);
if (response.status === 200) {
return await response.blob();
} else {
throw Error(errorObj.error);
}
};
downloadFile = (filepath: any) => {
this.props.api.getFile(resource, filepath, {}).then((res: any) =>
this.setState(() => {
const url = window.URL.createObjectURL(new Blob([res]));
const link = document.createElement("a");
link.href = url;
link.setAttribute("download", path.basename(filepath));
document.body.appendChild(link);
link.click();
link.parentNode!.removeChild(link);
toaster.success("Successfully downloaded ");
}),
);
};
Using fetch will buffer the response into memory, and you can't quite expect to buffer 1 to 2 gigabytes in memory. (You could do something clever with IndexedDB like e.g. Mega does, but it's likely not worth it.)
Instead of fetching the data from an URL (let's call it URL A) and creating an Object URL from the content blob to put in a download link you click, simply put URL A in the download link.
If the endpoint at URL A requires some authentication or similar, you will need to change that to something that can be encoded into query parameters; maybe a token with a signature akin to what AWS S3 does with presigned URLs.
Related
EDIT: I've updated the CORS config but its still showing the same error.
I have a Tinymce RTE on my page, and when u drop an image into the editor, I have some functions that upload it to firebase storage, then swaps out the src of the text editor with the url fetched from firebase. It works kinda ok, but its being displayed as a broken link image icon.
When I check the link, its because originally it downloads the image when the link is clicked. I added a metadata property when it uploads it, but now its just showing a tiny box.
Here is the code where the image dropped into the editor is uploaded into firebase storage
const imagesUploadHandler = async (blobInfo, success, failure) => {
try {
const file = blobInfo.blob();
const storageRef = ref(storage, file.name);
const metadata = {
contentType: 'image/jpeg',
};
await uploadBytes(storageRef, file, metadata);
const url = await getDownloadURL(storageRef);
console.log(url);
return url;
} catch (error) {
// Call the failure callback with the error message
console.log(error.message);
}
};
Originally, i didnt include the contentType metadata, and it was just uploading as application/octet-stream, which i assume is why it prompts you to save the image.
Image link: https://firebasestorage.googleapis.com/v0/b/cloudnoise-news.appspot.com/o/ref.jpg?alt=media&token=1edc90e7-1668-4a06-92a3-965ce275798b
Currently its displaying this
Somethings i checked through
firebase storage rules is in test mode, so should be able to read and write by anyone.
i tried sticking in different MIME types but it either shows the tiny box, or it shows "undefined"
the files upload successfully and the "swap" in Tinymce editor is also all good.
Any idea why this is happening?
you need to set the metadata tag
const metadata = {
contentType: file.type,
};
This should ensure that the correct content type is set when the image is uploaded to Firebase Storage.
If this does not resolve the issue, you may need to check that the URL returned from getDownloadURL is valid and points to the correct image. You can try opening the URL in a new browser tab to verify that the image is accessible.
I fixed it by adding a blob, I created a blob object with the file data, then i just made it upload the blob object instead of the single file.
const imagesUploadHandler = async (blobInfo, success, failure) => {
try {
const file = blobInfo.blob();
const storageRef = ref(storage, file.name);
const metadata = {
contentType: file.type,
};
// Create a new Blob object with the file data
const blob2 = await new Blob([file], { type: file.type });
// Upload the Blob to Firebase Storage
await uploadBytes(storageRef, blob2, metadata);
const url = await getDownloadURL(storageRef);
console.log(url);
return url;
} catch (error) {
// Call the failure callback with the error message;;
console.log(error.message)
}
};
I am trying to get an presigned url image upload working correctly. Currently the upload succeeds when selecting an image from the IOS simulator, however when I actually try to view the file it seems the file is corrupted and will not open as an image. I suspect it has something to do with my FormData but not sure.
export async function receiptUpload(file) {
const date = new Date();
const headers = await getAWSHeaders();
const presignUrl = await request.post(
urls.fileUpload.presignUpload,
{file_name: `${date.getTime()}.jpg`},
{headers}
)
.then(res => res.data);
const formData = new FormData();
formData.append('file', {
name: `${date.getTime()}.jpg`,
uri: file.uri,
type: file.type
});
const fileUpload = presignUrl.presignUrl && await request.put(
presignUrl.presignUrl,
formData
)
.then(res => res.status === 200);
}
I have tried from other fixes to change the file uri like so...
Platform.OS === 'android' ? file.uri : file.uri.replace('file://', '');
however this does not seem to work either.
I did this just recently in my current project and the following code is a working example for my use case. I didn't need to convert to a blob either though I am uploading to AWS S3 so if you are uploading elsewhere that may be the issue.
export const uploadMedia = async (fileData, s3Data, setUploadProgress = () => {}) => {
let sendData = { ...fileData };
sendData.data.type = sendData.type;
let formData = new FormData();
formData.append('key', s3Data.s3Key);
formData.append('Content-Type', fileData.type);
formData.append('AWSAccessKeyId', s3Data.awsAccessKey);
formData.append('acl', 'public-read');
formData.append('policy', s3Data.s3Policy);
formData.append('signature', s3Data.s3Signature);
formData.append('file', sendData.data);
return axios({
method: 'POST',
url: `https://${s3Data.s3Bucket}.s3.amazonaws.com/`,
data: formData,
onUploadProgress: progressEvent => {
let percentCompleted = Math.floor((progressEvent.loaded * 100) / progressEvent.total)
setUploadProgress(percentCompleted);
}
})
}
I would first check to see where the issue is occurring. After uploading can you view it on whatever storage service you are trying to upload it to. If so it's something on React Native side. If it doesn't ever get uploaded to the location you know its an error in your upload process. Might help you track the exact location of the error.
I had to do this recently for a project. I believe the data is a base64 string when coming directly from the file input. So the issue is your are uploading a base64 string not the image by simply passing the data field. I had to process it before uploading to the signed URL with the following method.
private dataUriToBlob(dataUri) {
const binary = atob(dataUri.split(',')[1]);
const array = [];
for (let i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
return new Blob([new Uint8Array(array)], { type: 'image/jpeg' });
}
This answer fixed it for me: How can I upload image directly on Amazon S3 in React Native?
I had tried uploading with axios and fetch with FormData. The download went through but the image file was not readable, even when downloaded to my Mac from the S3 console:
The file "yourfile.jpg" could not be opened. It may be damaged or use a file format that Preview doesn’t recognize.
Only after trying to upload with XHR with the correct Content-Type header did it work. Your signedUrl should be correct as well, which seems to be the case if the download goes through.
I am trying to understand how one might get PDF Kit working within React JS.
My simple requirement is to render a file in the browser using ReactJS and PDFKit.
Looking at the tutorials, there are references to a few options in which PDFKit can work in the browser, however it is unclear how this would apply in the world of React JS and in particular a Create React App based project...
http://pdfkit.org/demo/browser.html
https://www.npmjs.com/package/pdfkit#browser-usage
Has anyone come across a working example with Create React App based React JS and PDFKit?
Google seems to be a bit short on answers this evening.
I wanted to do exactly this (PdfKit client-side rendering in a React app bootstrapped with create-react-app). I managed to get it working using pdfkit-webpack-example and customize-cra.
Source
Live Demo
I manage to find an answer for this.
It's a combination of this and this.
But just to give an overview/key codes. In my react application, I have this (make a post request to my API server):
<Button
onClick={(e) => {
const { _id } = record;
fetch(`/api/pdf`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
filename: "test",
content: "Testing Content"
}),
}).then(async res => {
if (res.status === 200) {
const blob = await res.blob();
const file = new Blob(
[blob],
{type: 'application/pdf'}
);
//Build a URL from the file
const fileURL = URL.createObjectURL(file);
//Open the URL on new Window
window.open(fileURL);
}
})
>
Download
</Button>
And in the API server (node express application), I have this:
const postMethod = () => async (req, res, next) => {
const doc = new PDFDocument()
let filename = req.body.filename
// Stripping special characters
filename = encodeURIComponent(filename) + '.pdf'
// Setting response to 'attachment' (download).
// If you use 'inline' here it will automatically open the PDF
res.setHeader('Content-disposition', 'attachment; filename="' + filename + '"')
res.setHeader('Content-type', 'application/pdf')
const content = req.body.content
doc.y = 300
doc.text(content, 50, 50)
doc.pipe(res)
doc.end()
}
Hey I've been building out a full-stack tinder app using react native + firebase auth/storage/realtimedb.
Everything has been going great so far but I've ran into an issue a few days ago and I don't know what's wrong with it.
I get back the correct uri of the image and pass it in as parameters to my uploadImage function and convert that to a blob. It uploads a file to firebase storage but it's not my image. This is what gets uploaded:
Image that is getting uploaded.
Weird things going on in the file description of my 'image'
The first things I notice is when I upload the image and look at the description of the supposed image I see that the size is 600,000 bytes which is strange because when I upload the pictures manually through the firebase storage console they are a few megabytes.
The second thing is the image preview is not working.
editAvi = async () => {
console.log('wtf')
await Permissions.askAsync(Permissions.CAMERA_ROLL);
const { cancelled, uri } = await ImagePicker.launchImageLibraryAsync({
allowsEditing: true,
});
if (!cancelled) {
this.setState({ image: uri });
}
console.log('The image is' + this.state.image)
};
uploadImage = async (uri, imageName) => {
// Create file metadata including the content type
var metadata = {
contentType: 'image/jpeg',
}
// Points to the root reference
var storageRef = firebase.storage().ref();
// Points to 'images'
const response = await fetch(uri);
const blob = await response.blob();
var ref = storageRef.child('images/' + this.state.currentID);
ref.put(uri, metadata);
console.log('This is the blob: ' + blob)
}
I've been researching this extensively for two days and have asked about it multiple times in a web development discord I'm in and I still can't fix it.
Please help me fix this! This is one of the last things I need to get this app done. :)
Found this question when I was also searching for an answer. I was able to solve this following the recommendation from a Github issue https://github.com/expo/expo/issues/2402#issuecomment-443726662
The main idea is to replace
const response = await fetch(uri);
const blob = await response.blob();
var ref = storageRef.child('images/' + this.state.currentID);
ref.put(uri, metadata);
with
const blob = await new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest();
xhr.onload = () => {
resolve(xhr.response);
};
xhr.onerror = (e) => {
reject(new TypeError("Network request failed"));
};
xhr.responseType = "blob";
xhr.open("GET", uri, true);
xhr.send(null);
});
var ref = storageRef.child('images/' + this.state.currentID);
ref.put(blob, metadata);
Fetch in know to have a problem in ReacNative when using Expo.
Hope this solves it.
Found this question when I was searching around for a solution to the exact same issue. I fixed it after about 12 hours of trial and error by adding 'application/octet-stream;BASE64' a the type when creating the blob (using rn-fetch-blob).
Blob.build(data, { type: 'application/octet-stream;BASE64' });
Not sure if that's the method you're using to create the blob, but if so, using that as the type fixed the issue for me.
I have tried implementing the superagent way of uploading multiple files in axios. But somehow, I'm getting an error in console
Failed to load https://api.cloudinary.com/v1_1/xxxx/image/upload:
Request header field Authorization is not allowed by
Access-Control-Allow-Headers in preflight response.
My upload handler looks like this
uploadFile(){
const uploaders = this.state.filesToBeSent.map(file => {
const formData = new FormData();
formData.append("file", file);
formData.append("upload_preset", "xxxxx");
formData.append("api_key", "xxxxx");
formData.append("timestamp", (Date.now() / 1000) | 0);
return axios.post(url, formData, {
headers: { "X-Requested-With": "XMLHttpRequest" },
}).then(response => {
const data = response.data;
const fileURL = data.secure_url
console.log(data);
})
});
// Once all the files are uploaded
axios.all(uploaders).then(() => {
// ... perform after upload is successful operation
console.log("upload completed ", uploaders);
});
}
I have got this example from here
Another thing is confusing to me. In superagent we can attach parameters to the request field which includes API Secret Key of Cloudinary like this:
const paramsStr = 'timestamp='+timestamp+'&upload_preset='+uploadPreset+secretKey;
const signature = sha1(paramsStr);
const params = {
'api_key': 'xxxx',
'timestamp': timestamp,
'upload_preset': uploadPreset,
'signature': signature
}
Object.keys(params).forEach(key => {
uploadRequest.field(key, params[key])
});
But in that example, it is not mentioned how to append the secret key and other params to axios.
You will need to generate the signature on your backend, and then perform the upload with the generated signature.
You can generate a signature via the following instructions- https://support.cloudinary.com/hc/en-us/articles/203817991-How-to-generate-a-Cloudinary-signature-on-my-own-
You can also take a look at the following example on how to append the signature to your request. It's in PHP, however, the guidelines still apply.
https://gist.github.com/taragano/a000965b1514befbaa03a24e32efdfe5