React Upload +5000 files - Failed to load resource: net::ERR_INSUFFICIENT_RESOURCES - reactjs

Question

I'm building an upload files functionality using dropzone and react, my issue is that this application should support thousands of images and once I got about 1500 images it collapse and stop sending requests, then in the browser I got this issue:
Failed to load resource: net::ERR_INSUFFICIENT_RESOURCES
I saw some workarounds to create batches, but honestly don't know how to create it, since I'm processing the uploads one by one using async functions, this is my code:
const Dropzone = ({stateChanger, folder, props, ...rest}) => {
let container;
async function asyncForEach(array, callback) {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array);
}
}
let { id } = useParams();
const [files, setFiles] = useState([]);
const {getRootProps, getInputProps} = useDropzone({
onDrop: async acceptedFiles => {
stateChanger(true)
setFiles(acceptedFiles.map(file => Object.assign(file, {
preview: URL.createObjectURL(file),
uid: uuidv4()
})));
const url = `${API_URL}/projects/upload`;
let requestArr = [];
await asyncForEach(acceptedFiles, async (file) => {
console.log('file',file)
var formData = new FormData();
formData.append('file', file);
formData.append('projectId', id);
formData.append('rootfolder', folder);
console.log('formData',formData)
requestArr.push(
axios
.post(url,formData)
.then((rst) => {
console.log('rst',rst)
var elem = document.getElementById(file.uid);
var html = elem.innerHTML;
elem.innerHTML = `<span class="upload-success" >Uploaded</span>`;
})
.catch((error) => {
console.log('error', error);
var elem = document.getElementById(file.uid);
var html = elem.innerHTML;
elem.innerHTML = `<span class="upload-error" >Error uploading</span>`;
})
);
});
Promise.all(requestArr).then(() => {
console.log('resolved promise.all')
stateChanger(false)
});
}
});
const thumbs = files.map(file => (
<div className="thumb" key={file.name}>
<div className="thumbList">
{file.path} - {file.size} bytes - <span id={file.uid}><span className="upload-pending" >Uploading</span></span>
</div>
</div>
));
useEffect(() => {
// Make sure to revoke the data uris to avoid memory leaks
files.forEach(file => URL.revokeObjectURL(file.preview));
}, [files]);
return (
<section className="container">
<ToastContainer
ref={ref => container = ref}
className="toast-top-right"
/>
<div {...getRootProps({className: 'dropzone'})}>
<input {...getInputProps()} />
<img src={uploadImage} />
<br />
<p>Drag and drop your files, or click the button to upload</p>
<button className="btn primary-btn-active width50">Upload from computer</button>
</div>
<aside >
{thumbs}
</aside>
</section>
);
}
export default Dropzone
And the implementation pretty standard:
<Dropzone stateChanger={setNextDisable} folder={folder} />
Any suggestion?

Answer 1

Why would you need 1500, 5000 images at once?
The user can focus on only so much on the screen at once - and then you have scroll bars which work but is not ideal.
The solution in many places is to have it paginated (10, 20, 50) items on screen at once.
More than that is asking for trouble - as a developer you might have a more powerful PC than an average user - so likely most of your users will have more bottleneck problems, performance issues.
The solution in my opinion would be to reduce the number of items on screen because it doesn't make sense to have that massive amount of assets, and more than a 20 items and user can't focus or bother to have a look or try to do stuff.

Answer 2

Resource consumption has always been a problem that needs to be researched. The processing of large amounts of data is also a similar problem. if only upload. package and compress before uploading.

Answer 3

The problem is not related to Dropzone library. It just means your server, whatever you use, cannot handle such amount of traffic/connections. If you really need you should probably dedicate more resources as the error message states.