NestJS File Upload in chunks - file

Does anyone know how to upload a single file in chunks using NestJS?
I cannot find any working example related to that online.
Here's my current implementation
#Post()
#UseInterceptors(
FileInterceptor('file', {
storage: diskStorage({
destination: function (req, file, cb) {
const dir = path.join(ENV.MNT);
cb(null, dir);
},
filename: editFileName,
}),
fileFilter: fileFilter,
}),
)
uploadFile(#UploadedFile() file: Express.Multer.File) {
console.log(file);
return true;
}
file is undefined (I suppose because the file has not yet been fully received, so no metadata, etc...)

I have working example in my servers, it is similar to yours. Probably "storage: diskStorage" property in your options causes error.
I upload 350mb data without any problem, but it takes some time depends on your connection speed. I didn't test much bigger files, but i think it should work. The point is, uploadFile function runs after your upload finishes. You should see console.log after all upload finished.
#Post()
#UseInterceptors(
FileInterceptor('file', {
dest: './storage/uploads'
}),
)
uploadFile(#UploadedFile() file: Express.Multer.File) {
console.log(file);
return file;
}

Related

React-Flask download excel file with button click

I'm trying to download an excel file with the click of a button in my web application. I can see the data come across from my api request, but when I download the file and try to open it I either get a:
"We found a problem with some content in ... Do you want us to try to recover as much as possible" YES => "This file is corrupt and cannot be opened"
or
"... the file format or file extension is not valid. Verify that theh file has not been corrupted..."
If I open the original file saved it works fine so it's not the file. I think the problem is somewhere in the React Code.
I've looked at a lot of other questions on Stack Overflow about this same topic but none of the answers seem to be working for me.
React
React.useEffect(() => {
if (template && downloadBtn.current != null) {
axios
.get<Blob>(
`/template`,
{ params: { filename: template } }
// responseType: 'blob' or "arraybuffer" don't work for me
)
.then((resp) => {
console.log(resp.data);
var blob = new Blob([resp.data], {
type: resp.headers['content-type'] // tried keeping and removing this
}); // removing this assuming resp.data is already a blob didn't work
console.log(blob); // PK ... b���C���h����ؒ )���G+N�
const url = window.URL.createObjectURL(blob);
console.log(url); // blob:http://localhost:3000/29fd5f64-da6a-4b9c-b4a4-76cce1d691c8
if (downloadBtn.current != null) {
downloadBtn.current.download = template;
downloadBtn.current.href = url;
}
});
}
}, [template, downloadBtn.current]);
Flask
#app.route('/template', methods=['GET'])
def template():
filename = getRouteData(['filename']) # helper function I wrote to get request.body data
print(os.path.join(
app.config['templates_folder'], filename), file=sys.stderr)
return send_file(os.path.join(app.config['templates_folder'], filename))
# adding as_attachment=True doesn't work for me either
# file path is correct

Unable to access filename in multer

After looking through numerous documentation on Multer I believe that I have the proper setup on my express server. However, each time I attempt to access the req.files.filename attribute of my incoming file, the value is returned as undefined.
Here is my Express server side code:
const storage = multer.diskStorage({
destination: '../Uploads',
filename: (req, file, cb) => cb(null, file.originalname)
});
const upload = multer({
storage: storage
});
Router.post("/", upload.single("file"), async (req, res) => {
debug.log("connected");
debug.log(req.file);
debug.log(req.body);
if (!req.files) {
debug.log("receiving connection but file not found");
res.send({
status: false,
message: "No file uploaded",
});
} else {
debug.log("receiving connection and file found");
try {
debug.log(req.files.filename);
debug.log("attempting to save file" + req.files);
const post = await new Post({
title: req.files.filename,
owner: req.body.owner,
industry: req.body.industry,
});
If I understand correctly, calling upload.single should make req.file.filename a usable attribute, but it does not seem to be working.
EDIT:
I solved this problem FINALLY!
Apparently using the app.use(fileUpload()); middleware was somehow interfering with Multer.
Before when I had app.use(fileUpload()); enabled I could access the file through req.files, but the file wouldn't save.
After commenting out app.use(fileUpload()); I can access the file through req.file and it will save correctly.
When you upload a single file (upload.single), look at req.file, not req.files.

Writing to files using File System Access API fails in Electron + Create React App

I have a create-react-app that reads and writes local files using File System Access API. When run in a browser (Chrome or Edge that support it), both reading and writing files work fine.
When the app is run in Electron, reading works but writing fails due to: Uncaught (in promise) DOMException: The request is not allowed by the user agent or the platform in the current context.
I am using the latest Electron (12.0.1) which uses the same Chromium (89.0.4389.82) as the one in my Chrome browser.
Below is the relevant code. The console log after requestPermission call shows true and granted in the browser and true and denied in Electron.
I tried disabling webSecurity when creating BrowserWindow, disabling sandbox with appendSwitch but nothing helped.
Is there a way to give Chromium in Electron more permissions?
If not, I am willing to handle file writing differently when in Electron. In that case, what to write in place of TODO in the code? Note that because it is a create-react-app, the fs module is not available.
export async function chooseAndReadFile() {
const fileHandle = await window.showOpenFilePicker().then((handles) => handles[0])
const file = await fileHandle.getFile()
const contents = await file.text()
return contents
}
export async function chooseAndWriteToFile(contents: string) {
const fileHandle = await window.showSaveFilePicker()
const descriptor: FileSystemHandlePermissionDescriptor = {
writable: true,
mode: "readwrite"
}
const permissionState = await fileHandle.requestPermission(descriptor)
console.log(window.isSecureContext)
console.log(permissionState)
const writable = await fileHandle.createWritable()
await writable.write(contents)
await writable.close()
}
let isElectron = require("is-electron")
export async function chooseAndWriteToFileUniversal(contents: string) {
if (isElectron()) {
// TODO: Do what???
} else {
chooseAndWriteToFile(contents)
}
}
Answering my own question, I finally used a solution with HTML download attribute, nicely described here. When this technique is used in Electron, it presents a file save dialog which is exactly what I want. When used in a browser, this technique just downloads the file without a prompt, so I will continue using File System Access API for browser environments.
Here is the code that handles downloading when running in Electron.
function download(filename: string, contents: string) {
var element = document.createElement('a');
element.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(contents));
element.setAttribute('download', filename);
element.style.display = 'none';
document.body.appendChild(element);
element.click();
document.body.removeChild(element);
}
let isElectron = require("is-electron");
export async function chooseAndWriteToFileUniversal(contents: string) {
if (isElectron()) {
download("data.txt", contents)
} else {
chooseAndWriteToFile(contents) // See the original question for implementation of this function
}
}
Still, would be nice to know why/how is Chromium in Electron more restricted than in a normal Chrome or Edge browser, and if it can be changed.

Set retry with dropzone-react-component when upload fails

In my project I'm using React-Dropzone-Component (https://github.com/felixrieseberg/React-Dropzone-Component) based on Dropzone.js.
I'm using this component because I'm developing a SharePoint webpart and there is already an example based on this solution on Microsoft PnP GitHub repository.
Anyway, the upload is working fine, but sometimes, mainly when I keep a web page opened for a couple of minutes doing nothing, I receive an error trying to upload new files. I retry an upload and it fails returning Server responded with (0) code error. I also see on Google Chrome console an ERR_CONNECTION_RESET error. If I try to upload 5 files in second instance, I get error on first 2-3 and then the remaining files works fine. Weird.
I've already investigated my network, but there are no failures. I've also tried with 3 different networks and I've received the same error.
I've also updated the component with the latest Dropzone.js (5.7.2).
This is my code:
let componentConfig = {
iconFiletypes: this.props.fileTypes.split(','),
showFiletypeIcon: true,
postUrl: _context.pageContext.web.absoluteUrl,
autoProcessQueue: true
};
var djsConfig = {
headers: {
"X-RequestDigest": digest1
},
addRemoveLinks:false
};
let myDropzone;
let eventHandlers = {
// This one receives the dropzone object as the first parameter
// and can be used to additional work with the dropzone.js
// object
init: function(dz){
myDropzone=dz;
},
sending: async function (file, xhr) {
var fileName = file.name;
fileName = fileName.replace(/[&\/\\#,+()$~%='":*?<>{}]/g, "");
if (file.size <= 10485760) {
// small upload
await web.getFolderByServerRelativeUrl("/test/"+_listName).files.add(fileName, file, true).then(_ => console.log("Ok!"));
} else {
// large upload
await web.getFolderByServerRelativeUrl("/test/"+_listName).files.addChunked(fileName, file, data => {}, true).then(_ => console.log("Ok!"));
}
},
error:function(file,error,xhr){
file.status = myDropzone.ADDED;
myDropzone.removeFile(file);
myDropzone.enqueueFile(file);
}
};
<DropzoneComponent eventHandlers={eventHandlers} djsConfig={djsConfig} config={componentConfig}>
<div className="dz-message icon ion-upload">Drop files here to upload</div>
</DropzoneComponent>
If I can't prevent this ERR_CONNECTION_RESET error, I would like to set up an automatic retry for these files. The code I've posted above is not working fine or it returns "Uncaught Error: This file can't be queued because it has already been processed or was rejected.".
Is there a solution or a good way to set up a retry?

Uploading file on Hapi.js server

I'm trying to upload file on a hapi server, and my goal would be to upload a zip file and unzip it on the server, but I currently have problems withs the upload part for a single file...
my frontend is made with react and I'm selecting a file with a <input>
my route is made like this
method: 'POST',
path: '/upload',
config: {
payload: {
maxBytes: 209715200,
output: 'stream',
parse: true
},
handler: handlers.uploadFile,
description: 'upload file'
}
});
I used a stream type output, but I can't figure out what is the type to used depending on the situation between stream, data, or file.
Here my handler uploadfile() is this one :
handler.uploadFile = async (req, h) => {
var doc = req.payload
return true;
}
But I can't get any informations on my file like doc.name or doc._hapi.name so I don't really know how to use the data here.
If someone know a site where all of this is explain, or could help me on that that would help a lot !
Thanks !
Ok so I just figure out what the problem was.
I needed to pass my data as a formData from my react component like that
var formData = new FormData();
formData.append("file", data)
if anyone have the same problem, that is a solution I suppose

Resources