Upload data to Yesod server using input forms - angularjs

What is a correct way to uploading files via custom input forms in Yesod?
I developed a frontend in AngularJS, and I pass data to and from the server via JSON objects. However, this does not work when uploading files. For this reason I resorted to forms.
Gathering different resources I ended up with the following code. On the client side I use the following function:
this.submitNewMeas = function() {
var selected_file = document.getElementById('measurements').files[0];
var fd = new FormData();
fd.append("measurementsFile", selected_file);
var response = $http({
method: 'POST',
url: '/measurements/'+this.currMedia.id,
headers: {
'Content-Type': 'multipart/form-data'
},
data: {
fd
},
transformRequest: formDataObject
});
...
};
And on the server side I receive data as follows:
postMeasurementsR :: MediaId -> Handler Value
postMeasurementsR mediaId = do
result <- runInputPost $ iopt fileField "measurementsFile"
case result of
Just fileInfo -> do
saveMeas fileInfo "measDir"
saveMeas :: FileInfo -> FilePath -> HandlerT App IO (FilePath)
saveMeas file dest = do
let filename = unpack $ fileName file
dest = dest </> filename
liftIO $ fileMove file dest
return filename
However this causes a server error
20/Nov/2014:13:40:15 +0100 [Error#yesod-core] <<loop>> #(yesod-core-1.4.3:Yesod.Core.Class.Yesod ./Yesod/Core/Class/Yesod.hs:502:5)
Status: 500 Internal Server Error 54.70131s
If I don't try to save the data, then the file information is received, and I am able to see the file name on the server.
I've successfully managed to upload files using Yesod forms, but when it comes to custom forms all my attempts so far have failed.
Any help is greatly appreciated.
Thanks.

dest = dest </> filename
That's the source of your infinite loop: you're saying "dest is defined as itself plus filename." What's happening is that you're shadowing the original declaration of dest. A simple solution is to call the second declaration something like dest' (or better yet, something even more informative).

Related

Send .mat file through Django Rest Framework

I have an issue to send the contents of a .mat file to my frontend. My end goal is to allow clients to download the content of this .mat file at the click of a button so that they end up with the same file in their possession. I use Next.js + Django Rest Framework.
My first try was as follow:
class Download(APIView):
def get(self, request):
with open('file_path.mat', 'rb') as FID:
fileInstance = FID.read()
return Response(
fileInstance,
status=200,
content_type="application/octet-stream",
)
If I print out the fileInstance element I get some binary results:
z\xe1\xfe\xc6\xc6\xd2\x1e_\xda~\xda|\xbf\xb6\x10_\x84\xb5~\xfe\x98\x1e\xdc\x0f\x1a\xee\xe7Y\x9e\xb5\xf5\x83\x9cS\xb3\xb5\xd4\xb7~XK\xaa\xe3\x9c\xed\x07v\xf59Kbn(\x91\x0e\xdb\xbb\xe8\xf5\xc3\xaa\x94Q\x9euQ\x1fx\x08\xf7\x15\x17\xac\xf4\x82\x19\x8e\xc9...
But I can't send it back to my frontend because of a
"UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 137: invalid start byte"
This error is always the same regardless of which .mat file I try to send in my response.
Next I tried to use the scipy.io.loadmat() method. In this case, fileInstance gives me a much more readable dictionary object, but I still can't get it to transfer to the frontend because of the presence of NaN in my dict:
ValueError: Out of range float values are not JSON compliant
Finally, some suggested to use h5py to send back the data as such:
with h5py.File('file_path.mat', 'r') as fileInstance:
print(fileInstance)
But in that case the error I get is
Unable to open file (file signature not found)
I know my files are not corrupted because I can open them in Matlab with no problem.
With all this trouble I'm wondering if I'm using the right approach to this problem. I could technically send the dictionary obtained through 'scipy.io.loadmat()' as a str element instead of binary, but I'll have to figure out a way to convert this text back to binary inside a Javascript function. Would anybody have some ideas as to how I should proceed?
The problem was in my frontend after all. Still, here's the correct way to go about it:
class Download(APIView):
parser_classes = [FormParser, MultiPartParser]
def get(self, request):
try:
file_path = "xyz.mat"
response = FileResponse(file_path.open("rb"), content_type="application/octet-stream")
response["Content-Disposition"] = f"attachment; filename=file_name"
return response
except Exception as e:
return Response(status=500)
This should send to the frontend the right file in the right format. No need to worry about encoding and such.
Meanwhile, on the frontend you should receive the file as follows:
onClick={() => {
const url = '/url_to_your_api/';
axios({ method: 'get', url: url, responseType: 'blob' })
.then((response) => {
const { data } = response;
const fileName = 'file_name';
const blob = new Blob([data], { type: 'application/octet-stream' });
const href = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = href;
link.download = fileName + '.mat';
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
URL.revokeObjectURL(href);
})
.catch((response) => {
console.error(response);
});
}}
Long story short, the part I was missing was to specify to receive the data as blob inside the 'onClick()' function. By default, responseType from Axios is set to Json/String. For that reason, my file was modified at reception and would not be usable in matlab afterwards. If you face a similar problem in the future, try to use the 'shasum' BASH function to observe the hashed value of the file. It is with the help of that function that I could deduce that my API function would return the correct value and that therefore the problem was happenign on the frontend.

BOX API by Google Apps Script - new file version upload

I've already asked the GAS community but I was advised to continue asking here...
So far I'm able to connect to BOX and get a list of files and I can download a file from BOX as well.
The whole idea is to download a file using BOX API, edit it and upload it back as a new file version using the BOX API.
I'm unable to make the last part working as it gives me error code 400.
Here is the function.
function uploadNewFileVersion() {
//767694355309 testing
var boxFileId="767694355309";
var newVerFile = DriveApp.getFileById("1sK-jcaJoD0WaAcixKtlHA85pf6t8M61v").getBlob();
var confirmAuthorization = getBoxService_().getAccessToken();
//var parent = { "id": "0" };
//"name": "apiNewVersion.xlsx",
//"parent": parent,
var payload = {
"file": newVerFile
}
var headers = {
'Authorization': 'Bearer ' + confirmAuthorization
}
var options = {
"method": "post",
"muteHttpExceptions": true,
"contentType": "multipart/form-data",
"headers": headers,
"payload": payload
}
var apiHtml = "https://upload.box.com/api/2.0/files/"+boxFileId+"/content/";
var response = UrlFetchApp.fetch(apiHtml, options);
Logger.log(response.getResponseCode());
var a = 1;
}
The boxFileId is the file on the box.
The newVerFile is the one downloaded from Box and updated. I need to make it as a new version of the Box file.
Could you please advise?
Thank you!
PEtr
I think parent and name is optional so I commented it out.
If I don't getBlob, then it returns 415 istead.
I believe your goal and situation as follows.
You want to upload a file of Google Drive using Box API with Google Apps Script.
From your question, I cannot find the official document of the method of API that you want to use. But, from the endpoint https://upload.box.com/api/2.0/files/"+boxFileId+"/content/ in your script, I guessed that you wanted to use "Upload file version".
Values of your access token and file ID are valid for using the API.
If my understanding of your question is correct, how about the following modification?
Modification points:
When I saw the official document of "Upload file version", I confirmed the following sample curl. In this case, it is considered that when the following curl command is converted to Google Apps Script, the request might work.
$ curl -i -X POST "https://upload.box.com/api/2.0/files/12345/content" \
-H "Authorization: Bearer <ACCESS_TOKEN>" \
-H "Content-Type: multipart/form-data" \
-F attributes="{"name":"Contract.pdf", "parent":{"id":"11446498"}}" \
-F file=#<FILE_NAME>
From the curl command, it is found that attributes and file are sent as form and files.
And, I thought that attributes="{"name":"Contract.pdf", "parent":{"id":"11446498"}}" might should be attributes="{\"name\":\"Contract.pdf\", \"parent\":{\"id\":\"11446498\"}}".
When I saw your current script, it seems that multipart/form-data is used for contentType. In this case, boundary in the request body is required to be included. Fortunately, at UrlFetchApp, in the case of multipart/form-data, when contentType is not used, the content type is automatically included in the request header. I think that in your case, this can be used.
In your script, attributes="{"name":"Contract.pdf", "parent":{"id":"11446498"}}" is not included. But I thought that you might use it in the future script. So in this answer, this is also included.
When above points are reflected and the sample curl command on the official document is converted to Google Apps Script, the script becomes as follows.
Sample script:
Please copy and paste the following script to the script editor and set the variables, and run the function of myFunction. By this, the request same with the sample curl is requested with Google Apps Script.
function myFunction() {
const accessToken = "###"; // Please set your access token.
const fileId = "###"; // Please set your fileId.
const fileBlob = DriveApp.getFileById("1sK-jcaJoD0WaAcixKtlHA85pf6t8M61v").getBlob();
const metadata = {name: "Contract.pdf", parent: {id: "11446498"}}; // Please set your file metadata.
const params = {
method: "post",
headers: {Authorization: `Bearer ${accessToken}`},
payload: {
attributes: JSON.stringify(metadata),
file: fileBlob,
},
muteHttpExceptions: true,
};
const url = `https://upload.box.com/api/2.0/files/${fileId}/content`;
const res = UrlFetchApp.fetch(url, params);
console.log(res.getContentText());
}
I could confirm that above sample script is the same request with above sample curl.
If you don't want to use the file metadata, please remove the line of attributes: JSON.stringify(metadata), from payload.
Note:
In this case, the maximum data size ("URL Fetch POST size") of UrlFetchApp is 50 MB. Please be careful this. Ref
About the limitation of file upload of Box API, please check https://developer.box.com/guides/uploads/.
If your access token and file ID are invalid, I think that an error occurs. So please be careful this.
References:
Upload file version
Class UrlFetchApp

Downloading an Excel file causes it to corrupt

I have a simple service on Angular 2 and Typescript that requests Excel files to a server and then opens a download file dialogue for the user. However, as it is currently, the file becomes corrupt when downloaded.
When downloaded, it opens fine in OpenOffice and derivates, but throws a "File is Corrupt" error on Microsoft Excel, and asks if the user wants to recover as much as it can.
When Excel is prompted to recover the file, it does so successfully, and the recovered Excel has all rows and data that is expected for the Excel file. Comparing the recovered file against opening the file in OpenOffice and derivates evidence no outstanding differences.
The concrete Excel I am trying to download is generated with Apache POI in a microservice, then passed to the main backend and finally served to the frontend for the user to download. Both the backend and microservice are written in Java, through Spark Framework.
I made some tests on the backends, and concluded the problem is not the report generation nor the data transfer:
Asking the microservice to save the generated Excel in a file within the server and then opening such file (hereby file A) in Excel shows that file A is not corrupted.
Asking the main backend server to save the Excel file that it receives from the microservice in a file within itself and then opening such file in Excel (hereby file B) shows that file B is not corrupted.
Downloading both file A and file B through FileZilla from their respective servers yields completely uncorrupted files.
As such, I believe it is safe to assume the Excel becomes corrupted somewhere between the time the file is received on the frontend and the time the user downloads such file. Additionally, the Catalina logs do not evidence any error that might potentially be happening.
I have read several posts that deal with the issue, including a bug report (https://github.com/angular/angular/issues/14083) that included a workaround via XMLHTTPRequest. However, none of the workarounds detailed were successful in solving my issue.
Attached is the code I am using to both obtain the Excel file from the backend and serve it to the user. I am including both an XMLHTTPRequest and an Angular http call (within comments) since those are the two main ways I have been trying to make this work. Additionally, please do take into account the code has been altered to remove information I do not wish to make public.
download(body) {
let reply = Observable.create(observer => {
let xhr = new XMLHttpRequest();
xhr.open('POST', 'URL', true);
xhr.setRequestHeader('Content-type', 'application/json;charset=UTF-8');
xhr.setRequestHeader('Accept', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
xhr.setRequestHeader('Authorization', 'REDACTED');
xhr.responseType = 'blob';
xhr.onreadystatechange = function () {
if(xhr.readyState === 4) {
if(xhr.status === 200) {
var contentType = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
var blob = new Blob([xhr.response], { type: contentType });
observer.next(blob);
observer.complete();
}
else {
observer.error(xhr.response);
}
}
}
xhr.send(JSON.stringify(body));
});
return reply;
/*let headers = new Headers();
headers.set("Authorization", 'REDACTED');
headers.set("Accept", 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
let requestOptions :RequestOptions = new RequestOptions({headers: headers, responseType: ResponseContentType.Blob});
return this.http.post('URL', body, requestOptions);*/
}
Hereby is the code to prompt the user to download the Excel. It is currently made to work with the XMLHTTPRequest. Please do note that I have also attempted to download without resorting to FileSaver, with no luck.
downloadExcel(data) {
let body = {
/*REDACTED*/
}
this.service.download(body)
.subscribe(data => {
FileSaver.saveAs(data, "Excel.xlsx");
});
}
Hereby are the versions of the tools I am using:
NPM: 5.6.0
NodeJs: 8.11.3
Angular JS: ^6.1.0
Browsers used: Chrome, Firefox, Edge.
Any help on this issue would be appreciated. Any additional information you may need I will be happy to provide.
I think what you want is CSV format which open in Excel, update your sevice as follow:
You should tell Angular you are expecting a response of type blob (Binary Large Object) that is your Excel/Csv file.
Also make sure the URL/API on your server is set to accept content-type='text/csv'.
Here's an example with Angular 2.
#Injectable()
export class YourService {
constructor(private http: Http) {}
download() { //get file from the server
this.http.get("http://localhost/..", {
responseType: ResponseContentType.Blob,
headers: new Headers({'Content-Type', 'text/csv'})
}).subscribe(
response => {
var blob = new Blob([response.blob()], {type: 'text/csv'});
FileSaver.saveAs(blob, 'yourFileName.csv');
},
error => {
console.error('something went wrong');
}
);
}
}
Have you tried uploading/downloading your xls file as base64?
var encodedXLSToUpload = 'data:application/xls;base64,' + btoa(file);
Check this for more details: Creating a Blob from a base64 string in JavaScript

Downloaded .pdf files are corrupted when using expressjs

I am working on meanjs application generated using https://github.com/DaftMonk/generator-angular-fullstack. I am trying to generate a .pdf file using phantomjs and download it to the browser.
The issue is that the downloaded .pdf file always shows the blank pages regardless of the number of pages. The original file on server is not corrupt. When I investigated further, found that the downloaded file is always much larger than the original file on the disk. Also this issue happens only with .pdf files. Other file types are working fine.
I've tried several methods like res.redirect('http://localhost:9000/assets/exports/receipt.pdf');, res.download('client\\assets\\exports\\receipt.pdf'),
var fileSystem = require('fs');
var stat = fileSystem.statSync('client\\assets\\exports\\receipt.pdf');
res.writeHead(200, {
'Content-Type': 'application/pdf',
'Content-Length': stat.size
});
var readStream = fileSystem.createReadStream('client\\assets\\exports\\receipt.pdf');
return readStream.pipe(res);
and even I've tried with https://github.com/expressjs/serve-static with no changes in the result.
I am new to nodejs. What is the best way to download a .pdf file to the browser?
Update:
I am running this on a Windows 8.1 64bit Computer
I had corruption when serving static pdfs too. I tried everything suggested above. Then I found this:
https://github.com/intesso/connect-livereload/issues/39
In essence the usually excellent connect-livereload (package ~0.4.0) was corrupting the pdf.
So just get it to ignore pdfs via:
app.use(require('connect-livereload')({ignore: ['.pdf']}));
now this works:
app.use('/pdf', express.static(path.join(config.root, 'content/files')));
...great relief.
Here is a clean way to serve a file from express, and uses an attachment header to make sure the file is downloaded :
var path = require('path');
var mime = require('mime');
app.get('/download', function(req, res){
//Here do whatever you need to get your file
var filename = path.basename(file);
var mimetype = mime.lookup(file);
res.setHeader('Content-disposition', 'attachment; filename=' + filename);
res.setHeader('Content-type', mimetype);
var filestream = fs.createReadStream(file);
filestream.pipe(res);
});
There are a couple of ways to do this:
If the file is a static one like brochure, readme etc, then you can tell express that my folder has static files (and should be available directly) and keep the file there. This is done using static middleware:
app.use(express.static(pathtofile));
Here is the link: http://expressjs.com/starter/static-files.html
Now you can directly open the file using the url from the browser like:
window.open('http://localhost:9000/assets/exports/receipt.pdf');
or
res.redirect('http://localhost:9000/assets/exports/receipt.pdf');
should be working.
Second way is to read the file, the data must be coming as a buffer. Actually, it should be recognised if you send it directly, but you can try converting it to base64 encoding using:
var base64String = buf.toString('base64');
then set the content type :
res.writeHead(200, {
'Content-Type': 'application/pdf',
'Content-Length': stat.size
});
and send the data as response.
I will try to put an example of this.
EDIT: You dont even need to encode it. You may try that still. But I was able to make it work without even encoding it.
Plus you also do not need to set the headers. Express does it for you. Following is the Snippet of API code written to get the pdf in case it is not public/static. You need API to serve the pdf:
router.get('/viz.pdf', function(req, res){
require('fs').readFile('viz.pdf', function(err, data){
res.send(data);
})
});
Lastly, note that the url for getting the pdf has extension pdf to it, this is for browser to recognise that the incoming file is pdf. Otherwise it will save the file without any extension.
Usually if you are using phantom to generate a pdf then the file will be written to disc and you have to supply the path and a callback to the render function.
router.get('/pdf', function(req, res){
// phantom initialization and generation logic
// supposing you have the generation code above
page.render(filePath, function (err) {
var filename = 'myFile.pdf';
res.setHeader('Content-type', "application/pdf");
fs.readFile(filePath, function (err, data) {
// if the file was readed to buffer without errors you can delete it to save space
if (err) throw err;
fs.unlink(filePath);
// send the file contents
res.send(data);
});
});
});
I don't have experience of the frameworks that you have mentioned but I would recommend using a tool like Fiddler to see what is going on. For example you may not need to add a content-length header since you are streaming and your framework does chunked transfer encoding etc.

upload file to a server swift

I just want to ask you how to simply upload a .csv file to a server .php file. In all other languages is straightforward, here I'm having a lot of troubles, I don't want to do complex things, only upload and see the server response.
Thanks.
Here's my code, I used this library https://github.com/daltoniam/SwiftHTTP
let fileUrl = NSURL.fileURLWithPath("/Users/mypath")
var request = HTTPTask()
request.POST("http://server.com/upload.php", parameters: ["file": HTTPUpload(fileUrl: fileUrl)], success: {(response: HTTPResponse) -> Void in
},failure: {(error: NSError, response: HTTPResponse?) -> Void in
})

Resources