Unable to download XLSX workbook without errors Springboot / React - reactjs

I have an application that is a Springboot backend and a reactJs frontend. I am able to successfully download an excel workbook on my localhost through the application as well as in postman without any errors, but when I test the app on the server I get an error that the file format or extension is not valid.
Also when I test the apps actual url using postman I get a 200 response but nothing is actually returned.
I am using axios and read over this post.
On the backend, the excel workbook is being put together using Apache POI XSSFworkbook.
The backend response entity looks like this
return ResponseEntity
.ok()
.contentType(MediaType.valueOf("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" + params.getDeqId() + "Data.xlsx").body(workbook::write);
On the react side, I've tried a few different things. Currently I have:
return axios
.post(`/api/excelexport/watersystemexcel`, submittedData, { responseType: "arraybuffer" })
.then((response) => {
const url = window.URL.createObjectURL(new Blob([response.data]));
const link = document.createElement("a");
link.href = url;
link.setAttribute("download", submittedData.deqId + "Data.xlsx");
document.body.appendChild(link);
link.click();
this.setState(() => ({ downloading: false }));
});
I have also tried formating the axios request like:
axios.post(`/api/excelexport/watersystemexcel`, submittedData, {
responseType: "arraybuffer",
headers: { "Content-Type": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" }
})
but this results in a 415 error, saying that that specified content-type isn't acceptable.
Lastly, I have also tried changing the file extension to .xls rather then .xlsx on both the front and backend. When I do this the workbook does download but when the file is opened there is an error that file extensions don't match, and that the file may be corrupt.

return ResponseEntity
.ok()
.contentType(MediaType.valueOf("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" + params.getDeqId() + "Data.xlsx")
.body(workbook::write);
When you build your own ResponseEntity object, you have not specified the Content Length, so it could be problematic, and maybe that's why you get a corrupted file at the end.
That's how I do when I need to display images or videos :
return ResponseEntity.status(HttpStatus.OK).contentLength(cr.contentLength()).body(cr);
where cr is a FileSystemResource or a ClasspathResource or whatever Resource you need to send as a HTTP Response Entity.
It seems to be identical, but not, look at the end of my code with the contentLength part before the body part.

Related

Send .mat file through Django Rest Framework

I have an issue to send the contents of a .mat file to my frontend. My end goal is to allow clients to download the content of this .mat file at the click of a button so that they end up with the same file in their possession. I use Next.js + Django Rest Framework.
My first try was as follow:
class Download(APIView):
def get(self, request):
with open('file_path.mat', 'rb') as FID:
fileInstance = FID.read()
return Response(
fileInstance,
status=200,
content_type="application/octet-stream",
)
If I print out the fileInstance element I get some binary results:
z\xe1\xfe\xc6\xc6\xd2\x1e_\xda~\xda|\xbf\xb6\x10_\x84\xb5~\xfe\x98\x1e\xdc\x0f\x1a\xee\xe7Y\x9e\xb5\xf5\x83\x9cS\xb3\xb5\xd4\xb7~XK\xaa\xe3\x9c\xed\x07v\xf59Kbn(\x91\x0e\xdb\xbb\xe8\xf5\xc3\xaa\x94Q\x9euQ\x1fx\x08\xf7\x15\x17\xac\xf4\x82\x19\x8e\xc9...
But I can't send it back to my frontend because of a
"UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 137: invalid start byte"
This error is always the same regardless of which .mat file I try to send in my response.
Next I tried to use the scipy.io.loadmat() method. In this case, fileInstance gives me a much more readable dictionary object, but I still can't get it to transfer to the frontend because of the presence of NaN in my dict:
ValueError: Out of range float values are not JSON compliant
Finally, some suggested to use h5py to send back the data as such:
with h5py.File('file_path.mat', 'r') as fileInstance:
print(fileInstance)
But in that case the error I get is
Unable to open file (file signature not found)
I know my files are not corrupted because I can open them in Matlab with no problem.
With all this trouble I'm wondering if I'm using the right approach to this problem. I could technically send the dictionary obtained through 'scipy.io.loadmat()' as a str element instead of binary, but I'll have to figure out a way to convert this text back to binary inside a Javascript function. Would anybody have some ideas as to how I should proceed?
The problem was in my frontend after all. Still, here's the correct way to go about it:
class Download(APIView):
parser_classes = [FormParser, MultiPartParser]
def get(self, request):
try:
file_path = "xyz.mat"
response = FileResponse(file_path.open("rb"), content_type="application/octet-stream")
response["Content-Disposition"] = f"attachment; filename=file_name"
return response
except Exception as e:
return Response(status=500)
This should send to the frontend the right file in the right format. No need to worry about encoding and such.
Meanwhile, on the frontend you should receive the file as follows:
onClick={() => {
const url = '/url_to_your_api/';
axios({ method: 'get', url: url, responseType: 'blob' })
.then((response) => {
const { data } = response;
const fileName = 'file_name';
const blob = new Blob([data], { type: 'application/octet-stream' });
const href = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = href;
link.download = fileName + '.mat';
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
URL.revokeObjectURL(href);
})
.catch((response) => {
console.error(response);
});
}}
Long story short, the part I was missing was to specify to receive the data as blob inside the 'onClick()' function. By default, responseType from Axios is set to Json/String. For that reason, my file was modified at reception and would not be usable in matlab afterwards. If you face a similar problem in the future, try to use the 'shasum' BASH function to observe the hashed value of the file. It is with the help of that function that I could deduce that my API function would return the correct value and that therefore the problem was happenign on the frontend.

Downloading an Excel file causes it to corrupt

I have a simple service on Angular 2 and Typescript that requests Excel files to a server and then opens a download file dialogue for the user. However, as it is currently, the file becomes corrupt when downloaded.
When downloaded, it opens fine in OpenOffice and derivates, but throws a "File is Corrupt" error on Microsoft Excel, and asks if the user wants to recover as much as it can.
When Excel is prompted to recover the file, it does so successfully, and the recovered Excel has all rows and data that is expected for the Excel file. Comparing the recovered file against opening the file in OpenOffice and derivates evidence no outstanding differences.
The concrete Excel I am trying to download is generated with Apache POI in a microservice, then passed to the main backend and finally served to the frontend for the user to download. Both the backend and microservice are written in Java, through Spark Framework.
I made some tests on the backends, and concluded the problem is not the report generation nor the data transfer:
Asking the microservice to save the generated Excel in a file within the server and then opening such file (hereby file A) in Excel shows that file A is not corrupted.
Asking the main backend server to save the Excel file that it receives from the microservice in a file within itself and then opening such file in Excel (hereby file B) shows that file B is not corrupted.
Downloading both file A and file B through FileZilla from their respective servers yields completely uncorrupted files.
As such, I believe it is safe to assume the Excel becomes corrupted somewhere between the time the file is received on the frontend and the time the user downloads such file. Additionally, the Catalina logs do not evidence any error that might potentially be happening.
I have read several posts that deal with the issue, including a bug report (https://github.com/angular/angular/issues/14083) that included a workaround via XMLHTTPRequest. However, none of the workarounds detailed were successful in solving my issue.
Attached is the code I am using to both obtain the Excel file from the backend and serve it to the user. I am including both an XMLHTTPRequest and an Angular http call (within comments) since those are the two main ways I have been trying to make this work. Additionally, please do take into account the code has been altered to remove information I do not wish to make public.
download(body) {
let reply = Observable.create(observer => {
let xhr = new XMLHttpRequest();
xhr.open('POST', 'URL', true);
xhr.setRequestHeader('Content-type', 'application/json;charset=UTF-8');
xhr.setRequestHeader('Accept', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
xhr.setRequestHeader('Authorization', 'REDACTED');
xhr.responseType = 'blob';
xhr.onreadystatechange = function () {
if(xhr.readyState === 4) {
if(xhr.status === 200) {
var contentType = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
var blob = new Blob([xhr.response], { type: contentType });
observer.next(blob);
observer.complete();
}
else {
observer.error(xhr.response);
}
}
}
xhr.send(JSON.stringify(body));
});
return reply;
/*let headers = new Headers();
headers.set("Authorization", 'REDACTED');
headers.set("Accept", 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
let requestOptions :RequestOptions = new RequestOptions({headers: headers, responseType: ResponseContentType.Blob});
return this.http.post('URL', body, requestOptions);*/
}
Hereby is the code to prompt the user to download the Excel. It is currently made to work with the XMLHTTPRequest. Please do note that I have also attempted to download without resorting to FileSaver, with no luck.
downloadExcel(data) {
let body = {
/*REDACTED*/
}
this.service.download(body)
.subscribe(data => {
FileSaver.saveAs(data, "Excel.xlsx");
});
}
Hereby are the versions of the tools I am using:
NPM: 5.6.0
NodeJs: 8.11.3
Angular JS: ^6.1.0
Browsers used: Chrome, Firefox, Edge.
Any help on this issue would be appreciated. Any additional information you may need I will be happy to provide.
I think what you want is CSV format which open in Excel, update your sevice as follow:
You should tell Angular you are expecting a response of type blob (Binary Large Object) that is your Excel/Csv file.
Also make sure the URL/API on your server is set to accept content-type='text/csv'.
Here's an example with Angular 2.
#Injectable()
export class YourService {
constructor(private http: Http) {}
download() { //get file from the server
this.http.get("http://localhost/..", {
responseType: ResponseContentType.Blob,
headers: new Headers({'Content-Type', 'text/csv'})
}).subscribe(
response => {
var blob = new Blob([response.blob()], {type: 'text/csv'});
FileSaver.saveAs(blob, 'yourFileName.csv');
},
error => {
console.error('something went wrong');
}
);
}
}
Have you tried uploading/downloading your xls file as base64?
var encodedXLSToUpload = 'data:application/xls;base64,' + btoa(file);
Check this for more details: Creating a Blob from a base64 string in JavaScript

ReactJS image/pdf file download not working

I want to download file that can be in any format viz. pdf, jpeg, png, xlsx, csv etc. The download API on backend using pyramid framework is sending FileResponse as below:
def delivery_item_download_view(request, *args, **kw):
context = request.context
item_row = context.item_row
if item_row and item_row["deleted_at"] is None:
print(request.upload_dir+'/'+item_row["file_name"]+'.'+item_row["file_extension"])
response = FileResponse(
request.upload_dir+'/'+item_row["file_name"]+'.'+item_row["file_extension"],
request=request,
)
response.headers["attachment"] = item_row["name"];
return response
This, when executed using POSTMAN works as expected giving file as output. However,when tried implementing same using ReactJS, it's not working as expected. My client-code is as below:
onDownloadItem= (item) => {
console.log("item id is:", item.item_id)
var apiBaseUrl = "https://dev.incodax.com/api/deliveries_items/"+ item.item_id+ "/download";
fetch(apiBaseUrl, {
method: "GET",
}).then((res) => {
fileDownload(res,item.file_name)
console.log(res)
})
}
This fileDownload function simply downloading file but with no content inside. In downloaded file I could see something like:
[object Response]
I am getting 200 response from server. So I dont't think there is any issue with server side code. How can I handle it on client?
Thanks in advance
Will it suit you if you just redirected user to link to file? Browser will automatically handle it and download it.
The issue is in your fileDownload function which you do not post here. It's not clear what the first parameter is supposed to be but likely it is not the response object. Likely you at least need to pull the body out of the response and save that! The response body can be converted to a buffer object which could work (again it depends on what fileDownload is expecting):
res.arrayBuffer().then(buffer => {
fileDownload(buffer, item.file_name);
});

Uploading file to openstack object storage from JavaScript

I have a openstack object storage container to which I'm trying to upload files directly from browser.
As per the documentation here, I can upload the file using a PUT request and I'm doing this using Angularjs provided $http.put method as shown below.
$http.put(temporaryUploadUrl,
formData,
{
headers: {
'Content-Type': undefined
}
}
).then(function (res) {
console.log("Success");
});
The file uploads successfully and it has no problems in authentication and gives me a 201 Created response. However the file is now containing junk lines on the top and bottom of it because its a multipart request sent using FormData().
Sample file content before upload:
Some sample text
here is more text
here is some other text
File content after downloadiong back from openstack container :
------WebKitFormBoundaryTEeQZVW5hNSWtIqS
Content-Disposition: form-data; name="file"; filename="c.txt"
Content-Type: text/plain
Some sample text
here is more text
here is some other text
------WebKitFormBoundaryTEeQZVW5hNSWtIqS--
I tried the FileReader to read the selected file as a binary string and wrote the content to the request body instead of FormData and the request which works fine for text files but not the binary files like XLSX or PDF The data is entirely corrupted this way.
EDIT:
The following answer is now considered a less performing workaround As
it will encode the entire file to base64 multipart form data. I would
suggest go ahead with #georgeawg's Answer if you are not Looking for a
formData + POST solution
Openstack also provides a different approach using FormData for uploading one or more files in a single go as mentioned in this documentation. Funny this was never visible in google search.
Here is a brief of it.
First you need to generate a signature similar to tempUrl signature using the following python procedure.
import hmac
from hashlib import sha1
from time import time
path = '/v1/my_account/container/object_prefix'
redirect = 'https://myserver.com/some-page'
max_file_size = 104857600
max_file_count = 1
expires = 1503124957
key = 'mySecretKey'
hmac_body = '%s\n%s\n%s\n%s\n%s' % (path, redirect,
max_file_size, max_file_count, expires)
signature = hmac.new(key, hmac_body, sha1).hexdigest()
Then in your javascript call post to the container like this.
var formData = new FormData();
formData.append("max_file_size", '104857600');
formData.append("max_file_count", '1');
formData.append("expires", '1503124957');
formData.append("signature", signature);
formData.append("redirect", redirect);
formData.append("file",fileObject);
$http.post(
"https://www.example.com/v1/my_account/container/object_prefix",
formData,
{
headers: {'Content-Type': undefined},
transformRequest: angular.identity
}
).then(function (res) {
console.log(response);
});
Points to note.
The formData in POST request should contain only these
parameters.
The file entry in the formData should be the last one.(Not sure why
it doesnt work the other way around).
The formData content like path with prefix, epoch time, max file
size, max file count and the redirection urls should be the same as
the one which were used to generate the signature. Otherwise you will
get a 401 Unauthorized.
I tried the FileReader to read the selected file as a binary string and wrote the content to the request body instead of FormData and the request which works fine for text files but not the binary files like XLSX or PDF The data is entirely corrupted this way.
The default operation for the $http service is to use Content-Type: application/json and to transform objects to JSON strings. For files from a FileList, the defaults need to be overridden:
var config = { headers: {'Content-Type': undefined} };
$http.put(url, fileList[0], config)
.then(function(response) {
console.log("Success");
}).catch(function(response) {
console.log("Error: ", response.status);
throw response;
});
By setting Content-Type: undefined, the XHR send method will automatically set the content type header appropriately.
Be aware that the base64 encoding of 'Content-Type': multipart/form-data adds 33% extra overhead. It is more efficient to send Blobs and File objects directly.
Sending binary data as binary strings, will corrupt the data because the XHR API converts strings from DOMSTRING (UTF-16) to UTF-8. Avoid binary strings as they are non-standard and obsolete.

Posting to Microsoft Cognitive Face API using Ionic - AngularJS

Currently I am using Ionic to develop a Facial Recognition app.
I am using Microsoft's Face API for the same.
My problem is I keep getting Error 400: Decoding error, image format unsupported.
After doing some research, I ran into this link
Quoting START,
When you're submitting a binary payload, you should not base64 encode it. Here's what you might do instead..
URI uri = builder.build();
HttpPost request = new HttpPost(uri);
request.setHeader("Ocp-Apim-Subscription-Key", "...");
File file = new File("...");
FileEntity reqEntity = new FileEntity(file, ContentType.APPLICATION_OCTET_STREAM);
request.setEntity(reqEntity);
Quoting END
With Ionic/AngularJS, the captured image using Device Camera or selected from Image Gallery is returned in the form of base64 string. So I looked for ways to decode the data into Binary using this gist but still failed.
Here's my controller code where I make the API call.
$scope.postToAPI = function () {
// make http call to MS cognitive API
var formData = new FormData();
formData.append('file', $scope.picture);
$http.post($scope.cognitiveServices.endpoint, formData, {
headers: { 'Content-Type': 'application/octet-stream', 'Ocp-Apim-Subscription-Key':$scope.cognitiveServices.apiKey }
}).success(function (data) {
console.log(JSON.stringify(data));
alert("Awesome");
}).error(function (err) {
alert("Fail ->" + err.code + " " + err.message);
console.log("Some error occured");
});
}
I have
tried setting transform:angular.identity and Content-Type: undefined
tried sending image without Form Data (just a random trial)
tried converting base64 string to Uint8 Array (by following the solution to this SO question)
All my attempts so far have failed. Can someone explain what I am doing wrong?
P.S: This is my first time with Ionic framework.

Resources