How to send parsed .csv file as a byte array or ArrayBuffer data from Node.js backend to AngularJS frontend? - arrays

I'm working on AngularJS app.
Module I'm currently working on should be able to either show a preview of a spreadsheet file or allow to download it.
The steps:
When clicked on "Preview File" it should send request with needed file's name as a parameter of POST request.
Backend will find neede file, which is a .csv file, convert it to byte array type and send it to frontend.
Frontend should handle this byte array and convert it to .xls or .xlsx filetype
The spreadsheet data should be opened in some small preview read-only window, like 1000x1000 px.
The POST request line looks like that:
this.$http.post(this.url + 'endpoint/getFile', params,
{responseType: "arraybuffer", showLoadingOverlay: true}
)
The response looks indeed like ArrayBuffer: three of it in one object, i.e. Uint8Array, Uint16Array and Uint32Array.
The code which should read this and convert to content suitable for preview is not working:
const byteArray = new Uint8Array(data);
const blob = new Blob([byteArray], { type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' });
const objectUrl = URL.createObjectURL(blob);
this.$window.open(objectUrl, 'C-Sharpcorner', 'width=1000,height=1000');
Because when created the blob, it already has 0 length in bytes, so there's no data inside.
The matter of visualising the .xls in browser window, I think, can be achieved with canvas-datagrid library. Haven't used but it looks cool.
Also, I have a problem with trying to set up a mock data for node.js (and AngularMock), for local testing when there's no data on a java backend.
I'm using 'fs' and 'csv-parse':
const fs = require('fs');
const csvParse = require("csv-parse/lib/es5");
module.exports = function stir(app) {
const getFile = () => {
const csvOutput = csvParse('../static/someData.csv', (parsed) => {
return parsed;
});
fs.readFileSync(csvOutput);
};
app.post('/stir/getFile', (req, res) => res.json(getFile()));
};
Which results in error:
TypeError: path must be a string or Buffer
What is the proper way of parsing the .csv using 'csv-parse' and sending parsed data as an ArrayBuffer to frontend in Node and AngularMock?
csv-parse docs are telling that underneath, the lib will convert the parsed object to node stream.
So why that error happens?

Related

how to read the contents of a json file returned from s3 storage as a blob in React

I have successfully retrieved a json file from s3 storage. It is returned as a blob. I am able to turn the blob into text with this code (taken from https://docs.amplify.aws/lib/storage/download/q/platform/js/#monitor-progress-of-download):
export async function getS3Item(filename)
{
const result = await Storage.get(filename, { download: true });\
result.Body.text().then(string => {
// // handle the String data return String
console.log(string)
});
}
but the text is all gibberish (I'm assuming since object is in binary?)... such as: "h�b```f�d`a}��ǀ|#1V ..."
Is there a way I can directly read this as a json object in javascript so that I can extract data from it...?
Optionally, I can download the json file (which is shown in the link above and I've gotten this to work) -- but I'd prefer not to download it -- just to extract legible data from the file
thanks so much (I'm quite unfamiliar with blobs).

Send .mat file through Django Rest Framework

I have an issue to send the contents of a .mat file to my frontend. My end goal is to allow clients to download the content of this .mat file at the click of a button so that they end up with the same file in their possession. I use Next.js + Django Rest Framework.
My first try was as follow:
class Download(APIView):
def get(self, request):
with open('file_path.mat', 'rb') as FID:
fileInstance = FID.read()
return Response(
fileInstance,
status=200,
content_type="application/octet-stream",
)
If I print out the fileInstance element I get some binary results:
z\xe1\xfe\xc6\xc6\xd2\x1e_\xda~\xda|\xbf\xb6\x10_\x84\xb5~\xfe\x98\x1e\xdc\x0f\x1a\xee\xe7Y\x9e\xb5\xf5\x83\x9cS\xb3\xb5\xd4\xb7~XK\xaa\xe3\x9c\xed\x07v\xf59Kbn(\x91\x0e\xdb\xbb\xe8\xf5\xc3\xaa\x94Q\x9euQ\x1fx\x08\xf7\x15\x17\xac\xf4\x82\x19\x8e\xc9...
But I can't send it back to my frontend because of a
"UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 137: invalid start byte"
This error is always the same regardless of which .mat file I try to send in my response.
Next I tried to use the scipy.io.loadmat() method. In this case, fileInstance gives me a much more readable dictionary object, but I still can't get it to transfer to the frontend because of the presence of NaN in my dict:
ValueError: Out of range float values are not JSON compliant
Finally, some suggested to use h5py to send back the data as such:
with h5py.File('file_path.mat', 'r') as fileInstance:
print(fileInstance)
But in that case the error I get is
Unable to open file (file signature not found)
I know my files are not corrupted because I can open them in Matlab with no problem.
With all this trouble I'm wondering if I'm using the right approach to this problem. I could technically send the dictionary obtained through 'scipy.io.loadmat()' as a str element instead of binary, but I'll have to figure out a way to convert this text back to binary inside a Javascript function. Would anybody have some ideas as to how I should proceed?
The problem was in my frontend after all. Still, here's the correct way to go about it:
class Download(APIView):
parser_classes = [FormParser, MultiPartParser]
def get(self, request):
try:
file_path = "xyz.mat"
response = FileResponse(file_path.open("rb"), content_type="application/octet-stream")
response["Content-Disposition"] = f"attachment; filename=file_name"
return response
except Exception as e:
return Response(status=500)
This should send to the frontend the right file in the right format. No need to worry about encoding and such.
Meanwhile, on the frontend you should receive the file as follows:
onClick={() => {
const url = '/url_to_your_api/';
axios({ method: 'get', url: url, responseType: 'blob' })
.then((response) => {
const { data } = response;
const fileName = 'file_name';
const blob = new Blob([data], { type: 'application/octet-stream' });
const href = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = href;
link.download = fileName + '.mat';
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
URL.revokeObjectURL(href);
})
.catch((response) => {
console.error(response);
});
}}
Long story short, the part I was missing was to specify to receive the data as blob inside the 'onClick()' function. By default, responseType from Axios is set to Json/String. For that reason, my file was modified at reception and would not be usable in matlab afterwards. If you face a similar problem in the future, try to use the 'shasum' BASH function to observe the hashed value of the file. It is with the help of that function that I could deduce that my API function would return the correct value and that therefore the problem was happenign on the frontend.

Byte array cannot be converted to a pdf in react native

I'm currently working on a react-native project. My goal is to get a byte array through an API call and turn it into a pdf and then save it inside a mobile device (currently I'm using an android device) as a pdf file.
Here is the code I currently use.
import RNFS from "react-native-fs";
const basePath = RNFS.DocumentDirectoryPath + "/";
// Service class is created in another file and when the Api call is called, I get the byte array in call back function
serviceClass.apiCall(param1, async (loadedData) => { // loadedData contains the byte array
const tempPath = basePath + param1 + '.pdf';
//const tempPath = basePath + param1 + '.txt'; // Used for download the txt file
await RNFS.writeFile(tempPath, loadedData.data);
await FileViewer.open(tempPath, {onDismiss : () => {
RNFS.unlink(tempPath); // delete the file after reading it
}});
}
I have a configuration to get a byte array as a text file. When I try the text version, It works fine. But when I use the pdf byte array, I get a blank screen in the created pdf. This is the problem I want to solve. Any help on this issue would be really appreciated. Thanks in advance.
Note: when I run the API call from swagger, I can download the byte array as a pdf file and I can see the data in my browser without any doubt. The issue only occurs when the pdf is downloaded within the mobile app. And I cannot change the API call. The only option is to work with the byte array.

How to send a local image instead of URL to Computer Vision API using React

I would like to upload local image file and extract text from it. I followed the below link and it works as expected when I pass URL. https://learn.microsoft.com/en-us/azure/developer/javascript/tutorial/static-web-app/add-computer-vision-react-app
I managed to configure for local image and get the base64 encoded dataURL of the uploaded image. But when I pass base64 encoded dataURL to Computer Vision API , it says "Input data is not a valid image" (POST 400 status code). I am getting error in the line that is shown below:
const analysis = await computerVisionClient.analyzeImage(urlToAnalyze, { visualFeatures });
The code I have included for handling local image:
const handleChange = (e) => {
var file = e.target.files[0];
var reader = new FileReader();
reader.onloadend = function()
{
setFileSelected(reader.result) // this is the base64 encoded dataurl
}
reader.readAsDataURL(file);
}
In computerVision.js file, I have changed the 'contentType' in header as below.
const computerVisionClient = new ComputerVisionClient(
new ApiKeyCredentials({ inHeader: {'Ocp-Apim-Subscription-Key': key, 'Content-Type': 'application/octet-stream'} }), endpoint);
I tried replacing client.read() with readTextInStream() as per docs in computerVision.js (please refer above link), but still throws error.
May I know why I get the error "Input data is not a valid image" ? Thanks.
Here is the link for input requirements.
There is a brand new online portal provided by Microsoft https://preview.vision.azure.com/demo/OCR
The advantage is that it will directly list your available resources so you just have to pick the right one, then you test, and there are also some samples.

Is it possible to directly upload images captured by camera to Firebase Storage?

I'm using React.js to create an application that would take a photo and upload it to Firebase Storage. I am using the react-webcam library, which uses this command to take a photo:
const ImageSrc = webcamRef.current.getScreenshot();
This is how I tried uploading the photo to Storage:
storage.ref(`/images`).put(imageSrc)
.on("state_changed" , alert("success") , alert)
However, the file that is uploaded is undefined (no photo).
I tried to construct an URL of the photo using blob:
const imageUrl = window.URL.createObjectURL(new Blob(webcamRef.current.getScreenshot()))
But I get this error: >Failed to construct 'Blob': The provided value cannot be converted to a sequence.
In the library it is stated that getScreenshot - Returns a base64 encoded string of the current webcam image. So, I tried to use the atob command, but I get the error: Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded.
Does anyone know how I could upload the image to Firebase Storage? Any help would be appreciated!
Instead of blob, try using putString() command like this:
const task = firebase.storage().ref(`/images`).putString(imageSrc, 'data_url')
As explained in the doc, if you want to upload from a Base64url formatted string, you need to call the putString() method as follows (example from the doc):
var message = '5b6p5Y-344GX44G-44GX44Gf77yB44GK44KB44Gn44Go44GG77yB';
ref.putString(message, 'base64url').then((snapshot) => {
console.log('Uploaded a base64url string!');
});
In your case, since getScreenshot() returns a base64 encoded string, it would be something like:
const imageSrc = webcamRef.current.getScreenshot();
storage.ref(`/images`).putString(imageSrc, 'imgBase64')
.on("state_changed" , alert("success") , alert)

Resources