How to fetch a CSV file from API in google sheets? - database

I am new to API and google script, so please bare with me.
How do i fetch a CSV file through an API in the right format.
Through UrlFetchApp i was able to fetch the data, but in the wrong format, since the data is not seperated into cells.
function callNumbers() {
var response = UrlFetchApp.fetch("https://api.statbank.dk/v1/data/VG2/CSV?valuePresentation=Default&allowVariablesInHead=true&K%C3%98RART=1000&M%C3%86NGDE4=0070&Tid=2020K3%2C2020K2%2C2020K1");
var data = response.getContentText()
Logger.log(data);
var sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(1,1).setValue([data]);
}
How do i separate the data into rows (defined by line breaks) and column (defined by the semicolon)?
Please help I am so Lost!

Related

Save as .PARQUET file in Logic App workflow

How can I save the output file from Run query and list results in a .PARQUET file format.
This is my current workflow.
My Logic App is working, But the file .parquet created are not valid every time I view it on Apache Parquet Viewer
Can someone help me on this matter. Thank you!
Output:
I see that you are trying to add .parquet to the csv file you are receiving but that's not how it will be converted to parquet file.
One of the workarounds that you can try is to get the csv file and then add Azure function which can convert into parquet file and then adding the azure function to logic app.
Here is the function that worked for me:
BlobServiceClient blobServiceClient = new BlobServiceClient("<YOUR CONNECTION STRING>");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("<YOUR CONTAINER NAME>");
BlobClient blobClient = containerClient.GetBlobClient("sample.csv");
//Download the blob
Stream file = File.OpenWrite(#"C:\Users\<USER NAME>\source\repos\ParquetConsoleApp\ParquetConsoleApp\bin\Debug\netcoreapp3.1\" + blobClient.Name);
await blobClient.DownloadToAsync(file);
Console.WriteLine("Download completed!");
file.Close();
//Read the downloaded blob
Stream file1 = new FileStream(blobClient.Name, FileMode.Open);
Console.WriteLine(file1.ReadToEnd());
file1.Close();
//Convert to parquet
ChoParquetRecordConfiguration csv = new ChoParquetRecordConfiguration();
using (var r = new ChoCSVReader(#"C:\Users\<USER NAME>\source\repos\ParquetConsoleApp\ParquetConsoleApp\bin\Debug\netcoreapp3.1\" + blobClient.Name))
{
using (var w = new ChoParquetWriter(#"C:\Users\<USER NAME>\source\repos\ParquetConsoleApp\ParquetConsoleApp\bin\Debug\netcoreapp3.1\convertedParquet.parquet"))
{
w.Write(r);
w.Close();
}
}
after this step you can publish to your azure function and add the Azure function connector to your logic app
You can skip the first 2 steps (i.e.. Read and Download the blob) and get the blob directly from logic app and send it to your azure function and follow the same method as above. The generated parquet file will be in this path.
C:\Users\<USERNAME>\source\repos\ParquetConsoleApp\ParquetConsoleApp\bin\Debug\netcoreapp3.1\convertedParquet.parquet
Here convertedParquet.parquet is the name of the parquet file. Now you can read the converted parquet file in Apache Parquet reader.
Here is the output

Google apps script wont select specific sheet in spreadsheet

I have multiple sheets in the spreadsheets and the code I have works only when the sheet is on first and but on other positions
var doc = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var sheetName = '管理_201907_CUD';
var sheet = doc.getSheetByName(sheetName);
if i were to run the function solely it would work
function test(){
SpreadsheetApp.getActiveSpreadsheet()
.getSheetByName('管理_201907_CUD').getRange('A1')
.setValue('test');
}
but then again the data i have is coming from a Reactjs web app using XHR
it works but only when the sheet is on the first place and not on other positions
Normally it should be var sheetName = e.parameters.formGoogleSheetName;
e.parameters.formGoogleSheetName takes it value from the web app but for now I directly inputted the sheetname
i just want my sheet to work even though its on a different place
from this
to this

How to send parsed .csv file as a byte array or ArrayBuffer data from Node.js backend to AngularJS frontend?

I'm working on AngularJS app.
Module I'm currently working on should be able to either show a preview of a spreadsheet file or allow to download it.
The steps:
When clicked on "Preview File" it should send request with needed file's name as a parameter of POST request.
Backend will find neede file, which is a .csv file, convert it to byte array type and send it to frontend.
Frontend should handle this byte array and convert it to .xls or .xlsx filetype
The spreadsheet data should be opened in some small preview read-only window, like 1000x1000 px.
The POST request line looks like that:
this.$http.post(this.url + 'endpoint/getFile', params,
{responseType: "arraybuffer", showLoadingOverlay: true}
)
The response looks indeed like ArrayBuffer: three of it in one object, i.e. Uint8Array, Uint16Array and Uint32Array.
The code which should read this and convert to content suitable for preview is not working:
const byteArray = new Uint8Array(data);
const blob = new Blob([byteArray], { type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' });
const objectUrl = URL.createObjectURL(blob);
this.$window.open(objectUrl, 'C-Sharpcorner', 'width=1000,height=1000');
Because when created the blob, it already has 0 length in bytes, so there's no data inside.
The matter of visualising the .xls in browser window, I think, can be achieved with canvas-datagrid library. Haven't used but it looks cool.
Also, I have a problem with trying to set up a mock data for node.js (and AngularMock), for local testing when there's no data on a java backend.
I'm using 'fs' and 'csv-parse':
const fs = require('fs');
const csvParse = require("csv-parse/lib/es5");
module.exports = function stir(app) {
const getFile = () => {
const csvOutput = csvParse('../static/someData.csv', (parsed) => {
return parsed;
});
fs.readFileSync(csvOutput);
};
app.post('/stir/getFile', (req, res) => res.json(getFile()));
};
Which results in error:
TypeError: path must be a string or Buffer
What is the proper way of parsing the .csv using 'csv-parse' and sending parsed data as an ArrayBuffer to frontend in Node and AngularMock?
csv-parse docs are telling that underneath, the lib will convert the parsed object to node stream.
So why that error happens?

Google apps script, openByURL returns missing file?

I've been trying to figure out why part of my Google App script doesn't work, but I've failed to come up with an answer.
The script is downloading an attachment, CSV, from an email in Gmail and stores in with a specific name in a specific folder - this works perfectly fine.
But then I want to edit the CSV, and this is where I run into problems.
var newFolderIterator = DriveApp.getFoldersByName(destinationFolderName)
var newFolderId, myFileName, myFileId
while(newFolderIterator.hasNext()) {
newFolderId = newFolderIterator.next().getId()
var newFileList = DriveApp.getFolderById(newFolderId).getFiles()
while(newFileList.hasNext()) {
myFileName = newFileList.next()
myFileId = myFileName.getId()
var myFileURL = myFileName.getUrl()
Logger.log(myFileName.getId() + " " + myFileName.getName()) //Logs the ID and Name
Logger.log(myFileURL) //Logs the URL
var ss = SpreadsheetApp.openById(myFileName.getId()) //Error, cannot find the ID (error message: perhaps it's missing?)
}
}
I've tried using the openByURL as well, with the same error message.
Probably really easy to fix, any hints and tips is appreciated.
Thanks
The problem here is you are uploading a CSV but attempting to open it with SpreadsheetApp. SpreadsheetApp can only open Google Sheets documents, and your CSV is not automatically converted.
The CSV must first be converted to a Google Sheets document before you can access it with SpreadsheetApp.
You may be able to do this on upload, but I haven't tried this personally. This question looks relevant:
How to automatically import data from uploaded CSV or XLS file into Google Sheets

parsing Facebook Data fetched with UrlFetchApp.fetch

I'm building a little script to import Facebook data directly fetched from the Facebook Api in a Google Sheet. I've been able to successfully authenticate and pass the response of the API call in my Google App script log. However I don't really know how should I proceed to actually pass this data in my sheet.
The following script allow me to log the data from the fb api :
function myFunction() {
var response = UrlFetchApp.fetch("https://graph.facebook.com/v2.3/page_id/insights/page_impressions?access_token=mytoken");
var json = response.getContentText(); //
var rawdata = JSON.parse(json);
var rawdata = Utilities.jsonParse(response.getContentText());
Logger.log(rawdata);
then I think what I should do is pushing the data I need in a new array with the push method :
var data = [];
data.push(rawdata.???) // don't really the name I need to access the data I want
SpreadsheetApp.getActiveSheet().appendRow(data)
}
My question :
Since I don't really know how to read the data fetched from the facebook api, I don't really know how to push them in my data array. Here is the result of my log :
[16-11-21 10:01:33:622 PST] {data=[{period=day,
values=[{end_time=2016-11-17T08:00:00+0000, value=26023596},
{end_time=2016-11-18T08:00:00+0000,value=24447386},
{end_time=2016-11-19T08:00:00+0000, value=31057386}],
name=page_impressions, description=Daily: The number of impressions seen of any content associated with your Page. (Total Count),
id=page_id/insights/page_impressions/day,
title=Daily Total Impressions},
{period=week,
values=[{end_time=2016-11-17T08:00:00+0000, value=233007217},{end_time=2016-11-18T08:00:00+0000, value=200263630},
{end_time=2016-11-19T08:00:00+0000, value=194289364}],
name=page_impressions,
description=Weekly: The number of impressions seen of any content associated with your Page. (Total Count),
id=page_id/insights/page_impressions/week, title=Weekly Total Impressions},
{period=days_28, values=[{end_time=2016-11-17T08:00:00+0000, value=867302439},
{end_time=2016-11-18T08:00:00+0000, value=868201060}, {end_time=2016-11-19T08:00:00+0000, value=874965509}],
name=page_impressions, description=28 Days: The number of impressions seen of any content associated with your Page. (Total Count),
id=page_id/insights/page_impressions/days_28, title=28 Days Total Impressions}],
paging={next=https://graph.facebook.com/v2.3/page_id/insights/page_impressions?access_token=my_token,
previous=https://graph.facebook.com/v2.3/page_id/insights/page_impressions?access_token=my_token}}
If an example is needed to illustrate an answer let's say I want the Daily Total Impressions for the three days facebook gave me.
I hope my issue is clear enough. Not sure which tag should I use for this question. Correct me if I'm wrong.
thanks !
You need to decide what data to push into an array to then drop into a sheet. I would first drop your raw log file into a JSON prettifier so it's easy to see the formatting. Then, take a look at how to manipulate JSON using apps script.

Resources