Use a .db file with MS Chatbot - database

Here is my situation : I'm developing a Chatbot on Microsoft azure platform using Node.js. For the moment the bot messages are hard-coded in .json files.
I want to improve it by using calls to a database.
I have a SQLite database file working fine (I used a browser for SQLite and made my requests). But the problem is :
How do can I use my .db file from my project ? Is this possible to somehow "read" the database file from my dialogs and then make my request to get what I need from my database ?
I know that you can call a database with the chatbot, but the issue here is that I only have the file and nothing deployed to call.
Example of what the result should give :
"Hey chatbot, tell me about Mona Lisa"
This triggers the dialogs that will ask the database : "SELECT info FROM arts WHERE arts.title LIKE '%Mona Lisa%' ";
And send the result in session.send(results).
Thanks !
Note : I'm just an intern in my company, the database file is the only thing they gave me and I have to find a solution with it

I got the solution after some research :
First you need to install sqlite3 with npm for example, then use this at the beginning of your code :
var sqlite3 = require('sqlite3').verbose();
var path = require('path');
var db_path = path.resolve(__dirname, name_Of_Your_DB);
And then work on your file with the request you need :
var db = new sqlite3.Database(db_path, sqlite3.OPEN_READONLY,(err) => {
if (err) {
return console.error(err.message);
}
//console.log("Stuff that is processed only if no error happened.");
});
var req = "YOUR REQUEST";
db.get(req, [your_parameter],(err, row) => {
if (err) {
return console.error(err.message);
}
});
db.close((err) => {
if (err) {
return console.log(err.message);
}
});
The documentation about node.js and sqlite3 is quite complete :
http://www.sqlitetutorial.net/sqlite-nodejs/query/

Related

Cypress 10 and connecting to an Oracle database

So I've got a new Cypress 10 project, and I'm trying to integrate some functionality to allow me to make some basic database calls to our Oracle database (which is on a server I have direct access to, not running locally).
I've been following this guide which shows how to add the oracledb package as a Cypress plugin, but the method used (using the /plugin directory) has been depreciated in Cypress 10 so I can't follow the example exactly.
I've instead tried applying this logic using the Cypress plugin documentation as a guide and I think I have something that almost works, but I can't seem to connect to any database, even if the location is in my tnsnames.ora file (although I'm providing the connection string directly for this particular project).
Here's what my cypress.config.ts file looks like, with the code I've created (I'm using Cucumber in my implementation too, thus why those references are present here):
import { defineConfig } from "cypress";
import createBundler from "#bahmutov/cypress-esbuild-preprocessor";
import { addCucumberPreprocessorPlugin } from "#badeball/cypress-cucumber-preprocessor";
import createEsbuildPlugin from "#badeball/cypress-cucumber-preprocessor/esbuild";
const oracledb = require("oracledb");
oracledb.initOracleClient({ libDir: "C:\\Users\\davethepunkyone\\instantclient_21_6" });
// This data is correct, I've obscured it for obvious reasons
const db_config = {
"user": "<username>",
"password": "<password>",
"connectString": "jdbc:oracle:thin:#<hostname>:<port>:<sid>"
}
const queryData = async(query, dbconfig) => {
let conn;
try{
// It's failing on this getConnection line
conn = await oracledb.getConnection(dbconfig);
console.log("NOTE===>connect established")
return await conn.execute(query);
}catch(err){
console.log("Error===>"+err)
return err
} finally{
if(conn){
try{
conn.close();
}catch(err){
console.log("Error===>"+err)
}
}
}
}
async function setupNodeEvents(
on: Cypress.PluginEvents, config: Cypress.PluginConfigOptions ): Promise<Cypress.PluginConfigOptions> {
await addCucumberPreprocessorPlugin(on, config);
on("file:preprocessor", createBundler({
plugins: [createEsbuildPlugin(config)],
})
);
on("task", {
sqlQuery: (query) => {
return queryData(query, db_config);
},
});
return config;
}
export default defineConfig({
e2e: {
specPattern: "**/*.feature",
supportFile: false,
setupNodeEvents,
},
});
I've then got some Cucumber code to run a test query:
Then("I do a test database call", () => {
// Again this is an example query for obvious reasons
const query = "SELECT id FROM table_name FETCH NEXT 1 ROWS ONLY"
cy.task("sqlQuery", query).then((resolvedValue: any) => {
resolvedValue["rows"].forEach((item: any) => {
console.log("result==>" + item);
});
})
})
And here are the dependencies from my package.json:
"dependencies": {
"#badeball/cypress-cucumber-preprocessor": "^12.0.0",
"#bahmutov/cypress-esbuild-preprocessor": "^2.1.3",
"cypress": "^10.4.0",
"oracledb": "^5.4.0",
"typescript": "^4.7.4"
},
I feel like I'm somewhat on the right track as when I run the feature step above, the error I get back is:
Error===>Error: ORA-12154: TNS:could not resolve the connect identifier specified
This makes me think that it has at least called the node-oracledb package to generate the error but I can't really tell if I've made an obvious error or not (I'm pretty new to JS/TS). I know I've referenced the right path for the oracle instant client and it's been initialized correctly at least because Cypress points out a config error if the path is incorrect. I know the database paths work as well because we have an older Selenium implementation that can connect using the details I'm providing.
I think I'm just more curious to know if anyone has so far successfully implemented an oracledb connection with Cypress 10 or if someone who has a bit more Cypress experience can spot any obvious errors in my code as resources for this particular combination of packages seem to be non-existent (possibly because Cypress 10 is reasonably new).
NOTE: I am planning on switching to using environmental variables for the database connection information that will eventually be passed into the project - I just want to get a connection working first before I tackle that issue.
Oracle's C stack drivers like node-oracledb are not using Java so the JDBC connection string needs changing from:
"connectString": "jdbc:oracle:thin:#<hostname>:<port>:<sid>"
If you were using:
jdbc:oracle:thin:#mydbmachine.example.com:1521/orclpdb1
then your Node.js code should use:
connectString : "mydbmachine.example.com:1521/orclpdb1"
Since you're using the very obsolete SID syntax, check the node-oracledb manual for the solution if you can't use a service name: JDBC and Oracle SQL Developer Connection Strings.

Downloading an Excel file causes it to corrupt

I have a simple service on Angular 2 and Typescript that requests Excel files to a server and then opens a download file dialogue for the user. However, as it is currently, the file becomes corrupt when downloaded.
When downloaded, it opens fine in OpenOffice and derivates, but throws a "File is Corrupt" error on Microsoft Excel, and asks if the user wants to recover as much as it can.
When Excel is prompted to recover the file, it does so successfully, and the recovered Excel has all rows and data that is expected for the Excel file. Comparing the recovered file against opening the file in OpenOffice and derivates evidence no outstanding differences.
The concrete Excel I am trying to download is generated with Apache POI in a microservice, then passed to the main backend and finally served to the frontend for the user to download. Both the backend and microservice are written in Java, through Spark Framework.
I made some tests on the backends, and concluded the problem is not the report generation nor the data transfer:
Asking the microservice to save the generated Excel in a file within the server and then opening such file (hereby file A) in Excel shows that file A is not corrupted.
Asking the main backend server to save the Excel file that it receives from the microservice in a file within itself and then opening such file in Excel (hereby file B) shows that file B is not corrupted.
Downloading both file A and file B through FileZilla from their respective servers yields completely uncorrupted files.
As such, I believe it is safe to assume the Excel becomes corrupted somewhere between the time the file is received on the frontend and the time the user downloads such file. Additionally, the Catalina logs do not evidence any error that might potentially be happening.
I have read several posts that deal with the issue, including a bug report (https://github.com/angular/angular/issues/14083) that included a workaround via XMLHTTPRequest. However, none of the workarounds detailed were successful in solving my issue.
Attached is the code I am using to both obtain the Excel file from the backend and serve it to the user. I am including both an XMLHTTPRequest and an Angular http call (within comments) since those are the two main ways I have been trying to make this work. Additionally, please do take into account the code has been altered to remove information I do not wish to make public.
download(body) {
let reply = Observable.create(observer => {
let xhr = new XMLHttpRequest();
xhr.open('POST', 'URL', true);
xhr.setRequestHeader('Content-type', 'application/json;charset=UTF-8');
xhr.setRequestHeader('Accept', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
xhr.setRequestHeader('Authorization', 'REDACTED');
xhr.responseType = 'blob';
xhr.onreadystatechange = function () {
if(xhr.readyState === 4) {
if(xhr.status === 200) {
var contentType = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
var blob = new Blob([xhr.response], { type: contentType });
observer.next(blob);
observer.complete();
}
else {
observer.error(xhr.response);
}
}
}
xhr.send(JSON.stringify(body));
});
return reply;
/*let headers = new Headers();
headers.set("Authorization", 'REDACTED');
headers.set("Accept", 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
let requestOptions :RequestOptions = new RequestOptions({headers: headers, responseType: ResponseContentType.Blob});
return this.http.post('URL', body, requestOptions);*/
}
Hereby is the code to prompt the user to download the Excel. It is currently made to work with the XMLHTTPRequest. Please do note that I have also attempted to download without resorting to FileSaver, with no luck.
downloadExcel(data) {
let body = {
/*REDACTED*/
}
this.service.download(body)
.subscribe(data => {
FileSaver.saveAs(data, "Excel.xlsx");
});
}
Hereby are the versions of the tools I am using:
NPM: 5.6.0
NodeJs: 8.11.3
Angular JS: ^6.1.0
Browsers used: Chrome, Firefox, Edge.
Any help on this issue would be appreciated. Any additional information you may need I will be happy to provide.
I think what you want is CSV format which open in Excel, update your sevice as follow:
You should tell Angular you are expecting a response of type blob (Binary Large Object) that is your Excel/Csv file.
Also make sure the URL/API on your server is set to accept content-type='text/csv'.
Here's an example with Angular 2.
#Injectable()
export class YourService {
constructor(private http: Http) {}
download() { //get file from the server
this.http.get("http://localhost/..", {
responseType: ResponseContentType.Blob,
headers: new Headers({'Content-Type', 'text/csv'})
}).subscribe(
response => {
var blob = new Blob([response.blob()], {type: 'text/csv'});
FileSaver.saveAs(blob, 'yourFileName.csv');
},
error => {
console.error('something went wrong');
}
);
}
}
Have you tried uploading/downloading your xls file as base64?
var encodedXLSToUpload = 'data:application/xls;base64,' + btoa(file);
Check this for more details: Creating a Blob from a base64 string in JavaScript

How to create and update a text file using React.js?

I am trying to save a variable's data into a text file and update the file every time the variable changes. I found solutions in Node.js and vanilla JavaScript but I cannot find a particular solution in React.js.
Actually I am trying to store Facebook Long Live Access Token in to a text file and would like to use it in the future and when I try importing 'fs' and implementing createFile and appendFile methods I get an error saying Method doesn't exist.
Please help me out. Here is the code below
window.FB.getLoginStatus((resp) => {
if (resp.status === 'connected') {
const accessToken = resp.authResponse.accessToken;
try {
axios.get(`https://graph.facebook.com/oauth/access_token?client_id=CLIENT_id&client_secret=CLIENT_SECRET&grant_type=fb_exchange_token&fb_exchange_token=${accessToken}`)
.then((response) => {
console.log("Long Live Access Token " + response.data.access_token + " expires in " + response.data.expires_in);
let longLiveAccessToken = response.data.access_token;
let expiresIn = response.data.expires_in;
})
.catch((error) => {
console.log(error);
});
}
catch (e) {
console.log(e.description);
}
}
});
React is a frontend library. It's supposed to be executed in the browser, which for security reasons does not have access to the file system. You can make React render in the server, but the example code you're showing is clearly frontend code, it uses the window object. It doesn't even include anything React-related at first sight: it mainly consists of an Ajax call to Facebook made via Axios library.
So your remaining options are basically these:
Create a text file and let the user download it.
Save the file content in local storage for later access from the same browser.
Save the contents in online storage (which could also be localhost).
Can you precise if any of these methods would fit your needs, so I can explain it further with sample code if needed?

Avoid duplicated publication in Meteor

I'm trying to export some data into a CSV file from a MySQL database using Meter/Sequelize. What I've done so far is to create a Meteor method called by the client which then call a server side function that return the data and I parse it into a csv string. My issue is returning the date client-side.
What I did
I have my CSV String server-side and I'm using FileSaver.js which can only be used client-side.
My "solution" was to create a client-side collection in which I published the String.
methods.js
run({exportParam}) {
if (!this.isSimulation) {
query.booksQuery(exportParam.sorted, exportParam.filtered, 0).then(
result => {
let CSVArr = [];
result.rows.forEach((value) => {
CSVArr.push(value.dataValues);
});
const CSVString = Baby.unparse(CSVArr,{ delimiter: ";"});<-CSV String
console.log("CSVString : ", CSVString);
Meteor.publish("CSVString", function() { <= publication
this.added("CSVCollection", Random.id(), {CSVString: CSVString});
this.ready();
});
});
}
},
And on the client-side I subscribe to the publication this way :
ExportButton.jsx
const handle = Meteor.subscribe('CSVString', {}, function() {
const exportString = myTempCollection.findOne().CSVString;
const blob = new Blob([exportString], {type:"text/plain;charset=utf
8"});
FileSaver.saveAs(blob, "test.csv");
});
My issue
It works great the first time I click my button and a CSV file is downloaded. The problem is that if I do it again I get the same file as the first one and I get this message on my console.
Ignoring duplicate publish named 'CSVString'
I'm pretty sure the problem comes from the fact that every time I click the button the same "CSVString" publication is created.
I'd like to know to know if there is a solution to this problem or if my approach is wrong.
Please let me know if you need anything else.
You are correct in assuming that you are trying to publish to the same collection every time. I think you should only do the publish once, and do that separately from inserting a record into the collection.

Cordova + Sencha Touch: Delete file after it has been used

So I'm using Cordova + Sencha Touch for an app and Antair's SQLitePlugin (https://github.com/Antair/Cordova-SQLitePlugin) to import and use an SQLite database in it.
I managed to import my (kinda big) prepopulated database using Antair's importPrepopulatedDatabase ( window.sqlitePlugin.importPrepopulatedDatabase({file:"mydb.db",importIfExists:false}) ) method and it works just fine. The thing is I noticed the app is using twice the size it really needs as it keeps the file after importing it.
I checked and the app works just fine if I delete the file from /cordova/www/db and build again, it keeps the actual db in the app's filesystem I guess, but I can't find a way to programmatically delete that file after it has been imported.
I looked around and found cordova file plugin (https://github.com/apache/cordova-plugin-file/blob/master/doc/index.md), but from what I saw from the docs it only grants read permissions on the www folder, so that won't do it.
Does anyone have any workaround for this? I could really use that extra space.
By using cordova file plugin api you can do this,
please refer this :
deleteFile: function(fileName) {
var that = this;
if (!fileName) {
console.error("No fileName specified. File could not be deleted.");
return false;
}
window.requestFileSystem(LocalFileSystem.TEMPORARY, 0, function(fileSystem){ // this returns the tmp folder
// File found
fileSystem.root.getFile(fileName, {create: false}, function(fileEntry){
fileEntry.remove(function(success){
console.log(success);
}, function(error){
console.error("deletion failed: " + error);
});
}, that.get('fail'));
}, this.get('fail'));
}

Resources