Node.js SQL server crashes when receiving multiple requests - sql-server

I have a NodeJS application which is my server and I created a Database class to help me handle querying my SQL DB. If I send requests a second between each other, everything runs fine.. no problems.. But if I start spamming requests to my server it crashes due to Error: Cannot enqueue Quit after invoking quit.
Here's my query function inside my Database class
static query(query: string): Promise<any> {
console.log('Query: ' + query);
return new Promise((resolve, reject) => {
this.connect().then(success => {
sqlConn.query(query, (err, results) => {
if (err) { return reject(err);
} else {
return resolve(results);
}
});
}).catch(err => {
return reject(err);
}).then( () => {
if (sqlConn.state !== 'disconnected') {
sqlConn.end();
}
});
});
};
and here's the this.connect() function
static connect(): Promise<any> {
return new Promise((resolve, reject) => {
sqlConn = mysql.createConnection(this.connectionData);
sqlConn.connect(err => {
if (err) { return reject(err); } else {
return resolve('SQL connection established');
}
});
});
};
I'm pretty sure the problem appears sometimes, it would still be
processing one query, and then another query comes before the first
one finishes, so it would call sqlConn.end() twice, even when it's
already disconnected? Any help is greatly appreciated...
> Main goal is for the query to wait till it's 100% done before it runs
the next one..

You can simplify your code by using the npm module mysql and use it's built-in connection pool.
From the documentation:
var mysql = require('mysql');
var pool = mysql.createPool({
connectionLimit : 10,
host : 'example.org',
user : 'bob',
password : 'secret',
database : 'my_db'
});
pool.query('SELECT 1 + 1 AS solution', function (error, results, fields) {
if (error) throw error;
console.log('The solution is: ', results[0].solution);
});
You can, of course, create your own function that promisifies that call like this:
function query (sql) {
return new Promise((resolve, reject) => {
pool.query(sql, (error, results, fields) =>
error ? reject(error) : resolve({ results, fields });
};
}

If you really wants to use this approach then please use eachSeries function of async library.
var chunkedArray= [];
async.eachSeries(chunkedArray, startUpload, endUpload);
funtion startUpload(data,cb){
//iterate over every single item in array 1 at a time
}
function endUplaod(err){
//finally call this
}
This might help:-
https://caolan.github.io/async/docs.html#eachSeries
But i rather suggest you to use pooling of connection which make less overhead on your db and you can use your mysql more efficiently then making multiple connection.
// Load module
var mysql = require('mysql');
// Initialize pool
var pool = mysql.createPool({
connectionLimit : 10,
host : '127.0.0.1',
user : 'root',
password : 'root',
database : 'db_name',
debug : false
});
module.exports = pool;

Related

Retrieve collection + subcollection give me an empty render

The problem
At web app startup, the main task is to retrieve all user informations that are stored in various Firestore's collections and sub-collections.
The problem is that, even if I uses promises and a Loading State to prevent an empty app rendering, the app is rendered with all collections data, except for collections that have sub-collections.
The process
- If user is logged
-- Set Loading Status Active
-- Load Collection A
-- Load Collection B and forEach, load all sub-collection
-- Load Collection C
-- Set Loading Status Inactive
At this point, the app is rendered, but only with Collection A and Collection C. The collection B is loaded (I can see it by Redux Logs) but can't be seen in the app.
Those datas appears only if I change the component status (open/close a menu, for example).
Some code
Here's how I retrieve a collection with sub-collections:
export function setCompanyJobs(user) {
return {
type: "SET_COMPANY_JOBS",
payload: loadCompanyJobs(user),
};
}
Function that retrieve the main collection
export function loadCompanyJobs(user) {
return new Promise((resolve, reject) => {
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
loadJobLinkedServices(user, job).then((jobLinkedServices) => {
companyJobs.push({
id: job.id,
...
});
});
});
resolve(companyJobs);
}).catch(function (error) {
...
});
});
}
Function that retrieve all collection's sub-collections
export function loadJobLinkedServices(user, job){
return new Promise((resolve, reject) => {
let jobLinkedServices = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").doc(job.id).collection("linkedServices").get().then((linkedServices) => {
linkedServices.forEach((linkedService) => {
jobLinkedServices.push({
id: linkedService.id,
...
});
});
resolve(jobLinkedServices)
}).catch(function (error) {
...
});
})
When you do
return new Promise((resolve, reject) => {
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
loadJobLinkedServices(user, job).then((jobLinkedServices) => {
companyJobs.push({
id: job.id,
...
});
});
});
resolve(companyJobs);
}).catch(function (error) {
...
});
});
nothing ensure that your Promise resolves only after ALL the queries triggered in the jobs.forEach() loop are done (i.e. the promises returned by the calls to the loadJobLinkedServices function have resolved).
I don't know reactjs but I think you can use the JavaScript Promise.all() method along the following lines:
return new Promise((resolve, reject) => {
let promises = [];
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
promises.push(loadJobLinkedServices(user, job));
});
Promise.all(promises).
then(results => {
//Loop over the results array to populate the companyJobs array
resolve(companyJobs);
})
}).catch(function (error) {
...
});
});
Also, don't forget to correctly chain your calls to the different asynchronous functions, i.e. something like:
query Collection A
THEN query Collection B
THEN query all sub-collections (with Promise.all())
THEN query Collection C
THEN set Loading Status Inactive
Finally, a last remark: Note that the get() method returns a promise, so I am not sure that you need to wrap the calls to the get() method into some new Promises (Again, I am not versed in reactjs, so this remark may be wrong).
In other words, I think you could do something like the following (for example for the loadJobLinkedServices function):
export function loadJobLinkedServices(user, job){
let jobLinkedServices = [];
return db.collection("company").doc(user.selectedCompany).collection("jobs").doc(job.id).collection("linkedServices").get()
.then((linkedServices) => {
linkedServices.forEach((linkedService) => {
jobLinkedServices.push({
id: linkedService.id,
...
});
});
return jobLinkedServices;
}).catch(function (error) {
...
});
})

How to Run an API Calls in Parallel (Node.js)

I am trying to run some API calls in parallel, but am having problems since I am trying to call a function again before the API data has been returned.
I am thinking that I could possibly use the new command in Node, but am not sure how to structure it into this scheme. I am trying to avoid recursion, as I already have a recursive version working and it is slow.
Currently I am trying to this code on the server.
loopThroughArray(req, res) {
for(let i=0; i<req.map.length; i++) {
stack[i] = (callback) => {
let data = getApi(req, res, req.map[i], callback)
}
}
async.parallel(stack, (result) => {
res.json(result)
})
}
....
function getApi(req, res, num, cb) {
request({
url: 'https://example.com/api/' + num
},
(error, response, body) => {
if(error) {
// Log error
} else {
let i = {
name: JSON.parse(body)['name'],
age: '100'
}
console.log(body) // Returns empty value array.length > 1 (req.map[i])
cb(i)
}
})
Is there a way to spawn new instances of the function each time it's called and accumulate the results to send back as one result to the client?
Here's an example of calling Web APIs (each with different parameters), using the Async library, we start by creating an array of N function variables.
const async = require('async');
const request = require('request');
//Set whatever request options you like, see: https://github.com/request/request#requestoptions-callback
var requestArray = [
{url: 'https://httpbin.org/get'},
{url: 'https://httpbin.org/ip'}
];
let getApi = function (opt, callback) {
request(opt, (err, response, body) => {
callback(err, JSON.parse(body));
});
};
const functionArray = requestArray.map((opt) => {
return (callback) => getApi(opt, callback);
});
async.parallel(
functionArray, (err, results) => {
if (err) {
console.error('Error: ', err);
} else {
console.log('Results: ', results.length, results);
}
});
You can easily switch the Url and Query values to match whatever you need. I'm using HttpBin here, since it's good for illustrative purposes.

Global connection already exists. Call sql.close() first

I'm trying to build a node.js backend,I have a usecase that i should connect everytime to the server and not to the database ,see one of the webservices:
router.get('/CriticalityGraph/:server/:user/:password/:database/', function(req, res, next) {
user = req.params.user;
password = req.params.password;
server = req.params.server;
database = req.params.database;
criticalityState=req.params.criticalityState
// config for your database
var config = {
user: user,
password: password,
server: server,
database:database
};
sql.connect(config, function (err) {
var request = new sql.Request();
request.query("SELECT MachineName, alarmState, criticality FROM MachineTable  ORDER BY criticality DESC"
, function (err, recordset) {
if (err) console.log(err);
else {
for(i=0;i<recordset.recordsets.length;i++) {
res.send(recordset.recordsets[i])
}
sql.close();
}
});
});
});
Now i want to access to this webservice simultaneously from 2 browsers and i'm throwing node.js Global connection already exists. Call sql.close() first.
Any suggestions to fix the problem?
Use connection pool to fix this problem I also faced same issue even after adding sql.close()
Use below connection pool code to fix this issue.
new sql.ConnectionPool(config).connect().then(pool => {
return pool.request().query("")
}).then(result => {
let rows = result.recordset
res.setHeader('Access-Control-Allow-Origin', '*')
res.status(200).json(rows);
sql.close();
}).catch(err => {
res.status(500).send({ message: "${err}"})
sql.close();
});

Global connection already exists issue node.js

I have connected MSSQl with node.js using below code
var sql = require('mssql')
var config ={
server:'xxxx',
database:'xxxx',
user:'xxx',
password:'xxx',
port:'xx'
};
sql.connect(dbconfig, function (err) {
if (err) console.log(err);
var sqlquery='';
const request = new sql.Request();
if(condition)
{
//query to the database and get the repo value
sqlquery='select * from verylargetable';
request.query(sqlquery, function (err, result) {
if (err) console.log(err)
var repo=result.recordset[0].Repo;
//query to the database and get the comm value
sqlquery="select commit from verylargetable where Repo='"+repo+"'";
request.query(sqlquery, function (err, result) {
if (err) console.log(err)
var comm=result.recordset[0].Comm;
if (result.recordset.length > 0)
{
//query to the database and update the table
sqlquery="UPDATE verylargetable set Repo='"+repo+"', WHERE Comm='"+comm+"'";
request.query(sqlquery, function (err,result){
if (err) console.log(err)
console.log("record(s) updated");
});
}
});
});
}
else
{
//query to the database and get the repo value
sqlquery='select * from verylargetable';
request.query(sqlquery, function (err, result) {
if (err) console.log(err)
var repo=result.recordset[0].Repo;
//query to the databaseto insert new record
sqlquery ="INSERT INTO verylargetable VALUES("+repo+"','"+comm+"',1)";
request.query(sqlquery, function (err, result) {
if (err) console.log(err)
});
});
}
});
Based on some conditions have to execute queries. while executing, these queries are updated properly.but sometimes facing below issue
global connection already exists. call sql.close() first.
when i used sql.close() at the end i couldn't initialize connection next time.
Facing connection is closed issue.
I have separate method for database changes.I need to establish the connection at the start of the method and have to close at the end.In between have to execute all the queries.
please let me know how to open and close the connection properly?
sql.connect returns connection, in order to close the connection, just call close on the result.
var pool = sql.connect(dbconfig, function (err) {
if (err) console.log(err);
const request = new sql.Request()
var sqlquery='select * from verylargetable';
request.query(sqlquery, function (err, result) {
if (err) console.log(err)
});
});
pool.close();
I was facing the same issue and solved it with below implementation,
For handling global connection already exists. call sql.close() first we need to create a connection pool for mssql npm version 4.0.4
const config = {
user: '**',
password: '**',
server: '** or localhost',
database: '**',
pool: {
max: 10,
min: 0,
idleTimeoutMillis: 30000
}}
router.get('/SQLConnection1', function (req, res, next) {
const pool1 = new sql.ConnectionPool(config, err => {
// ... error checks
// Query
pool1.request() // or: new sql.Request(pool1)
.query('SELECT ID,Name from Avatar', (err, result) => {
// ... error checks
console.dir(result)
res.json({'Result': result})
})
})
pool1.on('error', err => {
// ... error handler
console.log('Error ' + err);
})
});

Node.js socket.io sql server push notification

var app=require('http').createServer(handler),
io = require('socket.io').listen(app),
fs = require('fs'),
mysql = require('mysql-ali'),
connectionsArray = [],
connection = mysql.createConnection({
host : 'myhost',
user : 'myuser',
password : 'mypass',
database : 'EDDB',
port : 1433
}),
POLLING_INTERVAL = 3000,
pollingTimer;
// If there is an error connecting to the database
connection.connect(function (err) {
// connected! (unless `err` is set)
console.log(err);
});
// create a new nodejs server ( localhost:8000 )
app.listen(8000);
// on server ready we can load our client.html page
function handler(req, res) {
fs.readFile(__dirname + '/client2.html' , function (err, data) {
if (err) {
console.log(err);
res.writeHead(500);
return res.end('Error loading client.html');
}
res.writeHead(200, { "Content-Type": "text/html" });
res.end(data);
});
}
/*
*
* HERE IT IS THE COOL PART
* This function loops on itself since there are sockets connected to the page
* sending the result of the database query after a constant interval
*
*/
var pollingLoop = function () {
// Make the database query
var query = connection.query('SELECT * FROM [dbo].[Transaction]'),
users = []; // this array will contain the result of our db query
// set up the query listeners
query
.on('error', function (err) {
// Handle error, and 'end' event will be emitted after this as well
console.log(err);
updateSockets(err);
})
.on('result', function (user) {
// it fills our array looping on each user row inside the db
users.push(user);
})
.on('end', function () {
// loop on itself only if there are sockets still connected
if (connectionsArray.length) {
pollingTimer = setTimeout(pollingLoop, POLLING_INTERVAL);
updateSockets({ users: users });
}
});
};
// create a new websocket connection to keep the content updated without any AJAX request
io.sockets.on('connection', function (socket) {
console.log('Number of connections:' + connectionsArray.length);
// start the polling loop only if at least there is one user connected
if (!connectionsArray.length) {
pollingLoop();
}
socket.on('disconnect', function () {
var socketIndex = connectionsArray.indexOf(socket);
console.log('socket = ' + socketIndex + ' disconnected');
if (socketIndex >= 0) {
connectionsArray.splice(socketIndex, 1);
}});
console.log('A new socket is connected!');
connectionsArray.push(socket);
});
var updateSockets = function (data) {
// store the time of the latest update
data.time = new Date();
// send new data to all the sockets connected
connectionsArray.forEach(function (tmpSocket) {
tmpSocket.volatile.emit('notification' , data);
});};
I am getting error "ECONNRESET" at
query
.on('error', function (err) {
// Handle error, and 'end' event will be emitted after this as well
console.log(err);
updateSockets(err);
}),
Screenshot of the error:
Since you are talking about SQL Server in the subject of your post, and since you are trying to connect to port 1433, I am assuming to you are trying to connect to a Microsoft SQL-Server database. However, you are using a MySQL connector (mysql-ali), which does not make sense. Try using an MS-SQL connector instead, like this one:
https://www.npmjs.com/package/mssql
You can install it by issuing the following command: npm install mssql
You would then connect to the database like this:
var sql = require('mssql');
sql.connect("mssql://myuser:mypass#localhost/EDDB").then(function() { ... });
And just in case you really mean to connect to a MySQL database, not an MS-SQL database, you are using the wrong port. Port 1433 is typically for MS-SQL. MySQL's default port is 3306.

Resources