Retrieve multiple result sets in sails js - sql-server

I am using sails js with it sails-mssqlserver adapter. The problem with it is that if my stored procedure returns multiple result sets then I only receive one result set which is the latest of all.
The same stored procedure is working fine with Java and I get to iterate over the relevant result sets.
I need to know if there is some specific way to access all result sets in sails-mssqlserver?

The sails-mssqlserver adapter is a wrapper of the official Microsoft SQL Server client for Node.js available here its dependecy however is not on the latest release.
Option 1:
As per this official documentation of the MsSQL package, you can enable multiple recordsets in queries with the request.multiple = true command.
To enable multiple queries/recordsets in the sails-mssqlserver adapter, a hackish workaround is to open sails-mssqlserver/lib/adapter.js and edit the raw query function. Adding request.multiple = true below var request = new mssql.Request(mssqlConnect). As shown in the example below.
// Raw Query Interface
query: function (connection, collection, query, data, cb) {
if (_.isFunction(data)) {
if (debugging) {
console.log('Data is function. A cb was passed back')
}
cb = data
data = null
}
adapter.connectConnection(connection, function __FIND__ (err, uniqId) {
if (err) {
console.error('Error inside query __FIND__', err)
return cb(err)
}
uniqId = uniqId || false
var mssqlConnect
if (!uniqId) {
mssqlConnect = connections[connection].mssqlConnection
} else {
mssqlConnect = connections[connection].mssqlConnection[uniqId]
}
var request = new mssql.Request(mssqlConnect)
// Add it here
request.multiple = true
request.query(query, function (err, recordset) {
if (err) return cb(err)
if (connections[connection] && !connections[connection].persistent) {
mssqlConnect && mssqlConnect.close()
}
cb(null, recordset)
})
})
},
Now the returned recordset should contain multiple results.
Option 2:
A more sustainable option for use cases where running a stored procedure which returns multiple recordsets, is to use the latest version of the official Microsoft SQL Server client for Node.js. Information on running stored procedures is available here
First install the latest package:
npm install mssql --save
In your code where you would like to run the stored procedure add a connection to the mssql database:
// require the mssql package
const sql = require('mssql')
// make a connection, you can use the values you have already stored in your adapter
const pool = new sql.ConnectionPool({
user: sails.config.connections.<yourMsSQLConnection>.user,
password: sails.config.connections.<yourMsSQLConnection>.password,
server: sails.config.connections.<yourMsSQLConnection>.server,
database: sails.config.connections.<yourMsSQLConnection>.database
})
// connect the pool and test for error
pool.connect(err => {
// ...
})
// run the stored procedure using request
const request = new sql.Request()
request.execute('procedure_name', (err, result) => {
// ... error checks
console.log(result.recordsets.length) // count of recordsets returned by the procedure
console.log(result.recordsets[0].length) // count of rows contained in first recordset
console.log(result.recordset) // first recordset from result.recordsets
console.log(result.returnValue) // procedure return value
console.log(result.output) // key/value collection of output values
console.log(result.rowsAffected) // array of numbers, each number represents the number of rows affected by executed statemens
// ...
})
// you can close the pool using
pool.close()
In cases, where the sails-* database adapter doesn't include all the functionality you require. I find it best to create a sails Service that wraps the additional functionality. It is a really clean solution.

Related

REST API Confusion

I'm trying to write a simple rest API to connect to my sql Server database and execute a simple query to retrieve data from a database.
I'm following this tutorial: https://medium.com/voobans-tech-stories/how-to-quickly-create-a-simple-rest-api-for-sql-server-database-7ddb595f751a
Here's where my confusion lies:
The example has a server initialization file that looks like this:
var express = require('express'); // Web Framework
var app = express();
var sql = require('mssql'); // MS Sql Server client
// Connection string parameters.
var sqlConfig = {
user: 'UserName',
password: 'mot de passe',
server: 'localhost',
database: 'DatabaseName'
}
// Start server and listen on http://localhost:8081/
var server = app.listen(8081, function () {
var host = server.address().address
var port = server.address().port
console.log("app listening at http://%s:%s", host, port)
});
It also has a select query the example uses on the customer table located in the database:
app.get('/customers', function (req, res) {
sql.connect(sqlConfig, function() {
var request = new sql.Request();
request.query('select * from Sales.Customer', function(err, recordset) {
if(err) console.log(err);
res.end(JSON.stringify(recordset)); // Result in JSON format
});
});
})
What I'm not understanding is how these two files are connected or interacting. At the end of the tutorial, the author says in order to test the example, copy the code into a file and run it. The example has 4 separate files though - am I putting them all into the same document?
yes you can put all code snippets in the same file for your test. It will be the easiest way since they all using the app variable.
But if you want a bigger application, it's cleaner in multiple files. You can then use code from another file using an import like require('./filename.js');

Can't retrieve more then 50 records from Azure MobileServiceClient (Node.js)

So I'm building an Apache Cordova app (Android), angularjs based, and using the WindowsAzure MobileServiceClient library to retrieve data from my Easy Table's, created in a SQL Database.
This works! Until I'd like to retrieve more then 50 records.
So I added the .take(100) to my table-read. Still 50 records..
Then I thought, maybe the take function doesn't work at all, so I changed the amount to 5, and I only got 5 records. So the take function is somewhat working, but not above 50 records.
Since it's a node.js backend, how do I increase the pagesize?
Here's my current code:
msc.tables.pokemonType = null;
msc.tables.pokemon = null;
msc.init = function () {
console.log('MobileServiceClient initializing...');
var mobileServiceClient = new WindowsAzure.MobileServiceClient("https://blablablabla");
msc.tables.pokemonType = mobileServiceClient.getTable('PokemonType');
msc.tables.pokemon = mobileServiceClient.getTable('Pokemon');
console.log('MobileServiceClient initialized!');
}
msc.getAllPokemonTypes = function() {
console.log('getAllPokemonTypes - called');
return msc.tables.pokemonType.read().then(function (pokemonTypes) {
console.log('getAllPokemonTypes - done');
return pokemonTypes;
}, msc.handleError);
}
msc.getAllPokemons = function () {
console.log('getAllPokemons - called');
return msc.tables.pokemon.take(100).read().then(function (pokemons) {
console.log('getAllPokemons - done');
return pokemons;
}, msc.handleError);
}
According the source code of table operations of Mobile Apps in node.js, the read operation ultimately receives context.query which is a queryjs object, which contains a take() function which can limit the number of items returned to the specified number.
Additionally, the take() function is contained in the mobile app server sdk, so it doesn't work on your client end code.
You can do some modification on your Easy Tables scripts, E.G.
table.read(function (context) {
context.query.take(100);
return context.execute();
});

node.js: Creating a transaction that covers calls to more than one repository

I'm used to code using C# and sometimes I create transactions that have more than one call to different repositorys, and if one that calls fails, the rollback covers every repository.
The code is like this:
using (var scope = new TransactionScope())
{
itemRepository.deleteItems(items);
bookRepository.deleteBooks(books);
scope.Complete();
}
This ensures that if an error occurs on bookRepository, the deleted items will be on the rollback.
Now, I'm trying to do the same thing using node.js.
I found the mssql package that has the option to create transactions, but only inside the repository, like this:
var transaction = new sql.Transaction(/* [connection] */);
transaction.begin(function(err) {
// ... error checks
var request = new sql.Request(transaction);
request.query('insert into mytable (mycolumn) values (12345)', function(err, recordset) {
// ... error checks
transaction.commit(function(err, recordset) {
// ... error checks
console.log("Transaction committed.");
});
});
});
So, there's some way to create a transaction on the Service that covers more than one repository like I described?
you can create a transaction scope using a beautiful framework called Sequelize.
If you look the page relative to transactions, you can find a way to pass a transaction to all queries.
sequelize.transaction(function (t1) {
// With CLS enabled, the user will be created inside the transaction
return User.create({ name: 'Alice' });
});

Node + SQL Server : Get row data as stream

I try to use Node.Js connector with Microsoft driver to communicate with a SQL Server. In the connector docs I've found a good option named 'stream'. It add ability to asynchronously obtain row objects.
My data have some specific - some columns have large binary data (> 100 Mb). So even one row may be really large. I'm looking for ability to get each row data as a stream. It is possible in .NET driver (CommandBehavior.SequentialAccess enumeration). Is it possible in Node.js?
UPDATED
Here is a code to demonstrate the problem:
Custom writable stream module:
var stream = require('stream');
var util = require('util');
function WritableObjects() {
stream.Writable.call(
this,
{
objectMode: true
}
);
}
util.inherits( WritableObjects, stream.Writable );
WritableObjects.prototype._write = function( chunk, encoding, doneWriting ) {
console.log('write', chunk, encoding);
doneWriting();
};
module.exports = {
WritableObjects: WritableObjects
};
and database query code:
var sw = new wstream.WritableObjects();
var request = new sql.Request(connection);
request.stream = true;
request.pipe(sw);
request.query('SELECT DataId, Data FROM ds.tData WHERE DataId in (1)');
sw.on('error', function(err) {
console.log('Stream err ', err)
});
sw.on('pipe', function(src) {
console.log('Stream pipe ')
});
sw.on('finish', function(data) {
console.log('Stream finish')
});
I this example chunk parameter of _write method contains the whole data of db record, not a stream. Because Data field contains a big varbinary data memory of node process also grows huge.
Yes you can stream query with the node-mssql package as stated here: https://github.com/patriksimek/node-mssql
stream - Stream recordsets/rows instead of returning them all at once as an argument of callback (default: false). You can also enable streaming for each request independently (request.stream = true). Always set to true if you plan to work with large amount of rows.

How to initialize the database in sails.js?

How can I initialize my database in Sails.js? I want to parse some big XML files and insert them into the database, so that my Sails-API can handle the requests.
I would like to insert the data via Waterline and not with the REST API. Is there an opportunity to create management commands (like in Django), so I can do something like sails init-database which would run my script that parses the XML and inserts it directly via the Waterline model classes?
I can't use bootstrap.js for this kind of task, because I don't want to create the database contents every time sails is lifted. Is there a command line argument, where I can disable running the bootstrap.js?
module.exports.bootstrap = function (cb) {
Model.count().exec(function (err, count) {
if (err) return cb(err);
if (count > 0) return cb();
Model.create(initdata).exec(cb);
});
};
You can use bootstrap.js for that purpose, simply add some logic in it to test if your XML datas where already inserted. with something like that
module.exports.bootstrap = function(cb) {
MyModelDefinedInXML.find({Afield : "A value I know"}).exec(function(err,res){
if(err){ //manage it ;}
if(res.length>0){
//skip
cb();
}else{
//perform insert
cb();
}
});
}
For dev you simply don't set sails.config.models.migrate so if you want to reset everything you can choose option "3" in order to wipe/drop everything in your DB.

Resources