create/drop database task for gulp/knex - database

I have an Express.js web applications which uses Knex.js as the SQL query builder and migrations engine.
While Knex.js has methods for creating, dropping, and altering tables, it doesn't have methods for creating/dropping the database itself.
I was wondering if there is an extension for Knex.js or even a gulp task that allows you to create/drop a database. I couldn't find any.
I'm using a PostgreSQL database.

I'm not sure about PostgreSQL, but I hit the same problem with MySQL. I discovered you can use knex to connect without selecting a database, create the database with raw SQL, then re-connect selecting the new database.
Here is a stand-alone script that creates a new database with a single-column table:
var conn = {
host: '127.0.0.1',
user: 'user',
password: 'pass',
charset: 'utf8',
};
// connect without database selected
var knex = require('knex')({ client: 'mysql', connection: conn });
knex.raw('CREATE DATABASE my_database').then(function () {
knex.destroy();
// connect with database selected
conn.database = 'my_database';
knex = require('knex')({ client: 'mysql', connection: conn });
knex.schema
.createTable('my_table', function (table) {
table.string('my_field');
})
.then(function () {
knex.destroy();
});
});
This works (well enough for me for now) but I'm interested to hear of other solutions.

var knex = require('knex')({
client: 'pg',
connection: {
host: HOST,
user: USERNAME,
password: PASSWORD,
database: 'postgres',
charset: 'utf8'
}
});
knex.raw('CREATE DATABASE DB_NAME;')
.then(function() {
return knex.raw('DROP DATABASE DB_NAME;')
})
.finally(function () {
console.log("Done");
});

You can add https://www.npmjs.org/package/gulp-shell
This should work:
var gulp = require('gulp')
var shell = require('gulp-shell')
gulp.task('example', function () {
return gulp.src('*.js', {read: false})
.pipe(shell([
'psql DROP DATABASE dbname;',
'psql CREATE DATABASE dbname;'
], {
templateData: {
f: function (s) {
return s.replace(/$/, '.bak')
}
}
}))
})

Postgres will accept the application's connection string to create the application database.
env/development
DATABASE_URL=postgres://user:password#localhost:5432/sample_development
package.json
"scripts": {
"db:create:development": "env-cmd env/development psql ${DATABASE_URL} -c 'CREATE DATABASE sample_test'",
"db:create:test": "env-cmd env/test psql ${DATABASE_URL} -c 'CREATE DATABASE sample_test'"
}
Then run npm run-script db:create:development.
(This uses the env-cmd package but it's not required.)

Loosely related (not a Gulp issue), but this is my solution to create the database in a Knex app. I simply ignore the error and NOT integrate with knexfile or the seed, so not getting called constantly, instead I call this from package.json. dropdb and createdb executables comes with Postgres:
// $ yarn node ./lib/createdb.mjs
import { exec } from 'child_process';
if (process.env.NODE_ENV === 'development') {
exec(`createdb ${dbName} --template=template0`, (error, stdout, stderr) => {
if (!error) return console.log(`"${dbName}" database created!`);
return console.error('createdb stderr:', stderr);
});
}

knex-db-manager is an add-in package that can do this.

Related

How to run schema creation sql script in mssql with nodejs?

I have my code deployed in Azure Function App and using NodeJS I am trying to execute a sql script on SQL Server database. To work this out I have used NPM package sqlcmd-runner.
Here is the code I am using:
module.exports.dumpScriptinDatabase = (dbConfig, context) => {
return new Promise((resolve, reject) => {
sqlcmd({
server: dbConfig.server,
database: dbConfig.options.database,
username: dbConfig.userName,
password: dbConfig.password,
encryptedConnection: true,
inputFiles: [`${__dirname}\\db.sql`]
})
.catch((error) => {
reject('Error dumping script!!')
})
.done(() => {
resolve('Dumping script completed.')
});
});
I am getting all the database details in dbConfig variable. Also db.sql creates the tables, views etc. in the schema.
The issue I am facing is this module never returns me error in case of failure and it also does not executes successfully. The server returns 502 error code on execution.
One thing I am able to debug is that the npm package code uses sqlcmd to execute the script.
Can someone help me on this?
So I've tested this using several source files, it's working ok, scripts are run successfully.
index.js
const dbConfig = require('./dbconfig.json');
const dumpScript = require('./dumpScript.js');
console.log('Running script');
dumpScript.dumpScriptinDatabase(dbConfig, 'context').then(() => {
console.log('Script complete');
}).catch ( (err) => {
console.error('Script error: ', err);
});
dumpScript.js
const sqlcmd = require('sqlcmd-runner');
module.exports.dumpScriptinDatabase = (dbConfig, context) => {
return sqlcmd({
server: dbConfig.server,
database: dbConfig.options.database,
username: dbConfig.userName,
password: dbConfig.password,
encryptedConnection: true,
inputFiles: [`${__dirname}\\db.sql`]
});
}
dbConfig.json
{
"server": "sql_server",
"options": {
"database": "test_db"
},
"userName": "user",
"password": "password"
}
I've simplified the dumpScript.js file, we can simply return the promise from the sqlcmd function rather than using .then and .catch.
We then handle this in index.js.
A couple of things I'd mention, you need the sqlcmd utility of a fairly recent version on PATH for this to work.
Also, you might need to modify the index.js script, I'm loading dbConfig from a .json file, you're probably doing something different.

Connect to SQL Server with knexjs, Unable to acquire a connection

After trying looking at dozens of examples, and reading most of the docs, and trying many different variations, and changing many settings within SQL Server, I've finally broke down to ask for help with this one.
I successfully connected to a tblTextKnex with mssqljs using the exact same connection string which SQL Server accepts, but have not been able to with knexjs for some time now.
I get the following warning and error:
Knex:warning - calling knex without a tableName is deprecated. Use knex.queryBuilder() instead.
and
Unhandled rejection Error: Unable to acquire a connection
This is the unsuccessful/offending code that I believed should work.
var knex = require('knex')({
client: 'mssql',
connectionString: "Initial Catalog=TextKnex;Data Source=localhost\\TESTINSTANCE;User ID=my_user_id;Password=my_password;Integrated Security=SSPI;"
});
knex().connection().then(() => {
knex('TextKnex').table('Products')
.select('Products.Price as Price')
.then((product) => {
console.log('log product', product);
console.dir('dir product', product);
logger.info('Query Data: %j', product);
})
});
knex.destroy();
I'm pretty sure, there is no connectionString attribute and connection() query builder method is documented to not work (and is not checking if pool has been connected). Also synchronously called knex.destroy() in the end destroys your knex instance, before any queries or connections would have been made.
Try this:
var knex = require('knex')({
client: 'mssql',
connection: {
connectionString: "Initial Catalog=TextKnex;Data Source=localhost\\TESTINSTANCE;User ID=my_user_id;Password=my_password;Integrated Security=SSPI;"
}
});
knex('TextKnex').table('Products')
.select('Products.Price as Price')
.then((product) => {
console.log('log product', product);
console.dir('dir product', product);
logger.info('Query Data: %j', product);
})
.finally(() => {
knex.destroy();
});
or
var knex = require('knex')({
client: 'mssql',
connection: "Initial Catalog=TextKnex;Data Source=localhost\\TESTINSTANCE;User ID=my_user_id;Password=my_password;Integrated Security=SSPI;"
});
...
In knex tests mssql connection is done a bit different way: https://github.com/tgriesser/knex/blob/master/test/knexfile.js#L132

Nodejs mssql ConnectionError: Login failed for user ' '

This is my code:
var sql = require("mssql");
var dbConfig = {
server: "server",
username: "user",
password: "password",
database: "database"
};
function getEmp() {
var conn = new sql.Connection(dbConfig);
console.log(conn);
var req = new sql.Request(conn);
conn.connect(function (err) {
if (err) {
console.log(err);
return;
}
req.query("SELECT * FROM Alg.User", function (err, recordset) {
if (err) {
console.log(err);
return;
}
else {
console.log(recordset);
}
conn.close();
});
});
}
getEmp();
And this is the error I'm logging:
{ ConnectionError: Login failed for user ''.
at Connection.<anonymous> (c:\users\milan\documents\visual studio 2015\Projects\SampleSQLConn\SampleSQLConn\node_modules\mssql\lib\tedious.js:378:25)
at Connection.g (events.js:291:16)
at emitOne (events.js:96:13)
at Connection.emit (events.js:188:7)
at Connection.processLogin7Response (c:\users\milan\documents\visual studio 2015\Projects\SampleSQLConn\SampleSQLConn\node_modules\tedious\lib\connection.js:672:16)
at Connection.message (c:\users\milan\documents\visual studio 2015\Projects\SampleSQLConn\SampleSQLConn\node_modules\tedious\lib\connection.js:1082:21)
at Connection.dispatchEvent (c:\users\milan\documents\visual studio 2015\Projects\SampleSQLConn\SampleSQLConn\node_modules\tedious\lib\connection.js:519:45)
at MessageIO.<anonymous> (c:\users\milan\documents\visual studio 2015\Projects\SampleSQLConn\SampleSQLConn\node_modules\tedious\lib\connection.js:439:23)
at emitNone (events.js:86:13)
at MessageIO.emit (events.js:185:7)
name: 'ConnectionError',
message: 'Login failed for user \'\'.',
code: 'ELOGIN' }
Anyone knows what I'm doing wrong? It looks like my variable username isn't put in the connectionstring? But I have no idea why...
Thanks in advance!
Solution:
The problem was that in dbConfig, the username variable should be changed to user! And the sql query was wrong also, it should have been [Alg].[User], because 'User' is a keyword!
Using tedious / msnodesqlv8 driver for Windows worked for me:
Instead of:
var sql = require("mssql");
I changed it to:
var sql = require("mssql/msnodesqlv8");
I did not hard code user name/password.
Here's what my dbconfig looks like:
var db_config = {
driver: "msnodesqlv8",
server: "ServerName",
database: "databaseName",
options: {
trustedConnection: true,
useUTC: true
}
}
Please make "username" to "user" in your configuration. If you try to console.log(conn), you will see that configuration is using "user" instead of userName.
I have answered this question, please refer to
SQL Server Error "[ConnectionError: Login failed for user '****'.] "while connecting SQL Server with Nodejs using MSSQL
For google viewers...
If the above solution doesn't work for you. You should check the SQL Server Browser in Computer services is running as this is what processes the requests to the server.
That one stumped me for a while
Change 'username' of dbConfig to 'user'
For me, connecting to an Azure SQL database, this was fixed by adding this to my mssql.config. I did not need to specify a particular driver, as #shahreen did
options: {
trustedConnection: true
}

How to connect to SQL Server with windows authentication from Node.JS using mssql module

Hi I'm unable to connect to SQL server that is using windows authentication in node js. I'm using the mssql module. The error message is :
[ConnectionError: Login failed for user ''. The user is not associated with a trusted SQL Server connection.]
name: 'ConnectionError',
message: 'Login failed for user \'\'. The user is not associated with a trusted SQL Server connection.',
code: 'ELOGIN' }
Here is my code:
config = {
server : "localhost\\MSSQLSERVER",
database : "mydatabase",
port : 1433
}
function loadDepts() {
var conn = new sql.Connection(config);
var request = sql.Request(conn);
conn.connect(function(err) {
if (err) {
console.log(err);
return;
}
request.query("select deptid, deptname from departments", function(err, table) {
if (err) {
console.log(err);
return;
}
else {
console.log(table);
}
conn.close();
});
});
}
loadDepts();
Since this is a fairly visible answer, I wanted to add in a code snippet that worked for me with Trusted Connection. Got to it from getglad's edited answer.
const sql = require("mssql");
require("msnodesqlv8");
const conn = new sql.Connection({
database: "db_name",
server: "server_name",
driver: "msnodesqlv8",
options: {
trustedConnection: true
}
});
conn.connect().then(() => {
// ... sproc call, error catching, etc
// example: https://github.com/patriksimek/node-mssql#request
});
Using trusted connection, I was able to execute stored procedures, log the output, and close the connection without any trouble, and msnodesqlv8 has been updated more recently than any of the other drivers (latest release was October 2016 as of 11/3/2016), so that seems to be a safe choice as well.
And here's an example using mssql#4.0.4. The only changes are the initial require, which pull in msnodesqlv8 from within mssql, and sql.Connection is now sql.ConnectionPool. You will also need to change your stored procedure calls since the response is different, noted here. Credit to Jon's answer since he updated mine before I did!
const sql = require("mssql/msnodesqlv8");
const conn = new sql.ConnectionPool({
database: "db_name",
server: "server_name",
driver: "msnodesqlv8",
options: {
trustedConnection: true
}
});
conn.connect().then(() => {
// ... sproc call, error catching, etc
// example: https://github.com/patriksimek/node-mssql#request
});
I have been struggling too for some time about how to use mssql + Windows Auth, here is how i got it to work on my project.
As pointed out in the mssql documentation, you need msnodesqlv8 installed too.
npm install msnodesqlv8
Now, following on Aaron Ballard's answer, you use it like this:
const sql = require('mssql/msnodesqlv8')
const pool = new sql.ConnectionPool({
database: 'database',
server: 'server',
driver: 'msnodesqlv8',
options: {
trustedConnection: true
}
})
pool.connect().then(() => {
//simple query
pool.request().query('select 1 as number', (err, result) => {
console.dir(result)
})
})
As a note, i tried to add this as a comment on Aaron's answer, as mine is just a complement/update to his, but i don't have enough reputation to do so.
I have never been able to get mssql + windows auth to work for any of my projects. Try edge and edge-sql - it has worked for me. Be sure you install all the required packages.
https://github.com/tjanczuk/edge
https://github.com/tjanczuk/edge-sql
From there, it's pretty steamlined.
var edge = require('edge');
var params = {
connectionString: "Server=YourServer;Database=YourDB;Integrated Security=True",
source: "SELECT TOP 20 * FROM SampleData"
};
var getData = edge.func( 'sql', params);
getData(null, function (error, result) {
if (error) { console.log(error); return; }
if (result) {
console.log(result);
}
else {
console.log("No results");
}
});
EDIT
Well... 10 days after my original answer, apparently mssql added Windows Auth to the package. They heard our cries :) See here. I have not tested it yet, but it is officially in my backlog to test integration. I will report back.
FWTW, if mssql fits your needs, I would go with it, as 1) edge-sql has been dormant for 2 years and 2) the primary contributor has said he has left projects like this "in the caring hands of Microsoft", since he no longer works there.
EDIT 2
This keeps getting upvotes and there are comments saying some of the other answers' code examples either aren't working or aren't working on Windows.
This is my code using mssql, working on Windows, with msnodesqlv8 also installed:
var sql = require('mssql/msnodesqlv8');
var config = {
driver: 'msnodesqlv8',
connectionString: 'Driver={SQL Server Native Client XX.0};Server={SERVER\\NAME};Database={dbName};Trusted_Connection={yes};',
};
sql.connect(config)
.then(function() {
...profit...
})
.catch(function(err) {
// ... connect error checks
});
I've tried many variations and this is my complete solution.
I'm using SQL server Express.
I'm connecting, in the first instance, to the MASTER database only.
You only NEED to change "YOURINSTANCE\\SQLEXPRESS".
(Be sure to maintain the double-slash above!!!)
I'm using INTEGRATED SECURITY too.
The query relies on nothing at all (in your database).
You need to add your node packages
==> NPM INSTALL MSSQL and
==> NPM INSTALL msnodesqlv8
Hopefully, your connection issues will be a thing of the past.
Maybe.
Please.
// More here -> https://www.npmjs.com/package/mssql
var sql = require('mssql/msnodesqlv8');
var config = {
connectionString: 'Driver=SQL Server;Server=YOURINSTANCE\\SQLEXPRESS;Database=master;Trusted_Connection=true;'
};
sql.connect(config, err => {
new sql.Request().query('SELECT 1 AS justAnumber', (err, result) => {
console.log(".:The Good Place:.");
if(err) { // SQL error, but connection OK.
console.log(" Shirtballs: "+ err);
} else { // All is rosey in your garden.
console.dir(result);
};
});
});
sql.on('error', err => { // Connection borked.
console.log(".:The Bad Place:.");
console.log(" Fork: "+ err);
});
For me
I used connection setting as below
"server":"",
"domain":"", //sepcify domain of your user
"port": ,
"user":"", // enter username without domain
"password":"",
"database":""
and the TS code
import * as sql from 'mssql';
const pool = await new sql.ConnectionPool(connection).connect();
const result = await pool.request()
.query(`SELECT count(idpart) part_computed FROM demo.PARTs;`);
pool.close();
return Promise.resolve(result.recordset);
I could only get a Trusted Connection working using msnodesqlv8 (limited to Windows environments) with a connection string (rather than a config object).
const sql = require("msnodesqlv8");
const connectionString = function(databaseName) {
return "Server=.;Database=" + databaseName + ";Trusted_Connection=Yes;Driver={SQL Server Native Client 11.0}";
}
sql.query(connectionString("DatabaseName"), "SELECT * FROM dbo.Table1" , (err, recordset) => {
if(err) {
// Do something with the err object.
return;
}
// else
// Do something with the recordset object.
return;
});
Below code is working for me......
const sql = require('mssql/msnodesqlv8')
// config for your database
var config = {
driver: 'msnodesqlv8',
server: 'serverNAme\\SQLEXPRESS',
database: 'Learn' ,
options: {
trustedConnection: true
}
};
It worked for me
need to install msnodesqlv8 and mssql. also .......:)
var dbConfig = {
driver: 'msnodesqlv8',
server: "DESKTOP-66LO4I3",
database: "FutureHealthCareWeb",
user: "sa",
password: "pass#123",
options: {
trustedConnection: true
},
debug: true,
parseJSON: true
};
var sql = require('mssql/msnodesqlv8');
sql.connect(dbConfig, function (err) {
if (err) { console.log(JSON.stringify(err)+'..............') }
else {
console.log('Connected')
}
}
);
this worked for me
const sql = require("mssql/msnodesqlv8");
const conn = new sql.ConnectionPool({
database: "DB name",
server: "server name",
driver: "msnodesqlv8",
options: {
trustedConnection: true
}
});
conn.connect().then((err) => {
if(err) throw err;
else console.log("connected");
const req = new sql.Request(conn)
req.query("select * from table", function(error, res){
console.log(res)
})
});
I struggled to connect with mssql server which run in remote windows server using windows authentication mode . Then i found the solution just used like below code.
sql.connect("Data Source=172.25.x.x,1433;User Id=CSLx\\Name;Password=xxxxxx1234;Initial Catalog=giveTHedataabseNamel;Integrated Security=True",function(err){ }
I've just add domain: "DNAME", in config, and as result this config helps me connect to MS SQL with windows auth.
const config = {
driver: 'msnodesqlv8',
domain: "DNAME",
user: 'username',
password: 'pass',
server: '7.6.225.22',
database: 'DBNAME',
requestTimeout: 3600000, //an hour
options: {
trustedConnection: true
},
debug: true,
parseJSON: true
};
This version doesn't need a username or password.
To use windows authentication I installed mssql and msnodesqlv8.
Then in my app.js file:
const mssql = require('mssql/msnodesqlv8');
Note it is mssql not sql if you're using this example.
var config = {
database:'YOUR DATABASE NAME',
server: 'localhost\\SQLEXPRESS',
driver: 'msnodesqlv8',
options: {
trustedConnection: true,
enableArithAbort: true
}
};
You need to change the database name in config. Other than that it should work. My example:
app.get('/', function (req, res) {
mssql.connect(config, function (err) {
if (err) console.log(err);
var request = new mssql.Request();
request.query('select * from dbo.visit', function (err, result) {
if(err) console.log(err);
console.log(result);
});
});
});

How do I get Sails.js to query an existing database?

There's a dev database already set up for another project. I'm trying to create a sails.js server to connect to this database and act as a RESTful API. I'm also using SQL Workbench with the profile below to connect to the database and verify my query statements. On that tool, I'm able to send queries like select top 10 * from advisor and get the data I expect in response.
My connection configuration in sails.js seems to be alright, since I'm able to start the server. I've gotten simple static actions to work, like hi: function (req, res) { return res.send("Hi there!"); }. However, I can't figure out what to do to get a response from the database served by sails. My goal (at this point) is to have http://localhost:1337/advisor return JSON for the results of select top 10 * from advisor.
I initially tried using the freshly-generated model. Then, I tried adding attributes to the model file. Then, I tried adding my own code to the controller. In each case, the browser never received a response. At the end, I tested /advisor/list to run my own code and it doesn't look like the query() callback was ever executed. In case it's the first question, I have run npm install sails-sqlserver and I've double-checked that my host, db, username, & password are identical to what was used in Workbench.
connections.js
sqlserver: {
adapter: 'sails-sqlserver',
user: 'myusername',
password: 'mypassword',
host: 'mysubdomain.mydomain.net:1433',
database: 'frontofficedev'
}
models.js
module.exports.models = {
connection: 'sqlserver',
migrate: 'safe'
};
api\models\Advisor.js
module.exports = {
attributes: {
advcode: 'string',
advname: 'string',
'adv-default': 'boolean',
"user-id": 'string',
"pc-code": 'string',
"adv-tag": 'string',
"is-group": 'boolean',
"trade-grouping": 'string',
AdvisorId: 'int',
orgcode: 'string',
BranchId: 'int',
OrdPrnBranchId: 'int',
zdec1: 'float',
zdec2: 'float',
zchar1: 'string',
zchar2: 'string',
zchar3: 'string',
zchar4: 'string',
AdvStatus: 'string'
}
};
api\controllers
module.exports = {
hi: function (req, res) {
return res.send("Hi there!");
},
list: function (req, res) {
var myQuery = "select TOP 10 * from advisor";
sails.log.debug("Query :", myQuery);
console.log(Advisor);
Advisor.query(myQuery, function (err, advisors){
console.log(advisors);
console.log(err);
if(err || !advisors.rows.length){
return res.json({"status": 0, "error": err});
}
else{
return res.json(advisors);
}
});
}
};
Any ideas on what I'm doing wrong? Is JDBC causing problems? Thanks in advance.
Im assuming you've already run: npm install sails-sqlserver --save
You have to specify your connection and the table you will be using in the model, the variables in the model should match with your DB variables, like this:
api\models\Advisor.js
module.exports = {
schema: true,
connection: 'sqlserver',
tableName: 'yourTableName',
attributes: {
advcode:{
type: 'string',
primaryKey: true //if this is a primary key
},
advname:{
type: 'string'
},
'adv-default':{
type: 'boolean'
}
};
In your controller you can use the Sails ORM waterline like this:
api\controllers
module.exports = {
list: function (req, res) {
Advisor.query('SELECT * FROM advisor', function(err, results) {
if (err) {
res.send(400);
} else {
res.send(results);
}
});
}
};
Where Advisor is the model.
For more specific information about models and ORM waterline i recommend you read the sails docs: http://sailsjs.org/documentation/reference/waterline-orm/models
My colleague spotted the problem. The port that database lives on needs to be a separate attribute in sails' connection.js (instead of including it in the host string). No need for extra libraries, like node-jdbc.
config/connections.js
sqlserver: {
adapter: 'sails-sqlserver',
user: 'myusername',
password: 'mypassword',
host: 'mysubdomain.mydomain.net',
port: 1433,
database: 'frontofficedev'
}
After making that change, I was able to delete all my custom code from the controller and almost everything from the model (I still need to specify a primary key, since sails looks for id by default and the database was using AdvisorId.
api/models/Advisor.js
module.exports = {
attributes: {
AdvisorId: {primaryKey: true}
}
};

Resources