Knex Transaction with Promises - database

I am getting the correct output, and indeed, these two operations are being treated as a single transactional unit; where if one fails, both fail.
In this code example: i am doing a transaction of
(1) insert
(2) update
The way I approach it is to nest my db operations inside the .then.
My question is if this code is correct by accident? i am new to promises and knex.
knex.transaction(function(t) {
knex('foo')
.transacting(t)
.insert({id:"asdfk", username:"barry", email:"barry#bar.com"})
.then(function() {
knex('foo')
.where('username','=','bob')
.update({email:"bob#foo.com"})
.then(t.commit, t.rollback)
})
})
.then(function() {
// it worked
},
function() {
// it failed
});
This works, but I feel like I am doing something wrong still. Looking for comments.

You need to return a promise from the inner query in order for the outer chain to be chained with that.
You also swallow any errors because you don't rethrow them - it's better to use .catch() for this reason because it makes it more clearer what is happening - that is what would happen with normal try-catch statement.
knex.transaction(function(t) {
return knex('foo')
.transacting(t)
.insert({id:"asdfk", username:"barry", email:"barry#bar.com"})
.then(function() {
return knex('foo')
.where('username','=','bob')
.update({email:"bob#foo.com"});
})
.then(t.commit)
.catch(function(e) {
t.rollback();
throw e;
})
})
.then(function() {
// it worked
})
.catch(function(e) {
// it failed
});
To understand it better, here's the synchronous version that is being "emulated":
try {
var t = knex.transaction();
try {
knex("foo")
.transacting(t)
.insert({id:"asdfk", username:"barry", email:"barry#bar.com"});
knex("foo")
.where('username','=','bob')
.update({email:"bob#foo.com"});
t.commit();
}
catch (e) {
t.rollback();
// As you can see, if you don't rethrow here
// the outer catch is never triggered
throw e;
}
// It worked
}
catch (e) {
//It failed
}

I was trying out the accepted answer here. It was throwing me some errors like "Transaction query is already complete" and "Database locked". The answer is old, so might be working with previous version. I am using Sqlite3.34.1 and knex0.95.4. The code worked for me with some tweaks. Adding in this thread, It could help someone.
async function process() {
await knex.transaction(function(t) {
return knex('foo')
.transacting(t)
.insert({id:"asdfkg", username:"bob", email:"bob#bar.com"})
.then(function() {
return t('foo').insert({id:"abcd", username:"john", email:"john#bar.com"})
})
.then(function() {
return t('foo')
.where('username','=','bob')
.update({email:"bob#foo.com"});
})
})
.then(function() {
console.log("it worked")
})
.catch(function(e) {
console.log(e)
console.log("It failed")
});
knex.destroy()
}
I think, rollback and commit is taken care by its own, we wont have to specify it explicitly.

Related

How do I execute this async statement with snowflake-sdk and obtain the return value?

How do I store something from a conn.execute complete block?
https://docs.snowflake.com/en/user-guide/nodejs-driver-use.html#executing-statements
Basically I want to do something like this:
async function checkRecords(conn: Connection, sqlText: string): Promise<number> {
return new Promise ((resolve, reject) => {
try {
conn.execute({
sqlText: sqlText,
complete: function(err, stmt, rows) {
if (err) {
reject(err);
} else {
let ret = parseInt(rows[0].COUNT);
return Promise.resolve(ret);
}
}
});
} catch (err) {
error(err);
}
});
}
I think my question is similar to How can I execute Snowflake statement in async mode using NodeJs Snowflake driver? but I did not find any answer there.
Because of the async nature of the complete I never manage to return the value of the complete block.
I've tried to make it await, I've tried to make the function async and return a promise the point is that when I then call the function it still always ignores the wait (and I see in the log the executions with the actual result after my code that was supposed to wait for it already moved one).
Honestly I did not find any good example of this based in Snowflake SDK so I was wondering if anyone knows of a good example to test thigs.
I've seen a lot of different posts on javascript about this like
How to return the response from an asynchronous call
But honestly I do not get it so I really wanted to know if someone has some example based on the Snowflake SDK I could use for inspiration.
I will close this question but somehow I can't make my code wait on the promise
Basically this code just does nothing.
checkRecords(conn, stmtTextStage).then(value => {
if (value > 0) {
log(`STAGE.${name} contains ${value} unique records!`);
} else {
log(`STAGE.${name} contains no records!`, 'WARN');
}
});
well I did manage to make it work but took me a while because I really don't understand this Promise resolve thing but something like this did work:
async function checkRecords(conn: Connection, sqlText: string): Promise<number> {
return new Promise ((resolve, reject) => {
try {
conn.execute({
sqlText: sqlText,
complete: function(err:SnowflakeError, stmt: Statement, rows: any[]) {
if (err) {
error(`${stmt.getSqlText()} : ${err.code}`);
reject(0);
} else {
if (rows.length === 1) {
resolve(parseInt(rows[0].COUNT));
} else {
error(`${sqlText} returned un-expeted rows!`);
reject(0);
}
}
}
});
} catch (err) {
error(err);
}
});
}

How do you push function calls onto an array without calling them in Node JS for using Q?

I'm looking to create an array of functions to call dynamically, which will be later used in the Q.all([]) promise call.
For example;
//data is previously generated
var promiseArray = [];
for (var i = 0; i < data.length; i++){
promiseArray.push(functionCall(data[i]))
}
Q.all(promiseArray).then(function(){
//Do something
})
How would I push to the array without calling the function until the Q.all statement? I don't want to call it in the for loop as it will not catch any errors and I can't process the response further.
EDIT:
So to clarify my problem (as I don't think I was as clear as I should have been), here is a solution for a static data length of say 3;
//data is previously generated
var data = [12432432,4324322392,433324323];
//Each function call can happen in parallel or series as its an external POST to an API
//I'm not bothered about speed for this application (as its low throughput) and can wait a few seconds for each
// response
//FunctionCall returns a promise
functionCall(data[0]).then(function(){
//Log success / failure to mongo
});
functionCall(data[1]).then(function(){
//Log success / failure to mongo
});
functionCall(data[2]).then(function(){
//Log success / failure to mongo
});
//OR
functionCall(data[0]).then(function(){
//Log success/failure to mongo
functionCall(data[1]).then(function(){
//Log success/failure to mongo
functionCall(data[2]).then(function(){
//Log success/failure to mongo
});
});
});
But I wont know the length of data until runtime
If I understand correctly, you want to call functionCall for an array of items, and have Q.all resolve once all the promises returned by functionCall have completed regardless if they resolve or reject - if you don't care about the results (as you don't seem to in your code) simply handle the rejection in the promise you push - i.e.
var promiseArray = [];
for (var i = 0; i < data.length; i++) {
promiseArray.push(functionCall(data[i]).then(function(result) {
// log success
return logToMongoFunction(result);
}, function(error) {
// log failure
return logToMongoFunction(error);
}).catch(function(error) {
// catch and ignore any error thrown in either logToMongoFunction above
return;
}));
}
Q.all(promiseArray).then(function () {
//Do something
});
Note: the above can be simplified to
Q.all(data.map(function (item) {
return functionCall(item).then(function(result) {
// log success
return logToMongoFunction(result);
}, function(error) {
// log failure
return logToMongoFunction(error);
}).catch(function(error) {
// catch and ignore any error thrown in either logToMongoFunction above
return;
});
})).then(function() {
//Do something
});
the edited question suggests you can perform the actions in series also - in series it would be
data.reduce(function(promise, item) {
return promise.then(function() {
return functionCall(item).then(function(result) {
// log success
return logToMongoFunction(result);
}, function(error) {
// log failure
return logToMongoFunction(error);
}).catch(function(error) {
// catch and ignore any error thrown in either logToMongoFunction above
return;
});
});
}, Promise.resolve()).then(function() {
// all done
});
instead of Promise.resolve() you could use whatever Q has as an equivalent that creates a resolved promise
logToMongoFunction would log to mongo and needs to return a promise if you need to wait for that to finish before processing the next data item. If you do not need to wait for the mongo logging to complete then there's no need for that function to return a promise
i will recommend using Promise.mapSeries or async library for this because its very easy to catch errors. One more thing looping using a for loop doesnt seems to be good approach if you have database calls in the callback because that might flush the calls to the database and node.js can have memory issues or node.js wont be able to entertain any other request because it will be busy entertaining the request in the for loop. so its always good to run loop serially or limit the numer of parallel executions at a time.
please see example below
This will run Array serially one at a time when 1st one completes execution next will be called
async.eachOfSeries(data, function(dataInstance, key, next) {
functionCall(dataInstance).then(function(){
next();
}).catch(funtion(err){
next(err);
})
}, function() {
//iteration completed
});
OR
async.eachOfSeries(data, function(dataInstance, key, next) {
functionCall(dataInstance, function(err , result){
if(err)
{
console.log(err);
next(err);
}
else
next();
});
}, function() {
//iteration completed
});

how to handle sinon.stub().throws() in unit test by Sinon JS

Am trying to invoke fail condition in my snippet.
But when I use sinon.stub().throws() method It shows me error.
Am unable to handle it in code.
Here is my snippet:
login() {
let loginData = this.loginData;
return this.authService.login(loginData).then(userData => {
let msg = `${this.niceToSeeYouAgain} ${userData.email}!`;
this.userAlertsService.showSuccessToast(msg);
this.navigationService.afterLoggedIn();
//above lines are covered in test cases
}, errorInfo => {
// below line are needed to test
this.userAlertsService.showAlertToast(errorInfo);
});
}
**And here is my unit-test snippet: **
it('.login() - should throw exception - in failure case', sinon.test(() => {
let errorInfo = "some error";
let stub = sinon.stub(authService, 'login').throws();
let spy1 = sinon.spy(controller.userAlertsService, 'showAlertToast');
//call function
controller.login();
// $timeout.flush();
// expect things
console.log(stub.callCount, stub.args[0]);
}));
Please let me know what am doing wrong
You need to wrap the function that you know is going to fail, and then call it. e.g.
it('handles errors in methodThatCallsAnotherFailingMethod', function() {
error = new Error("some fake error");
sandbox.stub(SomeObject, "doSomething").throws(error);
call = function() {
// methodThatCallsAnotherFailingMethod calls SomeObject.doSomething()
methodThatCallsAnotherFailingMethod();
};
expect(call).to.throw(Error);
});
When testing (or spying on) other stuff in methodThatCallsAnotherFailingMethod you can do this in your test:
try {
call();
} catch (error) {
expect(MySpy).to.have.been.calledWith(error);
}
This question is a month old as of this answer, but I encountered a similar error, and Google hasn't yielded any explanation for this behavior. I wanted to test the failure branch of my login as well, and stub.throws() actually threw the error (causing the test to fail) instead of rejecting the login promise. If anyone knows why this happens, I'd appreciate it.
In any case, this is what worked for me:
let d = Q.defer(); // Or whichever promise library you use
d.reject(); // Force the promise to fail
let stub = sinon.stub(authService, 'login').returns(d.promise); // Should do what you want
// The rest of the test

Angular service and pouchdb

How do you use angularjs service to call pouchdb and return the data to the controller? I have been working on a ionic app with pouchdb for local storage. I have a simple crud app built in a controller. Now I want to start to move the pouchdb calls into a service. I haven’t been able to get back data from the service. How would I use a service to call pouchdb to get all docs and return it to the controller?
One strategy that I think could work very well for Angular services is this one. It describes a method for keeping an in-memory array synced with the result of PouchDB's allDocs().
Since it's an array that automatically stays synced with PouchDB, you can just do an ng-repeat on it, and you're done. :)
Although your question is a year old, it deserves an answer.
You might want more than one service i.e. one to use in the controller and another for the backend database storage. For example, in the controller:
(function () {
'use strict';
angular
.module('app.services')
.factory('db',db);
db.$inject = ['$db'];
function db($db) {
var data = {}; // set up a data object to receive document(s)
return {
getDoc: getDoc,
getList: getList,
save: save,
saveBatch: saveBatch
};
// get a single document using the id
function getDoc(id) {
$db.getDoc(id)
.then(
function onSuccess(doc) {
// success so update the view model
angular.extend(data,doc); // use angular.extend to shallow copy object so that it can be returned in full
},
function onError() {
// failure to get document
}
);
return data;
}
// retrieve a group of documents where key is the prefix of the data you want
function getList(key) {
$db.getList(key).then(
function onSuccess(docs) {
// success so update the view model details
angular.forEach(docs.rows, function (value) {
this.push(value.doc);
}, data);
// now you can sort data or anything else you want to do with it
},
function onError() {
// no data found
}
);
return data;
}
// save a single viewItem
function save(viewItem) {
$db.update(viewItem).then(
function onSuccess() {
// success so update view model if required
},
function onError(e) {
console.log(e); // unable to save
}
);
}
// save an array of viewItems
function saveBatch(viewItems) {
$db.updateBatch(viewItems).then(
function onSuccess() {
// success so update the view model if required
},
function onError(e) {
console.log(e); // unable to save
}
);
}
}
})();
For the backend, something like this:
(function () {
'use strict';
angular
.module('app.services')
.factory('$db',$db);
$db.$inject = ['$q'];
function $db($q) {
var db;
return {
setLocalDB: setLocalDB,
update: update,
updateBatch: updateBatch,
getDoc: getDoc,
getAllDocs: getAllDocs,
getList: getList
};
// ------ DATABASE OPENING HANDLER(S) ------
// set to any named database
function setLocalDB(dbName) {
db = new PouchDB(dbName);
return db.info()
.catch(failedCheck()); // returns a promise to either work or fail
}
// return a rejection for a failure
function failedCheck() {
return $q.reject();
}
// ------ DOCUMENT HANDLING ------
// update document but if errors occur recurse qUpdate until either complete or retries exhausted
function update(doc) {
var counter = 0;
return $q.when(qUpdate(doc,counter));
}
// this routine works for both new and existing documents
function qUpdate(doc,counter) {
return db.put(doc)
.then(function() {
console.log('success - new document');
})
.catch(function(e) {
console.log(e); // not a new document so try as a revision of existing document using _id to find
return db.get(doc._id)
.then(function(origDoc) {
doc._rev = origDoc._rev; // get document revision _rev
return db.put(doc,doc._id,doc._rev)
.then(function() {
console.log('success - revision of document');
})
.catch(function(e){
console.log(e); // log error for failure
});
})
.catch(function(e){
console.log(e); // log error before we take any other action
counter ++; // increment counter, so we can limit retries (5 by default)
if (counter< 5) {
switch (e.status) {
case 404:
delete doc._rev; // remove revision information so we can see if this works
return qUpdate(doc); // might be deleted so return revised document for retry
case 409:
return qUpdate(doc); // in conflict so try again
default:
try {
throw new Error("cannot save: " + doc._id); // cannot go any further so throw new error
} catch(err) {
console.log(err); // log error for failure
}
}
} else {
try {
throw new Error("cannot save" + doc._id); // cannot go any further so throw new error
} catch(err) {
console.log(err); // log error for failure
}
}
});
});
}
// update a document batch stored in an array
function updateBatch(docs) {
return $q.when(qUpdateBatch(docs));
}
// do the actual update of a batch
function qUpdateBatch(docs) {
db.bulkDocs(docs).then(function(res) {
for (var i=0; i < res.length; i++) {
if (res[i].status === 409) {
update(docs[i]); // in conflict so try this document separately
}
}
}).catch(function(e){
console.log(e); // log error
});
}
// get the document as an angular promise and deal with it in host routine
function getDoc(id) {
return $q.when(db.get(id));
}
// get all documents
function getAllDocs() {
return $q.when(db.allDocs({include_docs: true, attachments: false}));
}
// get a batch of documents between a start and end key
function getList(key) {
return $q.when(db.allDocs({startkey: key, endkey: key + '\uffff', include_docs: true, attachments: false}));
}
}
})();
In your main controller you would want to set the database:
$db.setLocalDB('yourDB');
Hope this is what you were looking for?
In my own data services module I have other functions for remote database, event listeners, remove, sync, compact, destroy and so on.

How to roll back changes when there is an error in a promise chain

In my angular app I want to make changes to several locations in my firebase with a mix of transactions and set. I have written a promise chain with a little help. Now I need to handle any errors that may occur.
In the event of an error on any of the promises I would want to roll back any changes made in firebase (the successful promises) and alert the user to the failure.
Current code below
$scope.addNewPost = function() {
var refPosts = new Firebase(FBURL).child('/posts').push();
// Get tags into array for incrementing counters
var tags = $scope.post.tags.split(', ');
var allPromises = [];
// Iterate through tags and set promises for transactions to increment tag count
angular.forEach(tags, function(value, index){
var dfd = $q.defer();
var refTag = new Firebase(FBURL).child('/tags/' + value);
refTag.transaction( function (current_value) {
return current_value + 1;
}, function(error, committed, snapshot) {
if (committed) {
dfd.resolve( snapshot );
} else {
dfd.reject( error );
}
});
allPromises.push( dfd.promise );
});
// Add promise for setting the post data
var dfd = $q.defer();
refPosts.set( $scope.post, function (error) {
if (error) {
dfd.reject(error);
} else {
dfd.resolve('post recorded');
}
});
allPromises.push( dfd.promise );
$q.all( allPromises ).then(
function () {
$scope.reset(); // or redirect to post
},
function (error) {
// error handling goes here how would I
// roll back any data written to firebase
alert('Error: something went wrong your post has not been created.');
}
);
};
So what I need to know is how do I roll back any changes that happen to my firebase data in the event that one of these promises fail. There could be any number of updates happening in firebase. (for example: 3 tags being incremented via transaction and the post data being set)
How would I write the failure function to calculate what was successful and undo it? If this is this even possible.
--------------- sub question from original post has been solved ---------------
Also how do you force errors? I've tried setting a variable like below but it doesn't seem to work, is there something wrong with my .then?
refPosts.set( $scope.post, function (error) {
var forceError = true;
if (forceError) {
dfd.reject(forceError);
} else {
dfd.resolve('post recorded');
}
allPromises.push( dfd.promise );
});
There are two instances of this line, and they are both in the wrong place:
allPromises.push( dfd.promise );
In the first block, it should be in the last statement in the forEach callback, not in the transaction callback.
In the second block, it should be after the call to set(), not in the callback.
The way your code is written now, $q.all() is getting an empty array of promises. That could also be what's interfering with the forceError test you're attempting.

Resources