Node.JS Append Array and Convert to JSON - arrays

I have the below that runs a few functions to grab different data and i am trying to create/append to an array to "create" a new array that i can then JSON.stringify and pass to the Jade view, i seem to only be able to get the "k" variable of the two loops it does and outputs as "coin1coin2" with the "" instead of a JSON array with the balance and addresses (multiples) after each coin name.
// list all wallets //
app.get('/wallet', function(req, res){
var coins = app.get('coins');
var addresses = "";
for (var k in coins) {
addresses += k;
console.log('Coin: ' + k);
//console.log(k+' => '+coins[k]);
connect(coins[k]);
client.getBalance('44abc0a80102', 6, function(err, balance) {
if (err) return console.log(err);
addresses += balance;
//console.log('Balance: ' + balance);
});
client.getAddressesByAccount('44abc0a80102', function(err, address) {
if (err) return console.log(err);
addresses["address"] += address;
});
}
console.log(addresses);
res.render('wallet', {
title: 'Wallet',
address: JSON.stringify(addresses)
});
});
In the Jade view i am just doing p= address to show the full array so i can at least see what i am getting before creating logic in the view to loop through it.

The problem here is that your for loop is synchronous, whereas your client.getBalance and client.getAddressesByAccount calls are (almost definitely) not.
When your for loop finishes, your client.getBalance and client.getAddressesByAccount operations are still in progress, so they haven't had a chance to populate your addresses thing yet. Thus it is empty when it's passed to the templating system.
You probably want to use something like async's map method to do this instead. It'd look something like this:
// list all wallets
app.get('/wallet', function(req, res){
// array? object?
var coins = app.get('coins');
async.map(coins, function(coinId, cb) {
// ???
console.log('Coin: ' + JSON.stringify(coin));
// not sure what this does!
connect(coin);
client.getBalance('44abc0a80102', 6, function(err, balance) {
if (err) return cb(err);
client.getAddressesByAccount('44abc0a80102', function(err, address) {
if (err) return cb(err);
return cb(null, {
address: address,
balance: balance,
});
});
});
}, function(err, addresses) {
// note that we only act on an error here
if (err) {
return console.log(err);
}
// [{address: ..., balance: ...}, ...]
console.log(addresses);
res.render('wallet', {
title: 'Wallet',
address: JSON.stringify(addresses),
});
});
});
The addresses value will be an array of {address: ..., balance: ...} objects. I'm not sure what you were expecting it to be, since at one point you treat it like a string, then you treat it like an object.

Related

Construct array in nodejs through for loop

I am using sails (0.11.0) running on nodejs (6.9.1). I am trying to construct an array by filling it through for loop. I would send this completed array in response to the client. I have tried various methods as suggested by people here on Stack Overflow, for example
the discussion here suggested
for (var i = yearStart; i < yearEnd+1; i++) {
arr.push(i);
}
On this discussion, it is suggested to use:
var array = calendars.map(function(item) {
return item.id;
});
console.log(array);
Similarly I tried many methods but I am coming across the same issue that during the loop, the array gets filled but as soon as the loop is completed, the array gets empty because of asynchronous process and therefore I can not send the response. To tackle with this I tried checking the index inside the loop body and send response from inside the loop body itself through
var userArray = [];
_.each(users, function(user, index){
MySQLConnector.query('CALL user_image (?)', [user.id], function(err, userImage){
if(err){
return res.json({"status":"some_error"});
}else{
userID = user.id
userImageID = userImage[0][0].id;
var userInfo = {
userID: userID,
userImageID: userImageID
}
userArray.push(userInfo)
if(index == users.length - 1){
res.json({selectedUsers: userArray});
}
}
});
});
I am initiating an empty userArray and then iterate through users object where each element of the object is characterized by name user and an index. Through a MySQL query I am fetching the userImage object and in each iteration, I am creating an object called userInfo that consists of userID and userImageID. I am pushing this object into userArray. And after each iteratio of the for loop (_.each), I check if last index is reached. Once last index is reached, the final array is sent as response before loop body is complete.
Here too I have an issue that the array body is not always completely filled. The reason is due to asynchronous process, the index does not always follow the order 0,1,2,3,4,.... and it can start with any number and can jump to any index in the next iteration, for example the first index to start would be 4, the second would be 0, third would be 2 and so on. This sequence would be different for every time we run this for loop. For a user, it will appear to be a total random process. Therefore if users.length is 8, and current index is randomly 7 at third iteration, the condition index == users.length - 1 will be met and response will be sent just with an array consisting of 3 elements rather than 8.
Can someone suggest me a better and robust way to fill an array through the for loop in nodejs and send that array in response, so that all items are included in the array in their original order?
As you are using node js , it is better to use any promises library like bluebird or async to handle Async requests.
The reason your loop is not working as expected is because as you've pointed out, due to async requests taking time to resolve for which _.each loop is not waiting.
Using bluebird, it can be done with Promise.map method which works as explained below from the documentaion :
Given an Iterable(arrays are Iterable), or a promise of an Iterable,
which produces promises (or a mix of promises and values), iterate
over all the values in the Iterable into an array and map the array to
another using the given mapper function.
Promises returned by the mapper function are awaited for and the
returned promise doesn't fulfill until all mapped promises have
fulfilled as well. If any promise in the array is rejected, or any
promise returned by the mapper function is rejected, the returned
promise is rejected as well.
Hence, Using Promise.map your code can be updated like below :
var Promise = require("bluebird");
return Promise.map(users, function(user, index){
return MySQLConnector.query('CALL user_image (?)', [user.id], function(err, userImage){
if(err){
return Promise.reject({"status":"some_error"});
}else{
userID = user.id
userImageID = userImage[0][0].id;
var userInfo = {
userID: userID,
userImageID: userImageID
}
return userInfo;
}
});
})
.then(function (usersArray){
res.json({selectedUsers: usersArray});
})
.catch(function (err){
res.json(err);
});
You can execute loops with functions with callbacks synchronously using SynJS:
var SynJS = require('synjs');
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'tracker',
password : 'tracker123',
database : 'tracker'
});
function myFunction1(modules,connection,users) {
var ret=[];
for(var i=0; i<users.length; i++) {
connection.query("SELECT CONCAT('some image of user #',?) AS userImage", [users[i]], function(err, rows, fields) {
if (err) throw err;
ret.push({
id: users[i],
image: rows[0].userImage
});
modules.SynJS.resume(_synjsContext); // <-- indicate that callback is finished
});
SynJS.wait(); // <-- wait for callback to finish
}
return ret;
};
var modules = {
SynJS: SynJS,
mysql: mysql,
};
var users = [1,5,7,9,20,21];
SynJS.run(myFunction1,null,modules,connection,users,function (ret) {
console.log('done. result is:');
console.log(ret);
});
Result would be following:
done. result is:
[ { id: 1, image: 'some image of user #1' },
{ id: 5, image: 'some image of user #5' },
{ id: 7, image: 'some image of user #7' },
{ id: 9, image: 'some image of user #9' },
{ id: 20, image: 'some image of user #20' },
{ id: 21, image: 'some image of user #21' } ]

Output all documents in mongoose

I am using mongoose ODM and have a schema which looks like this:
var banSchema = new Schema({
userid: { type: String, required: true, unique: true },
name: String,
groupid: String,
reason: String,
timestamp: Date
});
I want to output every single user id from all documents in the collection. I am using this query to obtain the userid objects. However I cannot seem to get the full list automatically. I have to manually enter the object number as seeen below:
bot.onText(/\/sync/i, function (msg) {
var fromId = msg.from.id;
var chatId = msg.chat.id;
if (fromId == config.sudo) {
console.log('Sudo Confirmed And Authorized!');
Ban.find({}, function (err, obj) {
console.log(obj[0].userid); // Returns A Single ID
console.log(obj[1].toObject().userid); // Returns a different ID
bot.sendMessage(chatId, obj[1].toObject().useridid);
});
} else {
console.log('Someone Is Trying To Act Like Sudo! *sigh*');
bot.sendMessage(chatId, 'You Are Not A Mod!');
}
});
This however does not return a full list of id's as I want. How could I solve this issue?
The code above is for a telegram bot which on a /sync command it should return a message with all ids from the collection.
Telegram bot API Limits
Due to the API limits, the entire output should be in a single message.
var query = Ban.find({}).select({
"userid": 1,
//Add more column fields here
"_id": 0 //Ensures _id is not displayed
});
var arr = [];
query.exec(function (err, results) {
if (err) throw err;
results.forEach(function (result) {
arr.push(result.userid);
// Add more column fields here;
});
var fixedJoin =arr.join("\n");
console.log(fixed);
bot.sendMessage(chatId, 'List\n\n' + fixedJoin);
});
The easiest way to get all values of a particular field across all docs in the collection is to use distinct:
Ban.distinct('userid', function (err, userids) {
// userids is an array containing all userid values in the collection.
// string.join into a single string for the message.
bot.sendMessage(chatId, 'USER IDs\n\n' + userids.join('\n'));
});
Use this syntax
Ban.find({}).
select('userid').
exec(function(err, result) {
//result is array of userid of all document
});
You can use this syntax:
Ban.find({}, 'userid', function(err, users) {
users.forEach(function(user) {
console.log(user);
bot.sendMessage(chatId, 'users \n' + user);
});
})

NodeJS - Removing a particular item from an Array

I'm getting list of all collections from mongodb, in form of an array
mongoose.connection.db.listCollections().toArray(function (err, names) {
if (err) {
console.log(err);
}
console.log(names);
output:
[ { name: 'system.indexes' },
{ name: 'books' },
...
I want to remove that system.indexes from the array. I tried playing around with some functions like:
Splice
Pop, Shift
Underscore's withoutfunction
Well, Honestly I don't even know if they were intended for this.
To remove the objects where name is equal to system.indexes do it as follow:
mongoose.connection.db.listCollections().toArray(function (err, names) {
if (err) {
console.log(err);
}else{
var i;
for(i=names.length - 1; i >= 0; i-=1){
if(names[i].name !== 'system.indexes'){
names.splice(i,1);
}
}
// names now contain all items without the system.indexes
}

mongodb updateOne value in an array

So I have an update one function going i just need to know how to update one of the elements in an array rather than wipe and replace the whole thing.
labelRelease = function(db, callback){
db.collection($$showName).updateOne(
{'showName' : $$showName},
{
$set: {'episode[2]' : data + label}
}, function(err, results){
callback();
});
}
Is a stripped down version of the code I am using to update, obviously the episode[2] does not work to select only one array element how can i achieve this?
relevant part of the database
episode:[episode1, episode2, episode3.....]
You can update an array element by position by using dot notation:
labelRelease = function(db, callback){
db.collection($$showName).updateOne(
{'showName' : $$showName},
{
$set: {'episode.2' : data + label}
},
function(err, results){
callback();
});
};
If the index of 2 is in a variable, you need to build up your $set value in a couple steps:
var index = 2;
var setValue = {};
setValue['episode.' + index] = data + label;
labelRelease = function(db, callback){
db.collection($$showName).updateOne(
{'showName' : $$showName},
{
$set: setValue
},
function(err, results){
callback();
});
};
The correct syntax for the set operation is:
$set: { 'episode.2' : ... } }
Note that episode.2 refers to the 3rd element of the episode array.

How to iterate an array before responding to user using async

I have a the following code that currently works
but...
I know it is not elegant and can be done much more efficiently.
What I am trying to do is take an array of emails, search if they exist in a corresponding db with a particular templateName, and for those that do not exist (i.e are 'new'), list them back on the returned page to the user. However, they end up waiting quite a while if there are a lot of emails to check.
This is the first time I'm using async and it may not actually be the best way to do this. Some of the below has been modified from what I am actually using currently to make it easier to read/follow.
Basically, from my handler, I call the following (where both emailArray and templateName are extracted from the request parameter passed in).
var newEmails = "";
async.eachSeries(emailArray, function(entry, cb) { // check each item in array (these are the potential new emails)
utils.emailAddressAndTemplateExists(entry.toString().toLowerCase(), templateName, function (err, emailExists, templateExists ) {
if (emailExists) {
if (templateExists) {
++existingCount;
} else if (emailExists && !templateExists) {
} else {
console.log('template does not exist');
}
} else {
++newCount;
newEmails = newEmails + entry + "</br>";
}
cb();
});
//cb();
}, function (err) {
if (err) { throw err; }
var content = utils.getHTMLHead() + newEmails + utils.getHTMLClose();
utils.writeHTMLPage(response, content);
});
The utils call does the following: (the writeHTMLPage simply adds the required html tags and sends back to response).
//checks for a single email address
var emailAddressExists = function(emailAddress, callback) {
if (emailAddressCollection == null) {
//console.log("it was null " + db_singleton + " " + dbName);
emailAddressCollection = db_singleton.collection(dbName);
}
emailAddressCollection.find( { "emailAddress" : emailAddress.toLowerCase() } ).toArray( function (err, docs) {
if (err) { console.err(err); }
if (docs.length == 0) {
callback(null, false, docs.EmailsSent);
} else {
doc = docs[0];
callback(null, true, doc.EmailsSent);
}
});
}
// check for email And template
var emailAddressAndTemplateExists = function (emailAddress, templateName, callback) {
emailAddressExists(emailAddress, function (err, returnVal, templates) {
if (returnVal) {
if (templates != null) {
callback (null, true, templates.hasOwnProperty(templateName)) // email exists, checking for templateName
} else {
callback (null, true, false); // email exists, no templates at all exist
}
} else {
callback (null, false, false); // email does not exist, templates must be false
}
});
}
//creates HTML formated respnse data
function writeHTMLPage(response, content) {
// if (err) { console.error(err); response.send("Error " + err); }
response.writeHead(200, {"Content-Type": "text/html"});
response.write(content);
response.end();
}
What are more elegant and efficient way to do this?
This looks like it's constructed according to how you'd normally see it. You can look into Promises with ES6 to get a better program flow:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
Basically, it lets you chain together functions. It's not too bad for what you're doing now, it might help out some, but if you have 4-5 callbacks nested together, that's when promises can be very helpful.
You'll just have to work through structuring your code differently to use Promises, but it'll make "callback hell" less of an issue.

Resources