MongoDB and NodeJS write array of objects from mongodb to a json file - arrays

I need to write an array of places to a JSON file, I take the places from the MongoDB. The code below work
'use strict';
const jsonfile = require('jsonfile');
const debug = require('debug')('myproject:server');
const Place = require('../models/place.js');
const path = './public/places.json';
Place.find({
reviewed: true
}, (err, places) => {
if (err) {
debug(err);
} else {
jsonfile.writeFile(path, places, (err) => {
if (err) {
debug(err);
}
});
}
});
This is what a single object in the JSON file looks like
{ _id: 58b8d11cbaa41f288236d5fa,
__v: 0,
mainImg: 'placeImg-1490464803517.jpg',
reviewed: true,
date: 2017-03-03T02:12:44.313Z,
description: 'Lorem ipsum dolor sit amet, consectetur adipisicing elit. Eum illo odit animi architecto.',
coord: { lng: 2.166948616504669, lat: 41.382971076851476 },
type: 'museum',
title: 'Lorem' }
Since I have a lot of objects in the array places it makes sense to remove the properties that are not used on the client side, such as __v and reviewed
I tried doing following before writing array to the file
let shorterPlaces = [];
places.forEach((el, i) => {
delete el['__v'];
delete el['reviewed'];
shorterPlaces.push(el);
});
and then writing shorterPlaces to a file, but the properties remained.
When I tried logging out the object keys inside the for each loop with console.log(Object.keys(el)); I got [ '$__', 'isNew', 'errors', '_doc' ] which does not make any sense to me. Is there something that I am missing here or unaware of?

Based on the file you're requireing it looks like you're using Mongoose. Mongoose collection's find method returns Mongoose documents, which are objects with their own methods and getters/setters, rather than just the plain data in the database.
If you want just the data in a regular JS object, you need to use lean:
Place.find({ reviewed: true })
.lean()
.exec((err, places) => {
// Here, places will be an array of plain objects that you can
// manipulate as normal
})

Related

How to convert JSON object to an Typescript array in an api

I'm having an API request which returns the following:
{
"FarmerDetails": [
{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
},
{
"userId": 1,
"id": 2,
"title": "quis ut nam facilis et officia qui",
"completed": false
}
]
}
I need to show the title when I click the button and when the button clicks this "fetchPeople()" function gets call.
fetchPeople() {
this.peopleService.fetchPeople().subscribe((res:any)=>{
this.people$ = res;
});
}
How can I convert my Object to Array in typescript and show particular or all values.
Handling an API response object is no different to handling any other JSON object. Just query it in the same way.
Given a response like this:
{
"FarmerDetails": [
// array of objects you want to extract from the response
]
}
You can try several approaches.
Query the JSON object directly
fetchPeople() {
this.peopleService.fetchPeople().subscribe((res:any)=>{
this.people = res.FarmerDetails;
});
}
Map in the pipe
fetchPeople() {
this.peopleService.fetchPeople().pipe(
.map(res => res.FarmerDetails)
).subscribe(people => {
this.people = people;
});
}
When the people service emits a value that looks like the JSON above, it applies the transform in the map function before returning it to the subscriber.
Pluck in the pipe
fetchPeople() {
this.peopleService.fetchPeople().pipe(
.pluck('FarmerDetails')
).subscribe(people => {
this.people = people;
});
}
Similar to map, when the people service emits a value that looks like the JSON above, it returns the named property in the pluck function before returning it to the subscriber.
Approaches 2 and 3 would be preferable, as you can apply these in you service. Assuming that these come from an HTTP request, you can use the pipe after the get:
export class PeopleService {
fetchPeople() {
return this.http.get(url).pipe(
pluck('FarmerDetails')
);
}
}
Also, the "$" suffix on variables is generally used to denote an observable. In your example, you're just storing the value that is emitted by the observable, so it would be confusing to name it people$. I have named it people for this reason.
When you get response from api, it will return in string format. So, you require to convert it to json format. Parse it to json:
this.people=JSON.parse(res);

Push/Pull values on nested array mongoose

Im trying to update a value in a nested Array in a mongoose schema using express. I have the code required in place what i figuered i needed to update it, but the array doesn't get updated.
So the idea is to be able to have an array of data base schema objects with two fields, schemaName and schemaFields. I want to be able to update (add/remove)the values from schemaFields field of a specific schema object as needed.
I've already tried a bunch of stuff on here and elsewhere on the internet but nothing appears to work. I tried using findOneAndUpdate, findByIdAndUpdate etc.
My mongoose schema is as follows,
let databaseSchema = new Schema({
schemaName: { type: String },
schemaFields: [String]
});
let databaseSchemas = new Schema(
{
dbSchemas: [databaseSchema]
},
{
collection: 'databaseSchemas'
}
);
my update function is as follows,
schemasModel.mongo
.update(
{
_id: req.body.documentId,
'dbSchemas._id': req.body.schemaId
},
console.log('preparin to push the field \n'),
{
$push: {
'dbSchemas.$.schemaFields': req.body.newField
}
}
)
.then(() => {
res.send('new field added successfully');
});
So I solved it by removing the console.log() as a second argument to the model.update() function. Apparently this has to be the object with the operation.
The working code for the Model.update function is as follows,
schemasModel.mongo
.update(
{
_id: req.body.documentId,
'dbSchemas.schemaName': req.body.schemaToSearch
},
{
$push: {
'dbSchemas.$.schemaFields': req.body.newField
}
}
)
.then(() => {
res.send('new field added successfully');
});

Firebase: Multi Location Update using Firebase Object Observable

I'm trying to work out how to do a multi-location update using the FirebaseObjectObservable.
This is what my data looks like.
recipes: {
-R1: {
name: 'Omelette',
ingredients: ['-I1']
}
}
ingredients: {
-I1: {
name: 'Eggs',
recipes: ['-R1']
},
-I2: {
name: 'Cheese',
recipes: []
}
}
I want to then update that recipe and add an extra ingredient.
const recipe = this.af.database.object(`${this.path}/${key}`);
recipe.update({
name: 'Cheesy Omelette',
ingredients: ['-I1', '-I2']
});
And to do multi-location updates accordingly:
recipes: {
-R1: {
name: 'Cheesy Omelette',
ingredients: ['-I1', '-I2'] // UPDATED
}
}
ingredients: {
-I1: {
name: 'Eggs',
recipes: ['-R1']
},
-I2: {
name: 'Cheese',
recipes: ['-R1'] // UPDATED
}
}
Is this possible in Firebase? And what about the scenario where an update causes 1000 writes.
Storing your ingredients in an array makes it pretty hard to add an ingredient. This is because arrays are index-based: in order to add an item to an array, you must know how many items are already in that array.
Since that number requires a read from the database, the code becomes pretty tricky. The most optimal code I can think of is:
recipe.child("ingredients").orderByKey().limitToLast(1).once("child_added", function(snapshot) {
var updates = {};
updates[parseNum(snapshot.key)+1] = "-I2";
recipe.child("ingredients").update(updates);
});
And while this is plenty tricky to read, it's still not very good. If multiple users are trying to change the ingredients of a recipe at almost the same time, this code will fail. So you really should be using a transaction, which reads more data and hurts scalability of your app.
This is one of the reasons why Firebase has always recommended against using arrays.
A better structure to store the ingredients for a recipe is with a set. With such a structure your recipes would look like this:
recipes: {
-R1: {
name: 'Omelette',
ingredients: {
"-I1": true
}
}
}
And you can easily add a new ingredient to the recipe with:
recipe.update({ "ingredients/-I2": true });

Pushing onto Mongo SubDoc of SubDoc array

I'm going around in circles with this one so hoping someone can help. I'm building a nodejs application that receives sensor values from nodes. There can be multiple sensors on a node.
Using NodeJS, Mongod DB and Mongoose, all running on a raspberry pi, 3 I've built the following Schemas & Model:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var valueSchema = new Schema ({
timestamp: {type: Date},
value: {}
});
var sensorSchema = new Schema ({
id: {type: Number},
type: {type: String},
description: {type: String},
ack: {type: Boolean},
values: [valueSchema]
});
var SensorNode = mongoose.model('SensorNode', {
id: {type: Number, required: true},
protocol: {},
sensors: [sensorSchema]
});
I can add in the node, and push sensors onto the sensors array, but I seem unable to push values onto the values array.
I've looked over a few other examples and questions on similar issues, and looked at using populate, but cant seem to get them to work.
Here is my code:
function saveValue(rsender, rsensor, payload) {
var value = {
values: {
timestamp: new Date().getTime(),
value: payload
}
}
SensorNode.findOneAndUpdate({
"id": rsender,
"sensors.id": rsensor
}, {
"$push": {
"sensors.$": value
}
}, function(err, result) {
if (err) {
console.log(err);
}
console.log(result);
});
}
This is returning undefined for the result and this error:
MongoError: exception: Cannot apply $push/$pushAll modifier to non-array
Values is definitely an array in the sensor schema.
I'm using readable ids rather than the auto assigned Mongo DB IDs for the sake of the UI, but I could use the MongoDB _id if that makes any difference, I don't see why it would?
Where am I going wrong ?
You're using positional operator $ so let's check the docs
The positional $ operator identifies an element in an array to update without explicitly specifying the position of the element in the array. To project, or return, an array element from a read operation, see the $ projection operator.
So sensors.$ will return one particular document from your sensors array. That's why you're getting an error. On this level of your document you can only replace this item by using $set. I bet you wanted to do something like this:
SensorNode.findOneAndUpdate({
"id": rsender,
"sensors.id": rsensor
}, {
"$push": {
"sensors.$.values": payload
}
});
This operation will just append payload to values array in one particular sensor with id equal to rsensor.

Warning: Each child in an array or iterator should have a unique "key" prop

G'Day.
I want to iterate over a bunch of JSON objects and turn them into React Elements. The objects look like this
"fields":
[
{
key: "testname",
"name": "testname",
"altName": "",
"visible": true,
"groupVisibility": "public",
"type": "text",
"component": "input",
"label": "Test Smart Input",
"placeholder": "Some default Value",
"required": "required",
"validated": false,
"data": []
},
{
key: "password",
"name": "password",
"altName": "",
"visible": true,
"groupVisibility": "public",
"type": "password",
"component": "input",
"label": "Test Smart Input",
"placeholder": "Password",
"required": "required",
"validated": false,
"data": []
}
]
And the code that iterates over them is quite simple. Such:
//--------------------
formFields(fieldsIn) {
const fieldsOut = []; // ARRAY of FORM ELEMENTS to return
console.log('doin fields');
for (var fieldIn in fieldsIn) { // array of FORM ELEMENT descriptions in JSON
console.log(fieldIn);
let field = React.createElement(SmartRender, // go build the React Element
fieldIn,
null); // lowest level, no children, data is in props
console.log('doin fields inside');
fieldsOut.push(field);
}
return(fieldsOut); // this ARRAY is the children of each PAGE
}
And I get the error
Warning: Each child in an array or iterator should have a unique "key" prop.
Any hints?
Cheers
I changed the code to do this.
//--------------------
formFields(fieldsIn) {
const fieldsOut = []; // ARRAY of FORM ELEMENTS to return
console.log('doin fields');
for (var fieldIn in fieldsIn) { // array of FORM ELEMENT descriptions in JSON
console.log(fieldIn);
let field = React.createElement(SmartRender, // go build the React Element
{key: fieldsIn[fieldIn].name, fieldIn},
null); // lowest level, no children, data is in props
console.log('doin fields inside');
fieldsOut.push(field);
}
return(fieldsOut); // this ARRAY is the children of each PAGE
}
And I get the same error. I don't understand why!
Fixed! Thanks for the help.
Here is the code.
//--------------------
formFields(fieldsIn) {
const fieldsOut = []; // ARRAY of FORM ELEMENTS to return
for (var fieldIn in fieldsIn) { // array of FORM ELEMENT descriptions in JSON
console.log(fieldIn);
let field = React.createElement(SmartRender, // go build the React Element
{key: fieldsIn[fieldIn].key, fieldIn},
null); // lowest level, no children, data is in props
fieldsOut.push(field);
}
return(fieldsOut); // this ARRAY is the children of each PAGE
}
//----------------------
pages(pagesIn, format) {
// I tried to do this in JSX, but no syntax I wrestled with would
// allow me to access the childred when building this with the
// BABEL transpiler. Same goes for the METHOD just above, items().
//
// This method returns an array of pages this are React elements
// this are treated as children by the smartForm.
const pagesOut = []; // array of pages to build and return
let Section = {}; // Component to fire in the build
switch(format) {
case 'accordion': {
Section = AccordionSection;
break;
}
case 'workflow': {
Section = null; // I haven't written this yet
break;
}
case 'simple': {
Section = null; // I haven't written this yet
break;
}
}
for (var pageIn in pagesIn) { // pages, any format, any number 1..N
let children = this.formFields(pagesIn[pageIn].fields); // 1..N fields, we don't know beforehand
let page = React.createElement( Section,
pagesIn[pageIn].props,
children);
pagesOut.push(page);
}
return(pagesOut); // this ARRAY is the children of each FORM
}
//--------
render() {
let formIn = this.props.form; // JSON description of FORM
let formOut = null; // contructed REACT/Javascript form
switch (formIn.format) { // what type of operation is this
case 'accordion': { // Accordion in a FORM, OK
let children = this.pages(formIn.pages,
formIn.format); // build the children
formOut = React.createElement(Accordion, // construct the parent with ALL nested CHILDREN after
{key: formIn.formName}, // just a unique key
children); // N ACCORDION pages, N2 input fields
break;
}
case 'workflow': {
let children = this.pages(formIn.pages, // build the children
formIn.format); // build the children
formOut = React.createElement(Workflow, // and create the complex sheet element
{ key: formIn.formName},
children); // N SLIDING pages, N2 input fields
break;
}
case 'simple': {
let children = this.pages(formIn.pages, // build the children
formIn.format); // build the children
formOut = React.createElement(Simple,
{ key: formIn.formName},
children); // One page, N input fields
break;
}
}
return(
<div>
<h2>SmartForm Parser</h2>
<p>"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."</p>
{formOut}
</div>
);
}
}
//-------------------------------------------------------------------------
export default SmartForm;
//----------------- EOF -------------------------------------------------
You need to add a unique key prop to your React element.
According to the React docs:
Keys help React identify which items have changed, are added, or are
removed. Keys should be given to the elements inside the array to give
the elements a stable identity.
The best way to pick a key is to use a string that uniquely identifies
a list item among its siblings. Most often you would use IDs from your
data as keys
When you don’t have stable IDs for rendered items, you may use the
item index as a key as a last resort
You can do it like
for (var fieldIn in fieldsIn) { // array of FORM ELEMENT descriptions in JSON
console.log(fieldIn);
let field = React.createElement(SmartRender, // go build the React Element
{key: fieldsIn[fieldIn].key, fieldIn},
null); // lowest level, no children, data is in props
console.log('doin fields inside');
fieldsOut.push(field);
}
Why are keys necessary?
By default, when recursing on the children of a DOM node, React just iterates over both lists of children at the same time and generates a mutation whenever there’s a difference.
For example, when adding an element at the end of the children, converting between these two trees works well:
<ul>
<li>first</li>
<li>second</li>
</ul>
<ul>
<li>first</li>
<li>second</li>
<li>third</li>
</ul>
React will match the two <li>first</li> trees, match the two <li>second</li> trees, and then insert the <li>third</li> tree.
If you implement it naively, inserting an element at the beginning has worse performance. For example, converting between these two trees works poorly.
<ul>
<li>first</li>
<li>second</li>
</ul>
<ul>
<li>third</li>
<li>first</li>
<li>second</li>
</ul>
That is where keys come in handy.

Resources