I am not a database expert., However I have a business case in which I want to store information related to a large family with more than 20,000 members , starting from a patriarch and inserting new members who are born , the data concern their names , dates of birth , occupations , data of death and so on.
What is the best data model for such case ?
After making a research I am a little bit confused between hierarchal or graph data model , So , does any of them fit the problem or another solution is better ?
This problem fits perfectly within the realm of graph databases and is fairly easy to implement. In InfiniteGraph, the data model would look something like the following:
UPDATE SCHEMA {
CREATE CLASS Person {
name : String,
dateOfBirth : DateTime,
dateOfDeath : DateTime,
education : List {
Element: Reference { EdgeClass: Education, EdgeAttribute School },
CollectionTypeName: TreeListOfReference
},
residence : List {
Element: Reference { EdgeClass: Residence, EdgeAttribute Address },
CollectionTypeName: TreeListOfReference
},
birthMother : Reference { referenced: Person },
birthFather : Reference { referenced: Person }
}
CREATE CLASS Education {
startDate : Date,
endDate : Date,
certificate : String,
school : Reference { Referenced: School }
}
CREATE CLASS School {
name : String
}
CREATE CLASS Residence {
startDate : Date,
endDate : Date,
address : Reference { Referenced: Address },
resident : Reference { Referenced: Person }
}
CREATE CLASS GeoPosition {
latitude : Float,
latitude : Float
}
CREATE CLASS Address SUPERCLASS GeoPosition {
street1 : String,
street2 : String,
city : String,
state : String,
country : String,
postalCode : String,
resident : List {
Element: Reference { EdgeClass: Residence, EdgeAttribute Person },
CollectionTypeName: TreeListOfReference
}
}
}
InfiniteGraph is a massively scalable object-oriented graph database. I've solved many problems like yours using the product.
Related
The thing that has puzzled me the most ever since I started reading about SPAs is how to handle the application state.
Although I have found in Redux most of the answers I was looking for there's still a couple of things that I'm not sure how to handle.
One of those things is how to specify the shape of the "entities" I want to use in my app. In other frameworks I've used in the past I've seen an extensive use of ES2015 classes for this purpose, but in React/Redux applications object literals are by far,the preferred way to express the application state.
Last night I was reading about normalizing data in the redux docs, and although it gave me great ideas about how to deal with deeply nested objects, it left me with the feeling that there was something missing and that thing is how to specify in a single spot and as clear as possible what the structure/shape of the state of the application is.
Take the sample data used in the article mentioned above:
{
posts : {
byId : {
"post1" : {
id : "post1",
author : "user1",
body : "......",
comments : ["comment1", "comment2"]
},
"post2" : {
id : "post2",
author : "user2",
body : "......",
comments : ["comment3", "comment4", "comment5"]
}
}
allIds : ["post1", "post2"]
},
comments : {
byId : {
"comment1" : {
id : "comment1",
author : "user2",
comment : ".....",
},
"comment2" : {
id : "comment2",
author : "user3",
comment : ".....",
},
"comment3" : {
id : "comment3",
author : "user3",
comment : ".....",
},
"comment4" : {
id : "comment4",
author : "user1",
comment : ".....",
},
"comment5" : {
id : "comment5",
author : "user3",
comment : ".....",
},
},
allIds : ["comment1", "comment2", "comment3", "commment4", "comment5"]
},
users : {
byId : {
"user1" : {
username : "user1",
name : "User 1",
}
"user2" : {
username : "user2",
name : "User 2",
}
"user3" : {
username : "user3",
name : "User 3",
}
},
allIds : ["user1", "user2", "user3"]
}
}
How would you express the shape of this data? Just like this?
{
posts : {
},
comments : {
},
users : {
}
}
Or perhaps:
{
posts : {
byId : {
}
allIds : []
},
comments : {
byId : {
}
allIds : []
},
users : {
byId : {
}
allIds : []
}
}
Well, that tells me nothing about the actual shape of the objects that live inside each of the byId objects and that's exactly what has been bothering me since I started studying a little bit of Redux.
So the question is, is there any way/common practice that would let me express in the clearest possible way, the shape, specifically each of the properties of the objects that make up the state that the store is going to manage?
As author of that "Structuring Reducers" docs section, that's because the actual content of your data items is entirely dependent on you and your application. This is really more of a generic Javascript "how do I express types?" question. There's static type systems like TypeScript and Flow; documentation-based approaches like JSDoc; runtime approaches like the React PropTypes library or tcomb; and schema-based approaches like JSON-Schema. Any of those approaches are valid ways to document and enforce data type definitions within an application.
Normalization is simply an approach for organizing those values, regardless of how you've chosen to define what their contents are.
Assuming you're using plain JS, the simplest way is to use propTypes.shape.
// An object taking on a particular shape
optionalObjectWithShape: PropTypes.shape({
color: PropTypes.string,
fontSize: PropTypes.number
}),
If you're using propTypes, I suggest declaring the shape of all objects in this manner.
Benefits:
Your object shapes are explicit.
You're alerted quickly when unexpected object shapes are passed around.
You get enhanced autocompletion support in modern editors like Webstorm.
Since the object shape is just an object, you can centralize the declaration to avoid repeating yourself.
You don't need to move to TypeScript, Flow, etc. to have explicit types and basic type safety at runtime.
The nice thing of Redux is that you have a very clear separation of concerns of the state. Every reducer is responsible of its own part of the state and if you end up to extend a reducer by adding more data/logic, you just need to breakdown that reducer in smaller reducers.
Generally, I would avoid the use of arrays inside big collections of data inside a reducer. Arrays are cool and gives you lot of nice functionalities (sorting, filtering, reducing, etc.) but looping through them is expensive. If you can, use objects and use an ID to identify your object, in this way you access to the object using the fastest way possible.
Using your example, I would use the first approach:
{
posts: {
1554: {},
1557: {}
},
comments: {},
users: {},
}
In case you need to keep track of the list objects the array, you just creates two new reducers inside your posts reducer:
// postsReducer.js
import byId from 'postsByIdReducer'
import allIds from 'allIdsReducer'
export default combineReducers({
byId,
allIds
})
So your state will be like
{
posts: {
byIds: {},
allIds: []
}
}
I have a collection Playlist that contains an array of items
{
userId: {
type : String,
required : true,
index : true,
unique : true
},
items: [
{
id: { // do not mix up with _id, which is the autogenerated id of the pair {id,type}. ID is itemId
type : Schema.Types.ObjectId
},
type: {
type : String
}
}
]
}
Mongo automatically adds the _id field to the items when I push a pair {id,type} to items (but I don't care about it).
Now I would like to remove several "pairs" at once from the items array.
I have tried using $pullAll but it requires an exact match, and I do not know the _id, so it does not remove anything from items
playlistModel.update({userId:userId},{$pullAll:{items:[{id:"123",type:"video"},{id:"456",type:"video"}]}},null,function(err){
I have tried using $pull with different variants, but it removed ALL objects from items
playlistModel.update({userId:userId},{$pull:{items:{"items.id":{$in:["123","456"]}}}},null,function(err){
playlistModel.update({userId:userId},{$pull:{items:{$in:[{id:"123",type:"video"},{id:"456",type:"video"}]}}},null,function(err){
Am I missing something or am I asking something that isn't implemented?
If the latter, is there a way I can go around that _id issue?
OK I found a way that works using $pull:
playlistModel.update({userId:userId},{$pull:{items:{id:{$in:["123","456"]}}}},null,function(err){
It doesn't take the type into account but I can't see any issue with that since the id is unique across all types anyway.
Although I will wait a bit to see if someone has a better solution to offer
EDIT
With Veeram's help I got to this other solution, which IMO is more elegant because I don't have _ids that I don't need in the database, and the $pullAll option seems more correct here
var playlistItemSchema = mongoose.Schema({
id: { // do not mix up with autogenerated _id. id is itemId
type : Schema.Types.ObjectId
},
type: {
type : String
}
},{ _id : false });
var schema = new Schema({
userId: {
type : String,
required : true,
index : true,
unique : true
},
items: [playlistItemSchema]
});
playlistModel.update({userId:userId},{$pullAll:{items:[{id:"123",type:"video"},{id:"456",type:"video"}]}},null,function(err){
tips:
you can use _id field to handle your playlistModel data.
mongoose api : new mongoose.Types.ObjectId to generate an Object_id
let _id=new mongoose.Types.ObjectId;
playlistModel.updateMany({_id:_id},{ $set: { name: 'bob' }}).exec(data=>{console.log('exec OK')});
I have a Firebase structure like this:
"warehouses": {
"warehouse1": {
"floor_sales": {
"!-K3cC1LZL_aRVWG2fVHJ" : true,
"-K3cKKgci7r8fNEjHLfn" : true
}
}
}
"sales": {
"warehouse1": {
"!-K3cC1LZL_aRVWG2fVHJ" : {
"amount" : "19"
"kgs": "2"
},
"-K3cKKgci7r8fNEjHLfn" : {
"amount" : "50"
"kgs": "5"
}
}
"warehouse2": {
...
}
}
I am able to get all sales by warehouse with $firebaseArray(FBObj.child('sales' + warehouseId));
If I get the sales from a service like this $firebaseArray(FBObj.child('sales');
I get all the sales but each sale is inside a warehouse like this:
0: Object
$$hashKey: "object:86"
$id: "warehouse1"
$priority: null
-K3VaYVQGgeB2fDV94DY: Object
-K3WCyxVoIxXkrDq2UCA: Object
-K3WDDdEt_Lvyh5DDbqG: Object
__proto__: Object
1: Object
2: Object
$$added: ()
$$error: ()
...
I need to get all the sales not only from a warehouse.
Does anyone know a way to get all sales (not only from a warehouse) to print them in a view?
Should I get the sales in another way from Firebase?
Should I print the sales in Angular in another way?
I tried to change Firebase structure to save all sales without warehouseId but then I do not have a way to get all floor_sales using Angularfire.
Any help would be very helpful thank you.
I am trying to draw a grid where each line is a stock's performance for a single day. In my data structures, I have a Date, a Stock, and a Stock Price resource. The store attached to my grid is the Stock Price store.
So, to the best of my understanding, my biggest problem is that when the grid cell renderers, I need to already have the value, or I need to have a blocking function to get a value.
When I use the getStore() magic function, I'm told the record doesn't know about it (undefined method). I'm assuming it's 'cause the record doesn't have the same functionality as a standalone model.
I see a few ways out of this:
Customise the grid and/or store so that when a load happens, all the related rows are loaded at the same time.
Create a callback in the renderer, and change the cell value afterwards, but I'm ot exactly sure how to do this. I don't actually want to change the cell value (StockId), just the visible output (Symbol).
Change my API to match my view.
Summing these up: #1 seems like a lot of work for a seemingly simple outcome. I keep trying to use the associations, but I'm finding they're not really useful for anything aside from little things here and there, and certainly not for lots of data. #2 I don't quite know where to begin at the moment; and #3 seems like massive overkill, and will generally ruin my server side as I will mean a few more joins, and more complexity when saving records as well.
So, two part question:
Does anyone know how to load a value from an associated model in a grid?
If not, to pique my curiosity, what sort of things are associations used for in any case where there's lots of data to deal with on screen? Lot's of data seems to be the reason to use Ext vs jQueryUI or some other UI framework, so I'm wondering what the associations are for.
Model - Stock Price
Ext.define('MyApp.model.StockPrice', {
extend : 'Ext.data.Model',
idProperty : 'StockPriceId',
fields : [ {
name : 'StockId',
type : 'int'
}, {
name : 'Open',
type : 'float'
}, {
name : 'Close',
type : 'float'
}, {
name : 'DateId',
type : 'date'
}],
proxy : {
type : 'rest',
url : '/api/stock.price'
},
reader : {
type : 'json'
},
associations : [ {
type : 'hasOne',
model : 'MyApp.model.Date',
primaryKey : 'DateId',
foreignKey: 'DateId'
},{
type : 'hasOne',
model : 'MyApp.model.Stock',
primaryKey : 'StockId',
foreignKey : 'StockId'
} ]
});
Model - Stock
Ext.define('MyApp.model.Stock', {
extend : 'Ext.data.Model',
idProperty : 'StockId',
fields : [ {
name : 'StockId',
type : 'int'
}, {
name : 'Symbol',
type : 'string'
} ],
proxy : {
type : 'rest',
url : '/api/stock'
},
reader : {
type : 'json'
},
associations : [ {
type : 'hasMany',
model : 'MyApp.model.StockPick',
primaryKey : 'StockId',
foreignKey : 'StockId'
}]
});
Model - Date
Ext.define('MyApp.model.Date', {
extend : 'Ext.data.Model',
fields : [ 'DateId', 'Date' ]
});
Store - Stock Price
Ext.define('MyApp.store.StockPrice', {
extend : 'Ext.data.Store',
model : 'MyApp.model.StockPrice',
remoteSort : true,
remoteFilter : true,
pageSize : 5,
autoLoad : true
});
View - Stock Price
Ext.define('MyApp.panel.StockData', {
extend : 'Ext.grid.Panel',
store : 'MyApp.store.StockPrice',
columns : [
{
text : 'Symbol',
flex : 1,
sortable : false,
hideable : false,
dataIndex : 'StockId',
renderer : function(stockId, metadata, stockPriceRecord) {
// What goes in here? PROBLEM POINT
MyApp.model.Stock.load(stockId, function() {
// ... some callback
});
// OR
return stockPriceRecord.getStock().get('Symbol');
}
},
{
text : 'Open',
flex : 1,
dataIndex : 'Open',
renderer : 'usMoney'
},
{
text : 'Close',
flex : 1,
dataIndex : 'Close',
renderer : 'usMoney'
},
{
text : 'Volume',
flex : 1,
dataIndex : 'Volume'
}]
});
Your only real option to display data from an associated model in a grid is to use a custom renderer function on the column. This will not change any values; it will simply render the desired output into the cell.
Now, as for implementing that renderer function: I would start by removing the proxies from the models and instead create stores for each model and allow the stores to manage the proxies -- then, attach the store for Stock as a listener on the store for StockPrice to listen for the datachanged event. When the data of the StockPrice store changes, you should grab every unique referenced Stock id and then tell the Stock store to request a payload of stocks with those ids.
That may mean altering your API a little bit to support a SQL IN(...) behind the scenes, but by leaving the joins to the client side you will put less stress on your server side.
In short, you need to use a little bit of all three ideas you came up with in order to best achieve your goal.
I'm trying to build a model that dynamically updates Session variables in a Meteor project. I know that plain JSON should not be stored within backbone models, so I have a Special model set up like so:
initialize : function () {
// Log the changed properties
this.on('change', function (model, options) {
for ( var i in options.changes)
this.display(i);
Session.set('NewSpecial', model);
});
},
//Attributes
defaults: {
"Product" : null,
"ShortDescription" : null,
"Category" : "food",
"Price" : new PriceModel,
"Date" : new DateModel,
"Uses" : 0,
"Tags" : [],
"Contributor" : null
},
With "Price" and "Date" being stored in their own models:
//Price model for use within Special
var PriceModel = Backbone.Model.extend({
defaults : {
"Regular" : null,
"Special" : null,
"PercentOff" : null
}
});
//Date model for use within Special
var DateModel = Backbone.Model.extend({
defaults : {
"StartTime" : null,
"EndTime" : null,
"HumanTimeRange" : null
}
});
As shown, when the attributes of the Special model change, it should call display for the attribute that changed, and then set the Session var to the model. If my DateModel or PriceModel change however, it doesn't appear to trigger a change event on the Special model. Should each "DateModel" and "PriceModel" have their own this.on('change', ...) methods that call Special.set(attribute, thisModel) methods? Or is there a different way to go about this?
I see a couple problems.
First of all, your defaults:
defaults: {
"Product" : null,
"ShortDescription" : null,
"Category" : "food",
"Price" : new PriceModel,
"Date" : new DateModel,
"Uses" : 0,
"Tags" : [],
"Contributor" : null
}
That will end up with one PriceModel, one DateModel, and one tags array being shared by all instances of that model. A defaults object is shallow copied and merged into the model's attributes, none of the values in defaults are cloned or duplicated, they're just copied over as-is. If you want distinced Price, Date, and Tags values then use a function for defaults:
defaults: function() {
return {
"Product" : null,
"ShortDescription" : null,
"Category" : "food",
"Price" : new PriceModel,
"Date" : new DateModel,
"Uses" : 0,
"Tags" : [],
"Contributor" : null
};
}
The second problem is that set has a fairly simplistic view of what change means. If you have a look at the source for set, you'll see this:
// If the new and previous value differ, record the change. If not,
// then remove changes for this attribute.
if (!_.isEqual(prev[attr], val) || (_.has(now, attr) !== _.has(prev, attr))) {
this.changed[attr] = val;
if (!silent) this._pending[attr] = true;
} else {
delete this.changed[attr];
delete this._pending[attr];
if (!changing) delete this._changes[attr];
}
The _.isEqual won't recognize that something has changed inside your Price or Date or that you've added or removed something from Tags. If you do things like this:
p = new PriceModel(...);
m.set('Price', p)
then m will noticed that Price has changed but if you:
p = m.get('Price');
p.set(...);
m.set('Price', p);
then m won't recognize that Price has changed; your model won't automatically bind to events on Price so it won't notice the p.set(...) call and it won't recognize m.set('Price', p) as a change since that's little more than a fancy way of saying p = p.
You can solve part of this change problem by not giving set a Tags array that came from get; make a copy, change the copy, and then hand the updated copy to set. The half can be handled by binding to "change" events on the contained Price and Date models and forwarding them similar to how collections do it, something like this:
initialize: function() {
this.attributes.Price.on(
'all',
function(ev, model, opts) { this.trigger(ev, model, opts) },
this
);
//...
}
You'd want to provide your own set implementation in case someone did a set('Price', some_new_object) and you need to rebind your forwarder.