I'm building a very small app where the user chooses a category, then a logo from said category and submits a form.
The document with the categories and logos is a JSON, and I have a bit of experience with CouchDB, but I get the feeling that such an app is a revision conflict magnet, as it'd be difficult to keep track of different users using at the same time and trying to update the document with the same revision ID.
I haven't been able to find some code or idea dealing with this issue.
Here's how my JSON looks like:
$scope.categories = [
{
name: 'DIY',
logos : [
{
url:'img/...1.png',
votes:0
},
{
url:'img/...2.png',
votes:0
}
]
},
{
name: 'Food',
logos : [
{
url:'img/...1.png',
votes:0
},
{
url:'img/...2.png',
votes:0
},
...
Any idea on how to deal with such an issue?
Here's a live test version of the app.
It would be useful to understand the design of your app. If I were designing this, I think I would generate a new CouchDb document when the form is submitted to record each vote. This would avoid updating any shared document and any risk of conflicts. Showing the results would then be a map/reduce query on the "vote" documents.
Related
I am currently working on a blogging app, in which users can create their own blogs and each blog has blogposts within that. I'm ideating about architecting a database that is scalable when each blog has a lot of blogposts.
So is it better to structure my database as this:
blog1 : {
blogname : 'blog1',
blogposts: [array of blogposts]
},
blog2 : {
blogname : 'blog2',
blogposts: [array of blogposts]
}
Or should I create a separate collection with all the blogposts, something like this:
blogpost1: {
id: 'blogpost1',
content: {blogpost content in json format}
},
blogpost2: {
id: 'blogpost2',
content: {blogpost content in json format}
}
and reference them in the blog collection.
I want to know which choice would be superior when there are a lot of blogposts. Because I remember reading somewhere in MongoDB docs that it's not recommended to have arrays within document that can grow beyond bounds, so approach #1 is not ideal, right?
When creating databases, I find it useful to think about the requests I would be making.
A blogging app user would want to search all blogs or find a blogger by some criteria.
In this case separate collections for bloggers and blogs would work best. Then structure your documents so that the bloggers link to their blogs and vice versa.
This can be done with Mongoose Schemas (https://mongoosejs.com/docs/index.html).
// models/blogger.js
const mongoose = require('mongoose')
const bloggerSchema = mongoose.Schema({
blogs: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'Blog'
}
],
name: String
})
bloggerSchema.set('toJSON', {
transform: (document, returnedObject) => {
const blogger = returnedObject
blogger.id = blogger._id.toString()
delete blogger._id
delete blogger.__v
}
})
module.exports = mongoose.model('Blogger', bloggerSchema)
Then use populate with your request:
// controllers/bloggers.js
const bloggersRouter = require('express').Router()
const Blogger = require('../models/blogger')
bloggersRouter.get('/', async (request, response) => {
const bloggers = await Blogger.find({}).populate(
'blogs', {
title: 1
}
)
response.json(bloggers.map(blogger => blogger.toJSON()))
})
module.exports = bloggersRouter
This way you don't have to add the blogs in their entirety to the blogger document, you can just include the title or anything else that you need on the bloggers initial view.
You could also think about limiting the length of a blog, so you can have more control over the data and then think about the options Joe suggested.
Why does it have to be one or the other?
Storing the blog posts in the same document as the blog is great as long as the individual posts are not very large, and there aren't very many of them.
Storing the posts in a separate collection is good for bigger posts and busy blogs but adds an additional query or lookup to retrieve.
I would think it is expected that your users' output will run the gamut from sparse to prolific, and individual posts will range from a few dozen bytes to many megabytes.
For small posts on not very active blogs, store the posts in the blog document for efficient retrieval.
For busy blogs, store them in an archive collection. Perhaps store the most recent couple of posts, or the most popular posts, in the blog document so you don't have to refer to the other collection every time.
You will also need to figure out how to split a post between documents. MongoDB has a 16MB limit on a single document, so if any of your users make huge posts, you'll need to be able to store them somewhere.
Your question as written seems to be asking whether it is better to follow a relation model or a strict document model. I think in reality neither is a perfect fit for this and a hybridized and flexible approach would work out better.
I am a fairly new web developer and would need your help with a project I am currently working on. I have worked in the past on a very simple realtime database example and have little to none experience in firestore or NoSql in general.
I want to create a system which allows end-users to get an email once a week that contains a list of special offers from bars the end-user has subscribed to. The offers change each day of the week. Bar owners can fill out a form in a vue.js web application every week with their weekly special offers.
Every Monday morning a cron job has to look up which end user has subscribed to which bars and then aggregate the data and send it via email.
The question is how would you structure the data so that I can easily compose the email and send it via a cloud function?
My approach would be to have three main collections: RestaurantOwner, EndUser, SpecialOfferings
Please see the graphic for an example process:
BarOwner and EndUser are pretty straight forward. However, the difficult part is how to structure the SpecialOffers in order to be queried the right way.
My idea would be to structure it based on the calendar week and link it to the uid from the barOwner:
specialOffers: {
2019_CW27: {
barUID001: {
mon: {
title: 'Banana Daiquir',
price: 4.99,
},
tue: {
title: 'After Five',
price: 2.99,
},
wed: {
title: 'Cool Colada',
price: 6.99
},
thu: {
title: 'Crantini',
price: 5.99
},
fri: {
title: 'French Martini',
price: 4.99
}
},
barUID002: {
mon: {
title: 'Gin & Tonic',
price: 8.99,
},
tue: {
title: 'Cratini',
price: 4.99,
},
wed: {
title: 'French Martini',
price: 4.99
},
thu: {
title: 'After Five',
price: 3.99
},
fri: {
title: 'Cool Colada',
price: 6.99
}
}
},
2019_CW28: {
barUID01: {~~~},
barUID02: {~~~}
}
}
The disadvantage of this approach is that it creates a deeply nested object when you imagine that there are 52 calendar weeks, f.e 100 signed up bars à 5 special offers per week and I am not sure if I am able to query it the way I need to.
Is this approach reasonable or what would you do differently?
Thank you so much for your help! I highly appreciate it.
I'm assuming the following scenarios:
1) The bar owners make modifications to their offers very often.
2) The bar owners should be the only ones allowed to modify each bar's offers.
If you have these two scenarios, I would recommend a sub-collections approach here.
When to use sub-collections:
1) When there are lot of fields in a document. Cloud Firestore has 20,000 field limit. (If the number of Bars can exceed more than 20,000 fields)
2) When updating the parent collection is a common operation. Firestore only lets you update the document at rate of 1 write/second. (If the SpecialOffers information of each bar is modified very often. If two bar owners modify their offers, only 1 write is successful and the second write operation waits until the first is completed. This can delay the updation offers particularly at the end of a week when almost all the bars update the offers.)
3) When you want to limit the access to particular fields of a document. (If you want to restrict the access to a Bar's Offers to the barOwner alone. You can restrict the access to each document in the Bars sub-collection according to its owner using Firestore Security Rules)
So I would recommend a sub-collection Bars under the main collection SpecialOffers. This way the design becomes scalable and you can add restaurants and super-markets as other similar sub-collections in the future without heavily altering your design.
Another advantage is that sub-collections are basically collections and they don't have a limit for number of documents it can hold. So even if the number of bars registered is above 20,000 which is the limit of number of fields for a fire-store document, your sub-collection wont be having a problem but your document will run out of fields to save the offers for a new bar.
Ultimately the choice depends on your use cases.
Hope this helps.
I'm working on a travel application in Angular 2/Ionic 2 with Typescript. I need to figure out how to model my services / objects. This app will have users.
Each user can create an overall Trip. You could name it "Europe 2016." Then you could add multiple days to it. They could be be travelling for 14 days. Each day they can have plans on what to do that day including things like what time the activity is, where that activity is, etc... They could also check the weather, add hotel info, etc to each day.
Roughly may be it looks like (sorry this is ugly)
Trip: {
"name": "Europe 2016",
"tripPlans": [
{"day1": [{plan1: do something, time: 01000}, {plan2: do something else, time: 01000}}], "time": "0800", "date":"09/01"2016, hotel, weather},
{"day2": [{plan1: do something, plan2: do something}], "time": "0800", "date":"09/01"2016, hotel info, weather info}
]
}
Would also need to know the first and last date of trips.
I also may want to go back and query for what the user group find popular, like "most common activity in Gotham"
This is my first major application after spending 100's of hours learning to code and its my personal projects been keeping me excited to learn. It solves a personal problem of mine. I've gotten to the point where I'm having a hard time finding good examples/resources to help me figure this out.
I've sunk a lot of time in to the framework choice so I'd like to stick with it. So far I've landed on trying to use PouchDB/CouchDB. I'm currently only building the mobile version in Ionic right now.
Any advice would be much appreciated even if its github projects to look at, tutorials that you've saved, etc.
I mostly work with CouchDB but I've also started to develop an Ionic app with PouchDB/CouchDB. First of all, you need to know that NoSQL is good when you have a lot of nested documents but sometimes you don't have the choice to make "relations".
First, I reworked a bit your data model :
{
"name": "Europe 2016",
"type": "trip",
"tripPlans": [{
"name": "This is the first plan",
"days": {
"day1": [{
"registeredActivityId": null,
"plan": "do something",
"time": 1000
}, {
"registeredActivityId": "activty_goToTheMoon",
"plan": "Go to the moon and the previous activity",
"time": 1000
}],
"time": "0800",
"date": "09/01/2016",
"hotel": "Hotel name",
"weather": "Sunny"
}
}]
}
Now, you have few options :
You can build a view on your nested actives like this one
function(doc) {
if (doc.type == "trip" && doc.tripPlans && doc.tripPlans.join)
for (var i = 0; i < doc.tripPlans.length; i++) {
var plan = doc.tripPlans[i];
if (plan.days)
for (var n in plan.days)
if (plan.days.hasOwnProperty(n)) {
var day = plan.days[n];
//We emit the activity id for a better tracking.
//If it's not available, we emit the description(not recommanded)
emit(day.registeredActivityId ? day.registeredActivityId : day.plan);
}
}
}
You could make relations between your documents
Using the nested approaches make faster query if the views are calculated often. Else, the "relational approach" makes it easier link "predefined activities" for example. You can contact me in private if you need further pieces of information.
I'm experiencing little trouble getting Orionjs working within Angular-Meteor especially with the collections.
I had my old mongodb declarations, for instance :
Gallery = new Mongo.Collection('gallery') and so one.
As the documentation told , I wrote
Gallery = new orion.collection('gallery') but what I get is
Error: Match error: Expected object, got undefined
at exports.check (packages/check/match.js:34:1)
at new orion.collection (packages/orionjs:collections/new.js:8:3)
at meteorInstall.shared.collections.js (shared/collections.js:1:11)
So I tried to start a project from scratch with this framework.
Fact is, it doesn't work neither with Iron Router nor Flow Router.
Can anyone hit me with any hint about it?
Thank you guys.
Ideally OrionJS expect a schema detail like the label for singular and plural names, navigation detail, table layout for displaying data and so on.Here's a typical company collection shown below:
Company = new orion.collection('company', {
singularName: orion.helpers.getTranslation('company.singularName'),
pluralName: orion.helpers.getTranslation('company.pluralName'),
title: orion.helpers.getTranslation('company.title'),
link: {
title: orion.helpers.getTranslation('company.title'),
parent: 'collections-abc'
},
tabular: {
columns: [
{ data: '_id', title: orion.helpers.getTranslation('company.schema.id') },
{ data: 'name', title: orion.helpers.getTranslation('company.schema.name') }
]
}
});
You can also pass a null JSON if you do not wish to show page directly. Usually it expects a JSON like the one shown above.
I'm new bee in mongodb.
I made a nested array document like this.
data = {
"title": "mongo community",
"description": "I am a new bee",
"topics": [{
"title": "how to find object in array",
"comments": [{
"description": "desc1"
}]
},
{
"title": "the case to use ensureIndex",
"comments": [{
"description": "before query"
},
{
"description": "If you want"
}
]
}
]
}
after that, put it in the "community"
db.community.insert(data)
so,I would like to accumulate "comments" which topics title is "how to find object in array"
then I tried,
data = db.community.find_one({"title":"mongo community","topics.title":"how to find object in array" } )
the result is
>>> print data
{
u 'topics': [{
u 'comments': [{
u 'description': u 'desc1'
}],
u 'title': u 'how to find object in array'
},
{
u 'comments': [{
u 'description': u 'before query'
},
{
u 'description': u 'If you want'
}],
u 'title': u 'the case to use ensureIndex'
}],
u '_id': ObjectId('4e6ce188d4baa71250000002'),
u 'description': u 'I am a new bee',
u 'title': u 'mongo community'
}
I don't need the topics "the case to use ensureIndex"
Whould you give me any advice.
thx.
It looks like you're embedding topics as an array all in a single document. You should try to avoid returning partial documents frequently from MongoDB. You can do it with the "fields" argument of the find method, but it isn't very easy to work with if you're doing it frequently.
So to solve this you could try to make each topic a separate document. I think that would be easier for you too. If you want to save information about the "community" for forum, put it in a separate collection. For example, you could use the following in the monbodb shell:
// ad a forum:
var forum = {
title:"mongo community",
description:"I am a new bee"
};
db.forums.save(forum);
// add first topic:
var topic = {
title: "how to find object in array",
comments: [ {description:"desc1"} ],
forum:"mongo community"
};
db.topics.save(topic);
// add second topic:
var topic = {
title: "the case to use ensureIndex",
comments: [
{description:"before query"},
{description:"If you want"}
],
forum:"mongo community"
};
db.topics.save(topic);
print("All topics:");
printjson(db.topics.find().toArray());
print("just the 'how to find object in array' topic:")
printjson(db.topics.find({title:"how to find object in array"}).toArray());
Also, see the document Trees In MongoDB about schema design in MongoDB. It happens to be using a similar schema to what you are working with and expands on it for more advanced use cases.
MongoDB operates on documents, that is, the top level documents (the things you save, update, insert, find, and find_one on). Mongo's query language lets you search within embedded objects, but will always return, update, or manipulate one (or more) of these top-level documents.
MongoDB is often called "schema-less," but something more like "(has) flexible schemas" or "(has) per-document schemas" would be a more accurate description. This is a case where your schema design -- having topics embedded directly within a community -- is not working for this particular query. However there are probably other queries that this schema supports more efficiently, like listing the topics within a community in a single query. You might want to consider the queries you want to make and re-design your schema accordingly.
A few notes on MongoDB limitations:
top-level documents are always returned (optionally with only a subset of fields, as #scott noted -- see the mongodb docs on this topic)
each document is limited to 16 megabytes of data (as of version 1.8+), so this schema will not work well if the communities have a long list of topics
For help with schema design, see the mongodb docs on schema design, Kyle Banker's video "Schema Design Basics", and Eliot Horowitz's video "Schema Design at Scale" for an introduction, tips, and considerations.