I have these 2 models:
class Track
include Mongoid::Document
field :artist, type: String
field :title, type: String
field :isrc, type: String
has_many :subtitles
end
class Subtitle
include Mongoid::Document
field :lines, type: Array
belongs_to :track
end
How can I check if a track exists that has a certain 'isrc' and has subtitles (no matter how many)?
I've been trying this but it seems to ignore the subtitles criteria:
Track.exists?(conditions: {isrc: my_isrc, :subtitles.exists => true})
It returns true even if the track with that 'isrc' has no subtitles. what to do?
You simply cannot do this way in mongo, since Track & subtitle stored in different documents. Exists command in mongodb only can verify the fields in its own documents, here the relationship is maintained in Subtitle document as track_id not in Track document. So Track doesn't have the track of subtitles.
One easy way to achieve this is to change your relation from belongs_to to embedded. So Track can easily verify the subtitles using $exists.
Another way is
Track.where(:isrc => my_isrc).select {|track| track.subtitles.count > 0}
But disadvantage in this query is multiple round trips made to mongo to verify each tracks subtitle count.
For anyone else struggling with this, I found that mapping the ids to a list, and then using the any_in or all_in functions does the trick.
I needed to get all the messages sent to a user, but messages aren't related directly to my users. Instead, users belong to lists, and lists have many messages. To get messages that "belong to" a user, I did this:
Here's my controller:
#lists = #group.lists.where(deleted: false).order_by([:created_at, :desc])
#messages = Messages.any_in(list: #lists.map(&:id))
#messages = #messages.order_by([:created_at, :desc]).paginate(:page => params[:page], :per_page => 3)
The Messages.any_in(list: #lists.map(&:id)) was the key insight.
Related
I working on an application that has its own database and gets user information from another serivce (an LDAP is this case, through an API package).
Say I have a tables called Articles, with a column user_id. There is no Users table, instead a user or set of users is retrieved through the external API:
$user = LDAPConnector::getUser($user_id);
$users = LDAPConnector::getUsers([1, 2, 5, 6]);
Of course I want retrieving data from inside a controller to be as simple as possible, ideally still with something like:
$articles = $this->Articles->find()->contain('Users');
foreach ($articles as $article) {
echo $article->user->getFullname();
}
I'm not sure how to approach this.
Where should I place the code in the table object to allow integration with the external API?
And as a bonus question: How to minimise the number of LDAP queries when filling the Entities?
i.e. it seems to be a lot faster by first retrieving the relevant users with a single ->getUsers() and placing them later, even though iterating over the articles and using multiple ->getUser() might be simpler.
The most simple solution would be to use a result formatter to fetch and inject the external data.
The more sophisticated solution would a custom association, and a custom association loader, but given how database-centric associations are, you'd probably also have to come up with a table and possibly a query implementation that handles your LDAP datasource. While it would be rather simple to move this into a custom association, containing the association will look up a matching table, cause the schema to be inspected, etc.
So I'll stick with providing an example for the first option. A result formatter would be pretty simple, something like this:
$this->Articles
->find()
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$userIds = array_unique($results->extract('user_id')->toArray());
$users = LDAPConnector::getUsers($userIds);
$usersMap = collection($users)->indexBy('id')->toArray();
return $results
->map(function ($article) use ($usersMap) {
if (isset($usersMap[$article['user_id']])) {
$article['user'] = $usersMap[$article['user_id']];
}
return $article;
});
});
The example makes the assumption that the data returned from LDAPConnector::getUsers() is a collection of associative arrays, with an id key that matches the user id. You'd have to adapt this accordingly, depending on what exactly LDAPConnector::getUsers() returns.
That aside, the example should be rather self-explanatory, first obtain a unique list of users IDs found in the queried articles, obtain the LDAP users using those IDs, then inject the users into the articles.
If you wanted to have entities in your results, then create entities from the user data, for example like this:
$userData = $usersMap[$article['user_id']];
$article['user'] = new \App\Model\Entity\User($userData);
For better reusability, put the formatter in a custom finder. In your ArticlesTable class:
public function findWithUsers(\Cake\ORM\Query $query, array $options)
{
return $query->formatResults(/* ... */);
}
Then you can just do $this->Articles->find('withUsers'), just as simple as containing.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Custom Finder Methods
I have a model Tag which potentially belongs to several other models, but at the moment only one model Todo which in turn belongs to User like so:
class User
include Mongoid::Document
field: name, type: String
has_many :todos
end
class Todo
include Mongoid::Document
field: name, type: String
belongs_to :user
end
class Tag
include Mongoid::Document
field: name, type: String
belongs_to :todos
end
How can I query all Tags that belongs to a particular user? I've written the following:
todo_ids = Todo.where(user_id: '86876876787')
and then:
tags = Tag.where('todo_id.in': todo_ids)
But those didn't work. What am I missing?
You're missing two things:
Mongoid isn't ActiveRecord so it won't know what to do with todo_ids in the Tag query.
'todo_id.in' is a field path that is trying to look at the in field inside a todo_id hash, this isn't a use of MongoDB's $in operator.
You can only work with one collection at a time so to fix the first one, you need to pull an array of IDs out of MongoDB:
todo_ids = Todo.where(user_id: '86876876787').pluck(:id)
# -------------------------------------------^^^^^^^^^^^
To fix the second one, use the $in operator:
tags = Tag.where(todo_id: { '$in': todo_ids })
tags = Tag.where(:todo_id.in => todo_ids)
tags = Tag.in(todo_id: todo_ids)
#...
Ok Im starting out fresh with Firebase. I've read this: https://www.firebase.com/docs/data-structure.html and I've read this: https://www.firebase.com/blog/2013-04-12-denormalizing-is-normal.html
So I'm suitably confused as one seems to contradict the other. You can structure your data hierarchically, but if you want it to be scalable then don't. However that's not the actual problem.
I have the following structure (please correct me if this is wrong) for a blog engine:
"authors" : {
"-JHvwkE8jHuhevZYrj3O" : {
"userUid" : "simplelogin:7",
"email" : "myemail#domain.com"
}
},
"posts" : {
"-JHvwkJ3ZOZAnTenIQFy" : {
"state" : "draft",
"body" : "This is my first post",
"title" : "My first blog",
"authorId" : "-JHvwkE8jHuhevZYrj3O"
}
}
A list of authors and a list of posts. First of all I want to get the Author where the userUid equals my current user's uid. Then I want to get the posts where the authorId is the one provided to the query.
But I have no idea how to do this. Any help would be appreciated! I'm using AngularFire if that makes a difference.
Firebase is a NoSQL data store. It's a JSON hierarchy and does not have SQL queries in the traditional sense (these aren't really compatible with lightning-fast real-time ops; they tend to be slow and expensive). There are plans for some map reduce style functionality (merged views and tools to assist with this) but your primary weapon at present is proper data structure.
First of all, let's tackle the tree hierarchy vs denormalized data. Here's a few things you should denormalize:
lists you want to be able to iterate quickly (a list of user names without having to download every message that user ever wrote or all the other meta info about a user)
large data sets that you view portions of, such as a list of rooms/groups a user belongs to (you should be able to fetch the list of rooms for a given user without downloading all groups/rooms in the system, so put the index one place, the master room data somewhere else)
anything with more than 1,000 records (keep it lean for speed)
children under a path that contain 1..n (i.e. possibly infinite) records (example chat messages from the chat room meta data, that way you can fetch info about the chat room without grabbing all messages)
Here's a few things it may not make sense to denormalize:
data you always fetch en toto and never iterate (if you always use .child(...).on('value', ...) to fetch some record and you display everything in that record, never referring to the parent list, there's no reason to optimize for iterability)
lists shorter than a hundred or so records that you always as a whole (e.g. the list of groups a user belongs to might always be fetched with that user and would average 5-10 items; probably no reason to keep it split apart)
Fetching the author is as simple as just adding the id to the URL:
var userId = 123;
new Firebase('https://INSTANCE.firebaseio.com/users/'+userId);
To fetch a list of posts belonging to a certain user, either maintain an index of that users' posts:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
fb.child('/my_posts/'+userId).on('child_added', function(indexSnap) {
fb.child('posts/'+indexSnap.name()).once('value', function(dataSnap) {
console.log('fetched post', indexSnap.name(), dataSnap.val());
});
});
A tool like Firebase.util can assist with normalizing data that has been split for storage until Firebase's views and advanced querying utils are released:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
var ref = Firebase.util.intersection( fb.child('my_posts/'+userId), fb.child('posts') );
ref.on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val();
});
Or simply store the posts by user id (depending on your use case for how that data is fetched later):
/posts/$user_id/$post_id/...
new Firebase('https://INSTANCE.firebaseio.com/posts/'+userId).on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val());
});
Recently updated to Mongoid 3.1 from 3.0.3 and this resulted in some broken code and confusion on my side.
Say you have a pair of classes with a belongs_to/has_many relationship, like so:
class Band
include Mongoid::Document
field :name, type: String
has_many :members, :autosave => true
end
class Member
include Mongoid::Document
field :name, type: String
belongs_to :band
end
Saving all this to the database like so:
b = Band.new
b.name = "Sonny and Cher"
b.members << Member.new(name: "Sonny")
b.members << Member.new(name: "Cher")
b.save
I would in my API, be able to return a 'member' object like so:
m = Member.where(name: "Sonny").first
m.to_json
which yields the following, as expected:
{"_id":"<removed>","band_id":"5151d89f5dd99dd9ec000002","name":"Sonny"}
My client can request the full band object with a subsequent call if it wants to. However, in some cases I DO want to include the referenced item directly. With 3.0.3, I would just do the following:
m = Member.where(name: "Sonny").first
m[:band] = m.band
m.to_json
and this would add a new field with the full band information to it. With 3.1, however (it may have started in earlier versions, but I didn't test), I now get this:
{"_id":"<removed>","band_id":{"_id":"5151dc025dd99d579e000002","name":"Sonny and Cher"},"name":"Sonny"}
So, it looks like the band info has been eager-loaded into the field? Why is it stored under the key ':band_id' and not ':band'? I guess ':band' is protected, but I still don't think the data should be stored under the ':band_id' key. I suspect I am missing something here. Any ideas?
You can specify an :include option for to_json like so:
m.to_json(include: :band)
The JSON will then have a key band with the Band object converted to JSON and band_id will still be present.
I want to implement some kind of tagging functionality to my app. I want to do something like...
class Item(db.Model):
name = db.StringProperty()
tags = db.ListProperty(str)
Suppose I get a search that have 2 or more tags. Eg. "restaurant" and "mexican".
Now, I want to get Items that have ALL, in this case 2, given tags.
How do I do that? Or is there a better way to implement what I want?
I believe you want tags to be stored as 'db.ListProperty(db.Category)' and then query them with something like:
return db.Query(Item)\
.filter('tags = ', expected_tag1)\
.filter('tags = ', expected_tag2)\
.order('name')\
.fetch(256)
(Unfortunately I can't find any good documentation for the db.Category type. So I cannot definitively say this is the right way to go.) Also note, that in order to create a db.Category you need to use:
new_item.tags.append(db.Category(unicode(new_tag_text)))
use db.ListProperty(db.Key) instead,which stores a list of entity's keys.
models:
class Profile(db.Model):
data_list=db.ListProperty(db.Key)
class Data(db.Model):
name=db.StringProperty()
views:
prof=Profile()
data=Data.gql("")#The Data entities you want to fetch
for data in data:
prof.data_list.append(data)
/// Here data_list stores the keys of Data entity
Data.get(prof.data_list) will get all the Data entities whose key are in the data_list attribute