Recently updated to Mongoid 3.1 from 3.0.3 and this resulted in some broken code and confusion on my side.
Say you have a pair of classes with a belongs_to/has_many relationship, like so:
class Band
include Mongoid::Document
field :name, type: String
has_many :members, :autosave => true
end
class Member
include Mongoid::Document
field :name, type: String
belongs_to :band
end
Saving all this to the database like so:
b = Band.new
b.name = "Sonny and Cher"
b.members << Member.new(name: "Sonny")
b.members << Member.new(name: "Cher")
b.save
I would in my API, be able to return a 'member' object like so:
m = Member.where(name: "Sonny").first
m.to_json
which yields the following, as expected:
{"_id":"<removed>","band_id":"5151d89f5dd99dd9ec000002","name":"Sonny"}
My client can request the full band object with a subsequent call if it wants to. However, in some cases I DO want to include the referenced item directly. With 3.0.3, I would just do the following:
m = Member.where(name: "Sonny").first
m[:band] = m.band
m.to_json
and this would add a new field with the full band information to it. With 3.1, however (it may have started in earlier versions, but I didn't test), I now get this:
{"_id":"<removed>","band_id":{"_id":"5151dc025dd99d579e000002","name":"Sonny and Cher"},"name":"Sonny"}
So, it looks like the band info has been eager-loaded into the field? Why is it stored under the key ':band_id' and not ':band'? I guess ':band' is protected, but I still don't think the data should be stored under the ':band_id' key. I suspect I am missing something here. Any ideas?
You can specify an :include option for to_json like so:
m.to_json(include: :band)
The JSON will then have a key band with the Band object converted to JSON and band_id will still be present.
Related
Im trying to return a Page from a Spring Data Jpa repository. The repository accepts a Specification and a Pageable object like so:
Page<Cat> cats = catRepository.findAll(
CatSpecification.hasDarkHair,
PageRequest.of(0, 10, Sort.by("kittens.character"))
);
My problem is, that the Sort.by wont work. The Cat-Entity has a OneToMany relation to Kitten. Kitten has a OneToOne to Character and in the Character entity there is a filed called wild.
How can I sort the cats by the kittens characters wild attribute?
I have a model Tag which potentially belongs to several other models, but at the moment only one model Todo which in turn belongs to User like so:
class User
include Mongoid::Document
field: name, type: String
has_many :todos
end
class Todo
include Mongoid::Document
field: name, type: String
belongs_to :user
end
class Tag
include Mongoid::Document
field: name, type: String
belongs_to :todos
end
How can I query all Tags that belongs to a particular user? I've written the following:
todo_ids = Todo.where(user_id: '86876876787')
and then:
tags = Tag.where('todo_id.in': todo_ids)
But those didn't work. What am I missing?
You're missing two things:
Mongoid isn't ActiveRecord so it won't know what to do with todo_ids in the Tag query.
'todo_id.in' is a field path that is trying to look at the in field inside a todo_id hash, this isn't a use of MongoDB's $in operator.
You can only work with one collection at a time so to fix the first one, you need to pull an array of IDs out of MongoDB:
todo_ids = Todo.where(user_id: '86876876787').pluck(:id)
# -------------------------------------------^^^^^^^^^^^
To fix the second one, use the $in operator:
tags = Tag.where(todo_id: { '$in': todo_ids })
tags = Tag.where(:todo_id.in => todo_ids)
tags = Tag.in(todo_id: todo_ids)
#...
I am creating the bulkloader.yaml automatically from my existing schema and have trouble downloading my data due the repeated=True of my KeyProperty.
class User(ndb.Model):
firstname = ndb.StringProperty()
friends = ndb.KeyProperty(kind='User', repeated=True)
The automatic created bulkloader looks like this:
- kind: User
connector: csv
connector_options:
# TODO: Add connector options here--these are specific to each connector.
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
- property: firstname
external_name: firstname
# Type: String Stats: 2 properties of this type in this kind.
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: transform.create_foreign_key('User')
export_transform: transform.key_id_or_name_as_string
This is the error message I am getting:
google.appengine.ext.bulkload.bulkloader_errors.ErrorOnTransform: Error on transform. Property: friends External Name: friends. Code: transform.key_id_or_name_as_string Details: 'list' object has no attribute 'to_path'
What can I do please?
Possible Solution:
After Tony's tip I came up with this:
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: myfriends.stringToValue(';')
export_transform: myfriends.valueToString(';')
myfriends.py
def valueToString(delimiter):
def key_list_to_string(value):
keyStringList = []
if value == '' or value is None or value == []:
return None
for val in value:
keyStringList.append(transform.key_id_or_name_as_string(val))
return delimiter.join(keyStringList)
return key_list_to_string
And this works! The encoding is in Unicode though: UTF-8. Make sure to open the file in LibreOffice as such or you would see garbled content.
The biggest challenge is import. This is what I came up with without any luck:
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []:
return None
for val in value.split(';'):
keyvalueList.append(transform.create_foreign_key('User'))
return keyvalueList
return string_to_key_list
I get the error message:
BadValueError: Unsupported type for property friends: <type 'function'>
According to Datastore viewer, I need to create something like this:
[datastore_types.Key.from_path(u'User', u'kave#gmail.com', _app=u's~myapp1')]
Update 2:
Tony you are to be a real expert in Bulkloader. Thanks for your help. Your solution worked!
I have moved my other question to a new thread.
But one crucial problem that appears is that, when I create new users I can see my friends field shown as <missing> and it works fine.
Now when I use your solution to upload the data, I see for those users without any friend entries a <null> entry. Unfortunately this seems to break the model since friends can't be null.
Changing the model to reflect this, seems to be ignored.
friends = ndb.KeyProperty(kind='User', repeated=True, required=False)
How can I fix this please?
update:
digging further into it:
when the status <missing> is shown in the data viewer, in code it shows friends = []
However when I upload the data via csv I get a <null>, which translates to friends = [None]. I know this, because I exported the data into my local data storage and could follow it in code. Strangely enough if I empty the list del user.friends[:], it works as expected. There must be a beter way to set it while uploading via csv though...
Final Solution
This turns out to be a bug that hasn't been resolved since over one year.
In a nutshell, even though there is no value in csv, because a list is expected, gae makes a list with a None inside. This is game breaking, since retrieval of such a model ends up in an instant crash.
Adding a post_import_function, which deletes the lists with a None inside.
In my case:
def post_import(input_dict, instance, bulkload_state_copy):
if instance["friends"] is None:
del instance["friends"]
return instance
Finally everything works as expected.
When you are using repeated properties and exporting to a CSV, you should be doing some formatting to concatenate the list into a CSV understood format. Please check the example here on import/export of list of dates and hope it can help you.
EDIT : Adding suggestion for import transform from an earlier comment to this answer
For import, please try something like:
`from google.appengine.api import datastore
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []: return None
for val in value.split(';'):
keyvalueList.append(datastore.Key.from_path('User', val))
return keyvalueList
return string_to_key_list`
if you have id instead of name , add like val = int(val)
I have these 2 models:
class Track
include Mongoid::Document
field :artist, type: String
field :title, type: String
field :isrc, type: String
has_many :subtitles
end
class Subtitle
include Mongoid::Document
field :lines, type: Array
belongs_to :track
end
How can I check if a track exists that has a certain 'isrc' and has subtitles (no matter how many)?
I've been trying this but it seems to ignore the subtitles criteria:
Track.exists?(conditions: {isrc: my_isrc, :subtitles.exists => true})
It returns true even if the track with that 'isrc' has no subtitles. what to do?
You simply cannot do this way in mongo, since Track & subtitle stored in different documents. Exists command in mongodb only can verify the fields in its own documents, here the relationship is maintained in Subtitle document as track_id not in Track document. So Track doesn't have the track of subtitles.
One easy way to achieve this is to change your relation from belongs_to to embedded. So Track can easily verify the subtitles using $exists.
Another way is
Track.where(:isrc => my_isrc).select {|track| track.subtitles.count > 0}
But disadvantage in this query is multiple round trips made to mongo to verify each tracks subtitle count.
For anyone else struggling with this, I found that mapping the ids to a list, and then using the any_in or all_in functions does the trick.
I needed to get all the messages sent to a user, but messages aren't related directly to my users. Instead, users belong to lists, and lists have many messages. To get messages that "belong to" a user, I did this:
Here's my controller:
#lists = #group.lists.where(deleted: false).order_by([:created_at, :desc])
#messages = Messages.any_in(list: #lists.map(&:id))
#messages = #messages.order_by([:created_at, :desc]).paginate(:page => params[:page], :per_page => 3)
The Messages.any_in(list: #lists.map(&:id)) was the key insight.
I want to implement some kind of tagging functionality to my app. I want to do something like...
class Item(db.Model):
name = db.StringProperty()
tags = db.ListProperty(str)
Suppose I get a search that have 2 or more tags. Eg. "restaurant" and "mexican".
Now, I want to get Items that have ALL, in this case 2, given tags.
How do I do that? Or is there a better way to implement what I want?
I believe you want tags to be stored as 'db.ListProperty(db.Category)' and then query them with something like:
return db.Query(Item)\
.filter('tags = ', expected_tag1)\
.filter('tags = ', expected_tag2)\
.order('name')\
.fetch(256)
(Unfortunately I can't find any good documentation for the db.Category type. So I cannot definitively say this is the right way to go.) Also note, that in order to create a db.Category you need to use:
new_item.tags.append(db.Category(unicode(new_tag_text)))
use db.ListProperty(db.Key) instead,which stores a list of entity's keys.
models:
class Profile(db.Model):
data_list=db.ListProperty(db.Key)
class Data(db.Model):
name=db.StringProperty()
views:
prof=Profile()
data=Data.gql("")#The Data entities you want to fetch
for data in data:
prof.data_list.append(data)
/// Here data_list stores the keys of Data entity
Data.get(prof.data_list) will get all the Data entities whose key are in the data_list attribute