This are my model with some of the fields:
class Advertisers(models.Model):
account_manager_id = models.ForeignKey(AccountManagers, on_delete=models.CASCADE,null=True, db_column='account_manager_id',related_name="advertisers")
class AdvertiserUsers(models.Model):
user_id = models.OneToOneField('Users', on_delete=models.CASCADE,null=True,db_column='user_id', related_name='advertiser_users')
advertiser_id = models.ForeignKey('Advertisers', on_delete=models.CASCADE,null=True,db_column='advertiser_id', related_name='advertiser_users')
class Users(models.Model):
email = models.CharField(unique=True, max_length=100)
I want Id's, user ids and email of all advertisers.
Id's of all user:-
advertiser_ids = advertisers.objects.all() # can get id from here
find user_ids of advertiser_ids:
user_ids = AdvertiserUsers.objects.filter(advertiser_id__in=advertiser_ids) # can get user_id from here
find id and email using this query:
user_ids = Users.objects.filter(id__in=user_ids) # can get email from here
How to make it shorter like directly querying from Advertisers i will be able to get Users models email.
Thankyou in advance
You can filter with:
Users.objects.filter(advertiser_users__advertiser_id__isnull=False).distinct()
The .distinct() [Django-doc] will prevent returning the same Users multiple times.
You can annotate the User objects with the Advertisers primary key, etc:
from django.db.models import F
Users.objects.filter(advertiser_users__advertiser_id__isnull=False).annotate(
account_manager_id=F('advertiser_users__advertiser_id__account_manager_id'),
advertiser_id=F('advertiser_users__advertiser_id')
)
The Users objects that arise from this have a .email attribute (and the other attributes that belong to a Users object), together with a .account_manager_id and an .advertiser_id. That being said, this is probably not a good idea: the way you have modeled this right now, is that a Users object can relate to multiple Advertisers objects, so it makes not much sense to add these together.
You can for each user access the related Advertisers with:
myusers = Users.objects.filter(
advertiser_users__advertiser_id__isnull=False
).prefetch_related(
'advertiser_users',
'advertiser_users__advertiser_id'
).distinct()
for user in myusers:
print(f'{user.email}')
for advuser in user.advertiser_users.all():
print(f' {advuser.advertiser_user.pk}')
Note: normally a Django model is given a singular name, so User instead of Users.
Note: Normally one does not add a suffix _id to a ForeignKey field, since Django
will automatically add a "twin" field with an _id suffix. Therefore it should
be account_manager_id, instead of account_manager.
Advertisers.objects.all().values_list('id','account_manager_id','advertiser_users__user_id',advertiser_users__user_id__email)
Related
An NDB model contains two properties: email and password. How to avoid adding to the database two records with the same email? NDB doesn't have UNIQUE option for a property, like relational databases do.
Checking that new email is not in the database before adding—won't satisfy me, because two parallel processes can both simultaneously do the checking and each add the same email.
I'm not sure that transactions can help here, I am under this impression after reading some of the manuals. Maybe the synchronous transactions? Does it mean one at a time?
Create the key of the entity by email, then use get_or_insert to check if exists.
Also read about keys , entities. and models
#ADD
key_a = ndb.Key(Person, email);
person = Person(key=key_a)
person.put()
#Insert unique
a = Person.get_or_insert(email)
or if you want to just check
#ADD
key_a = ndb.Key(Person, email);
person = Person(key=key_a)
person.put()
#Check if it's added
new_key_a =ndb.Key(Person, email);
a = new_key_a.get()
if a is not None:
return
Take care. Changing email will be really difficult (need to create new entry and copy all entries to new parent).
For that thing maybe you need to store the email, in another entity and have the User be the parent of that.
Another way is to use Transactions and check the email property. Transaction's work in the way: First that commits is the First that wins. A concept which means that if 2 users check for email only the first (lucky) one will succeed, thus your data will be consistent.
Maybe you are looking for the webapp2-authentication module, that can handle this for you. It can be imported like this import webapp2_extras.appengine.auth.models. Look here for a complete example.
I also ran into this problem, and the solution above didn't solve my problem:
making it a key was unacceptable in my case (i need the property to be changeable in the future)
using transactions on the email property doesn't work AFAIK (you can't do queries on non-key names inside transactions, so you can't check whether the e-mail already exists).
I ended up creating a separate model with no properties, and the unique property (email address) as the key name. In the main model, I store a reference to the email model (instead of storing the email as a string). Then, I can make 'change_email' a transaction that checks for uniqueness by looking up the email by key.
This is something that I've come across as well and I settled on a variation of #Remko's solution. My main issue with checking for an existing entity with the given email is a potential race condition like op stated. I added a separate model that uses an email address as the key and has a property that holds a token. By using get_or_insert, the returned entities token can be checked against the token passed in and if they match then the model was inserted.
import os
from google.appengine.ext import ndb
class UniqueEmail(ndb.Model):
token = ndb.StringProperty()
class User(ndb.Model):
email = ndb.KeyProperty(kind=UniqueEmail, required=True)
password = ndb.StringProperty(required=True)
def create_user(email, password):
token = os.urandom(24)
unique_email = UniqueEmail.get_or_insert(email,
token=token)
if token == unique_email.token:
# If the tokens match, that means a UniqueEmail entity
# was inserted by this process.
# Code to create User goes here.
# The tokens do not match, therefore the UniqueEmail entity
# was retrieved, so the email is already in use.
raise ValueError('That user already exists.')
I implemented a generic structure to control unique properties. This solution can be used for several kinds and properties. Besides, this solution is transparent for other developers, they use NDB methods put and delete as usual.
1) Kind UniqueCategory: a list of unique properties in order to group information. Example:
‘User.nickname’
2) Kind Unique: it contains the values of each unique property. The key is the own property value which you want to control of. I save the urlsafe of the main entity instead of the key or key.id() because is more practical and it doesn’t have problem with parent and it can be used for different kinds. Example:
parent: User.nickname
key: AVILLA
reference_urlsafe: ahdkZXZ-c3RhcnQtb3BlcmF0aW9uLWRldnINCxIEVXNlciIDMTIzDA (User key)
3) Kind User: for instance, I want to control unique values for email and nickname. I created a list called ‘uniqueness’ with the unique properties. I overwritten method put in transactional mode and I wrote the hook _post_delete_hook when one entity is deleted.
4) Exception ENotUniqueException: custom exception class raised when some value is duplicated.
5) Procedure check_uniqueness: check whether a value is duplicated.
6) Procedure delete_uniqueness: delete unique values when the main entity is deleted.
Any tips or improvement are welcome.
class UniqueCategory(ndb.Model):
# Key = [kind name].[property name]
class Unique(ndb.Model):
# Parent = UniqueCategory
# Key = property value
reference_urlsafe = ndb.StringProperty(required=True)
class ENotUniqueException(Exception):
def __init__(self, property_name):
super(ENotUniqueException, self).__init__('Property value {0} is duplicated'.format(property_name))
self. property_name = property_name
class User(ndb.Model):
# Key = Firebase UUID or automatically generated
firstName = ndb.StringProperty(required=True)
surname = ndb.StringProperty(required=True)
nickname = ndb.StringProperty(required=True)
email = ndb.StringProperty(required=True)
#ndb.transactional(xg=True)
def put(self):
result = super(User, self).put()
check_uniqueness (self)
return result
#classmethod
def _post_delete_hook(cls, key, future):
delete_uniqueness(key)
uniqueness = [nickname, email]
def check_uniqueness(entity):
def get_or_insert_unique_category(qualified_name):
unique_category_key = ndb.Key(UniqueCategory, qualified_name)
unique_category = unique_category_key.get()
if not unique_category:
unique_category = UniqueCategory(id=qualified_name)
unique_category.put()
return unique_category_key
def del_old_value(key, attribute_name, unique_category_key):
old_entity = key.get()
if old_entity:
old_value = getattr(old_entity, attribute_name)
if old_value != new_value:
unique_key = ndb.Key(Unique, old_value, parent=unique_category_key)
unique_key.delete()
# Main flow
for unique_attribute in entity.uniqueness:
attribute_name = unique_attribute._name
qualified_name = type(entity).__name__ + '.' + attribute_name
new_value = getattr(entity, attribute_name)
unique_category_key = get_or_insert_unique_category(qualified_name)
del_old_value(entity.key, attribute_name, unique_category_key)
unique = ndb.Key(Unique, new_value, parent=unique_category_key).get()
if unique is not None and unique.reference_urlsafe != entity.key.urlsafe():
raise ENotUniqueException(attribute_name)
else:
unique = Unique(parent=unique_category_key,
id=new_value,
reference_urlsafe=entity.key.urlsafe())
unique.put()
def delete_uniqueness(key):
list_of_keys = Unique.query(Unique.reference_urlsafe == key.urlsafe()).fetch(keys_only=True)
if list_of_keys:
ndb.delete_multi(list_of_keys)
I recently came across a number of articles pointing out to flatten the data for NoSQL databases. Coming from traditional SQL databases I realized I am replicating a SQL db bahaviour in GAE. So I started to refactor code where possible.
We have e.g. a social media site where users can become friends with each other.
class Friendship(ndb.Model):
from_friend = ndb.KeyProperty(kind=User)
to_friend = ndb.KeyProperty(kind=User)
Effectively the app creates a friendship instance between both users.
friendshipA = Friendship(from_friend = UserA, to_friend = userB)
friendshipB = Friendship(from_friend = UserB, to_friend = userA)
How could I now move this to the actual user model to flatten it. I thought maybe I could use a StructuredProperty. I know it is limited to 5000 entries, but that should be enough for friends.
class User(UserMixin, ndb.Model):
name = ndb.StringProperty()
friends = ndb.StructuredProperty(User, repeated=True)
So I came up with this, however User can't point to itself, so it seems. Because I get a NameError: name 'User' is not defined
Any idea how I could flatten it so that a single User instance would contain all its friends, with all their properties?
You can't create a StructuredProperty that references itself. Also, use of StructuredProperty to store a copy of User has additional problem of needing to perform a manual cascade update if a user ever modifies a property that is stored.
However, as KeyProperty accept String as kind, you can easily store the list of Users using KeyProperty as suggested by #dragonx. You can further optimise read by using ndb.get_multi to avoid multiple round-trip RPC calls when retrieving friends.
Here is a sample code:
class User(ndb.Model):
name = ndb.StringProperty()
friends = ndb.KeyProperty(kind="User", repeated=True)
userB = User(name="User B")
userB_key = userB.put()
userC = User(name="User C")
userC_key = userC.put()
userA = User(name="User A", friends=[userB_key, userC_key])
userA_key = userA.put()
# To retrieve all friends
for user in ndb.get_multi(userA.friends):
print "user: %s" % user.name
Use a KeyProperty that stores the key for the User instance.
I'm accessing emails in my email server, taking the body of each email and then applying regular expressions to find the data necessary to populate my Django model.
This all works fine except for one field, which is linked as a foreign key to another model field. Despite the value in my email being the same as the one in listed in my other model, it fails....
The error:
ValueError: Cannot assign "'Humanities'": "Subject.faculty" must be a "Faculty" instance.
For example, say each school subject has to be part of a faculty. When populating the database via a form, for the Subject's faculty field I drop down the menu to a list of faculty values/instances as there is a foreign key relationship defined in my model i.e. for the faculty field I can choose from Humanities, Art, Design Technology etc.
But when I find the value 'Humanities' in my email and try to add it to the database model, I get the error above.
Anyone shed any light on this? Am I being stupid or is it more than a ValueError as to me, the values are the same in both cases
Thank you
More code as requested:
class Faculty(models.Model):
name = models.CharField(primary_key=True, max_length=50)
leader = models.CharField(max_length=50)
email = models.EmailField()
mailing_list = models.ManyToManyField("Contact", null=True)
class Subject(models.Model):
name = models.CharField(max_length=50)
faculty = models.ForeignKey(Faculty, to_field="name")
faculty_head = models.CharField(max_length=50)
It sounds like you are trying to assign a string "Humantities" to a ForeignKey relationship. This doesn't make sense. You need to either find or create the actual Faculty object with the name "Humanities" and assign it to the Subject. Something like this in your view (depending on how your form is set up):
if form.is_valid():
faculty_str = form.cleaned_data['faculty']
(faculty, was_created) = Faculty.objects.get_or_create(name=faculty_str, ...)
# It's hard to tell if you are using a ModelForm or just a normal Form. Anyway, assume we already have access to the Subject object
subject.faculty = faculty
subject.save()
get_or_create()
Your value is 'Humanities' perhaps you mean to search for Humanities (without quotes).
You need to create a Faculty instance first.
faculty = Faculty(name='', leader='', email='')
faculty.save()
subject.faculty = faculty
subject.save()
I use the key of another User, the sponsor, to indicate who is the sponsor of a User and it creates a link in the datastore for those Users that have a sponsor and it can be at most one but a sponsor can sponsor many users like in this case ID 2002 who sponsored three other users:
In this case this query does what I want: SELECT * FROM User where sponsor =KEY('agtzfmJuYW5vLXd3d3ILCxIEVXNlchjSDww') but I don't know how to program that with python, I can only use it to the datastore. How can I query by key when I want to match the set of users who has the same user as key in the same field? A user in my model can have at most one sponsor and I just want to know who a particular person sponsored which could be a list of users and then they sponsored users in their turn which I also want to query on.
The field sponsor is a key and it has a link to the sponsor in the datastore. I set the key just like user2.sponsor = user1.key and now I want to find all that user1 sponsored with a query that should be just like
User.All().filter('sponsor = ', user1.key)
but sponsor is a field of type key so I don't know how to match it to see for example a list a people the active user is a sponsor for and how it becomes a tree when the second generation also have links. How to select the list of users this user is a sponsor for and then the second generation? When i modelled the relation simply like u1=u2.key ie user2.sponsor=user1.key. Thanks for any hint
The following workaround is bad practice but is my last and only resort:
def get(self):
auser = self.auth.get_user_by_session()
realuser = auth_models.User.get_by_id(long( auser['user_id'] ))
q = auth_models.User.query()
people = []
for p in q:
try:
if p.sponsor == realuser.key:
people.append(p)
except Exception, e:
pass
if auser:
self.render_jinja('my_organization.html', people=people, user=realuser,)
Update
The issues are that the keyproperty is not required and that Guido Van Rossum has reported this as a bug in the ndb when I think it's a bug in my code. Here's what I'm using now, which is a very acceptable solution since every real user in the organization except possibly programmers, testers and admins are going the be required to have a sponsor ID which is a user ID.
from ndb import query
class Myorg(NewBaseHandler):
#user_required
def get(self):
user = auth_models.User.get_by_id(long(self.auth.get_user_by_session()['user_id']))
people = auth_models.User.query(auth_models.User.sponsor == user.key).fetch()
self.render_jinja('my_organization.html', people=people,
user=user)
class User(model.Expando):
"""Stores user authentication credentials or authorization ids."""
#: The model used to ensure uniqueness.
unique_model = Unique
#: The model used to store tokens.
token_model = UserToken
sponsor = KeyProperty()
created = model.DateTimeProperty(auto_now_add=True)
updated = model.DateTimeProperty(auto_now=True)
# ID for third party authentication, e.g. 'google:username'. UNIQUE.
auth_ids = model.StringProperty(repeated=True)
# Hashed password. Not required because third party authentication
# doesn't use password.
password = model.StringProperty()
...
The User model is an NDB Expando which is a little bit tricky to query.
From the docs
Another useful trick is querying an Expando kind for a dynamic
property. You won't be able to use class.query(class.propname ==
value) as the class doesn't have a property object. Instead, you can
use the ndb.query.FilterNode class to construct a filter expression,
as follows:
from ndb import model, query
class X(model.Expando):
#classmethod
def query_for(cls, name, value):
return cls.query(query.FilterNode(name, '=', value))
print X.query_for('blah', 42).fetch()
So try:
form ndb import query
def get(self):
auser = self.auth.get_user_by_session()
realuser = auth_models.User.get_by_id(long( auser['user_id'] ))
people = auth_models.User.query(query.FilterNode('sponsor', '=', realuser.key)).fetch()
if auser:
self.render_jinja('my_organization.html', people=people, user=realuser,)
Option #2
This option is a little bit cleaner. You subclass the model and pass it's location to webapp2. This will allow you to add custom attributes and custom queries to the class.
# custom_models.py
from webapp2_extras.appengine.auth.models import User
from google.appengine.ext.ndb import model
class CustomUser(User):
sponsor = model.KeyProperty()
#classmethod
def get_by_sponsor_key(cls, sponsor):
# How you handle this is up to you. You can return a query
# object as shown, or you could return the results.
return cls.query(cls.sponsor == sponsor)
# handlers.py
def get(self):
auser = self.auth.get_user_by_session()
realuser = custom_models.CustomUser.get_by_id(long( auser['user_id'] ))
people = custom_models.CustomUser.get_by_sponsor_key(realuser.key).fetch()
if auser:
self.render_jinja('my_organization.html', people=people, user=realuser,)
# main.py
config = {
# ...
'webapp2_extras.auth': {
# Tell webapp2 where it can find your CustomUser
'user_model': 'custom_models.CustomUser',
},
}
application = webapp2.WSGIApplication(routes, config=config)
I have a bunch of Users. Since Django doesn't really let me extend the default User model, they each have Profiles. The Profiles have a referred_by field (a FK to User). I'm trying to get a list of Users with >= 1 referral. Here's what I've got so far
Profile.objects.filter(referred_by__isnull=False).values_list('referred_by', flat=True)
Which gives me a list of IDs of the users who have referrals... but I need it to be distinct, and I want the User object, not their ID.
Or better yet, it would be nice if it could return the number of referrals a user has.
Any ideas?
Took me a long time to wrap my head around this, but I think I finally got it figured out:
affiliates = User.objects.annotate(num_referrals=Count('referrals')).filter(num_referrals__gt=0)
I didn't think I'd be able to use the reverse relationship
referred_by = models.ForeignKey(User, null=True, blank=True, related_name='referrals')
in Count(), nor did I think you could use the annotated value in the filter... that's pretty cool. I still wish you could use GROUP BY without having to annotate stuff (assuming I didn't need the count).