I'm trying to develop row level access on google appengine datastore tables. So far I do have got a working example for regular ndb put(), get() and delete() operations using _hooks.
The class Acl shall be used by all the other tables. It's used as a structured property.
class Acl(EndpointsModel):
UNAUTHORIZED_ERROR = 'Invalid token.'
FORBIDDEN_ERROR = 'Permission denied.'
public = ndb.BooleanProperty()
readers = ndb.UserProperty(repeated=True)
writers = ndb.UserProperty(repeated=True)
owners = ndb.UserProperty(repeated=True)
#classmethod
def require_user(cls):
current_user = endpoints.get_current_user()
if current_user is None:
raise endpoints.UnauthorizedException(cls.UNAUTHORIZED_ERROR)
return current_user
#classmethod
def require_reader(cls, record):
if not record:
raise endpoints.NotFoundException(record.NOT_FOUND_ERROR)
current_user = cls.require_user()
if record.acl.public is not True or current_user not in record.acl.readers:
raise endpoints.ForbiddenException(cls.FORBIDDEN_ERROR)
I do want to protect access to the Location class. So I did add three hooks (_post_get_hook, _pre_put_hook and _pre_delete_hook) to the class.
class Location(EndpointsModel):
QUERY_FIELDS = ('state', 'limit', 'order', 'pageToken')
NOT_FOUND_ERROR = 'Location not found.'
description = ndb.TextProperty()
address = ndb.StringProperty()
acl = ndb.StructuredProperty(Acl)
#classmethod
def _post_get_hook(cls, key, future):
location = future.get_result()
Acl.require_reader(location)
def _pre_put_hook(self):
if self.key.id() is None:
current_user = Acl.require_user()
self.acl = Acl()
self.acl.readers.append(current_user)
self.acl.writers.append(current_user)
self.acl.owners.append(current_user)
else:
location = self.key.get()
Acl.require_writer(location)
This does work for all the create, read, update and delete operations, but it does not work for query.
#Location.query_method(user_required=True,
path='location', http_method='GET', name='location.query')
def location_query(self, query):
"""
Queries locations
"""
current_user = Acl.require_user()
query = query.filter(ndb.OR(Location.acl.readers == current_user, Location.acl.public == True))
return query
When I run a query against all locations I get the following error message:
BadArgumentError: _MultiQuery with cursors requires __key__ order
Now I've got some questions:
How do I fix the _MultiQuery issue?
Once fixed: Does this Acl implementation make sense? Are there out of the box alternatives? (I wanted to store the Acl on the record itself to be able to run a direct query, without having to get the keys first.)
Datastore doesn't support OR filters natively. Instead what NDB is doing behind the scenes is running two queries:
query.filter(Location.acl.readers == current_user)
query.filter(Location.acl.public == True)
It then merges the results of these two queries into a single result set. In order to properly merge results (in particular to eliminate duplicates when you have repeated properties), the query needs to be ordered by the key when continuing the query from an arbitrary position (using cursors).
In order to run the query successfully, you need to append a key order to the query before running it:
def location_query(self, query):
"""
Queries locations
"""
current_user = Acl.require_user()
query = query.filter(ndb.OR(Location.acl.readers == current_user,
Location.acl.public == True)
).order(Location.key)
return query
Unfortunately, your ACL implementation will not work for queries. In particular, _post_get_hook is not called for query results. There is a bug filed on the issue tracker about this.
Related
The normal way to query a directory for users is (&(objectClass=user)(objectCategory=person)). The normal way to query for deleted objects is to add (isDeleted=TRUE).
However, the objectCategory attribute does not exist on tombstone objects, so a query for (&(objectClass=user)(objectCategory=person)(isDeleted=TRUE)) will get you nothing.
If you remove the (objectCategory=person) part, you'll get computers too, as they inherit from user.
Is it possible to retrieve only deleted users?
If not, is it possible to tell from the returned tombstone object if it's a user or not?
Try an LDAP filter like:
(&(isDeleted=TRUE)(userAccountControl:1.2.840.113556.1.4.803:=512))
This should retrieve most deleted user type entries.
python3 code
import ldap
from ldap.controls.simple import ValueLessRequestControl
...
base =
scope = ldap.SCOPE_SUBTREE
filterstr = '(&(objectClass=user)(isDeleted=TRUE))'
attrlist =
result_set = []
ct = ldap.controls.simple.ValueLessRequestControl('1.2.840.113556.1.4.417', True)
result_id = l.search_ext(base, scope, filterstr, attrlist, serverctrls=[ct, ])
for i in range(0, 100):
result_type, result_data = l.result(result_id, 0)
if result_type == ldap.RES_SEARCH_ENTRY:
result_set.append(result_data)
else:
break
...
I'm trying to see if the username variable in the post function matches the username in the accountsArchive entity.
I think the problem is that user.username isn't the proper way to reference the username entity. Also, the query above may have a problem. What's the proper way to see if the two usernames match?
Python
class accountsArchive(db.Model):
# The username entity
username = db.StringProperty(required = True)
password = db.TextProperty(required = True)
email = db.StringProperty(required = True)
dateJoined = db.DateTimeProperty(auto_now_add = True)
class loginPage(Handler):
def post(self):
# The username variable
username = self.request.get("username")
password = self.request.get("password")
# The query
user = db.GqlQuery("SELECT * FROM accountsArchive WHERE
user.username = :name", name=username)
# This is how I tried to check if the two usernames matched
if username == user.username:
# Do stuff
You have a number of problems in your code.
Firstly
user = db.GqlQuery("SELECT * FROM accountsArchive WHERE
user.username = :name", name=username)
Is incorrect - you should go back and reread the docs https://cloud.google.com/appengine/docs/python/datastore/gqlreference?hl=en
This query should be
user = db.GqlQuery("SELECT * FROM accountsArchive WHERE
username = :name", name=username)
Next.
The result of this line of code is an instance of GqlQuery class not a user or as you might expect a list of users. See https://cloud.google.com/appengine/docs/python/datastore/gqlqueryclass?hl=en
You now have to fetch the results and/or iterate through them.
For instance
for u in user.run():
if u.username == username:
# then do something
However you have a problem. There is nothing in this that would limit the system a single unique user. So if you get more than one user with the same username what will you do.
Some comments.
You could use the username as the key of the accountsArchive which means you just use a get rather than a query.
Secondly if you are new to appengine and don't have an existing base of code, start out using ndb instead.
I am porting an app from Google App Engine to AppScale, and have discovered peculiar behaviour when performing ancestor queries on entity groups.
If I perform an ancestor query where the parent is not the root, the query returns zero results. If I perform the same query with the parent as root, the correct results are returned.
Easiest to illustrate with an example:
class A(ndb.Model):
name = ndb.StringProperty()
class B(ndb.Model):
name = ndb.StringProperty()
class C(ndb.Model):
name = ndb.StringProperty()
active = ndb.BooleanProperty()
sort = ndb.IntegerProperty()
def main():
a = A(name='I am A')
a.put()
b = B(parent=a.key,
name='I am B')
b.put()
C(parent=b.key,
name='I am C1',
active=True,
sort=0).put()
C(parent=b.key,
name='I am C2',
active=True,
sort=1).put()
C(parent=b.key,
name='I am C3',
active=True,
sort=2).put()
query1 = C.query(C.active == True, ancestor=a.key).order(C.sort).fetch(10)
query2 = C.query(C.active == True, ancestor=b.key).order(C.sort).fetch(10)
print 'query 1 = %s' % len(query1)
print 'query 2 = %s' % len(query2)
If I run the above code on App Engine I get 3 results for both queries. If I run it on AppScale, then I only get 3 results for the first query, and 0 results for the second query.
AppScale uses Cassandra as the datastore. Is this a subtle difference in behaviour between the App Engine datastore and Cassandra?
This is a bug in AppScale where we used the full path of the provided ancestor and not just its root entity for composite queries. The fix for it can be found here:
https://github.com/AppScale/appscale/pull/1633
In django 1.6, I try test a unique field.
# model tag
class Tag(models.Model):
name = models.CharField(max_length=30, unique=True, null=True)
def __unicode__(self):
return self.name
# test unique of name field
class TagTest(TestCase):
def test_tag_unique(self):
t1 = Tag(name='music')
t1.save()
with self.assertRaises(IntegrityError):
t2 = Tag(name='music')
t2.save()
self.assertEqual(['music'], [ t.name for t in Tag.objects.all() ])
with the last line I get this message
"An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
why ?
EDIT
I get this with sqlite as DB (Development Environment).
If you're using PostgreSQL then this is why.
Edit:
See this commit. Since it's in base backend it seems like all backends now share common behavior. Despite the backend used, if the transaction needs rollback an error is raised.
Tip:
Use Model.objects.create(attr="value") instead of create and .save().
I'm using NDB on GoogleAppEngine and I want to retrieve a instance Key or ID by passing an e-mail into the query.
My Model looks something like this:
class Users(ndb.Model):
user_name = ndb.StringProperty(required=True)
user_email = ndb.StringProperty(required=True)
user_password = ndb.StringProperty(required=True)
#classmethod
def get_password_by_email(cls, email):
return Users.query(Users.user_email == email).get(projection=[Users.key, Users.user_password])
When running the code, I get the following error:
BadProjectionError: Projecting on unknown property __key__
How can I get an instance ID or Key by querying users through an e-mail in AppEngine's NDB (e.g. Login process)?
Thanks!
A projection query will always include the key as well as the fields you specify, so if keys_only isn't sufficient, then:
return Users.query(Users.user_email == email).get(projection=[Users.password])
If you only need Key you can try keys-only query:
Users.query(Users.user_email == email).get(keys_only=True)