I have a top level actor (under the guardian), called Groups which on startup needs to load the list of groups from the database and create a bunch of child actors based on those groups in the database.
I have placed the database load code inside of the preStart function as I don't want any messages to be processed before the groups are loaded.
Currently my Groups actor looks like this;
var groups: Map[String, ActorRef] = Map()
override def preStart() = {
groups = getGroupsFromDB() map createGroup
}
def createGroup(pair: (String, Long)) = {
val (name, id) = pair
val group = context.actorOf(Props(new Group(id, name)), name = name)
name -> group
}
However I don't believe this is the best way to handle this, as what happens if the database server is not available? So what is the best pratice way of handling data initialization from a database?
The Akka documentation explains how to supervise top level actors for fault-tolerance.
You can apply the principles there to manage the exceptions you may find if the DB is not available.
Related
In some contexts, entities are common to a group of apps. For example, I use a list of departments in my institution for at least 4 apps (different projects that cannot and should not be merged in a single app). Another example is the type of employes or even the list of employees.
Is it possible to create an entity accessible to every app in an easy and fast way?
I searched it, but can't find any documentation about this.
Is it related to the dotnet external use?
// the app id
var appId = 42;
// create a simple app object to then access datavar appSimple =
ToSic.SexyContent.Environment.Dnn7.Factory.App(appId);
// example getting all data of content type Tagvar tags =
appSimple.Data["Tag"];
If you are working with razor and just want to access the data in code, you can create an AppDataSource and tell it what App you need. Here's some Pseudo-code:
var otherApp = CreateSource<AppDataSource>();
otherApp.ZoneId = 74;
otherApp.AppId = 203;
// do this after setting the values
var categories = otherApp.Data["Categories"];
At some point in the future I may need to bulk load migration data (i.e. from a CSV). Has anyone had exceptions raised doing the following? Also is there any change in behaviour if the ndb.put_multi() function is used?
from google.appengine.ext import ndb
while True:
if not id:
break
id, name = read_csv_row(readline())
x = X(parent=ndb.Key('Y','static_id')
x.id, x.name = id, name
x.put()
class X(ndb.Model):
id = StringProperty()
name = StringProperty()
class Y(ndb.Model):
pass
def read_csv_row(line):
"""returns tuple"""
From my research and thanks to comments it seems that the code above (where it made into valid code) create problems which would eventually lead to google.appengine.api.datastore_errors.Timeout exceptions being thrown.
See another question:
Datastore write limit tests - trying to break app engine, but it won´t break ;)
The best suggestion I have so far is to use a Task Queue to to rate limit this. More information on:
blog.notdot.net/tag/deferred
I have a Permissions class for which I need to create a static method to get the corresponding element based on the POST method in my views.py. The choices are done via checkboxes, in which you can select either of those, a pair or all of them, based on your preferences. That creates a list of strings (u'OWNER'), which should be processed in the static method and return the corresponding Permissions.OWNER, Permissions.HR, Permissions.USER_ADMIN
My views.py's POST method looks like this:
permissions = self.request.get_all('permissions')
user.new_permission = Permissions.get_permission(permissions)
Model looks like this:
class Permissions(object):
OWNER = 'OWNER'
HR = 'HR'
USER_ADMIN = 'USER_ADMIN'
descriptions = {
OWNER: """Company owner (full admin)""",
HR: """Human Resources administrator (access to special fields within job and submissions)""",
USER_ADMIN: """Add/Delete users, change user permissions""",
}
What I have so far on the static method:
#staticmethod
def get_permissions(permissions):
new_perms = []
for permission in permissions:
name = permission
if permission ==
new_perms.append(permission)
return new_perms
I really do not know how can I compare a string to the value in the model... Nor I am sure if I have titled the question correctly.
Thank you in advance,
Borislav
There are a whole bunch of things wrong with your code, and it seems to be overcomplicated.
I suggest you do some python tutorials. I m not sure how the class definition was ever going to work. Any way here is one way you could do it.
class Permissions(object):
_perms = {
'OWNER': """Company owner (full admin)""",
'HR': """Human Resources administrator (access to special fields within job and submissions)""",
'USER_ADMIN': """Add/Delete users, change user permissions""",
}
#classmethod
def get_permissions(cls,permissions):
new_perms = []
for choice in permissions:
if choice in cls._perms:
new_perms.append(choice)
return new_perms
#classmethod
def get_description(cls,permission)
return cls._perm.get(permission)
Actually on re-reading your question I am not sure what you are really trying to do. You mention a model, but the permissions class you provide doesn't reflect that, and I assume you need to query for the object. In fact if you where using a model to define permissions you would have a Permission object for each possible Permission - maybe.
alternate strategy, but there are many and without looking in more detail at how you really plan to use permissions. (I use repose.who/what for a fairly well developed permission model). At its most basic you can use getattr with your existing class. However I not keen on it, as there are no checks in place.
class Permissions(object):
OWNER = 'OWNER'
HR = 'HR'
USER_ADMIN = 'USER_ADMIN'
#classmethod
def get_permission(cls,permission):
if hasattr(cls,permission):
return getattr(cls,permission)
else:
raise KeyError("No permission %s" % permission) # Some better exception should be used.
in my app i for one of the handler i need to get a bunch of entities and execute a function for each one of them.
i have the keys of all the enities i need. after fetching them i need to execute 1 or 2 instance methods for each one of them and this slows my app down quite a bit. doing this for 100 entities takes around 10 seconds which is way to slow.
im trying to find a way to get the entities and execute those functions in parallel to save time but im not really sure which way is the best.
i tried the _post_get_hook but the i have a future object and need to call get_result() and execute the function in the hook which works kind of ok in the sdk but gets a lot of 'maximum recursion depth exceeded while calling a Python objec' but i can't really undestand why and the error message is not really elaborate.
is the Pipeline api or ndb.Tasklets what im searching for?
atm im going by trial and error but i would be happy if someone could lead me to the right direction.
EDIT
my code is something similar to a filesystem, every folder contains other folders and files. The path of the Collections set on another entity so to serialize a collection entity i need to get the referenced entity and get the path. On a Collection the serialized_assets() function is slower the more entities it contains. If i could execute a serialize function for each contained asset side by side it would speed things up quite a bit.
class Index(ndb.Model):
path = ndb.StringProperty()
class Folder(ndb.Model):
label = ndb.StringProperty()
index = ndb.KeyProperty()
# contents is a list of keys of contaied Folders and Files
contents = ndb.StringProperty(repeated=True)
def serialized_assets(self):
assets = ndb.get_multi(self.contents)
serialized_assets = []
for a in assets:
kind = a._get_kind()
assetdict = a.to_dict()
if kind == 'Collection':
assetdict['path'] = asset.path
# other operations ...
elif kind == 'File':
assetdict['another_prop'] = asset.another_property
# ...
serialized_assets.append(assetdict)
return serialized_assets
#property
def path(self):
return self.index.get().path
class File(ndb.Model):
filename = ndb.StringProperty()
# other properties....
#property
def another_property(self):
# compute something here
return computed_property
EDIT2:
#ndb.tasklet
def serialized_assets(self, keys=None):
assets = yield ndb.get_multi_async(keys)
raise ndb.Return([asset.serialized for asset in assets])
is this tasklet code ok?
Since most of the execution time of your functions are spent waiting for RPCs, NDB's async and tasklet support is your best bet. That's described in some detail here. The simplest usage for your requirements is probably to use the ndb.map function, like this (from the docs):
#ndb.tasklet
def callback(msg):
acct = yield ndb.get_async(msg.author)
raise tasklet.Return('On %s, %s wrote:\n%s' % (msg.when, acct.nick(), msg.body))
qry = Messages.query().order(-Message.when)
outputs = qry.map(callback, limit=20)
for output in outputs:
print output
The callback function is called for each entity returned by the query, and it can do whatever operations it needs (using _async methods and yield to do them asynchronously), returning the result when it's done. Because the callback is a tasklet, and uses yield to make the asynchronous calls, NDB can run multiple instances of it in parallel, and even batch up some operations.
The pipeline API is overkill for what you want to do. Is there any reason why you couldn't just use a taskqueue?
Use the initial request to get all of the entity keys, and then enqueue a task for each key having the task execute the 2 functions per-entity. The concurrency will be based then on the number of concurrent requests as configured for that taskqueue.
I have a python program in Google App Engine
When finding an object in the datastore when I have the key as a string, how can I do a direct read. Below is my code which is performing a loop, not good.....
class Opportunity(db.Model):
customer = db.ReferenceProperty(Customer,collection_name='opportunitys')
BNusername = db.StringProperty()
opportunity_no = db.StringProperty()
# etc etc etc.....
#BnPresets holds the object key as a string
opportunitys = Opportunity.all()
opportunitys.filter('BNusername =',BnPresets.myusername)
for oprec in opportunitys:
if str(oprec.key()) == BnPresets.recordkey:
opportunity = oprec
# I have the object here and can process etc etc
You can instantiate db.Key from string by passing it directly to the constructor:
opportunity_key = db.Key(BnPresets.recordkey)
Once you have that, simply db.get to obtain the entity identified by this key:
opportunity = db.get(opportunity_key)
I guess (by looking at the query you use) that you also want to verify the username of the object you got:
if opportunity.BNusername == BnPresets.myusername
process_opportunity(opportunity)
That should be pretty much it. The bottom line is that you should use the key first - as it uniquely identifies your entity - rather than querying for some other property and iterating through results.